Merge lp:~sergei.glushchenko/percona-xtrabackup/bug711166_part-1.6 into lp:percona-xtrabackup/1.6

Proposed by Sergei Glushchenko on 2012-02-01
Status: Rejected
Rejected by: Alexey Kopytov on 2012-11-16
Proposed branch: lp:~sergei.glushchenko/percona-xtrabackup/bug711166_part-1.6
Merge into: lp:percona-xtrabackup/1.6
Diff against target: 756 lines (+482/-154)
9 files modified
innobackupex (+27/-11)
test/inc/ib_part.sh (+76/-0)
test/t/ib_part_databases.sh (+42/-0)
test/t/ib_part_include.sh (+51/-0)
test/t/ib_part_include_stream.sh (+42/-0)
test/t/ib_part_tf_innodb.sh (+49/-0)
test/t/ib_part_tf_innodb_stream.sh (+43/-0)
test/t/ib_part_tf_myisam.sh (+42/-0)
xtrabackup.c (+110/-143)
To merge this branch: bzr merge lp:~sergei.glushchenko/percona-xtrabackup/bug711166_part-1.6
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) 2012-02-01 Needs Fixing on 2012-02-17
Review via email: mp+91013@code.launchpad.net

Description of the change

Bug 711166
Implement full support for backups of partitioned tables.

Fixed support for partitioned tables (both innobackupex and xtrabackup.c)
when --tables-file option specified. This also fixes support for
partitioned tables when --databases option specified. Two testcases
added. First one for MyISAM partitioned table with --tables-file option
and DATA DIRECTORY/INDEX DIRECTORY specified, second one for InnoDB
partitioned table with --tables-file option.

To post a comment you must log in.
Alexey Kopytov (akopytov) wrote :

Sergei,

Partial backups can also be created with the --include option of innobackupex (or its xtrabackup equivalent, --tables). That does not seem to be covered by the fix.

I've amended the bug report so we take that into account in the fix.

Some minor comments:

- innobackupex should be used in tests rather than IB_BIN directly

- please use --copy-back to restored when innobackupex is used to create a backup, so we also test that functionality

- you can use the checksum_table() function from common.sh

- xtrabackup.c (mostly) follows the InnoDB coding style. so the closin */ in comments should be on the last comment line, rather than on a separate one.

- typo ("Cheking")

- typo ("parttition")

review: Needs Fixing

Alexei,

> Sergei,
>
> Partial backups can also be created with the --include option of innobackupex
> (or its xtrabackup equivalent, --tables). That does not seem to be covered by
> the fix.
>

--include option passed directly to --tables option of xtrabackup.
Each database.table value passed to --tables option considered as regular
expression. So if we pass --tables=test.test, all partitions of database test
would be backed up because of test.test matches all files test/test#P#*
I think it's a good behavior.

Thanks for your comments! I'll do my best to fix tescases.

Alexey Kopytov (akopytov) wrote :

Sergei,

On 01.02.12 21:20, Sergei Glushchenko wrote:
> Alexei,
>
>> Sergei,
>>
>> Partial backups can also be created with the --include option of innobackupex
>> (or its xtrabackup equivalent, --tables). That does not seem to be covered by
>> the fix.
>>
>
> --include option passed directly to --tables option of xtrabackup.

Not always. It is handled in innobackupex for streaming backups.

> Each database.table value passed to --tables option considered as regular
> expression. So if we pass --tables=test.test, all partitions of database test
> would be backed up because of test.test matches all files test/test#P#*
> I think it's a good behavior.
>

Consider a case when I have tables 'test' and 'test_table'. So to be
able to backup only 'test', but not 'test_table', I have to use
--tables='test$'. But that won't work if 'test' is partitioned, right?

Bug 711166
Partitioned tables are not correctly handled by the --databases, --include,
--tables-file options of innobackupex, and by the --tables and
--tables-file options of xtrabackup.
Solution is to remove partition suffix (#P#...) before doing filtering.
Testcases cover variants of using filtering options with MyISAM and
InnoDB tables (to test both innobackupex and xtrabackup) with either stream
mode turned on and turned off.

Since last MP request:
 added --include option handling
 added more testcases

Jenkins build:
lp:~sergei.glushchenko/percona-xtrabackup/bug711166_part-1.6

Alexey Kopytov (akopytov) wrote :

Sergei,

   - I think you can avoid some code duplication in test cases by
     putting the table definition to a common file in the inc/ directory
     and using the default_storage_engine server variable.
   - you can also reduce the test sizes considerably by not messing with
     information_schema and performance_schema. I_S is actually not a
     storage engine, so it doesn't have any files in the data dir. And
     you could just drop the test database before shutting down the
     server and restoring, rather than just remove the entire data dir,
     so you don't have to care about performance_schema and mysql DBs. I
     think that would make test cases much smaller and easier to read,
     but without any impact on test coverage.
   - you can also make tests faster by not inserting 500 rows into the
     table. Inserting that many rows one-by-one by invoking the client
     each time actually take significant time. I think the following
     would be sufficient for your purposes:
       INSERT INTO test VALUES (1), (101), (201), (301), (401);
   - Unlike the core server, InnoDB code encloses "if" blocks in braces
     even if the block consists of only 1 statement.
   - I think changes in xtrabackup.c have to much duplication and
     unnecessary work. The whole point of those transformations is to
     check whether a specific table should be skipped. Please take a
     look at my changes to that code here
     http://bazaar.launchpad.net/~akopytov/percona-xtrabackup/compact-backups/view/head:/src/xtrabackup.c#L1550
     Perhaps it even makes sense to backport check_if_skip_table() from
     compact backups branch to 1.6/trunk and base your fix on that
     function?

review: Needs Fixing

Alexey,

> - I think you can avoid some code duplication in test cases by
> putting the table definition to a common file in the inc/ directory
> and using the default_storage_engine server variable.

yes. You're right about that.

> - you can also reduce the test sizes considerably by not messing with
> information_schema and performance_schema. I_S is actually not a
> storage engine, so it doesn't have any files in the data dir. And
> you could just drop the test database before shutting down the
> server and restoring, rather than just remove the entire data dir,
> so you don't have to care about performance_schema and mysql DBs. I
> think that would make test cases much smaller and easier to read,
> but without any impact on test coverage.

We can't just drop database `test` and keep other databases in their places.
copy-back function expects that data directory is empty when we are
restoring backup.

> - you can also make tests faster by not inserting 500 rows into the
> table. Inserting that many rows one-by-one by invoking the client
> each time actually take significant time. I think the following
> would be sufficient for your purposes:
> INSERT INTO test VALUES (1), (101), (201), (301), (401);

OK. I'll follow your advice.

> - Unlike the core server, InnoDB code encloses "if" blocks in braces
> even if the block consists of only 1 statement.

Thanks. I should print InnoDB code guidelines somewhere and put it somewhere over my eyes.

> - I think changes in xtrabackup.c have to much duplication and
> unnecessary work. The whole point of those transformations is to
> check whether a specific table should be skipped. Please take a
> look at my changes to that code here
> http://bazaar.launchpad.net/~akopytov/percona-xtrabackup/compact-
> backups/view/head:/src/xtrabackup.c#L1550
> Perhaps it even makes sense to backport check_if_skip_table() from
> compact backups branch to 1.6/trunk and base your fix on that
> function?

As I can see check_if_skip_table does pretty much the same thing, but does
in-place patching instead of copying. I tried to avoid in-place patching,
because of it is not good from my point of view to modify data which is
not belongs to us. You should look at xtrabackup_stats_func in your
branch. It contains pretty much the same code as check_if_skip_table.

I'm surely can write something similar to check_if_skip_table, but avoiding
in-place patching. What do you think about that?

Alexey Kopytov (akopytov) wrote :

Sergei,

On 17.02.12 14:05, Sergei Glushchenko wrote:
>> - you can also reduce the test sizes considerably by not messing with
>> information_schema and performance_schema. I_S is actually not a
>> storage engine, so it doesn't have any files in the data dir. And
>> you could just drop the test database before shutting down the
>> server and restoring, rather than just remove the entire data dir,
>> so you don't have to care about performance_schema and mysql DBs. I
>> think that would make test cases much smaller and easier to read,
>> but without any impact on test coverage.
>
> We can't just drop database `test` and keep other databases in their places.
> copy-back function expects that data directory is empty when we are
> restoring backup.
>

Right, which means --copy-back is useless for partial backups. It either
creates an unusable datadir, if it was empty originally, or just fails,
if it wasn't empty.

Which in turn means users are expected to restore manually from a
partial backups. And so we should do the same in the test suite?

>
>> - Unlike the core server, InnoDB code encloses "if" blocks in braces
>> even if the block consists of only 1 statement.
>
> Thanks. I should print InnoDB code guidelines somewhere and put it somewhere over my eyes.
>

The problem is that there's no such thing as InnoDB code guidelines. If
you find them, let me know! :)

>> - I think changes in xtrabackup.c have to much duplication and
>> unnecessary work. The whole point of those transformations is to
>> check whether a specific table should be skipped. Please take a
>> look at my changes to that code here
>> http://bazaar.launchpad.net/~akopytov/percona-xtrabackup/compact-
>> backups/view/head:/src/xtrabackup.c#L1550
>> Perhaps it even makes sense to backport check_if_skip_table() from
>> compact backups branch to 1.6/trunk and base your fix on that
>> function?
>
> As I can see check_if_skip_table does pretty much the same thing, but does
> in-place patching instead of copying. I tried to avoid in-place patching,
> because of it is not good from my point of view to modify data which is
> not belongs to us. You should look at xtrabackup_stats_func in your
> branch. It contains pretty much the same code as check_if_skip_table.
>
> I'm surely can write something similar to check_if_skip_table, but avoiding
> in-place patching. What do you think about that?
>

Sure, my point was that encapsulating the entire logic to decide whether
a table should be skipped or not is better than implementing some
auxiliary functions and passing the buffers around.

In-place modifications are certainly not nice, I just didn't have time
to fix that too, as I was after something completely different, so I
left that work for better times. Such as now :)

But thanks for pointing out the same code is in xtrabackup_stats_func().
I didn't notice it, because the main object of refactoring was
xtrabackup_copy_datafile().

OTOH I think malloc()s can be avoided and replaced with a
stack-allocated bufer.

Tescases were reduced and improved a little. It allowed to find mistake in testcase and implementation. (tar command in stream mode didn't relsolve symbolic likns). check_if_skip_table was implemented instead of auxiliary functions.

Jenkins:
http://jenkins.percona.com/view/Percona%20Xtrabackup/job/percona-xtrabackup-1.6-param/119/

Alexey Kopytov (akopytov) wrote :

Sergei,

As discussed previously, we will not be fixing this bug in 1.6. Please create a 2.0 MP.

Unmerged revisions

348. By Sergei Glushchenko on 2012-09-18

Partitioned tables are not correctly handled by the --databases, --include,
--tables-file options of innobackupex, and by the --tables and
--tables-file options of xtrabackup.
Solution is to remove partition suffix (#P#...) before doing filtering.
Testcases cover variants of using filtering options with MyISAM and
InnoDB tables (to test both innobackupex and xtrabackup) with either stream
mode turned on and turned off.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'innobackupex'
2--- innobackupex 2012-06-08 18:46:19 +0000
3+++ innobackupex 2012-09-18 13:09:33 +0000
4@@ -975,11 +975,9 @@
5 if($option_include) {
6 my $table_name;
7
8- $table_name = substr($file, rindex($file, '/'));
9- $table_name = substr($table_name, 1, rindex($table_name, '.') - 1);
10- $table_name = $subdir . "." . $table_name;
11+ $table_name = get_table_name($file);
12
13- if (!($table_name =~ /$option_include/)) {
14+ if (!("$subdir.$table_name" =~ /$option_include/)) {
15 print STDERR "'$file' is skipped.\n";
16 next;
17 }
18@@ -1984,7 +1982,8 @@
19 next unless check_if_required($database, $file);
20
21 if($option_include) {
22- if (!("$database.$file" =~ /$option_include/)) {
23+ my $table_name = get_table_name($file);
24+ if (!("$database.$table_name" =~ /$option_include/)) {
25 print STDERR "$database.$file is skipped because it does not match $option_include.\n";
26 next;
27 }
28@@ -2013,7 +2012,7 @@
29 my $ret = 0;
30 my $file_name = substr($file, rindex($file, '/') + 1);
31 $file_name=~s/([\$\\\" ])/\\$1/g;
32- $ret = system("cd $source_dir; tar cf - $database/$file_name") >> 8;
33+ $ret = system("cd $source_dir; tar chf - $database/$file_name") >> 8;
34 if ($ret == 1) {
35 print STDERR "$prefix If you use GNU tar, this warning can be ignored.\n";
36 # Check for non-zero exit code
37@@ -2303,6 +2302,27 @@
38 return ${$group_hash_ref}{$option_name};
39 }
40
41+# get_table_name subroutine returns table name of specified file.
42+# Parameters:
43+# $_[0] table path
44+# Return value:
45+# 1 table name
46+#
47+sub get_table_name {
48+ my $table_path = shift;
49+ my $filename;
50+ my $table;
51+
52+ # get the last component in the table pathname
53+ $filename = (reverse(split(/\//, $table_path)))[0];
54+ # get name of the table by removing file suffix
55+ $table = (split(/\./, $filename))[0];
56+ # and partition suffix
57+ $table = (split('#P#', $table))[0];
58+
59+ return $table;
60+}
61+
62 # check_if_required subroutine returns 1 if the specified database and
63 # table needs to be backed up.
64 # Parameters:
65@@ -2315,7 +2335,6 @@
66 my ( $db, $table_path ) = @_;
67 my $db_count = scalar keys %databases_list;
68 my $tbl_count = scalar keys %table_list;
69- my $filename;
70 my $table;
71
72 if ( $db_count == 0 && $tbl_count == 0 ) {
73@@ -2325,10 +2344,7 @@
74 }
75 else {
76 if ( $table_path ) {
77- # get the last component in the table pathname
78- $filename = (reverse(split(/\//, $table_path)))[0];
79- # get name of the table by removing file suffix
80- $table = (split(/\./, $filename))[0];
81+ $table = get_table_name($table_path);
82 }
83 }
84
85
86=== added file 'test/inc/ib_part.sh'
87--- test/inc/ib_part.sh 1970-01-01 00:00:00 +0000
88+++ test/inc/ib_part.sh 2012-09-18 13:09:33 +0000
89@@ -0,0 +1,76 @@
90+
91+function ib_part_schema()
92+{
93+ topdir=$1
94+ engine=$2
95+
96+ cat <<EOF
97+CREATE TABLE test (
98+ a int(11) DEFAULT NULL
99+) ENGINE=$engine DEFAULT CHARSET=latin1
100+PARTITION BY RANGE (a)
101+(PARTITION p0 VALUES LESS THAN (100) ENGINE = $engine,
102+ PARTITION P1 VALUES LESS THAN (200) ENGINE = $engine,
103+ PARTITION p2 VALUES LESS THAN (300)
104+ DATA DIRECTORY = '$topdir/ext' INDEX DIRECTORY = '$topdir/ext'
105+ ENGINE = $engine,
106+ PARTITION p3 VALUES LESS THAN (400)
107+ DATA DIRECTORY = '$topdir/ext' INDEX DIRECTORY = '$topdir/ext'
108+ ENGINE = $engine,
109+ PARTITION p4 VALUES LESS THAN MAXVALUE ENGINE = $engine);
110+EOF
111+}
112+
113+function ib_part_data()
114+{
115+ echo 'INSERT INTO test VALUES (1), (101), (201), (301), (401);';
116+}
117+
118+function ib_part_init()
119+{
120+ topdir=$1
121+ engine=$2
122+
123+ if [ -d $topdir/ext ] ; then
124+ rm -rf $topdir/ext
125+ fi
126+ mkdir -p $topdir/ext
127+
128+ ib_part_schema $topdir $engine | run_cmd $MYSQL $MYSQL_ARGS test
129+ ib_part_data $topdir $engine | run_cmd $MYSQL $MYSQL_ARGS test
130+}
131+
132+function ib_part_restore()
133+{
134+ topdir=$1
135+ mysql_datadir=$2
136+
137+ # Remove database
138+ rm -rf $mysql_datadir/test/*
139+ rm -rf $topdir/ext/*
140+ vlog "Original database removed"
141+
142+ # Restore database from backup
143+ cp -rv $topdir/backup/test/* $mysql_datadir/test
144+ vlog "database restored from backup"
145+
146+}
147+
148+function ib_part_assert_checksum()
149+{
150+ checksum_a=$1
151+
152+ vlog "Checking checksums"
153+ checksum_b=`checksum_table test test`
154+
155+ vlog "Checksums are $checksum_a and $checksum_b"
156+
157+ if [ "$checksum_a" != "$checksum_b" ]
158+ then
159+ vlog "Checksums are not equal"
160+ exit -1
161+ fi
162+
163+ vlog "Checksums are OK"
164+
165+}
166
167=== added file 'test/t/ib_part_databases.sh'
168--- test/t/ib_part_databases.sh 1970-01-01 00:00:00 +0000
169+++ test/t/ib_part_databases.sh 2012-09-18 13:09:33 +0000
170@@ -0,0 +1,42 @@
171+########################################################################
172+# Bug #711166: Partitioned tables are not correctly handled by the
173+# --databases and --tables-file options of innobackupex,
174+# and by the --tables option of xtrabackup.
175+# Testcase covers using --databases option with MyISAM
176+# database
177+########################################################################
178+
179+. inc/common.sh
180+. inc/ib_part.sh
181+
182+start_server
183+
184+if ! ${MYSQL} ${MYSQL_ARGS} -e "SHOW VARIABLES LIKE '%partition%';" | grep YES ; then
185+ echo "Requires partition support" > $SKIPPED_REASON
186+ exit $SKIPPED_EXIT_CODE
187+fi
188+
189+# Create MyISAM partitioned table with some partitions in
190+# different location
191+ib_part_init $topdir MyISAM
192+
193+# Saving the checksum of original table
194+checksum_a=`checksum_table test test`
195+
196+# Take a backup
197+cat >$topdir/databases_file <<EOF
198+test.test
199+EOF
200+innobackupex --no-timestamp --databases=$topdir/databases_file $topdir/backup
201+innobackupex --apply-log $topdir/backup
202+vlog "Backup taken"
203+
204+stop_server
205+
206+# Restore partial backup
207+ib_part_restore $topdir $mysql_datadir
208+
209+start_server
210+
211+# compare checksum
212+ib_part_assert_checksum $checksum_a
213
214=== added file 'test/t/ib_part_include.sh'
215--- test/t/ib_part_include.sh 1970-01-01 00:00:00 +0000
216+++ test/t/ib_part_include.sh 2012-09-18 13:09:33 +0000
217@@ -0,0 +1,51 @@
218+########################################################################
219+# Bug #711166: Partitioned tables are not correctly handled by the
220+# --databases and --tables-file options of innobackupex,
221+# and by the --tables option of xtrabackup.
222+# Testcase covers using --include option with InnoDB
223+# database
224+########################################################################
225+
226+. inc/common.sh
227+. inc/ib_part.sh
228+
229+start_server --innodb_file_per_table
230+
231+if ! ${MYSQL} ${MYSQL_ARGS} -e "SHOW VARIABLES LIKE '%partition%';" | grep YES ; then
232+ echo "Requires partition support" > $SKIPPED_REASON
233+ exit $SKIPPED_EXIT_CODE
234+fi
235+
236+# Create InnoDB partitioned table
237+ib_part_init $topdir InnoDB
238+
239+# Saving the checksum of original table
240+checksum_a=`checksum_table test test`
241+
242+# Take a backup
243+# Only backup of test.test table will be taken
244+cat >$topdir/tables <<EOF
245+test.test
246+EOF
247+innobackupex --no-timestamp --include='test.test$' $topdir/backup
248+innobackupex --apply-log $topdir/backup
249+vlog "Backup taken"
250+
251+# also test xtrabackup --stats work with --tables-file
252+COUNT=`xtrabackup --stats --tables='test.test$' --datadir=$topdir/backup 2>&1 \
253+ | grep table: | awk '{print $2}' | sort -u | wc -l`
254+
255+if [ $COUNT != 5 ] ; then
256+ xtrabackup --stats --tables='test.test$' --datadir=$topdir/backup
257+ vlog "xtrabackup --stats does not work. count = $COUNT"
258+ exit -1
259+fi
260+
261+stop_server
262+
263+# Restore partial backup
264+ib_part_restore $topdir $mysql_datadir
265+
266+start_server
267+
268+ib_part_assert_checksum $checksum_a
269
270=== added file 'test/t/ib_part_include_stream.sh'
271--- test/t/ib_part_include_stream.sh 1970-01-01 00:00:00 +0000
272+++ test/t/ib_part_include_stream.sh 2012-09-18 13:09:33 +0000
273@@ -0,0 +1,42 @@
274+########################################################################
275+# Bug #711166: Partitioned tables are not correctly handled by the
276+# --databases and --tables-file options of innobackupex,
277+# and by the --tables option of xtrabackup.
278+# Testcase covers using --include option with InnoDB
279+# database and --stream mode
280+########################################################################
281+
282+. inc/common.sh
283+. inc/ib_part.sh
284+
285+start_server --innodb_file_per_table
286+
287+if ! ${MYSQL} ${MYSQL_ARGS} -e "SHOW VARIABLES LIKE '%partition%';" | grep YES ; then
288+ echo "Requires partition support" > $SKIPPED_REASON
289+ exit $SKIPPED_EXIT_CODE
290+fi
291+
292+# Create MyISAM partitioned table
293+ib_part_init $topdir MyISAM
294+
295+# Saving the checksum of original table
296+checksum_a=`checksum_table test test`
297+
298+# Take a backup
299+mkdir -p $topdir/backup
300+innobackupex --stream=tar --include='test.test$' $topdir/backup > $topdir/backup/backup.tar
301+$TAR ixvf $topdir/backup/backup.tar -C $topdir/backup
302+$TAR cvhf $topdir/backup/backup11.tar $mysql_datadir/test/*
303+
304+innobackupex --apply-log $topdir/backup
305+
306+vlog "Backup taken"
307+
308+stop_server
309+
310+# Restore partial backup
311+ib_part_restore $topdir $mysql_datadir
312+
313+start_server
314+
315+ib_part_assert_checksum $checksum_a
316
317=== added file 'test/t/ib_part_tf_innodb.sh'
318--- test/t/ib_part_tf_innodb.sh 1970-01-01 00:00:00 +0000
319+++ test/t/ib_part_tf_innodb.sh 2012-09-18 13:09:33 +0000
320@@ -0,0 +1,49 @@
321+########################################################################
322+# Bug #711166: Partitioned tables are not correctly handled by the
323+# --databases and --tables-file options of innobackupex,
324+# and by the --tables option of xtrabackup.
325+# Testcase covers using --tables-file option with InnoDB
326+# database
327+########################################################################
328+
329+. inc/common.sh
330+. inc/ib_part.sh
331+
332+start_server --innodb_file_per_table
333+
334+if ! ${MYSQL} ${MYSQL_ARGS} -e "SHOW VARIABLES LIKE '%partition%';" | grep YES ; then
335+ echo "Requires partition support" > $SKIPPED_REASON
336+ exit $SKIPPED_EXIT_CODE
337+fi
338+
339+# Create InnoDB partitioned table
340+ib_part_init $topdir InnoDB
341+
342+# Saving the checksum of original table
343+checksum_a=`checksum_table test test`
344+
345+# Take a backup
346+# Only backup of test.test table will be taken
347+cat >$topdir/tables <<EOF
348+test.test
349+EOF
350+innobackupex --no-timestamp --tables-file=$topdir/tables $topdir/backup
351+innobackupex --apply-log $topdir/backup
352+vlog "Backup taken"
353+
354+COUNT=`xtrabackup --stats --tables-file=$topdir/tables --datadir=$topdir/backup 2>&1 \
355+ | grep table: | awk '{print $2}' | sort -u | wc -l`
356+
357+if [ $COUNT != 5 ] ; then
358+ vlog "xtrabackup --stats does not work"
359+ exit -1
360+fi
361+
362+stop_server
363+
364+# Restore partial backup
365+ib_part_restore $topdir $mysql_datadir
366+
367+start_server
368+
369+ib_part_assert_checksum $checksum_a
370
371=== added file 'test/t/ib_part_tf_innodb_stream.sh'
372--- test/t/ib_part_tf_innodb_stream.sh 1970-01-01 00:00:00 +0000
373+++ test/t/ib_part_tf_innodb_stream.sh 2012-09-18 13:09:33 +0000
374@@ -0,0 +1,43 @@
375+########################################################################
376+# Bug #711166: Partitioned tables are not correctly handled by the
377+# --databases and --tables-file options of innobackupex,
378+# and by the --tables option of xtrabackup.
379+# Testcase covers using --tables-file option with InnoDB
380+# database and --stream mode
381+########################################################################
382+
383+. inc/common.sh
384+. inc/ib_part.sh
385+
386+start_server --innodb_file_per_table
387+
388+if ! ${MYSQL} ${MYSQL_ARGS} -e "SHOW VARIABLES LIKE '%partition%';" | grep YES ; then
389+ echo "Requires partition support" > $SKIPPED_REASON
390+ exit $SKIPPED_EXIT_CODE
391+fi
392+
393+# Create InnoDB partitioned table
394+ib_part_init $topdir InnoDB
395+
396+# Saving the checksum of original table
397+checksum_a=`checksum_table test test`
398+
399+# Take a backup
400+# Only backup of test.test table will be taken
401+cat >$topdir/tables <<EOF
402+test.test
403+EOF
404+mkdir -p $topdir/backup
405+innobackupex --stream=tar --no-timestamp --tables-file=$topdir/tables $topdir/backup > $topdir/backup/backup.tar
406+$TAR ixvf $topdir/backup/backup.tar -C $topdir/backup
407+innobackupex --apply-log $topdir/backup
408+vlog "Backup taken"
409+
410+stop_server
411+
412+# Restore partial backup
413+ib_part_restore $topdir $mysql_datadir
414+
415+start_server
416+
417+ib_part_assert_checksum $checksum_a
418
419=== added file 'test/t/ib_part_tf_myisam.sh'
420--- test/t/ib_part_tf_myisam.sh 1970-01-01 00:00:00 +0000
421+++ test/t/ib_part_tf_myisam.sh 2012-09-18 13:09:33 +0000
422@@ -0,0 +1,42 @@
423+########################################################################
424+# Bug #711166: Partitioned tables are not correctly handled by the
425+# --databases and --tables-file options of innobackupex,
426+# and by the --tables option of xtrabackup.
427+# Testcase covers using --tables-file option with MyISAM
428+# database
429+########################################################################
430+
431+. inc/common.sh
432+. inc/ib_part.sh
433+
434+start_server
435+
436+if ! ${MYSQL} ${MYSQL_ARGS} -e "SHOW VARIABLES LIKE '%partition%';" | grep YES ; then
437+ echo "Requires partition support" > $SKIPPED_REASON
438+ exit $SKIPPED_EXIT_CODE
439+fi
440+
441+# Create MyISAM partitioned table with some partitions in
442+# different location
443+ib_part_init $topdir MyISAM
444+
445+# Saving the checksum of original table
446+checksum_a=`checksum_table test test`
447+
448+# Take a backup
449+# Only backup of test.test table will be taken
450+cat >$topdir/tables <<EOF
451+test.test
452+EOF
453+innobackupex --no-timestamp --tables-file=$topdir/tables $topdir/backup
454+innobackupex --apply-log $topdir/backup
455+vlog "Backup taken"
456+
457+stop_server
458+
459+# Restore partial backup
460+ib_part_restore $topdir $mysql_datadir
461+
462+start_server
463+
464+ib_part_assert_checksum $checksum_a
465
466=== modified file 'xtrabackup.c'
467--- xtrabackup.c 2012-05-28 16:37:00 +0000
468+++ xtrabackup.c 2012-09-18 13:09:33 +0000
469@@ -576,6 +576,15 @@
470
471 #endif /* INNODB_VERSION_SHORT */
472
473+#ifdef INNODB_VERSION_SHORT
474+#define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \
475+ HASH_SEARCH(NAME, TABLE, FOLD, xtrabackup_tables_t*, DATA, ASSERTION, \
476+ TEST)
477+#else
478+#define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \
479+ HASH_SEARCH(NAME, TABLE, FOLD, DATA, TEST)
480+#endif
481+
482 typedef struct {
483 ulint page_size;
484 } xb_delta_info_t;
485@@ -2518,6 +2527,73 @@
486 return(r);
487 }
488
489+static my_bool
490+check_if_skip_table(const char *path, const char *suffix)
491+{
492+ char buf[FN_REFLEN];
493+ const char *dbname, *tbname;
494+ const char *ptr;
495+ char *eptr;
496+ int dbname_len;
497+
498+ if (xtrabackup_tables == NULL && xtrabackup_tables_file == NULL) {
499+ return(FALSE);
500+ }
501+
502+ dbname= NULL;
503+ tbname= path;
504+ while ((ptr= strstr(tbname, SRV_PATH_SEPARATOR_STR)) != NULL) {
505+ dbname= tbname;
506+ tbname= ptr + 1;
507+ }
508+
509+ if (dbname == NULL) {
510+ return(TRUE);
511+ }
512+
513+ strncpy(buf, dbname, FN_REFLEN);
514+ buf[FN_REFLEN - 1]= 0;
515+ buf[tbname - 1 - dbname]= '.';
516+
517+ dbname_len= strlen(dbname) - strlen(suffix);
518+ if (dbname_len < 1) {
519+ return(TRUE);
520+ }
521+ buf[dbname_len - 1]= 0;
522+
523+ if ((eptr= strstr(buf, "#P")) != NULL) {
524+ *eptr= 0;
525+ }
526+
527+ if (xtrabackup_tables) {
528+ int regres = REG_NOMATCH;
529+ int i;
530+ for (i = 0; i < tables_regex_num; i++) {
531+ regres = regexec(&tables_regex[i], buf, 1,
532+ tables_regmatch, 0);
533+ if (regres != REG_NOMATCH) {
534+ break;
535+ }
536+ }
537+ if (regres == REG_NOMATCH) {
538+ return(TRUE);
539+ }
540+ }
541+
542+ if (xtrabackup_tables_file) {
543+ xtrabackup_tables_t* table;
544+
545+ XB_HASH_SEARCH(name_hash, tables_hash, ut_fold_string(buf),
546+ table, ut_ad(table->name),
547+ !strcmp(table->name, buf));
548+ if (!table) {
549+ return(TRUE);
550+ }
551+ }
552+
553+ return(FALSE);
554+}
555+
556 /***********************************************************************
557 Read meta info for an incremental delta.
558 @return TRUE on success, FALSE on failure. */
559@@ -2639,102 +2715,38 @@
560 info.page_size = 0;
561
562 #ifdef XTRADB_BASED
563- if (xtrabackup_tables && (!trx_sys_sys_space(node->space->id)))
564-#else
565- if (xtrabackup_tables && (node->space->id != 0))
566-#endif
567- { /* must backup id==0 */
568- char *p;
569- int p_len, regres= 0;
570- char *next, *prev;
571- char tmp;
572- int i;
573-
574- p = node->name;
575- prev = NULL;
576- while ((next = strstr(p, SRV_PATH_SEPARATOR_STR)) != NULL)
577- {
578- prev = p;
579- p = next + 1;
580- }
581- p_len = strlen(p) - strlen(".ibd");
582-
583- if (p_len < 1) {
584- /* unknown situation: skip filtering */
585- goto skip_filter;
586- }
587-
588- /* TODO: Fix this lazy implementation... */
589- tmp = p[p_len];
590- p[p_len] = 0;
591- *(p - 1) = '.';
592-
593- for (i = 0; i < tables_regex_num; i++) {
594- regres = regexec(&tables_regex[i], prev, 1, tables_regmatch, 0);
595- if (regres != REG_NOMATCH)
596- break;
597- }
598-
599- p[p_len] = tmp;
600- *(p - 1) = SRV_PATH_SEPARATOR;
601-
602- if ( regres == REG_NOMATCH ) {
603- printf("[%02u] Copying %s is skipped.\n",
604- thread_n, node->name);
605- return(FALSE);
606- }
607- }
608-
609-#ifdef XTRADB_BASED
610- if (xtrabackup_tables_file && (!trx_sys_sys_space(node->space->id)))
611-#else
612- if (xtrabackup_tables_file && (node->space->id != 0))
613-#endif
614- { /* must backup id==0 */
615- xtrabackup_tables_t* table;
616- char *p;
617- int p_len;
618- char *next, *prev;
619- char tmp;
620-
621- p = node->name;
622- prev = NULL;
623- while ((next = strstr(p, SRV_PATH_SEPARATOR_STR)) != NULL)
624- {
625- prev = p;
626- p = next + 1;
627- }
628- p_len = strlen(p) - strlen(".ibd");
629-
630- if (p_len < 1) {
631- /* unknown situation: skip filtering */
632- goto skip_filter;
633- }
634-
635- /* TODO: Fix this lazy implementation... */
636- tmp = p[p_len];
637- p[p_len] = 0;
638-
639- HASH_SEARCH(name_hash, tables_hash, ut_fold_string(prev),
640-#ifdef INNODB_VERSION_SHORT
641- xtrabackup_tables_t*,
642-#endif
643- table,
644-#ifdef INNODB_VERSION_SHORT
645- ut_ad(table->name),
646-#endif
647- !strcmp(table->name, prev));
648-
649- p[p_len] = tmp;
650-
651- if (!table) {
652- printf("[%02u] Copying %s is skipped.\n",
653- thread_n, node->name);
654- return(FALSE);
655- }
656- }
657-
658-skip_filter:
659+ if (xtrabackup_tables && (!trx_sys_sys_space(node->space->id))
660+#else
661+ if (xtrabackup_tables && (node->space->id != 0)
662+#endif
663+ && check_if_skip_table(node->name, "ibd")) {
664+ printf("[%02u] Copying %s is skipped.\n",
665+ thread_n, node->name);
666+ return(FALSE);
667+ }
668+
669+#ifndef INNODB_VERSION_SHORT
670+ page_size = UNIV_PAGE_SIZE;
671+ page_size_shift = UNIV_PAGE_SIZE_SHIFT;
672+#else
673+ zip_size = fil_space_get_zip_size(node->space->id);
674+ if (zip_size == ULINT_UNDEFINED) {
675+ goto skip;
676+ } else if (zip_size) {
677+ page_size = zip_size;
678+ page_size_shift = get_bit_shift(page_size);
679+ fprintf(stderr, "[%02u] %s is compressed with page size = "
680+ "%lu bytes\n", thread_n, node->name, page_size);
681+ if (page_size_shift < 10 || page_size_shift > 14) {
682+ fprintf(stderr, "[%02u] xtrabackup: Error: Invalid "
683+ "page size.\n", thread_n);
684+ ut_error;
685+ }
686+ } else {
687+ page_size = UNIV_PAGE_SIZE;
688+ page_size_shift = UNIV_PAGE_SIZE_SHIFT;
689+ }
690+#endif
691
692 #ifdef XTRADB_BASED
693 if (trx_sys_sys_space(node->space->id))
694@@ -4610,47 +4622,8 @@
695 table = dict_table_get_low(table_name);
696 mem_free(table_name);
697
698-
699- if (xtrabackup_tables) {
700- char *p;
701- int regres= 0;
702- int i;
703-
704- p = strstr(table->name, SRV_PATH_SEPARATOR_STR);
705-
706- if (p)
707- *p = '.';
708-
709- for (i = 0; i < tables_regex_num; i++) {
710- regres = regexec(&tables_regex[i], table->name, 1, tables_regmatch, 0);
711- if (regres != REG_NOMATCH)
712- break;
713- }
714-
715- if (p)
716- *p = SRV_PATH_SEPARATOR;
717-
718- if ( regres == REG_NOMATCH )
719- goto skip;
720- }
721-
722- if (xtrabackup_tables_file) {
723- xtrabackup_tables_t* xtable;
724-
725- HASH_SEARCH(name_hash, tables_hash, ut_fold_string(table->name),
726-#ifdef INNODB_VERSION_SHORT
727- xtrabackup_tables_t*,
728-#endif
729- xtable,
730-#ifdef INNODB_VERSION_SHORT
731- ut_ad(xtable->name),
732-#endif
733- !strcmp(xtable->name, table->name));
734-
735- if (!xtable)
736- goto skip;
737- }
738-
739+ if (table && check_if_skip_table(table->name, ""))
740+ goto skip;
741
742 if (table == NULL) {
743 fputs("InnoDB: Failed to load table ", stderr);
744@@ -6174,12 +6147,6 @@
745 break;
746 }
747
748- while (*p != '\0') {
749- if (*p == '.') {
750- *p = '/';
751- }
752- p++;
753- }
754 p = strchr(name_buf, '\n');
755 if (p)
756 {

Subscribers

People subscribed via source and target branches