Merge lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug569387 into lp:percona-xtrabackup/2.1

Proposed by Sergei Glushchenko
Status: Rejected
Rejected by: Alexey Kopytov
Proposed branch: lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug569387
Merge into: lp:percona-xtrabackup/2.1
Diff against target: 1052 lines (+686/-143)
9 files modified
doc/source/innobackupex/partial_backups_innobackupex.rst (+1/-5)
doc/source/xtrabackup_bin/partial_backups.rst (+6/-1)
doc/source/xtrabackup_bin/xbk_option_reference.rst (+8/-0)
innobackupex.pl (+8/-0)
src/innodb_int.h (+2/-2)
src/xtrabackup.cc (+338/-135)
test/t/ib_databases.sh (+48/-0)
test/t/ib_databases_file.sh (+57/-0)
test/t/xb_databases_options.sh (+218/-0)
To merge this branch: bzr merge lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug569387
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Disapprove
Review via email: mp+231931@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) wrote :

Looking at the diff, changes like these are too invasive for a CS release. Will review & merge only the 2.2 MP.

review: Disapprove

Unmerged revisions

757. By Sergei Glushchenko

Bug 569387: innobackupex ignores --databases without --stream
Changes:
- two options have been added for xtrabackup:
- "--databases" is a space separated list of entries
database_name[.table_name]
- "--databases-file" is a name of file which contains entries
database_name[.table_name] one entry per line
- check_if_table_matches_filters has been modified to check whether
database is enabled first and then check the table
- databases and tables filters initialization has been refactored in
order to avoid code duplication
- innnobackupex's --databases option is mapped to one of these
options, depending on value.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'doc/source/innobackupex/partial_backups_innobackupex.rst'
2--- doc/source/innobackupex/partial_backups_innobackupex.rst 2013-11-14 12:17:21 +0000
3+++ doc/source/innobackupex/partial_backups_innobackupex.rst 2014-08-22 16:46:12 +0000
4@@ -42,7 +42,7 @@
5 Using the :option:`--databases` option
6 --------------------------------------
7
8-This option is specific to |innobackupex| and accepts either a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form - or a file containing the list at one element per line.
9+This option accepts either a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form - or a file containing the list at one element per line.
10
11 For example, ::
12
13@@ -50,10 +50,6 @@
14
15 The command above will create a timestamped directory with the usual files that |innobackupex| creates, but only containing the data-files related to ``mytable`` in the ``mydatabase`` directory and the ``mysql`` directory with the entire ``mysql`` database.
16
17-.. note::
18-
19- Currently in |Percona XtraBackup| the --databases option has no effect for InnoDB files for both local and streaming backups, i.e. all InnoDB files are always backed up. Currently, only .frm and non-InnoDB tables are limited by that option.
20-
21 Preparing Partial Backups
22 =========================
23
24
25=== modified file 'doc/source/xtrabackup_bin/partial_backups.rst'
26--- doc/source/xtrabackup_bin/partial_backups.rst 2013-04-29 11:08:57 +0000
27+++ doc/source/xtrabackup_bin/partial_backups.rst 2014-08-22 16:46:12 +0000
28@@ -2,7 +2,7 @@
29 Partial Backups
30 =================
31
32-|xtrabackup| supports taking partial backups when the :term:`innodb_file_per_table` option is enabled. There are two ways to create partial backups: matching the tables' names with a regular expression or providing a list of them in a file.
33+|xtrabackup| supports taking partial backups when the :term:`innodb_file_per_table` option is enabled. There are two ways to create partial backups: matching the tables' names with a regular expression or providing a list of them in a file or providing a list of databases.
34
35 .. warning:: If any of the matched or listed tables is deleted during the backup, |xtrabackup| will fail.
36
37@@ -31,6 +31,11 @@
38 $ echo "mydatabase.mytable" > /tmp/tables.txt
39 $ xtrabackup --backup --tables-file=/tmp/tables.txt
40
41+Using the :option:`--databases` and :option:`--databases-file` options
42+======================================================================
43+
44+The ``--databases`` option accepts a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form. The ``--databases-file`` option specifies a file that can contain multiple databases and tables in the ``databasename[.tablename]`` form, one element name per line in the file. Only named databases and tables will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching.
45+
46 Preparing the Backup
47 ====================
48
49
50=== modified file 'doc/source/xtrabackup_bin/xbk_option_reference.rst'
51--- doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-05-18 08:23:31 +0000
52+++ doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-08-22 16:46:12 +0000
53@@ -125,6 +125,14 @@
54
55 Don't read default options from any option file. Must be given as the first option on the command-line.
56
57+.. option:: --databases=#
58+
59+ This option specifies the list of databases and tables that should be backed up. The option accepts the list of the form ``"databasename1[.table_name1] databasename2[.table_name2] . . ."``.
60+
61+.. option:: --databases-file=#
62+
63+ This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form ``databasename1[.table_name1]``, one element per line.
64+
65 .. option:: --parallel=#
66
67 This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer).
68
69=== modified file 'innobackupex.pl'
70--- innobackupex.pl 2014-07-21 15:16:00 +0000
71+++ innobackupex.pl 2014-08-22 16:46:12 +0000
72@@ -2811,6 +2811,14 @@
73 $options = $options . " --compact";
74 }
75
76+ if ($option_databases) {
77+ if ($option_databases =~ /^\//) {
78+ $options = $options . " --databases_file='$option_databases'";
79+ } else {
80+ $options = $options . " --databases='$option_databases'";
81+ }
82+ }
83+
84 $cmdline = "$option_ibbackup_binary $options";
85
86 # run ibbackup as a child process
87
88=== modified file 'src/innodb_int.h'
89--- src/innodb_int.h 2014-01-10 11:13:00 +0000
90+++ src/innodb_int.h 2014-08-22 16:46:12 +0000
91@@ -305,8 +305,8 @@
92 #endif
93
94 #define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \
95- HASH_SEARCH(NAME, TABLE, FOLD, xtrabackup_tables_t*, DATA, ASSERTION, \
96- TEST)
97+ HASH_SEARCH(NAME, TABLE, FOLD, xtrabackup_filter_entry_t*, DATA, \
98+ ASSERTION, TEST)
99
100 /* The following constants have been copied from fsp0fsp.c */
101 #define FSP_HEADER_OFFSET FIL_PAGE_DATA /* Offset of the space header
102
103=== modified file 'src/xtrabackup.cc'
104--- src/xtrabackup.cc 2014-06-24 09:58:44 +0000
105+++ src/xtrabackup.cc 2014-08-22 16:46:12 +0000
106@@ -58,6 +58,7 @@
107 #endif
108
109 #include <fcntl.h>
110+#include <string.h>
111
112 #ifdef __linux__
113 # include <sys/prctl.h>
114@@ -122,21 +123,33 @@
115
116 lsn_t xtrabackup_archived_to_lsn = 0; /* for --archived-to-lsn */
117
118-char *xtrabackup_tables = NULL;
119-int tables_regex_num;
120-xb_regex_t *tables_regex;
121-xb_regmatch_t tables_regmatch[1];
122-
123-char *xtrabackup_tables_file = NULL;
124-hash_table_t* tables_hash;
125-
126-hash_table_t* inc_dir_tables_hash;
127-
128-struct xtrabackup_tables_struct{
129+static char *xtrabackup_tables = NULL;
130+
131+/* List of regular expressions for filtering */
132+typedef struct xb_regex_list_node_struct xb_regex_list_node_t;
133+struct xb_regex_list_node_struct {
134+ UT_LIST_NODE_T(xb_regex_list_node_t) regex_list;
135+ xb_regex_t regex;
136+};
137+static UT_LIST_BASE_NODE_T(xb_regex_list_node_t) regex_list;
138+
139+static xb_regmatch_t tables_regmatch[1];
140+
141+static char *xtrabackup_tables_file = NULL;
142+static hash_table_t* tables_hash = NULL;
143+
144+static char *xtrabackup_databases = NULL;
145+static char *xtrabackup_databases_file = NULL;
146+static hash_table_t* databases_hash = NULL;
147+
148+static hash_table_t* inc_dir_tables_hash;
149+
150+struct xtrabackup_filter_entry_struct{
151 char* name;
152+ ibool has_tables;
153 hash_node_t name_hash;
154 };
155-typedef struct xtrabackup_tables_struct xtrabackup_tables_t;
156+typedef struct xtrabackup_filter_entry_struct xtrabackup_filter_entry_t;
157
158 #ifdef XTRADB_BASED
159 static ulint thread_nr[SRV_MAX_N_IO_THREADS + 6 + 64];
160@@ -408,6 +421,8 @@
161 OPT_XTRA_ARCHIVED_TO_LSN,
162 OPT_XTRA_TABLES,
163 OPT_XTRA_TABLES_FILE,
164+ OPT_XTRA_DATABASES,
165+ OPT_XTRA_DATABASES_FILE,
166 OPT_XTRA_CREATE_IB_LOGFILE,
167 OPT_XTRA_PARALLEL,
168 OPT_XTRA_STREAM,
169@@ -583,6 +598,13 @@
170 {"tables_file", OPT_XTRA_TABLES_FILE, "filtering by list of the exact database.table name in the file.",
171 (G_PTR*) &xtrabackup_tables_file, (G_PTR*) &xtrabackup_tables_file,
172 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
173+ {"databases", OPT_XTRA_DATABASES, "filtering by list of databases.",
174+ (G_PTR*) &xtrabackup_databases, (G_PTR*) &xtrabackup_databases,
175+ 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
176+ {"databases_file", OPT_XTRA_TABLES_FILE,
177+ "filtering by list of databases in the file.",
178+ (G_PTR*) &xtrabackup_databases_file, (G_PTR*) &xtrabackup_databases_file,
179+ 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
180 {"create-ib-logfile", OPT_XTRA_CREATE_IB_LOGFILE, "** not work for now** creates ib_logfile* also after '--prepare'. ### If you want create ib_logfile*, only re-execute this command in same options. ###",
181 (G_PTR*) &xtrabackup_create_ib_logfile, (G_PTR*) &xtrabackup_create_ib_logfile,
182 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
183@@ -1742,16 +1764,15 @@
184 static my_bool
185 check_if_table_matches_filters(const char *name)
186 {
187- int regres;
188- int i;
189- xtrabackup_tables_t* table;
190-
191- if (xtrabackup_tables) {
192-
193- regres = REG_NOMATCH;
194-
195- for (i = 0; i < tables_regex_num; i++) {
196- regres = xb_regexec(&tables_regex[i], name, 1,
197+ int regres;
198+ xtrabackup_filter_entry_t* table;
199+ xb_regex_list_node_t* node;
200+
201+ if (UT_LIST_GET_LEN(regex_list)) {
202+ /* Check against regular expressions list */
203+ for (node = UT_LIST_GET_FIRST(regex_list); node;
204+ node = UT_LIST_GET_NEXT(regex_list, node)) {
205+ regres = xb_regexec(&node->regex, name, 1,
206 tables_regmatch, 0);
207 if (regres != REG_NOMATCH) {
208
209@@ -1760,8 +1781,8 @@
210 }
211 }
212
213- if (xtrabackup_tables_file) {
214-
215+ if (tables_hash) {
216+ /* Finally, check against full qualified tables list */
217 XB_HASH_SEARCH(name_hash, tables_hash, ut_fold_string(name),
218 table, (void) 0,
219 !strcmp(table->name, name));
220@@ -1791,7 +1812,9 @@
221 const char *ptr;
222 char *eptr;
223
224- if (xtrabackup_tables == NULL && xtrabackup_tables_file == NULL) {
225+ if (UT_LIST_GET_LEN(regex_list) == 0 &&
226+ tables_hash == NULL &&
227+ databases_hash == NULL) {
228 return(FALSE);
229 }
230
231@@ -1807,6 +1830,26 @@
232 }
233
234 strncpy(buf, dbname, FN_REFLEN);
235+ buf[tbname - 1 - dbname] = 0;
236+
237+ if (databases_hash) {
238+ /* There are some filters for databases, check them */
239+ xtrabackup_filter_entry_t* database;
240+
241+ XB_HASH_SEARCH(name_hash, databases_hash, ut_fold_string(buf),
242+ database, (void) 0,
243+ !strcmp(database->name, buf));
244+ /* Table's database isn't found, skip the table */
245+ if (!database) {
246+ return(TRUE);
247+ }
248+ /* There aren't tables specified for the database,
249+ it should be backed up entirely */
250+ if (!database->has_tables) {
251+ return(FALSE);
252+ }
253+ }
254+
255 buf[FN_REFLEN - 1] = '\0';
256 buf[tbname - 1 - dbname] = '.';
257
258@@ -2789,120 +2832,279 @@
259 return(FALSE);
260 }
261
262+/***********************************************************************
263+Allocate and initialize the entry for databases and tables filtering
264+hash tables. If memory allocation is not successful, terminate program.
265+@return pointer to the created entry. */
266+static
267+xtrabackup_filter_entry_t *
268+xb_new_filter_entry(
269+/*================*/
270+ const char* name) /*!< in: name of table/database */
271+{
272+ xtrabackup_filter_entry_t *entry;
273+ ulint namelen = strlen(name);
274+
275+ /* Length of name shouldn't be greater than 129 character,
276+ which isn't actually 129 bytes as some of UTF-8 characters
277+ for example can be encoded by 4 bytes. */
278+ ut_a(namelen <= NAME_LEN * 2 + 1);
279+
280+ entry = static_cast<xtrabackup_filter_entry_t *>
281+ (ut_malloc(sizeof(xtrabackup_filter_entry_t) + namelen + 1));
282+ memset(entry, '\0', sizeof(xtrabackup_filter_entry_t) + namelen + 1);
283+ entry->name = ((char*)entry) + sizeof(xtrabackup_filter_entry_t);
284+ strcpy(entry->name, name);
285+ entry->has_tables = FALSE;
286+
287+ return entry;
288+}
289+
290+/***********************************************************************
291+Add entry to hash table. If hash table is NULL, allocate and initialize
292+new hash table */
293+static
294+xtrabackup_filter_entry_t*
295+xb_add_filter_entry_to_hash(
296+/*========================*/
297+ const char* name, /*!< in: name of table/database */
298+ hash_table_t** hash) /*!< in/out: hash to insert into */
299+{
300+ xtrabackup_filter_entry_t* entry;
301+
302+ entry = xb_new_filter_entry(name);
303+
304+ if (UNIV_UNLIKELY(*hash == NULL)) {
305+ *hash = hash_create(1000);
306+ }
307+ HASH_INSERT(xtrabackup_filter_entry_t,
308+ name_hash, *hash,
309+ ut_fold_string(entry->name),
310+ entry);
311+
312+ return entry;
313+}
314+
315+/***********************************************************************
316+Validate name of table or database. If name is invalid, program will
317+be finished with error code */
318+static
319+void
320+xb_validate_name(
321+/*=============*/
322+ const char* name, /*!< in: name */
323+ size_t len) /*!< in: length of name */
324+{
325+ const char* p;
326+
327+ /* perform only basic validation. validate length and
328+ path symbols */
329+ if (len > NAME_LEN) {
330+ msg("xtrabackup: name `%s` is too long.\n", name);
331+ exit(EXIT_FAILURE);
332+ }
333+ p = strpbrk(name, "/\\~");
334+ if (p && p - name < NAME_LEN) {
335+ msg("xtrabackup: name `%s` is not valid.\n", name);
336+ exit(EXIT_FAILURE);
337+ }
338+}
339+
340+/***********************************************************************
341+Register new filter entry which can be either database
342+or table name. */
343+static
344+void
345+xb_register_filter_entry(
346+/*=====================*/
347+ const char* name) /*!< in: name */
348+{
349+ const char* p;
350+ size_t namelen;
351+ xtrabackup_filter_entry_t* db_entry = NULL;
352+
353+ namelen = strlen(name);
354+ if ((p = strchr(name, '.')) != NULL) {
355+ char dbname[NAME_LEN + 1];
356+
357+ xb_validate_name(name, p - name);
358+ xb_validate_name(p + 1, namelen - (p - name));
359+
360+ strncpy(dbname, name, p - name);
361+ dbname[p - name] = 0;
362+
363+ if (databases_hash) {
364+ XB_HASH_SEARCH(name_hash, databases_hash,
365+ ut_fold_string(dbname),
366+ db_entry, (void) 0,
367+ !strcmp(db_entry->name, dbname));
368+ }
369+ if (!db_entry) {
370+ db_entry = xb_add_filter_entry_to_hash(dbname,
371+ &databases_hash);
372+ }
373+ db_entry->has_tables = TRUE;
374+ xb_add_filter_entry_to_hash(name, &tables_hash);
375+ } else {
376+ xb_validate_name(name, namelen);
377+
378+ xb_add_filter_entry_to_hash(name, &databases_hash);
379+ }
380+}
381+
382+/***********************************************************************
383+Register new table for the filter. */
384+static
385+void
386+xb_register_table(
387+/*==============*/
388+ const char* name) /*!< in: name of table */
389+{
390+ if (strchr(name, '.') == NULL) {
391+ msg("xtrabackup: `%s` is not fully qualified name.\n", name);
392+ exit(EXIT_FAILURE);
393+ }
394+
395+ xb_register_filter_entry(name);
396+}
397+
398+/***********************************************************************
399+Register new regex for the filter. */
400+static
401+void
402+xb_register_regex(
403+/*==============*/
404+ const char* regex) /*!< in: regex */
405+{
406+ xb_regex_list_node_t* node;
407+ char errbuf[100];
408+ int ret;
409+
410+ node = static_cast<xb_regex_list_node_t *>
411+ (ut_malloc(sizeof(xb_regex_list_node_t)));
412+
413+ ret = xb_regcomp(&node->regex, regex, REG_EXTENDED);
414+ if (ret != 0) {
415+ xb_regerror(ret, &node->regex, errbuf, sizeof(errbuf));
416+ msg("xtrabackup: error: tables regcomp(%s): %s\n",
417+ regex, errbuf);
418+ exit(EXIT_FAILURE);
419+ }
420+
421+ UT_LIST_ADD_LAST(regex_list, regex_list, node);
422+}
423+
424+typedef void (*insert_entry_func_t)(const char*);
425+
426+/***********************************************************************
427+Scan string and load filter entries from it. */
428+static
429+void
430+xb_load_list_string(
431+/*================*/
432+ char* list, /*!< in: string representing a list */
433+ const char* delimiters, /*!< in: delimiters of entries */
434+ insert_entry_func_t ins) /*!< in: callback to add entry */
435+{
436+ char* p;
437+ char* saveptr;
438+
439+ p = strtok_r(list, delimiters, &saveptr);
440+ while (p) {
441+
442+ ins(p);
443+
444+ p = strtok_r(NULL, delimiters, &saveptr);
445+ }
446+}
447+
448+/***********************************************************************
449+Scan file and load filter entries from it. */
450+static
451+void
452+xb_load_list_file(
453+/*==============*/
454+ const char* filename, /*!< in: name of file */
455+ insert_entry_func_t ins) /*!< in: callback to add entry */
456+{
457+ char name_buf[NAME_LEN*2+2];
458+ FILE *fp;
459+
460+ /* read and store the filenames */
461+ fp = fopen(filename, "r");
462+ if (!fp) {
463+ msg("xtrabackup: cannot open %s\n",
464+ filename);
465+ exit(EXIT_FAILURE);
466+ }
467+ for (;;) {
468+ char* p = name_buf;
469+
470+ if (fgets(name_buf, sizeof(name_buf), fp) == 0) {
471+ break;
472+ }
473+
474+ p = strchr(name_buf, '\n');
475+ if (p) {
476+ *p = '\0';
477+ } else {
478+ msg("xtrabackup: `%s...` name is too long", name_buf);
479+ exit(EXIT_FAILURE);
480+ }
481+
482+ ins(name_buf);
483+ }
484+
485+ fclose(fp);
486+}
487+
488+
489 static
490 void
491 xb_filters_init()
492 {
493+ UT_LIST_INIT(regex_list);
494+
495+ if (xtrabackup_databases) {
496+ xb_load_list_string(xtrabackup_databases, " \t",
497+ xb_register_filter_entry);
498+ }
499+
500+ if (xtrabackup_databases_file) {
501+ xb_load_list_file(xtrabackup_databases_file,
502+ xb_register_filter_entry);
503+ }
504+
505 if (xtrabackup_tables) {
506- /* init regexp */
507- char *p, *next;
508- int i;
509- char errbuf[100];
510-
511- tables_regex_num = 1;
512-
513- p = xtrabackup_tables;
514- while ((p = strchr(p, ',')) != NULL) {
515- p++;
516- tables_regex_num++;
517- }
518-
519- tables_regex = static_cast<xb_regex_t *>
520- (ut_malloc(sizeof(xb_regex_t) * tables_regex_num));
521-
522- p = xtrabackup_tables;
523- for (i=0; i < tables_regex_num; i++) {
524- int rc;
525-
526- next = strchr(p, ',');
527- ut_a(next || i == tables_regex_num - 1);
528-
529- next++;
530- if (i != tables_regex_num - 1)
531- *(next - 1) = '\0';
532-
533- rc = xb_regcomp(&tables_regex[i], p,
534- REG_EXTENDED);
535-
536- if (rc) {
537-
538- xb_regerror(rc, &tables_regex[i], errbuf,
539- sizeof(errbuf));
540- msg("xtrabackup: regcomp(%s) failed: %s\n",
541- p, errbuf);
542-
543- exit(EXIT_FAILURE);
544- }
545-
546- if (i != tables_regex_num - 1)
547- *(next - 1) = ',';
548- p = next;
549- }
550+ xb_load_list_string(xtrabackup_tables, ",",
551+ xb_register_regex);
552 }
553
554 if (xtrabackup_tables_file) {
555- char name_buf[NAME_LEN*2+2];
556- FILE *fp;
557-
558- name_buf[NAME_LEN*2+1] = '\0';
559-
560- /* init tables_hash */
561- tables_hash = hash_create(1000);
562-
563- /* read and store the filenames */
564- fp = fopen(xtrabackup_tables_file,"r");
565- if (!fp) {
566- msg("xtrabackup: cannot open %s\n",
567- xtrabackup_tables_file);
568- exit(EXIT_FAILURE);
569- }
570- for (;;) {
571- xtrabackup_tables_t* table;
572- char* p = name_buf;
573-
574- if ( fgets(name_buf, NAME_LEN*2+1, fp) == 0 ) {
575- break;
576- }
577-
578- p = strchr(name_buf, '\n');
579- if (p)
580- {
581- *p = '\0';
582- }
583-
584- table = static_cast<xtrabackup_tables_t *>
585- (ut_malloc(sizeof(xtrabackup_tables_t)
586- + strlen(name_buf) + 1));
587- memset(table, '\0', sizeof(xtrabackup_tables_t) + strlen(name_buf) + 1);
588- table->name = ((char*)table) + sizeof(xtrabackup_tables_t);
589- strcpy(table->name, name_buf);
590-
591- HASH_INSERT(xtrabackup_tables_t, name_hash, tables_hash,
592- ut_fold_string(table->name), table);
593- }
594+ xb_load_list_file(xtrabackup_tables_file, xb_register_table);
595 }
596 }
597
598 static
599 void
600-xb_tables_hash_free(hash_table_t* hash)
601+xb_filter_hash_free(hash_table_t* hash)
602 {
603 ulint i;
604
605 /* free the hash elements */
606 for (i = 0; i < hash_get_n_cells(hash); i++) {
607- xtrabackup_tables_t* table;
608+ xtrabackup_filter_entry_t* table;
609
610- table = static_cast<xtrabackup_tables_t *>
611+ table = static_cast<xtrabackup_filter_entry_t *>
612 (HASH_GET_FIRST(hash, i));
613
614 while (table) {
615- xtrabackup_tables_t* prev_table = table;
616+ xtrabackup_filter_entry_t* prev_table = table;
617
618- table = static_cast<xtrabackup_tables_t *>
619+ table = static_cast<xtrabackup_filter_entry_t *>
620 (HASH_GET_NEXT(name_hash, prev_table));
621
622- HASH_DELETE(xtrabackup_tables_t, name_hash, hash,
623+ HASH_DELETE(xtrabackup_filter_entry_t, name_hash, hash,
624 ut_fold_string(prev_table->name), prev_table);
625 ut_free(prev_table);
626 }
627@@ -2918,18 +3120,19 @@
628 void
629 xb_filters_free()
630 {
631- if (xtrabackup_tables) {
632- /* free regexp */
633- int i;
634-
635- for (i = 0; i < tables_regex_num; i++) {
636- xb_regfree(&tables_regex[i]);
637- }
638- ut_free(tables_regex);
639- }
640-
641- if (xtrabackup_tables_file) {
642- xb_tables_hash_free(tables_hash);
643+ while (UT_LIST_GET_LEN(regex_list) > 0) {
644+ xb_regex_list_node_t* node = UT_LIST_GET_FIRST(regex_list);
645+ UT_LIST_REMOVE(regex_list, regex_list, node);
646+ xb_regfree(&node->regex);
647+ ut_free(node);
648+ }
649+
650+ if (tables_hash) {
651+ xb_filter_hash_free(tables_hash);
652+ }
653+
654+ if (databases_hash) {
655+ xb_filter_hash_free(databases_hash);
656 }
657 }
658
659@@ -4271,7 +4474,7 @@
660 fil_space_t* fil_space;
661 os_file_t file = 0;
662 ulint tablespace_flags;
663- xtrabackup_tables_t* table;
664+ xtrabackup_filter_entry_t* table;
665
666 ut_a(dbname != NULL ||
667 trx_sys_sys_space(space_id) ||
668@@ -4311,13 +4514,13 @@
669 }
670
671 /* remember space name for further reference */
672- table = static_cast<xtrabackup_tables_t *>
673- (ut_malloc(sizeof(xtrabackup_tables_t) +
674+ table = static_cast<xtrabackup_filter_entry_t *>
675+ (ut_malloc(sizeof(xtrabackup_filter_entry_t) +
676 strlen(dest_space_name) + 1));
677
678- table->name = ((char*)table) + sizeof(xtrabackup_tables_t);
679+ table->name = ((char*)table) + sizeof(xtrabackup_filter_entry_t);
680 strcpy(table->name, dest_space_name);
681- HASH_INSERT(xtrabackup_tables_t, name_hash, inc_dir_tables_hash,
682+ HASH_INSERT(xtrabackup_filter_entry_t, name_hash, inc_dir_tables_hash,
683 ut_fold_string(table->name), table);
684
685 mutex_enter(&fil_system->mutex);
686@@ -4651,8 +4854,8 @@
687 const char* file_name, /*!<in: file name with suffix */
688 void* arg __attribute__((unused)))
689 {
690- char name[FN_REFLEN];
691- xtrabackup_tables_t* table;
692+ char name[FN_REFLEN];
693+ xtrabackup_filter_entry_t* table;
694
695 snprintf(name, FN_REFLEN, "%s%s/%s",
696 xb_dict_prefix, db_name, file_name);
697@@ -5434,7 +5637,7 @@
698
699 if(!xtrabackup_apply_deltas()) {
700 xb_data_files_close();
701- xb_tables_hash_free(inc_dir_tables_hash);
702+ xb_filter_hash_free(inc_dir_tables_hash);
703 goto error;
704 }
705 }
706@@ -5447,7 +5650,7 @@
707
708 xb_process_datadir("./", ".ibd", rm_if_not_found, NULL);
709
710- xb_tables_hash_free(inc_dir_tables_hash);
711+ xb_filter_hash_free(inc_dir_tables_hash);
712 }
713 sync_close();
714 sync_initialized = FALSE;
715
716=== added file 'test/t/ib_databases.sh'
717--- test/t/ib_databases.sh 1970-01-01 00:00:00 +0000
718+++ test/t/ib_databases.sh 2014-08-22 16:46:12 +0000
719@@ -0,0 +1,48 @@
720+########################################################################
721+# Bug #569387: innobackupex ignores --databases
722+# Testcase covers using --databases option with InnoDB
723+# database (list is specified in option itself)
724+########################################################################
725+
726+. inc/common.sh
727+
728+start_server --innodb-file-per-table
729+
730+cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
731+
732+ CREATE DATABASE test1;
733+
734+ CREATE TABLE test1.a (a INT PRIMARY KEY);
735+ CREATE TABLE test1.b (b INT PRIMARY KEY);
736+ CREATE TABLE test1.c (c INT PRIMARY KEY);
737+
738+ CREATE TABLE test.a (a INT PRIMARY KEY);
739+ CREATE TABLE test.b (b INT PRIMARY KEY);
740+ CREATE TABLE test.c (c INT PRIMARY KEY);
741+
742+EOF
743+
744+# This is a workaround to pass --databases='test test1.b test1.c ... ' (containing spaces)
745+$IB_BIN $IB_ARGS --no-timestamp --databases='test test1.b test1.c mysql performance_schema' $topdir/backup
746+innobackupex --apply-log $topdir/backup
747+vlog "Backup taken"
748+
749+stop_server
750+
751+# Restore partial backup
752+# Remove database
753+rm -rf $mysql_datadir/*
754+vlog "Original database removed"
755+
756+# Restore database from backup
757+innobackupex --copy-back $topdir/backup
758+vlog "database restored from backup"
759+
760+start_server
761+
762+OUT=`run_cmd $MYSQL $MYSQL_ARGS -e "USE test; SHOW TABLES; USE test1; SHOW TABLES;" | tr -d '\n'`
763+
764+if [ $OUT != "Tables_in_testabcTables_in_test1bc" ] ; then
765+ vlog "Backed up tables set doesn't match filter" ;
766+ exit 1;
767+fi
768
769=== added file 'test/t/ib_databases_file.sh'
770--- test/t/ib_databases_file.sh 1970-01-01 00:00:00 +0000
771+++ test/t/ib_databases_file.sh 2014-08-22 16:46:12 +0000
772@@ -0,0 +1,57 @@
773+########################################################################
774+# Bug #569387: innobackupex ignores --databases
775+# Testcase covers using --databases option with InnoDB
776+# database (list is specified in file which option
777+# points to)
778+########################################################################
779+
780+. inc/common.sh
781+
782+start_server --innodb-file-per-table
783+
784+cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
785+
786+ CREATE DATABASE test1;
787+
788+ CREATE TABLE test1.a (a INT PRIMARY KEY) engine=InnoDB;
789+ CREATE TABLE test1.b (b INT PRIMARY KEY) engine=InnoDB;
790+ CREATE TABLE test1.c (c INT PRIMARY KEY) engine=InnoDB;
791+
792+ CREATE TABLE test.a (a INT PRIMARY KEY) engine=InnoDB;
793+ CREATE TABLE test.b (b INT PRIMARY KEY) engine=InnoDB;
794+ CREATE TABLE test.c (c INT PRIMARY KEY) engine=InnoDB;
795+
796+EOF
797+
798+# Take a backup
799+# Backup the whole test and b,c from test1
800+cat >$topdir/databases_file <<EOF
801+mysql
802+performance_schema
803+test
804+test1.b
805+test1.c
806+EOF
807+innobackupex --no-timestamp --databases=$topdir/databases_file $topdir/backup
808+innobackupex --apply-log $topdir/backup
809+vlog "Backup taken"
810+
811+stop_server
812+
813+# Restore partial backup
814+# Remove database
815+rm -rf $mysql_datadir/*
816+vlog "Original database removed"
817+
818+# Restore database from backup
819+innobackupex --copy-back $topdir/backup
820+vlog "database restored from backup"
821+
822+start_server
823+
824+OUT=`run_cmd $MYSQL $MYSQL_ARGS -e "USE test; SHOW TABLES; USE test1; SHOW TABLES;" | tr -d '\n'`
825+
826+if [ $OUT != "Tables_in_testabcTables_in_test1bc" ] ; then
827+ vlog "Backed up tables set don't match filter" ;
828+ exit 1;
829+fi
830
831=== added file 'test/t/xb_databases_options.sh'
832--- test/t/xb_databases_options.sh 1970-01-01 00:00:00 +0000
833+++ test/t/xb_databases_options.sh 2014-08-22 16:46:12 +0000
834@@ -0,0 +1,218 @@
835+########################################################################
836+# Bug #569387: xtrabackup ignores --databases, i.e. when --databases
837+#
838+# Test following xtrabackup options
839+# --databases
840+# --databases-file
841+# --tables
842+# --tables-file
843+########################################################################
844+
845+. inc/common.sh
846+
847+start_server --innodb-file-per-table
848+
849+function setup_test {
850+
851+ cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
852+
853+ create database database1;
854+ create database database2;
855+ create database test1;
856+ create database test2;
857+ create database thisisadatabase;
858+ create database testdatabase;
859+
860+ create table database1.thisisatable (a int primary key) engine=InnoDB;
861+ create table database1.t (a int primary key) engine=InnoDB;
862+ create table database1.t1 (a int primary key) engine=InnoDB;
863+ create table database1.t2 (a int primary key) engine=InnoDB;
864+ create table database1.ancor (a int primary key) engine=InnoDB;
865+ create table database1.glow (a int primary key) engine=InnoDB;
866+
867+ create table database2.thisisatable (a int primary key) engine=InnoDB;
868+ create table database2.t (a int primary key) engine=InnoDB;
869+ create table database2.t1 (a int primary key) engine=InnoDB;
870+ create table database2.t2 (a int primary key) engine=InnoDB;
871+ create table database2.ancor (a int primary key) engine=InnoDB;
872+ create table database2.glow (a int primary key) engine=InnoDB;
873+
874+ create table test1.thisisatable (a int primary key) engine=InnoDB;
875+ create table test1.t (a int primary key) engine=InnoDB;
876+ create table test1.t1 (a int primary key) engine=InnoDB;
877+ create table test1.t2 (a int primary key) engine=InnoDB;
878+ create table test1.ancor (a int primary key) engine=InnoDB;
879+ create table test1.glow (a int primary key) engine=InnoDB;
880+
881+ create table test2.thisisatable (a int primary key) engine=InnoDB;
882+ create table test2.t (a int primary key) engine=InnoDB;
883+ create table test2.t1 (a int primary key) engine=InnoDB;
884+ create table test2.t2 (a int primary key) engine=InnoDB;
885+ create table test2.ancor (a int primary key) engine=InnoDB;
886+ create table test2.glow (a int primary key) engine=InnoDB;
887+
888+ create table thisisadatabase.thisisatable (a int primary key) engine=InnoDB;
889+ create table thisisadatabase.t (a int primary key) engine=InnoDB;
890+ create table thisisadatabase.t1 (a int primary key) engine=InnoDB;
891+ create table thisisadatabase.t2 (a int primary key) engine=InnoDB;
892+ create table thisisadatabase.ancor (a int primary key) engine=InnoDB;
893+ create table thisisadatabase.glow (a int primary key) engine=InnoDB;
894+
895+ create table testdatabase.thisisatable (a int primary key) engine=InnoDB;
896+ create table testdatabase.t (a int primary key) engine=InnoDB;
897+ create table testdatabase.t1 (a int primary key) engine=InnoDB;
898+ create table testdatabase.t2 (a int primary key) engine=InnoDB;
899+ create table testdatabase.ancor (a int primary key) engine=InnoDB;
900+ create table testdatabase.glow (a int primary key) engine=InnoDB;
901+
902+EOF
903+
904+}
905+
906+function ls_dir {
907+ ( cd $1 ; find . -name '*.ibd' -type f | sort -d )
908+}
909+
910+setup_test
911+
912+backup_dir=$topdir/backup_dir
913+mkdir -p $backup_dir
914+
915+# invalid characters
916+xtrabackup --backup --databases=a/b/c --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is not valid'
917+
918+# too long name
919+xtrabackup --backup --databases=verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
920+
921+# too long name
922+xtrabackup --backup --databases=test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
923+
924+# not fully qualified name
925+xtrabackup --backup --tables-file=<(echo verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow) --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is not fully qualified'
926+
927+# again too long name
928+xtrabackup --backup --tables-file=<(echo test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow) --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
929+
930+# should go fine, we cannot validate regex against length
931+xtrabackup --backup --tables=verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir
932+rm -rf $backup_dir/*
933+
934+# should go fine, we cannot validate regex against length
935+xtrabackup --backup --tables=test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir
936+rm -rf $backup_dir/*
937+
938+vlog "Testing with --databases=..."
939+$XB_BIN $XB_ARGS --backup --databases='database database2 test1.t test1.ancor thisisadatabase testdatabase.t testdatabase.t7' --target-dir=$backup_dir --datadir=$mysql_datadir
940+diff -u <(ls_dir $backup_dir) - <<EOF
941+./database2/ancor.ibd
942+./database2/glow.ibd
943+./database2/t1.ibd
944+./database2/t2.ibd
945+./database2/thisisatable.ibd
946+./database2/t.ibd
947+./test1/ancor.ibd
948+./test1/t.ibd
949+./testdatabase/t.ibd
950+./thisisadatabase/ancor.ibd
951+./thisisadatabase/glow.ibd
952+./thisisadatabase/t1.ibd
953+./thisisadatabase/t2.ibd
954+./thisisadatabase/thisisatable.ibd
955+./thisisadatabase/t.ibd
956+EOF
957+rm -rf $backup_dir/*
958+
959+vlog "Testing with --databases-file=..."
960+cat >$topdir/list <<EOF
961+database
962+database2
963+test1.t
964+test1.ancor
965+thisisadatabase
966+testdatabase.t
967+testdatabase.t7
968+EOF
969+xtrabackup --backup --databases-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
970+diff -u <(ls_dir $backup_dir) - <<EOF
971+./database2/ancor.ibd
972+./database2/glow.ibd
973+./database2/t1.ibd
974+./database2/t2.ibd
975+./database2/thisisatable.ibd
976+./database2/t.ibd
977+./test1/ancor.ibd
978+./test1/t.ibd
979+./testdatabase/t.ibd
980+./thisisadatabase/ancor.ibd
981+./thisisadatabase/glow.ibd
982+./thisisadatabase/t1.ibd
983+./thisisadatabase/t2.ibd
984+./thisisadatabase/thisisatable.ibd
985+./thisisadatabase/t.ibd
986+EOF
987+rm -rf $backup_dir/*
988+
989+vlog "Testing failure with --tables-file=..."
990+cat >$topdir/list <<EOF
991+database
992+database2
993+test1.t
994+test1.ancor
995+thisisadatabase
996+testdatabase.t
997+testdatabase.t7
998+EOF
999+run_cmd_expect_failure $XB_BIN $XB_ARGS --backup --tables-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
1000+diff -u <(ls_dir $backup_dir) - <<EOF
1001+EOF
1002+rm -rf $backup_dir/*
1003+
1004+vlog "Testing with --tables-file=..."
1005+cat >$topdir/list <<EOF
1006+test1.t
1007+test1.ancor
1008+testdatabase.t
1009+testdatabase.t7
1010+EOF
1011+xtrabackup --backup --tables-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
1012+diff -u <(ls_dir $backup_dir) - <<EOF
1013+./test1/ancor.ibd
1014+./test1/t.ibd
1015+./testdatabase/t.ibd
1016+EOF
1017+rm -rf $backup_dir/*
1018+
1019+vlog "Testing with --tables=..."
1020+xtrabackup --backup --tables='dat.base,test1.t,test1.a' --target-dir=$backup_dir --datadir=$mysql_datadir
1021+diff -u <(ls_dir $backup_dir) - <<EOF
1022+./database1/ancor.ibd
1023+./database1/glow.ibd
1024+./database1/t1.ibd
1025+./database1/t2.ibd
1026+./database1/thisisatable.ibd
1027+./database1/t.ibd
1028+./database2/ancor.ibd
1029+./database2/glow.ibd
1030+./database2/t1.ibd
1031+./database2/t2.ibd
1032+./database2/thisisatable.ibd
1033+./database2/t.ibd
1034+./test1/ancor.ibd
1035+./test1/t1.ibd
1036+./test1/t2.ibd
1037+./test1/thisisatable.ibd
1038+./test1/t.ibd
1039+./testdatabase/ancor.ibd
1040+./testdatabase/glow.ibd
1041+./testdatabase/t1.ibd
1042+./testdatabase/t2.ibd
1043+./testdatabase/thisisatable.ibd
1044+./testdatabase/t.ibd
1045+./thisisadatabase/ancor.ibd
1046+./thisisadatabase/glow.ibd
1047+./thisisadatabase/t1.ibd
1048+./thisisadatabase/t2.ibd
1049+./thisisadatabase/thisisatable.ibd
1050+./thisisadatabase/t.ibd
1051+EOF
1052+rm -rf $backup_dir/*

Subscribers

People subscribed via source and target branches