Merge lp:~sergei.glushchenko/percona-xtrabackup/2.2-xb-bug569387 into lp:percona-xtrabackup/2.2

Proposed by Sergei Glushchenko
Status: Merged
Approved by: Alexey Kopytov
Approved revision: no longer in the source branch.
Merged at revision: 4995
Proposed branch: lp:~sergei.glushchenko/percona-xtrabackup/2.2-xb-bug569387
Merge into: lp:percona-xtrabackup/2.2
Diff against target: 1051 lines (+674/-143)
9 files modified
storage/innobase/xtrabackup/doc/source/innobackupex/partial_backups_innobackupex.rst (+1/-5)
storage/innobase/xtrabackup/doc/source/xtrabackup_bin/partial_backups.rst (+6/-1)
storage/innobase/xtrabackup/doc/source/xtrabackup_bin/xbk_option_reference.rst (+8/-0)
storage/innobase/xtrabackup/innobackupex.pl (+8/-0)
storage/innobase/xtrabackup/src/xtrabackup.cc (+329/-135)
storage/innobase/xtrabackup/test/inc/common.sh (+2/-2)
storage/innobase/xtrabackup/test/t/ib_databases.sh (+46/-0)
storage/innobase/xtrabackup/test/t/ib_databases_file.sh (+56/-0)
storage/innobase/xtrabackup/test/t/xb_databases_options.sh (+218/-0)
To merge this branch: bzr merge lp:~sergei.glushchenko/percona-xtrabackup/2.2-xb-bug569387
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Approve
Review via email: mp+231932@code.launchpad.net

Description of the change

To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) :
review: Needs Fixing
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

What will happened with inline comments after repush?

Revision history for this message
Alexey Kopytov (akopytov) wrote :

Good question. Let's try and see? :)

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

They will be available: see for example https://code.launchpad.net/~hrvojem/percona-server/rn-5.1.73-14.2-5.1/+merge/228677, there is a "Show diff comments" link on the top right corner of the "Needs Fixing" box

Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

http://jenkins.percona.com/view/PXB%202.2/job/percona-xtrabackup-2.2-param/182/
I updated the branch.
Changes are:
- xtrabackup_filter_entry_ -> xb_filter_entry_
- indentation fixes
- System character set is restricted to... comment removed
- Why not just "while (fgets(...))"? -- rewritten with while
- char* p = name_buf; is indeed redundant, removed
- Did you try replacing $@ with "$@" in innobackupex() and quoting the argument, i.e. "--databases='...'"? -- I didm't but I suspected it might work, done and it works
- Could just use die() here and other tests. -- done in 2 places
- xb_add_filter_entry_to_hash shortened to xb_add_filter

Revision history for this message
Alexey Kopytov (akopytov) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'storage/innobase/xtrabackup/doc/source/innobackupex/partial_backups_innobackupex.rst'
2--- storage/innobase/xtrabackup/doc/source/innobackupex/partial_backups_innobackupex.rst 2013-11-14 12:17:21 +0000
3+++ storage/innobase/xtrabackup/doc/source/innobackupex/partial_backups_innobackupex.rst 2014-08-26 10:16:47 +0000
4@@ -42,7 +42,7 @@
5 Using the :option:`--databases` option
6 --------------------------------------
7
8-This option is specific to |innobackupex| and accepts either a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form - or a file containing the list at one element per line.
9+This option accepts either a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form - or a file containing the list at one element per line.
10
11 For example, ::
12
13@@ -50,10 +50,6 @@
14
15 The command above will create a timestamped directory with the usual files that |innobackupex| creates, but only containing the data-files related to ``mytable`` in the ``mydatabase`` directory and the ``mysql`` directory with the entire ``mysql`` database.
16
17-.. note::
18-
19- Currently in |Percona XtraBackup| the --databases option has no effect for InnoDB files for both local and streaming backups, i.e. all InnoDB files are always backed up. Currently, only .frm and non-InnoDB tables are limited by that option.
20-
21 Preparing Partial Backups
22 =========================
23
24
25=== modified file 'storage/innobase/xtrabackup/doc/source/xtrabackup_bin/partial_backups.rst'
26--- storage/innobase/xtrabackup/doc/source/xtrabackup_bin/partial_backups.rst 2013-04-29 11:08:57 +0000
27+++ storage/innobase/xtrabackup/doc/source/xtrabackup_bin/partial_backups.rst 2014-08-26 10:16:47 +0000
28@@ -2,7 +2,7 @@
29 Partial Backups
30 =================
31
32-|xtrabackup| supports taking partial backups when the :term:`innodb_file_per_table` option is enabled. There are two ways to create partial backups: matching the tables' names with a regular expression or providing a list of them in a file.
33+|xtrabackup| supports taking partial backups when the :term:`innodb_file_per_table` option is enabled. There are three ways to create partial backups: matching the tables' names with a regular expression, providing a list of them in a file or providing a list of databases.
34
35 .. warning:: If any of the matched or listed tables is deleted during the backup, |xtrabackup| will fail.
36
37@@ -31,6 +31,11 @@
38 $ echo "mydatabase.mytable" > /tmp/tables.txt
39 $ xtrabackup --backup --tables-file=/tmp/tables.txt
40
41+Using the :option:`--databases` and :option:`--databases-file` options
42+======================================================================
43+
44+The ``--databases`` option accepts a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form. The ``--databases-file`` option specifies a file that can contain multiple databases and tables in the ``databasename[.tablename]`` form, one element name per line in the file. Only named databases and tables will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching.
45+
46 Preparing the Backup
47 ====================
48
49
50=== modified file 'storage/innobase/xtrabackup/doc/source/xtrabackup_bin/xbk_option_reference.rst'
51--- storage/innobase/xtrabackup/doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-05-18 08:23:31 +0000
52+++ storage/innobase/xtrabackup/doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-08-26 10:16:47 +0000
53@@ -125,6 +125,14 @@
54
55 Don't read default options from any option file. Must be given as the first option on the command-line.
56
57+.. option:: --databases=#
58+
59+ This option specifies the list of databases and tables that should be backed up. The option accepts the list of the form ``"databasename1[.table_name1] databasename2[.table_name2] . . ."``.
60+
61+.. option:: --databases-file=#
62+
63+ This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form ``databasename1[.table_name1]``, one element per line.
64+
65 .. option:: --parallel=#
66
67 This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer).
68
69=== modified file 'storage/innobase/xtrabackup/innobackupex.pl'
70--- storage/innobase/xtrabackup/innobackupex.pl 2014-08-22 13:10:43 +0000
71+++ storage/innobase/xtrabackup/innobackupex.pl 2014-08-26 10:16:47 +0000
72@@ -2850,6 +2850,14 @@
73 $options = $options . " --compact";
74 }
75
76+ if ($option_databases) {
77+ if ($option_databases =~ /^\//) {
78+ $options = $options . " --databases_file='$option_databases'";
79+ } else {
80+ $options = $options . " --databases='$option_databases'";
81+ }
82+ }
83+
84 $cmdline = "$option_ibbackup_binary $options";
85
86 # run ibbackup as a child process
87
88=== modified file 'storage/innobase/xtrabackup/src/xtrabackup.cc'
89--- storage/innobase/xtrabackup/src/xtrabackup.cc 2014-06-24 09:59:50 +0000
90+++ storage/innobase/xtrabackup/src/xtrabackup.cc 2014-08-26 10:16:47 +0000
91@@ -49,6 +49,7 @@
92 #include <mysqld.h>
93
94 #include <fcntl.h>
95+#include <string.h>
96
97 #ifdef __linux__
98 # include <sys/prctl.h>
99@@ -128,21 +129,33 @@
100
101 lsn_t xtrabackup_archived_to_lsn = 0; /* for --archived-to-lsn */
102
103-char *xtrabackup_tables = NULL;
104-int tables_regex_num;
105-xb_regex_t *tables_regex;
106-xb_regmatch_t tables_regmatch[1];
107-
108-char *xtrabackup_tables_file = NULL;
109-hash_table_t* tables_hash;
110-
111-hash_table_t* inc_dir_tables_hash;
112-
113-struct xtrabackup_tables_struct{
114+static char *xtrabackup_tables = NULL;
115+
116+/* List of regular expressions for filtering */
117+typedef struct xb_regex_list_node_struct xb_regex_list_node_t;
118+struct xb_regex_list_node_struct {
119+ UT_LIST_NODE_T(xb_regex_list_node_t) regex_list;
120+ xb_regex_t regex;
121+};
122+static UT_LIST_BASE_NODE_T(xb_regex_list_node_t) regex_list;
123+
124+static xb_regmatch_t tables_regmatch[1];
125+
126+static char *xtrabackup_tables_file = NULL;
127+static hash_table_t* tables_hash = NULL;
128+
129+static char *xtrabackup_databases = NULL;
130+static char *xtrabackup_databases_file = NULL;
131+static hash_table_t* databases_hash = NULL;
132+
133+static hash_table_t* inc_dir_tables_hash;
134+
135+struct xb_filter_entry_struct{
136 char* name;
137+ ibool has_tables;
138 hash_node_t name_hash;
139 };
140-typedef struct xtrabackup_tables_struct xtrabackup_tables_t;
141+typedef struct xb_filter_entry_struct xb_filter_entry_t;
142
143 static ulint thread_nr[SRV_MAX_N_IO_THREADS + 6];
144 static os_thread_id_t thread_ids[SRV_MAX_N_IO_THREADS + 6];
145@@ -399,6 +412,8 @@
146 OPT_XTRA_ARCHIVED_TO_LSN,
147 OPT_XTRA_TABLES,
148 OPT_XTRA_TABLES_FILE,
149+ OPT_XTRA_DATABASES,
150+ OPT_XTRA_DATABASES_FILE,
151 OPT_XTRA_CREATE_IB_LOGFILE,
152 OPT_XTRA_PARALLEL,
153 OPT_XTRA_STREAM,
154@@ -540,6 +555,13 @@
155 {"tables_file", OPT_XTRA_TABLES_FILE, "filtering by list of the exact database.table name in the file.",
156 (G_PTR*) &xtrabackup_tables_file, (G_PTR*) &xtrabackup_tables_file,
157 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
158+ {"databases", OPT_XTRA_DATABASES, "filtering by list of databases.",
159+ (G_PTR*) &xtrabackup_databases, (G_PTR*) &xtrabackup_databases,
160+ 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
161+ {"databases_file", OPT_XTRA_TABLES_FILE,
162+ "filtering by list of databases in the file.",
163+ (G_PTR*) &xtrabackup_databases_file, (G_PTR*) &xtrabackup_databases_file,
164+ 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
165 {"create-ib-logfile", OPT_XTRA_CREATE_IB_LOGFILE, "** not work for now** creates ib_logfile* also after '--prepare'. ### If you want create ib_logfile*, only re-execute this command in same options. ###",
166 (G_PTR*) &xtrabackup_create_ib_logfile, (G_PTR*) &xtrabackup_create_ib_logfile,
167 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
168@@ -1611,15 +1633,14 @@
169 check_if_table_matches_filters(const char *name)
170 {
171 int regres;
172- int i;
173- xtrabackup_tables_t* table;
174-
175- if (xtrabackup_tables) {
176-
177- regres = REG_NOMATCH;
178-
179- for (i = 0; i < tables_regex_num; i++) {
180- regres = xb_regexec(&tables_regex[i], name, 1,
181+ xb_filter_entry_t* table;
182+ xb_regex_list_node_t* node;
183+
184+ if (UT_LIST_GET_LEN(regex_list)) {
185+ /* Check against regular expressions list */
186+ for (node = UT_LIST_GET_FIRST(regex_list); node;
187+ node = UT_LIST_GET_NEXT(regex_list, node)) {
188+ regres = xb_regexec(&node->regex, name, 1,
189 tables_regmatch, 0);
190 if (regres != REG_NOMATCH) {
191
192@@ -1628,10 +1649,9 @@
193 }
194 }
195
196- if (xtrabackup_tables_file) {
197-
198+ if (tables_hash) {
199 HASH_SEARCH(name_hash, tables_hash, ut_fold_string(name),
200- xtrabackup_tables_t*,
201+ xb_filter_entry_t*,
202 table, (void) 0,
203 !strcmp(table->name, name));
204 if (table) {
205@@ -1660,7 +1680,9 @@
206 const char *ptr;
207 char *eptr;
208
209- if (xtrabackup_tables == NULL && xtrabackup_tables_file == NULL) {
210+ if (UT_LIST_GET_LEN(regex_list) == 0 &&
211+ tables_hash == NULL &&
212+ databases_hash == NULL) {
213 return(FALSE);
214 }
215
216@@ -1676,6 +1698,27 @@
217 }
218
219 strncpy(buf, dbname, FN_REFLEN);
220+ buf[tbname - 1 - dbname] = 0;
221+
222+ if (databases_hash) {
223+ /* There are some filters for databases, check them */
224+ xb_filter_entry_t* database;
225+
226+ HASH_SEARCH(name_hash, databases_hash, ut_fold_string(buf),
227+ xb_filter_entry_t*,
228+ database, (void) 0,
229+ !strcmp(database->name, buf));
230+ /* Table's database isn't found, skip the table */
231+ if (!database) {
232+ return(TRUE);
233+ }
234+ /* There aren't tables specified for the database,
235+ it should be backed up entirely */
236+ if (!database->has_tables) {
237+ return(FALSE);
238+ }
239+ }
240+
241 buf[FN_REFLEN - 1] = '\0';
242 buf[tbname - 1 - dbname] = '.';
243
244@@ -2597,120 +2640,270 @@
245 return(FALSE);
246 }
247
248+/***********************************************************************
249+Allocate and initialize the entry for databases and tables filtering
250+hash tables. If memory allocation is not successful, terminate program.
251+@return pointer to the created entry. */
252+static
253+xb_filter_entry_t *
254+xb_new_filter_entry(
255+/*================*/
256+ const char* name) /*!< in: name of table/database */
257+{
258+ xb_filter_entry_t *entry;
259+ ulint namelen = strlen(name);
260+
261+ ut_a(namelen <= NAME_LEN * 2 + 1);
262+
263+ entry = static_cast<xb_filter_entry_t *>
264+ (ut_malloc(sizeof(xb_filter_entry_t) + namelen + 1));
265+ memset(entry, '\0', sizeof(xb_filter_entry_t) + namelen + 1);
266+ entry->name = ((char*)entry) + sizeof(xb_filter_entry_t);
267+ strcpy(entry->name, name);
268+ entry->has_tables = FALSE;
269+
270+ return entry;
271+}
272+
273+/***********************************************************************
274+Add entry to hash table. If hash table is NULL, allocate and initialize
275+new hash table */
276+static
277+xb_filter_entry_t*
278+xb_add_filter(
279+/*========================*/
280+ const char* name, /*!< in: name of table/database */
281+ hash_table_t** hash) /*!< in/out: hash to insert into */
282+{
283+ xb_filter_entry_t* entry;
284+
285+ entry = xb_new_filter_entry(name);
286+
287+ if (UNIV_UNLIKELY(*hash == NULL)) {
288+ *hash = hash_create(1000);
289+ }
290+ HASH_INSERT(xb_filter_entry_t,
291+ name_hash, *hash,
292+ ut_fold_string(entry->name),
293+ entry);
294+
295+ return entry;
296+}
297+
298+/***********************************************************************
299+Validate name of table or database. If name is invalid, program will
300+be finished with error code */
301+static
302+void
303+xb_validate_name(
304+/*=============*/
305+ const char* name, /*!< in: name */
306+ size_t len) /*!< in: length of name */
307+{
308+ const char* p;
309+
310+ /* perform only basic validation. validate length and
311+ path symbols */
312+ if (len > NAME_LEN) {
313+ msg("xtrabackup: name `%s` is too long.\n", name);
314+ exit(EXIT_FAILURE);
315+ }
316+ p = strpbrk(name, "/\\~");
317+ if (p && p - name < NAME_LEN) {
318+ msg("xtrabackup: name `%s` is not valid.\n", name);
319+ exit(EXIT_FAILURE);
320+ }
321+}
322+
323+/***********************************************************************
324+Register new filter entry which can be either database
325+or table name. */
326+static
327+void
328+xb_register_filter_entry(
329+/*=====================*/
330+ const char* name) /*!< in: name */
331+{
332+ const char* p;
333+ size_t namelen;
334+ xb_filter_entry_t* db_entry = NULL;
335+
336+ namelen = strlen(name);
337+ if ((p = strchr(name, '.')) != NULL) {
338+ char dbname[NAME_LEN + 1];
339+
340+ xb_validate_name(name, p - name);
341+ xb_validate_name(p + 1, namelen - (p - name));
342+
343+ strncpy(dbname, name, p - name);
344+ dbname[p - name] = 0;
345+
346+ if (databases_hash) {
347+ HASH_SEARCH(name_hash, databases_hash,
348+ ut_fold_string(dbname),
349+ xb_filter_entry_t*,
350+ db_entry, (void) 0,
351+ !strcmp(db_entry->name, dbname));
352+ }
353+ if (!db_entry) {
354+ db_entry = xb_add_filter(dbname, &databases_hash);
355+ }
356+ db_entry->has_tables = TRUE;
357+ xb_add_filter(name, &tables_hash);
358+ } else {
359+ xb_validate_name(name, namelen);
360+
361+ xb_add_filter(name, &databases_hash);
362+ }
363+}
364+
365+/***********************************************************************
366+Register new table for the filter. */
367+static
368+void
369+xb_register_table(
370+/*==============*/
371+ const char* name) /*!< in: name of table */
372+{
373+ if (strchr(name, '.') == NULL) {
374+ msg("xtrabackup: `%s` is not fully qualified name.\n", name);
375+ exit(EXIT_FAILURE);
376+ }
377+
378+ xb_register_filter_entry(name);
379+}
380+
381+/***********************************************************************
382+Register new regex for the filter. */
383+static
384+void
385+xb_register_regex(
386+/*==============*/
387+ const char* regex) /*!< in: regex */
388+{
389+ xb_regex_list_node_t* node;
390+ char errbuf[100];
391+ int ret;
392+
393+ node = static_cast<xb_regex_list_node_t *>
394+ (ut_malloc(sizeof(xb_regex_list_node_t)));
395+
396+ ret = xb_regcomp(&node->regex, regex, REG_EXTENDED);
397+ if (ret != 0) {
398+ xb_regerror(ret, &node->regex, errbuf, sizeof(errbuf));
399+ msg("xtrabackup: error: tables regcomp(%s): %s\n",
400+ regex, errbuf);
401+ exit(EXIT_FAILURE);
402+ }
403+
404+ UT_LIST_ADD_LAST(regex_list, regex_list, node);
405+}
406+
407+typedef void (*insert_entry_func_t)(const char*);
408+
409+/***********************************************************************
410+Scan string and load filter entries from it. */
411+static
412+void
413+xb_load_list_string(
414+/*================*/
415+ char* list, /*!< in: string representing a list */
416+ const char* delimiters, /*!< in: delimiters of entries */
417+ insert_entry_func_t ins) /*!< in: callback to add entry */
418+{
419+ char* p;
420+ char* saveptr;
421+
422+ p = strtok_r(list, delimiters, &saveptr);
423+ while (p) {
424+
425+ ins(p);
426+
427+ p = strtok_r(NULL, delimiters, &saveptr);
428+ }
429+}
430+
431+/***********************************************************************
432+Scan file and load filter entries from it. */
433+static
434+void
435+xb_load_list_file(
436+/*==============*/
437+ const char* filename, /*!< in: name of file */
438+ insert_entry_func_t ins) /*!< in: callback to add entry */
439+{
440+ char name_buf[NAME_LEN*2+2];
441+ FILE* fp;
442+
443+ /* read and store the filenames */
444+ fp = fopen(filename, "r");
445+ if (!fp) {
446+ msg("xtrabackup: cannot open %s\n",
447+ filename);
448+ exit(EXIT_FAILURE);
449+ }
450+ while (fgets(name_buf, sizeof(name_buf), fp) != NULL) {
451+ char* p = strchr(name_buf, '\n');
452+ if (p) {
453+ *p = '\0';
454+ } else {
455+ msg("xtrabackup: `%s...` name is too long", name_buf);
456+ exit(EXIT_FAILURE);
457+ }
458+
459+ ins(name_buf);
460+ }
461+
462+ fclose(fp);
463+}
464+
465+
466 static
467 void
468 xb_filters_init()
469 {
470+ UT_LIST_INIT(regex_list);
471+
472+ if (xtrabackup_databases) {
473+ xb_load_list_string(xtrabackup_databases, " \t",
474+ xb_register_filter_entry);
475+ }
476+
477+ if (xtrabackup_databases_file) {
478+ xb_load_list_file(xtrabackup_databases_file,
479+ xb_register_filter_entry);
480+ }
481+
482 if (xtrabackup_tables) {
483- /* init regexp */
484- char *p, *next;
485- int i;
486- char errbuf[100];
487-
488- tables_regex_num = 1;
489-
490- p = xtrabackup_tables;
491- while ((p = strchr(p, ',')) != NULL) {
492- p++;
493- tables_regex_num++;
494- }
495-
496- tables_regex = static_cast<xb_regex_t *>
497- (ut_malloc(sizeof(xb_regex_t) * tables_regex_num));
498-
499- p = xtrabackup_tables;
500- for (i=0; i < tables_regex_num; i++) {
501- int rc;
502-
503- next = strchr(p, ',');
504- ut_a(next || i == tables_regex_num - 1);
505-
506- next++;
507- if (i != tables_regex_num - 1)
508- *(next - 1) = '\0';
509-
510- rc = xb_regcomp(&tables_regex[i], p,
511- REG_EXTENDED);
512-
513- if (rc) {
514-
515- xb_regerror(rc, &tables_regex[i], errbuf,
516- sizeof(errbuf));
517- msg("xtrabackup: regcomp(%s) failed: %s\n",
518- p, errbuf);
519-
520- exit(EXIT_FAILURE);
521- }
522-
523- if (i != tables_regex_num - 1)
524- *(next - 1) = ',';
525- p = next;
526- }
527+ xb_load_list_string(xtrabackup_tables, ",",
528+ xb_register_regex);
529 }
530
531 if (xtrabackup_tables_file) {
532- char name_buf[NAME_LEN*2+2];
533- FILE *fp;
534-
535- name_buf[NAME_LEN*2+1] = '\0';
536-
537- /* init tables_hash */
538- tables_hash = hash_create(1000);
539-
540- /* read and store the filenames */
541- fp = fopen(xtrabackup_tables_file,"r");
542- if (!fp) {
543- msg("xtrabackup: cannot open %s\n",
544- xtrabackup_tables_file);
545- exit(EXIT_FAILURE);
546- }
547- for (;;) {
548- xtrabackup_tables_t* table;
549- char* p = name_buf;
550-
551- if ( fgets(name_buf, NAME_LEN*2+1, fp) == 0 ) {
552- break;
553- }
554-
555- p = strchr(name_buf, '\n');
556- if (p)
557- {
558- *p = '\0';
559- }
560-
561- table = static_cast<xtrabackup_tables_t *>
562- (ut_malloc(sizeof(xtrabackup_tables_t)
563- + strlen(name_buf) + 1));
564- memset(table, '\0', sizeof(xtrabackup_tables_t) + strlen(name_buf) + 1);
565- table->name = ((char*)table) + sizeof(xtrabackup_tables_t);
566- strcpy(table->name, name_buf);
567-
568- HASH_INSERT(xtrabackup_tables_t, name_hash, tables_hash,
569- ut_fold_string(table->name), table);
570- }
571+ xb_load_list_file(xtrabackup_tables_file, xb_register_table);
572 }
573 }
574
575 static
576 void
577-xb_tables_hash_free(hash_table_t* hash)
578+xb_filter_hash_free(hash_table_t* hash)
579 {
580 ulint i;
581
582 /* free the hash elements */
583 for (i = 0; i < hash_get_n_cells(hash); i++) {
584- xtrabackup_tables_t* table;
585+ xb_filter_entry_t* table;
586
587- table = static_cast<xtrabackup_tables_t *>
588+ table = static_cast<xb_filter_entry_t *>
589 (HASH_GET_FIRST(hash, i));
590
591 while (table) {
592- xtrabackup_tables_t* prev_table = table;
593+ xb_filter_entry_t* prev_table = table;
594
595- table = static_cast<xtrabackup_tables_t *>
596+ table = static_cast<xb_filter_entry_t *>
597 (HASH_GET_NEXT(name_hash, prev_table));
598
599- HASH_DELETE(xtrabackup_tables_t, name_hash, hash,
600+ HASH_DELETE(xb_filter_entry_t, name_hash, hash,
601 ut_fold_string(prev_table->name), prev_table);
602 ut_free(prev_table);
603 }
604@@ -2726,18 +2919,19 @@
605 void
606 xb_filters_free()
607 {
608- if (xtrabackup_tables) {
609- /* free regexp */
610- int i;
611-
612- for (i = 0; i < tables_regex_num; i++) {
613- xb_regfree(&tables_regex[i]);
614- }
615- ut_free(tables_regex);
616- }
617-
618- if (xtrabackup_tables_file) {
619- xb_tables_hash_free(tables_hash);
620+ while (UT_LIST_GET_LEN(regex_list) > 0) {
621+ xb_regex_list_node_t* node = UT_LIST_GET_FIRST(regex_list);
622+ UT_LIST_REMOVE(regex_list, regex_list, node);
623+ xb_regfree(&node->regex);
624+ ut_free(node);
625+ }
626+
627+ if (tables_hash) {
628+ xb_filter_hash_free(tables_hash);
629+ }
630+
631+ if (databases_hash) {
632+ xb_filter_hash_free(databases_hash);
633 }
634 }
635
636@@ -4239,7 +4433,7 @@
637 fil_space_t* fil_space;
638 os_file_t file = 0;
639 ulint tablespace_flags;
640- xtrabackup_tables_t* table;
641+ xb_filter_entry_t* table;
642
643 ut_a(dbname != NULL ||
644 !fil_is_user_tablespace_id(space_id) ||
645@@ -4278,13 +4472,13 @@
646 }
647
648 /* remember space name for further reference */
649- table = static_cast<xtrabackup_tables_t *>
650- (ut_malloc(sizeof(xtrabackup_tables_t) +
651+ table = static_cast<xb_filter_entry_t *>
652+ (ut_malloc(sizeof(xb_filter_entry_t) +
653 strlen(dest_space_name) + 1));
654
655- table->name = ((char*)table) + sizeof(xtrabackup_tables_t);
656+ table->name = ((char*)table) + sizeof(xb_filter_entry_t);
657 strcpy(table->name, dest_space_name);
658- HASH_INSERT(xtrabackup_tables_t, name_hash, inc_dir_tables_hash,
659+ HASH_INSERT(xb_filter_entry_t, name_hash, inc_dir_tables_hash,
660 ut_fold_string(table->name), table);
661
662 mutex_enter(&fil_system->mutex);
663@@ -4610,14 +4804,14 @@
664 void* arg __attribute__((unused)))
665 {
666 char name[FN_REFLEN];
667- xtrabackup_tables_t* table;
668+ xb_filter_entry_t* table;
669
670 snprintf(name, FN_REFLEN, "%s/%s", db_name, file_name);
671 /* Truncate ".ibd" */
672 name[strlen(name) - 4] = '\0';
673
674 HASH_SEARCH(name_hash, inc_dir_tables_hash, ut_fold_string(name),
675- xtrabackup_tables_t*,
676+ xb_filter_entry_t*,
677 table, (void) 0,
678 !strcmp(table->name, name));
679
680@@ -5370,7 +5564,7 @@
681
682 if(!xtrabackup_apply_deltas()) {
683 xb_data_files_close();
684- xb_tables_hash_free(inc_dir_tables_hash);
685+ xb_filter_hash_free(inc_dir_tables_hash);
686 goto error;
687 }
688 }
689@@ -5383,7 +5577,7 @@
690
691 xb_process_datadir("./", ".ibd", rm_if_not_found, NULL);
692
693- xb_tables_hash_free(inc_dir_tables_hash);
694+ xb_filter_hash_free(inc_dir_tables_hash);
695 }
696 sync_close();
697 sync_initialized = FALSE;
698
699=== modified file 'storage/innobase/xtrabackup/test/inc/common.sh'
700--- storage/innobase/xtrabackup/test/inc/common.sh 2013-11-27 10:55:41 +0000
701+++ storage/innobase/xtrabackup/test/inc/common.sh 2014-08-26 10:16:47 +0000
702@@ -2,12 +2,12 @@
703
704 function innobackupex()
705 {
706- run_cmd $IB_BIN $IB_ARGS $*
707+ run_cmd $IB_BIN $IB_ARGS "$@"
708 }
709
710 function xtrabackup()
711 {
712- run_cmd $XB_BIN $XB_ARGS $*
713+ run_cmd $XB_BIN $XB_ARGS "$@"
714 }
715
716 function vlog
717
718=== added file 'storage/innobase/xtrabackup/test/t/ib_databases.sh'
719--- storage/innobase/xtrabackup/test/t/ib_databases.sh 1970-01-01 00:00:00 +0000
720+++ storage/innobase/xtrabackup/test/t/ib_databases.sh 2014-08-26 10:16:47 +0000
721@@ -0,0 +1,46 @@
722+########################################################################
723+# Bug #569387: innobackupex ignores --databases
724+# Testcase covers using --databases option with InnoDB
725+# database (list is specified in option itself)
726+########################################################################
727+
728+. inc/common.sh
729+
730+start_server --innodb-file-per-table
731+
732+cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
733+
734+ CREATE DATABASE test1;
735+
736+ CREATE TABLE test1.a (a INT PRIMARY KEY);
737+ CREATE TABLE test1.b (b INT PRIMARY KEY);
738+ CREATE TABLE test1.c (c INT PRIMARY KEY);
739+
740+ CREATE TABLE test.a (a INT PRIMARY KEY);
741+ CREATE TABLE test.b (b INT PRIMARY KEY);
742+ CREATE TABLE test.c (c INT PRIMARY KEY);
743+
744+EOF
745+
746+innobackupex --no-timestamp --databases=test\ test1.b\ test1.c\ mysql\ performance_schema $topdir/backup
747+innobackupex --apply-log $topdir/backup
748+vlog "Backup taken"
749+
750+stop_server
751+
752+# Restore partial backup
753+# Remove database
754+rm -rf $mysql_datadir/*
755+vlog "Original database removed"
756+
757+# Restore database from backup
758+innobackupex --copy-back $topdir/backup
759+vlog "database restored from backup"
760+
761+start_server
762+
763+OUT=`run_cmd $MYSQL $MYSQL_ARGS -e "USE test; SHOW TABLES; USE test1; SHOW TABLES;" | tr -d '\n'`
764+
765+if [ $OUT != "Tables_in_testabcTables_in_test1bc" ] ; then
766+ die "Backed up tables set doesn't match filter"
767+fi
768
769=== added file 'storage/innobase/xtrabackup/test/t/ib_databases_file.sh'
770--- storage/innobase/xtrabackup/test/t/ib_databases_file.sh 1970-01-01 00:00:00 +0000
771+++ storage/innobase/xtrabackup/test/t/ib_databases_file.sh 2014-08-26 10:16:47 +0000
772@@ -0,0 +1,56 @@
773+########################################################################
774+# Bug #569387: innobackupex ignores --databases
775+# Testcase covers using --databases option with InnoDB
776+# database (list is specified in file which option
777+# points to)
778+########################################################################
779+
780+. inc/common.sh
781+
782+start_server --innodb-file-per-table
783+
784+cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
785+
786+ CREATE DATABASE test1;
787+
788+ CREATE TABLE test1.a (a INT PRIMARY KEY) engine=InnoDB;
789+ CREATE TABLE test1.b (b INT PRIMARY KEY) engine=InnoDB;
790+ CREATE TABLE test1.c (c INT PRIMARY KEY) engine=InnoDB;
791+
792+ CREATE TABLE test.a (a INT PRIMARY KEY) engine=InnoDB;
793+ CREATE TABLE test.b (b INT PRIMARY KEY) engine=InnoDB;
794+ CREATE TABLE test.c (c INT PRIMARY KEY) engine=InnoDB;
795+
796+EOF
797+
798+# Take a backup
799+# Backup the whole test and b,c from test1
800+cat >$topdir/databases_file <<EOF
801+mysql
802+performance_schema
803+test
804+test1.b
805+test1.c
806+EOF
807+innobackupex --no-timestamp --databases=$topdir/databases_file $topdir/backup
808+innobackupex --apply-log $topdir/backup
809+vlog "Backup taken"
810+
811+stop_server
812+
813+# Restore partial backup
814+# Remove database
815+rm -rf $mysql_datadir/*
816+vlog "Original database removed"
817+
818+# Restore database from backup
819+innobackupex --copy-back $topdir/backup
820+vlog "database restored from backup"
821+
822+start_server
823+
824+OUT=`run_cmd $MYSQL $MYSQL_ARGS -e "USE test; SHOW TABLES; USE test1; SHOW TABLES;" | tr -d '\n'`
825+
826+if [ $OUT != "Tables_in_testabcTables_in_test1bc" ] ; then
827+ die "Backed up tables set doesn't match filter"
828+fi
829
830=== added file 'storage/innobase/xtrabackup/test/t/xb_databases_options.sh'
831--- storage/innobase/xtrabackup/test/t/xb_databases_options.sh 1970-01-01 00:00:00 +0000
832+++ storage/innobase/xtrabackup/test/t/xb_databases_options.sh 2014-08-26 10:16:47 +0000
833@@ -0,0 +1,218 @@
834+########################################################################
835+# Bug #569387: xtrabackup ignores --databases, i.e. when --databases
836+#
837+# Test following xtrabackup options
838+# --databases
839+# --databases-file
840+# --tables
841+# --tables-file
842+########################################################################
843+
844+. inc/common.sh
845+
846+start_server --innodb-file-per-table
847+
848+function setup_test {
849+
850+ cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
851+
852+ create database database1;
853+ create database database2;
854+ create database test1;
855+ create database test2;
856+ create database thisisadatabase;
857+ create database testdatabase;
858+
859+ create table database1.thisisatable (a int primary key) engine=InnoDB;
860+ create table database1.t (a int primary key) engine=InnoDB;
861+ create table database1.t1 (a int primary key) engine=InnoDB;
862+ create table database1.t2 (a int primary key) engine=InnoDB;
863+ create table database1.ancor (a int primary key) engine=InnoDB;
864+ create table database1.glow (a int primary key) engine=InnoDB;
865+
866+ create table database2.thisisatable (a int primary key) engine=InnoDB;
867+ create table database2.t (a int primary key) engine=InnoDB;
868+ create table database2.t1 (a int primary key) engine=InnoDB;
869+ create table database2.t2 (a int primary key) engine=InnoDB;
870+ create table database2.ancor (a int primary key) engine=InnoDB;
871+ create table database2.glow (a int primary key) engine=InnoDB;
872+
873+ create table test1.thisisatable (a int primary key) engine=InnoDB;
874+ create table test1.t (a int primary key) engine=InnoDB;
875+ create table test1.t1 (a int primary key) engine=InnoDB;
876+ create table test1.t2 (a int primary key) engine=InnoDB;
877+ create table test1.ancor (a int primary key) engine=InnoDB;
878+ create table test1.glow (a int primary key) engine=InnoDB;
879+
880+ create table test2.thisisatable (a int primary key) engine=InnoDB;
881+ create table test2.t (a int primary key) engine=InnoDB;
882+ create table test2.t1 (a int primary key) engine=InnoDB;
883+ create table test2.t2 (a int primary key) engine=InnoDB;
884+ create table test2.ancor (a int primary key) engine=InnoDB;
885+ create table test2.glow (a int primary key) engine=InnoDB;
886+
887+ create table thisisadatabase.thisisatable (a int primary key) engine=InnoDB;
888+ create table thisisadatabase.t (a int primary key) engine=InnoDB;
889+ create table thisisadatabase.t1 (a int primary key) engine=InnoDB;
890+ create table thisisadatabase.t2 (a int primary key) engine=InnoDB;
891+ create table thisisadatabase.ancor (a int primary key) engine=InnoDB;
892+ create table thisisadatabase.glow (a int primary key) engine=InnoDB;
893+
894+ create table testdatabase.thisisatable (a int primary key) engine=InnoDB;
895+ create table testdatabase.t (a int primary key) engine=InnoDB;
896+ create table testdatabase.t1 (a int primary key) engine=InnoDB;
897+ create table testdatabase.t2 (a int primary key) engine=InnoDB;
898+ create table testdatabase.ancor (a int primary key) engine=InnoDB;
899+ create table testdatabase.glow (a int primary key) engine=InnoDB;
900+
901+EOF
902+
903+}
904+
905+function ls_dir {
906+ ( cd $1 ; find . -name '*.ibd' -type f | sort -d )
907+}
908+
909+setup_test
910+
911+backup_dir=$topdir/backup_dir
912+mkdir -p $backup_dir
913+
914+# invalid characters
915+xtrabackup --backup --databases=a/b/c --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is not valid'
916+
917+# too long name
918+xtrabackup --backup --databases=verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
919+
920+# too long name
921+xtrabackup --backup --databases=test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
922+
923+# not fully qualified name
924+xtrabackup --backup --tables-file=<(echo verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow) --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is not fully qualified'
925+
926+# again too long name
927+xtrabackup --backup --tables-file=<(echo test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow) --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
928+
929+# should go fine, we cannot validate regex against length
930+xtrabackup --backup --tables=verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir
931+rm -rf $backup_dir/*
932+
933+# should go fine, we cannot validate regex against length
934+xtrabackup --backup --tables=test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir
935+rm -rf $backup_dir/*
936+
937+vlog "Testing with --databases=..."
938+$XB_BIN $XB_ARGS --backup --databases='database database2 test1.t test1.ancor thisisadatabase testdatabase.t testdatabase.t7' --target-dir=$backup_dir --datadir=$mysql_datadir
939+diff -u <(ls_dir $backup_dir) - <<EOF
940+./database2/ancor.ibd
941+./database2/glow.ibd
942+./database2/t1.ibd
943+./database2/t2.ibd
944+./database2/thisisatable.ibd
945+./database2/t.ibd
946+./test1/ancor.ibd
947+./test1/t.ibd
948+./testdatabase/t.ibd
949+./thisisadatabase/ancor.ibd
950+./thisisadatabase/glow.ibd
951+./thisisadatabase/t1.ibd
952+./thisisadatabase/t2.ibd
953+./thisisadatabase/thisisatable.ibd
954+./thisisadatabase/t.ibd
955+EOF
956+rm -rf $backup_dir/*
957+
958+vlog "Testing with --databases-file=..."
959+cat >$topdir/list <<EOF
960+database
961+database2
962+test1.t
963+test1.ancor
964+thisisadatabase
965+testdatabase.t
966+testdatabase.t7
967+EOF
968+xtrabackup --backup --databases-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
969+diff -u <(ls_dir $backup_dir) - <<EOF
970+./database2/ancor.ibd
971+./database2/glow.ibd
972+./database2/t1.ibd
973+./database2/t2.ibd
974+./database2/thisisatable.ibd
975+./database2/t.ibd
976+./test1/ancor.ibd
977+./test1/t.ibd
978+./testdatabase/t.ibd
979+./thisisadatabase/ancor.ibd
980+./thisisadatabase/glow.ibd
981+./thisisadatabase/t1.ibd
982+./thisisadatabase/t2.ibd
983+./thisisadatabase/thisisatable.ibd
984+./thisisadatabase/t.ibd
985+EOF
986+rm -rf $backup_dir/*
987+
988+vlog "Testing failure with --tables-file=..."
989+cat >$topdir/list <<EOF
990+database
991+database2
992+test1.t
993+test1.ancor
994+thisisadatabase
995+testdatabase.t
996+testdatabase.t7
997+EOF
998+run_cmd_expect_failure $XB_BIN $XB_ARGS --backup --tables-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
999+diff -u <(ls_dir $backup_dir) - <<EOF
1000+EOF
1001+rm -rf $backup_dir/*
1002+
1003+vlog "Testing with --tables-file=..."
1004+cat >$topdir/list <<EOF
1005+test1.t
1006+test1.ancor
1007+testdatabase.t
1008+testdatabase.t7
1009+EOF
1010+xtrabackup --backup --tables-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
1011+diff -u <(ls_dir $backup_dir) - <<EOF
1012+./test1/ancor.ibd
1013+./test1/t.ibd
1014+./testdatabase/t.ibd
1015+EOF
1016+rm -rf $backup_dir/*
1017+
1018+vlog "Testing with --tables=..."
1019+xtrabackup --backup --tables='dat.base,test1.t,test1.a' --target-dir=$backup_dir --datadir=$mysql_datadir
1020+diff -u <(ls_dir $backup_dir) - <<EOF
1021+./database1/ancor.ibd
1022+./database1/glow.ibd
1023+./database1/t1.ibd
1024+./database1/t2.ibd
1025+./database1/thisisatable.ibd
1026+./database1/t.ibd
1027+./database2/ancor.ibd
1028+./database2/glow.ibd
1029+./database2/t1.ibd
1030+./database2/t2.ibd
1031+./database2/thisisatable.ibd
1032+./database2/t.ibd
1033+./test1/ancor.ibd
1034+./test1/t1.ibd
1035+./test1/t2.ibd
1036+./test1/thisisatable.ibd
1037+./test1/t.ibd
1038+./testdatabase/ancor.ibd
1039+./testdatabase/glow.ibd
1040+./testdatabase/t1.ibd
1041+./testdatabase/t2.ibd
1042+./testdatabase/thisisatable.ibd
1043+./testdatabase/t.ibd
1044+./thisisadatabase/ancor.ibd
1045+./thisisadatabase/glow.ibd
1046+./thisisadatabase/t1.ibd
1047+./thisisadatabase/t2.ibd
1048+./thisisadatabase/thisisatable.ibd
1049+./thisisadatabase/t.ibd
1050+EOF
1051+rm -rf $backup_dir/*

Subscribers

People subscribed via source and target branches