Merge lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug569387 into lp:percona-xtrabackup/2.1

Proposed by Sergei Glushchenko
Status: Rejected
Rejected by: Alexey Kopytov
Proposed branch: lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug569387
Merge into: lp:percona-xtrabackup/2.1
Diff against target: 1052 lines (+686/-143)
9 files modified
doc/source/innobackupex/partial_backups_innobackupex.rst (+1/-5)
doc/source/xtrabackup_bin/partial_backups.rst (+6/-1)
doc/source/xtrabackup_bin/xbk_option_reference.rst (+8/-0)
innobackupex.pl (+8/-0)
src/innodb_int.h (+2/-2)
src/xtrabackup.cc (+338/-135)
test/t/ib_databases.sh (+48/-0)
test/t/ib_databases_file.sh (+57/-0)
test/t/xb_databases_options.sh (+218/-0)
To merge this branch: bzr merge lp:~sergei.glushchenko/percona-xtrabackup/2.1-xb-bug569387
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Disapprove
Review via email: mp+231931@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) wrote :

Looking at the diff, changes like these are too invasive for a CS release. Will review & merge only the 2.2 MP.

review: Disapprove

Unmerged revisions

757. By Sergei Glushchenko

Bug 569387: innobackupex ignores --databases without --stream
Changes:
- two options have been added for xtrabackup:
- "--databases" is a space separated list of entries
database_name[.table_name]
- "--databases-file" is a name of file which contains entries
database_name[.table_name] one entry per line
- check_if_table_matches_filters has been modified to check whether
database is enabled first and then check the table
- databases and tables filters initialization has been refactored in
order to avoid code duplication
- innnobackupex's --databases option is mapped to one of these
options, depending on value.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'doc/source/innobackupex/partial_backups_innobackupex.rst'
--- doc/source/innobackupex/partial_backups_innobackupex.rst 2013-11-14 12:17:21 +0000
+++ doc/source/innobackupex/partial_backups_innobackupex.rst 2014-08-22 16:46:12 +0000
@@ -42,7 +42,7 @@
42Using the :option:`--databases` option42Using the :option:`--databases` option
43--------------------------------------43--------------------------------------
4444
45This option is specific to |innobackupex| and accepts either a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form - or a file containing the list at one element per line.45This option accepts either a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form - or a file containing the list at one element per line.
4646
47For example, ::47For example, ::
4848
@@ -50,10 +50,6 @@
5050
51The command above will create a timestamped directory with the usual files that |innobackupex| creates, but only containing the data-files related to ``mytable`` in the ``mydatabase`` directory and the ``mysql`` directory with the entire ``mysql`` database.51The command above will create a timestamped directory with the usual files that |innobackupex| creates, but only containing the data-files related to ``mytable`` in the ``mydatabase`` directory and the ``mysql`` directory with the entire ``mysql`` database.
5252
53.. note::
54
55 Currently in |Percona XtraBackup| the --databases option has no effect for InnoDB files for both local and streaming backups, i.e. all InnoDB files are always backed up. Currently, only .frm and non-InnoDB tables are limited by that option.
56
57Preparing Partial Backups53Preparing Partial Backups
58=========================54=========================
5955
6056
=== modified file 'doc/source/xtrabackup_bin/partial_backups.rst'
--- doc/source/xtrabackup_bin/partial_backups.rst 2013-04-29 11:08:57 +0000
+++ doc/source/xtrabackup_bin/partial_backups.rst 2014-08-22 16:46:12 +0000
@@ -2,7 +2,7 @@
2 Partial Backups2 Partial Backups
3=================3=================
44
5|xtrabackup| supports taking partial backups when the :term:`innodb_file_per_table` option is enabled. There are two ways to create partial backups: matching the tables' names with a regular expression or providing a list of them in a file.5|xtrabackup| supports taking partial backups when the :term:`innodb_file_per_table` option is enabled. There are two ways to create partial backups: matching the tables' names with a regular expression or providing a list of them in a file or providing a list of databases.
66
7.. warning:: If any of the matched or listed tables is deleted during the backup, |xtrabackup| will fail.7.. warning:: If any of the matched or listed tables is deleted during the backup, |xtrabackup| will fail.
88
@@ -31,6 +31,11 @@
31 $ echo "mydatabase.mytable" > /tmp/tables.txt31 $ echo "mydatabase.mytable" > /tmp/tables.txt
32 $ xtrabackup --backup --tables-file=/tmp/tables.txt 32 $ xtrabackup --backup --tables-file=/tmp/tables.txt
3333
34Using the :option:`--databases` and :option:`--databases-file` options
35======================================================================
36
37The ``--databases`` option accepts a space-separated list of the databases and tables to backup - in the ``databasename[.tablename]`` form. The ``--databases-file`` option specifies a file that can contain multiple databases and tables in the ``databasename[.tablename]`` form, one element name per line in the file. Only named databases and tables will be backed up. Names are matched exactly, case-sensitive, with no pattern or regular expression matching.
38
34Preparing the Backup39Preparing the Backup
35====================40====================
3641
3742
=== modified file 'doc/source/xtrabackup_bin/xbk_option_reference.rst'
--- doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-05-18 08:23:31 +0000
+++ doc/source/xtrabackup_bin/xbk_option_reference.rst 2014-08-22 16:46:12 +0000
@@ -125,6 +125,14 @@
125125
126 Don't read default options from any option file. Must be given as the first option on the command-line.126 Don't read default options from any option file. Must be given as the first option on the command-line.
127127
128.. option:: --databases=#
129
130 This option specifies the list of databases and tables that should be backed up. The option accepts the list of the form ``"databasename1[.table_name1] databasename2[.table_name2] . . ."``.
131
132.. option:: --databases-file=#
133
134 This option specifies the path to the file containing the list of databases and tables that should be backed up. The file can contain the list elements of the form ``databasename1[.table_name1]``, one element per line.
135
128.. option:: --parallel=#136.. option:: --parallel=#
129137
130 This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer).138 This option specifies the number of threads to use to copy multiple data files concurrently when creating a backup. The default value is 1 (i.e., no concurrent transfer).
131139
=== modified file 'innobackupex.pl'
--- innobackupex.pl 2014-07-21 15:16:00 +0000
+++ innobackupex.pl 2014-08-22 16:46:12 +0000
@@ -2811,6 +2811,14 @@
2811 $options = $options . " --compact";2811 $options = $options . " --compact";
2812 }2812 }
28132813
2814 if ($option_databases) {
2815 if ($option_databases =~ /^\//) {
2816 $options = $options . " --databases_file='$option_databases'";
2817 } else {
2818 $options = $options . " --databases='$option_databases'";
2819 }
2820 }
2821
2814 $cmdline = "$option_ibbackup_binary $options";2822 $cmdline = "$option_ibbackup_binary $options";
28152823
2816 # run ibbackup as a child process2824 # run ibbackup as a child process
28172825
=== modified file 'src/innodb_int.h'
--- src/innodb_int.h 2014-01-10 11:13:00 +0000
+++ src/innodb_int.h 2014-08-22 16:46:12 +0000
@@ -305,8 +305,8 @@
305#endif305#endif
306306
307#define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \307#define XB_HASH_SEARCH(NAME, TABLE, FOLD, DATA, ASSERTION, TEST) \
308 HASH_SEARCH(NAME, TABLE, FOLD, xtrabackup_tables_t*, DATA, ASSERTION, \308 HASH_SEARCH(NAME, TABLE, FOLD, xtrabackup_filter_entry_t*, DATA, \
309 TEST)309 ASSERTION, TEST)
310310
311/* The following constants have been copied from fsp0fsp.c */311/* The following constants have been copied from fsp0fsp.c */
312#define FSP_HEADER_OFFSET FIL_PAGE_DATA /* Offset of the space header312#define FSP_HEADER_OFFSET FIL_PAGE_DATA /* Offset of the space header
313313
=== modified file 'src/xtrabackup.cc'
--- src/xtrabackup.cc 2014-06-24 09:58:44 +0000
+++ src/xtrabackup.cc 2014-08-22 16:46:12 +0000
@@ -58,6 +58,7 @@
58#endif58#endif
5959
60#include <fcntl.h>60#include <fcntl.h>
61#include <string.h>
6162
62#ifdef __linux__63#ifdef __linux__
63# include <sys/prctl.h>64# include <sys/prctl.h>
@@ -122,21 +123,33 @@
122123
123lsn_t xtrabackup_archived_to_lsn = 0; /* for --archived-to-lsn */124lsn_t xtrabackup_archived_to_lsn = 0; /* for --archived-to-lsn */
124125
125char *xtrabackup_tables = NULL;126static char *xtrabackup_tables = NULL;
126int tables_regex_num;127
127xb_regex_t *tables_regex;128/* List of regular expressions for filtering */
128xb_regmatch_t tables_regmatch[1];129typedef struct xb_regex_list_node_struct xb_regex_list_node_t;
129130struct xb_regex_list_node_struct {
130char *xtrabackup_tables_file = NULL;131 UT_LIST_NODE_T(xb_regex_list_node_t) regex_list;
131hash_table_t* tables_hash;132 xb_regex_t regex;
132133};
133hash_table_t* inc_dir_tables_hash;134static UT_LIST_BASE_NODE_T(xb_regex_list_node_t) regex_list;
134135
135struct xtrabackup_tables_struct{136static xb_regmatch_t tables_regmatch[1];
137
138static char *xtrabackup_tables_file = NULL;
139static hash_table_t* tables_hash = NULL;
140
141static char *xtrabackup_databases = NULL;
142static char *xtrabackup_databases_file = NULL;
143static hash_table_t* databases_hash = NULL;
144
145static hash_table_t* inc_dir_tables_hash;
146
147struct xtrabackup_filter_entry_struct{
136 char* name;148 char* name;
149 ibool has_tables;
137 hash_node_t name_hash;150 hash_node_t name_hash;
138};151};
139typedef struct xtrabackup_tables_struct xtrabackup_tables_t;152typedef struct xtrabackup_filter_entry_struct xtrabackup_filter_entry_t;
140153
141#ifdef XTRADB_BASED154#ifdef XTRADB_BASED
142static ulint thread_nr[SRV_MAX_N_IO_THREADS + 6 + 64];155static ulint thread_nr[SRV_MAX_N_IO_THREADS + 6 + 64];
@@ -408,6 +421,8 @@
408 OPT_XTRA_ARCHIVED_TO_LSN,421 OPT_XTRA_ARCHIVED_TO_LSN,
409 OPT_XTRA_TABLES,422 OPT_XTRA_TABLES,
410 OPT_XTRA_TABLES_FILE,423 OPT_XTRA_TABLES_FILE,
424 OPT_XTRA_DATABASES,
425 OPT_XTRA_DATABASES_FILE,
411 OPT_XTRA_CREATE_IB_LOGFILE,426 OPT_XTRA_CREATE_IB_LOGFILE,
412 OPT_XTRA_PARALLEL,427 OPT_XTRA_PARALLEL,
413 OPT_XTRA_STREAM,428 OPT_XTRA_STREAM,
@@ -583,6 +598,13 @@
583 {"tables_file", OPT_XTRA_TABLES_FILE, "filtering by list of the exact database.table name in the file.",598 {"tables_file", OPT_XTRA_TABLES_FILE, "filtering by list of the exact database.table name in the file.",
584 (G_PTR*) &xtrabackup_tables_file, (G_PTR*) &xtrabackup_tables_file,599 (G_PTR*) &xtrabackup_tables_file, (G_PTR*) &xtrabackup_tables_file,
585 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},600 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
601 {"databases", OPT_XTRA_DATABASES, "filtering by list of databases.",
602 (G_PTR*) &xtrabackup_databases, (G_PTR*) &xtrabackup_databases,
603 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
604 {"databases_file", OPT_XTRA_TABLES_FILE,
605 "filtering by list of databases in the file.",
606 (G_PTR*) &xtrabackup_databases_file, (G_PTR*) &xtrabackup_databases_file,
607 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
586 {"create-ib-logfile", OPT_XTRA_CREATE_IB_LOGFILE, "** not work for now** creates ib_logfile* also after '--prepare'. ### If you want create ib_logfile*, only re-execute this command in same options. ###",608 {"create-ib-logfile", OPT_XTRA_CREATE_IB_LOGFILE, "** not work for now** creates ib_logfile* also after '--prepare'. ### If you want create ib_logfile*, only re-execute this command in same options. ###",
587 (G_PTR*) &xtrabackup_create_ib_logfile, (G_PTR*) &xtrabackup_create_ib_logfile,609 (G_PTR*) &xtrabackup_create_ib_logfile, (G_PTR*) &xtrabackup_create_ib_logfile,
588 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},610 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
@@ -1742,16 +1764,15 @@
1742static my_bool1764static my_bool
1743check_if_table_matches_filters(const char *name)1765check_if_table_matches_filters(const char *name)
1744{1766{
1745 int regres;1767 int regres;
1746 int i;1768 xtrabackup_filter_entry_t* table;
1747 xtrabackup_tables_t* table;1769 xb_regex_list_node_t* node;
17481770
1749 if (xtrabackup_tables) {1771 if (UT_LIST_GET_LEN(regex_list)) {
17501772 /* Check against regular expressions list */
1751 regres = REG_NOMATCH;1773 for (node = UT_LIST_GET_FIRST(regex_list); node;
17521774 node = UT_LIST_GET_NEXT(regex_list, node)) {
1753 for (i = 0; i < tables_regex_num; i++) {1775 regres = xb_regexec(&node->regex, name, 1,
1754 regres = xb_regexec(&tables_regex[i], name, 1,
1755 tables_regmatch, 0);1776 tables_regmatch, 0);
1756 if (regres != REG_NOMATCH) {1777 if (regres != REG_NOMATCH) {
17571778
@@ -1760,8 +1781,8 @@
1760 }1781 }
1761 }1782 }
17621783
1763 if (xtrabackup_tables_file) {1784 if (tables_hash) {
17641785 /* Finally, check against full qualified tables list */
1765 XB_HASH_SEARCH(name_hash, tables_hash, ut_fold_string(name),1786 XB_HASH_SEARCH(name_hash, tables_hash, ut_fold_string(name),
1766 table, (void) 0,1787 table, (void) 0,
1767 !strcmp(table->name, name));1788 !strcmp(table->name, name));
@@ -1791,7 +1812,9 @@
1791 const char *ptr;1812 const char *ptr;
1792 char *eptr;1813 char *eptr;
17931814
1794 if (xtrabackup_tables == NULL && xtrabackup_tables_file == NULL) {1815 if (UT_LIST_GET_LEN(regex_list) == 0 &&
1816 tables_hash == NULL &&
1817 databases_hash == NULL) {
1795 return(FALSE);1818 return(FALSE);
1796 }1819 }
17971820
@@ -1807,6 +1830,26 @@
1807 }1830 }
18081831
1809 strncpy(buf, dbname, FN_REFLEN);1832 strncpy(buf, dbname, FN_REFLEN);
1833 buf[tbname - 1 - dbname] = 0;
1834
1835 if (databases_hash) {
1836 /* There are some filters for databases, check them */
1837 xtrabackup_filter_entry_t* database;
1838
1839 XB_HASH_SEARCH(name_hash, databases_hash, ut_fold_string(buf),
1840 database, (void) 0,
1841 !strcmp(database->name, buf));
1842 /* Table's database isn't found, skip the table */
1843 if (!database) {
1844 return(TRUE);
1845 }
1846 /* There aren't tables specified for the database,
1847 it should be backed up entirely */
1848 if (!database->has_tables) {
1849 return(FALSE);
1850 }
1851 }
1852
1810 buf[FN_REFLEN - 1] = '\0';1853 buf[FN_REFLEN - 1] = '\0';
1811 buf[tbname - 1 - dbname] = '.';1854 buf[tbname - 1 - dbname] = '.';
18121855
@@ -2789,120 +2832,279 @@
2789 return(FALSE);2832 return(FALSE);
2790}2833}
27912834
2835/***********************************************************************
2836Allocate and initialize the entry for databases and tables filtering
2837hash tables. If memory allocation is not successful, terminate program.
2838@return pointer to the created entry. */
2839static
2840xtrabackup_filter_entry_t *
2841xb_new_filter_entry(
2842/*================*/
2843 const char* name) /*!< in: name of table/database */
2844{
2845 xtrabackup_filter_entry_t *entry;
2846 ulint namelen = strlen(name);
2847
2848 /* Length of name shouldn't be greater than 129 character,
2849 which isn't actually 129 bytes as some of UTF-8 characters
2850 for example can be encoded by 4 bytes. */
2851 ut_a(namelen <= NAME_LEN * 2 + 1);
2852
2853 entry = static_cast<xtrabackup_filter_entry_t *>
2854 (ut_malloc(sizeof(xtrabackup_filter_entry_t) + namelen + 1));
2855 memset(entry, '\0', sizeof(xtrabackup_filter_entry_t) + namelen + 1);
2856 entry->name = ((char*)entry) + sizeof(xtrabackup_filter_entry_t);
2857 strcpy(entry->name, name);
2858 entry->has_tables = FALSE;
2859
2860 return entry;
2861}
2862
2863/***********************************************************************
2864Add entry to hash table. If hash table is NULL, allocate and initialize
2865new hash table */
2866static
2867xtrabackup_filter_entry_t*
2868xb_add_filter_entry_to_hash(
2869/*========================*/
2870 const char* name, /*!< in: name of table/database */
2871 hash_table_t** hash) /*!< in/out: hash to insert into */
2872{
2873 xtrabackup_filter_entry_t* entry;
2874
2875 entry = xb_new_filter_entry(name);
2876
2877 if (UNIV_UNLIKELY(*hash == NULL)) {
2878 *hash = hash_create(1000);
2879 }
2880 HASH_INSERT(xtrabackup_filter_entry_t,
2881 name_hash, *hash,
2882 ut_fold_string(entry->name),
2883 entry);
2884
2885 return entry;
2886}
2887
2888/***********************************************************************
2889Validate name of table or database. If name is invalid, program will
2890be finished with error code */
2891static
2892void
2893xb_validate_name(
2894/*=============*/
2895 const char* name, /*!< in: name */
2896 size_t len) /*!< in: length of name */
2897{
2898 const char* p;
2899
2900 /* perform only basic validation. validate length and
2901 path symbols */
2902 if (len > NAME_LEN) {
2903 msg("xtrabackup: name `%s` is too long.\n", name);
2904 exit(EXIT_FAILURE);
2905 }
2906 p = strpbrk(name, "/\\~");
2907 if (p && p - name < NAME_LEN) {
2908 msg("xtrabackup: name `%s` is not valid.\n", name);
2909 exit(EXIT_FAILURE);
2910 }
2911}
2912
2913/***********************************************************************
2914Register new filter entry which can be either database
2915or table name. */
2916static
2917void
2918xb_register_filter_entry(
2919/*=====================*/
2920 const char* name) /*!< in: name */
2921{
2922 const char* p;
2923 size_t namelen;
2924 xtrabackup_filter_entry_t* db_entry = NULL;
2925
2926 namelen = strlen(name);
2927 if ((p = strchr(name, '.')) != NULL) {
2928 char dbname[NAME_LEN + 1];
2929
2930 xb_validate_name(name, p - name);
2931 xb_validate_name(p + 1, namelen - (p - name));
2932
2933 strncpy(dbname, name, p - name);
2934 dbname[p - name] = 0;
2935
2936 if (databases_hash) {
2937 XB_HASH_SEARCH(name_hash, databases_hash,
2938 ut_fold_string(dbname),
2939 db_entry, (void) 0,
2940 !strcmp(db_entry->name, dbname));
2941 }
2942 if (!db_entry) {
2943 db_entry = xb_add_filter_entry_to_hash(dbname,
2944 &databases_hash);
2945 }
2946 db_entry->has_tables = TRUE;
2947 xb_add_filter_entry_to_hash(name, &tables_hash);
2948 } else {
2949 xb_validate_name(name, namelen);
2950
2951 xb_add_filter_entry_to_hash(name, &databases_hash);
2952 }
2953}
2954
2955/***********************************************************************
2956Register new table for the filter. */
2957static
2958void
2959xb_register_table(
2960/*==============*/
2961 const char* name) /*!< in: name of table */
2962{
2963 if (strchr(name, '.') == NULL) {
2964 msg("xtrabackup: `%s` is not fully qualified name.\n", name);
2965 exit(EXIT_FAILURE);
2966 }
2967
2968 xb_register_filter_entry(name);
2969}
2970
2971/***********************************************************************
2972Register new regex for the filter. */
2973static
2974void
2975xb_register_regex(
2976/*==============*/
2977 const char* regex) /*!< in: regex */
2978{
2979 xb_regex_list_node_t* node;
2980 char errbuf[100];
2981 int ret;
2982
2983 node = static_cast<xb_regex_list_node_t *>
2984 (ut_malloc(sizeof(xb_regex_list_node_t)));
2985
2986 ret = xb_regcomp(&node->regex, regex, REG_EXTENDED);
2987 if (ret != 0) {
2988 xb_regerror(ret, &node->regex, errbuf, sizeof(errbuf));
2989 msg("xtrabackup: error: tables regcomp(%s): %s\n",
2990 regex, errbuf);
2991 exit(EXIT_FAILURE);
2992 }
2993
2994 UT_LIST_ADD_LAST(regex_list, regex_list, node);
2995}
2996
2997typedef void (*insert_entry_func_t)(const char*);
2998
2999/***********************************************************************
3000Scan string and load filter entries from it. */
3001static
3002void
3003xb_load_list_string(
3004/*================*/
3005 char* list, /*!< in: string representing a list */
3006 const char* delimiters, /*!< in: delimiters of entries */
3007 insert_entry_func_t ins) /*!< in: callback to add entry */
3008{
3009 char* p;
3010 char* saveptr;
3011
3012 p = strtok_r(list, delimiters, &saveptr);
3013 while (p) {
3014
3015 ins(p);
3016
3017 p = strtok_r(NULL, delimiters, &saveptr);
3018 }
3019}
3020
3021/***********************************************************************
3022Scan file and load filter entries from it. */
3023static
3024void
3025xb_load_list_file(
3026/*==============*/
3027 const char* filename, /*!< in: name of file */
3028 insert_entry_func_t ins) /*!< in: callback to add entry */
3029{
3030 char name_buf[NAME_LEN*2+2];
3031 FILE *fp;
3032
3033 /* read and store the filenames */
3034 fp = fopen(filename, "r");
3035 if (!fp) {
3036 msg("xtrabackup: cannot open %s\n",
3037 filename);
3038 exit(EXIT_FAILURE);
3039 }
3040 for (;;) {
3041 char* p = name_buf;
3042
3043 if (fgets(name_buf, sizeof(name_buf), fp) == 0) {
3044 break;
3045 }
3046
3047 p = strchr(name_buf, '\n');
3048 if (p) {
3049 *p = '\0';
3050 } else {
3051 msg("xtrabackup: `%s...` name is too long", name_buf);
3052 exit(EXIT_FAILURE);
3053 }
3054
3055 ins(name_buf);
3056 }
3057
3058 fclose(fp);
3059}
3060
3061
2792static3062static
2793void3063void
2794xb_filters_init()3064xb_filters_init()
2795{3065{
3066 UT_LIST_INIT(regex_list);
3067
3068 if (xtrabackup_databases) {
3069 xb_load_list_string(xtrabackup_databases, " \t",
3070 xb_register_filter_entry);
3071 }
3072
3073 if (xtrabackup_databases_file) {
3074 xb_load_list_file(xtrabackup_databases_file,
3075 xb_register_filter_entry);
3076 }
3077
2796 if (xtrabackup_tables) {3078 if (xtrabackup_tables) {
2797 /* init regexp */3079 xb_load_list_string(xtrabackup_tables, ",",
2798 char *p, *next;3080 xb_register_regex);
2799 int i;
2800 char errbuf[100];
2801
2802 tables_regex_num = 1;
2803
2804 p = xtrabackup_tables;
2805 while ((p = strchr(p, ',')) != NULL) {
2806 p++;
2807 tables_regex_num++;
2808 }
2809
2810 tables_regex = static_cast<xb_regex_t *>
2811 (ut_malloc(sizeof(xb_regex_t) * tables_regex_num));
2812
2813 p = xtrabackup_tables;
2814 for (i=0; i < tables_regex_num; i++) {
2815 int rc;
2816
2817 next = strchr(p, ',');
2818 ut_a(next || i == tables_regex_num - 1);
2819
2820 next++;
2821 if (i != tables_regex_num - 1)
2822 *(next - 1) = '\0';
2823
2824 rc = xb_regcomp(&tables_regex[i], p,
2825 REG_EXTENDED);
2826
2827 if (rc) {
2828
2829 xb_regerror(rc, &tables_regex[i], errbuf,
2830 sizeof(errbuf));
2831 msg("xtrabackup: regcomp(%s) failed: %s\n",
2832 p, errbuf);
2833
2834 exit(EXIT_FAILURE);
2835 }
2836
2837 if (i != tables_regex_num - 1)
2838 *(next - 1) = ',';
2839 p = next;
2840 }
2841 }3081 }
28423082
2843 if (xtrabackup_tables_file) {3083 if (xtrabackup_tables_file) {
2844 char name_buf[NAME_LEN*2+2];3084 xb_load_list_file(xtrabackup_tables_file, xb_register_table);
2845 FILE *fp;
2846
2847 name_buf[NAME_LEN*2+1] = '\0';
2848
2849 /* init tables_hash */
2850 tables_hash = hash_create(1000);
2851
2852 /* read and store the filenames */
2853 fp = fopen(xtrabackup_tables_file,"r");
2854 if (!fp) {
2855 msg("xtrabackup: cannot open %s\n",
2856 xtrabackup_tables_file);
2857 exit(EXIT_FAILURE);
2858 }
2859 for (;;) {
2860 xtrabackup_tables_t* table;
2861 char* p = name_buf;
2862
2863 if ( fgets(name_buf, NAME_LEN*2+1, fp) == 0 ) {
2864 break;
2865 }
2866
2867 p = strchr(name_buf, '\n');
2868 if (p)
2869 {
2870 *p = '\0';
2871 }
2872
2873 table = static_cast<xtrabackup_tables_t *>
2874 (ut_malloc(sizeof(xtrabackup_tables_t)
2875 + strlen(name_buf) + 1));
2876 memset(table, '\0', sizeof(xtrabackup_tables_t) + strlen(name_buf) + 1);
2877 table->name = ((char*)table) + sizeof(xtrabackup_tables_t);
2878 strcpy(table->name, name_buf);
2879
2880 HASH_INSERT(xtrabackup_tables_t, name_hash, tables_hash,
2881 ut_fold_string(table->name), table);
2882 }
2883 }3085 }
2884}3086}
28853087
2886static3088static
2887void3089void
2888xb_tables_hash_free(hash_table_t* hash)3090xb_filter_hash_free(hash_table_t* hash)
2889{3091{
2890 ulint i;3092 ulint i;
28913093
2892 /* free the hash elements */3094 /* free the hash elements */
2893 for (i = 0; i < hash_get_n_cells(hash); i++) {3095 for (i = 0; i < hash_get_n_cells(hash); i++) {
2894 xtrabackup_tables_t* table;3096 xtrabackup_filter_entry_t* table;
28953097
2896 table = static_cast<xtrabackup_tables_t *>3098 table = static_cast<xtrabackup_filter_entry_t *>
2897 (HASH_GET_FIRST(hash, i));3099 (HASH_GET_FIRST(hash, i));
28983100
2899 while (table) {3101 while (table) {
2900 xtrabackup_tables_t* prev_table = table;3102 xtrabackup_filter_entry_t* prev_table = table;
29013103
2902 table = static_cast<xtrabackup_tables_t *>3104 table = static_cast<xtrabackup_filter_entry_t *>
2903 (HASH_GET_NEXT(name_hash, prev_table));3105 (HASH_GET_NEXT(name_hash, prev_table));
29043106
2905 HASH_DELETE(xtrabackup_tables_t, name_hash, hash,3107 HASH_DELETE(xtrabackup_filter_entry_t, name_hash, hash,
2906 ut_fold_string(prev_table->name), prev_table);3108 ut_fold_string(prev_table->name), prev_table);
2907 ut_free(prev_table);3109 ut_free(prev_table);
2908 }3110 }
@@ -2918,18 +3120,19 @@
2918void3120void
2919xb_filters_free()3121xb_filters_free()
2920{3122{
2921 if (xtrabackup_tables) {3123 while (UT_LIST_GET_LEN(regex_list) > 0) {
2922 /* free regexp */3124 xb_regex_list_node_t* node = UT_LIST_GET_FIRST(regex_list);
2923 int i;3125 UT_LIST_REMOVE(regex_list, regex_list, node);
29243126 xb_regfree(&node->regex);
2925 for (i = 0; i < tables_regex_num; i++) {3127 ut_free(node);
2926 xb_regfree(&tables_regex[i]);3128 }
2927 }3129
2928 ut_free(tables_regex);3130 if (tables_hash) {
2929 }3131 xb_filter_hash_free(tables_hash);
29303132 }
2931 if (xtrabackup_tables_file) {3133
2932 xb_tables_hash_free(tables_hash);3134 if (databases_hash) {
3135 xb_filter_hash_free(databases_hash);
2933 }3136 }
2934}3137}
29353138
@@ -4271,7 +4474,7 @@
4271 fil_space_t* fil_space;4474 fil_space_t* fil_space;
4272 os_file_t file = 0;4475 os_file_t file = 0;
4273 ulint tablespace_flags;4476 ulint tablespace_flags;
4274 xtrabackup_tables_t* table;4477 xtrabackup_filter_entry_t* table;
42754478
4276 ut_a(dbname != NULL ||4479 ut_a(dbname != NULL ||
4277 trx_sys_sys_space(space_id) ||4480 trx_sys_sys_space(space_id) ||
@@ -4311,13 +4514,13 @@
4311 }4514 }
43124515
4313 /* remember space name for further reference */4516 /* remember space name for further reference */
4314 table = static_cast<xtrabackup_tables_t *>4517 table = static_cast<xtrabackup_filter_entry_t *>
4315 (ut_malloc(sizeof(xtrabackup_tables_t) +4518 (ut_malloc(sizeof(xtrabackup_filter_entry_t) +
4316 strlen(dest_space_name) + 1));4519 strlen(dest_space_name) + 1));
43174520
4318 table->name = ((char*)table) + sizeof(xtrabackup_tables_t);4521 table->name = ((char*)table) + sizeof(xtrabackup_filter_entry_t);
4319 strcpy(table->name, dest_space_name);4522 strcpy(table->name, dest_space_name);
4320 HASH_INSERT(xtrabackup_tables_t, name_hash, inc_dir_tables_hash,4523 HASH_INSERT(xtrabackup_filter_entry_t, name_hash, inc_dir_tables_hash,
4321 ut_fold_string(table->name), table);4524 ut_fold_string(table->name), table);
43224525
4323 mutex_enter(&fil_system->mutex);4526 mutex_enter(&fil_system->mutex);
@@ -4651,8 +4854,8 @@
4651 const char* file_name, /*!<in: file name with suffix */4854 const char* file_name, /*!<in: file name with suffix */
4652 void* arg __attribute__((unused)))4855 void* arg __attribute__((unused)))
4653{4856{
4654 char name[FN_REFLEN];4857 char name[FN_REFLEN];
4655 xtrabackup_tables_t* table;4858 xtrabackup_filter_entry_t* table;
46564859
4657 snprintf(name, FN_REFLEN, "%s%s/%s",4860 snprintf(name, FN_REFLEN, "%s%s/%s",
4658 xb_dict_prefix, db_name, file_name);4861 xb_dict_prefix, db_name, file_name);
@@ -5434,7 +5637,7 @@
54345637
5435 if(!xtrabackup_apply_deltas()) {5638 if(!xtrabackup_apply_deltas()) {
5436 xb_data_files_close();5639 xb_data_files_close();
5437 xb_tables_hash_free(inc_dir_tables_hash);5640 xb_filter_hash_free(inc_dir_tables_hash);
5438 goto error;5641 goto error;
5439 }5642 }
5440 }5643 }
@@ -5447,7 +5650,7 @@
54475650
5448 xb_process_datadir("./", ".ibd", rm_if_not_found, NULL);5651 xb_process_datadir("./", ".ibd", rm_if_not_found, NULL);
54495652
5450 xb_tables_hash_free(inc_dir_tables_hash);5653 xb_filter_hash_free(inc_dir_tables_hash);
5451 }5654 }
5452 sync_close();5655 sync_close();
5453 sync_initialized = FALSE;5656 sync_initialized = FALSE;
54545657
=== added file 'test/t/ib_databases.sh'
--- test/t/ib_databases.sh 1970-01-01 00:00:00 +0000
+++ test/t/ib_databases.sh 2014-08-22 16:46:12 +0000
@@ -0,0 +1,48 @@
1########################################################################
2# Bug #569387: innobackupex ignores --databases
3# Testcase covers using --databases option with InnoDB
4# database (list is specified in option itself)
5########################################################################
6
7. inc/common.sh
8
9start_server --innodb-file-per-table
10
11cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
12
13 CREATE DATABASE test1;
14
15 CREATE TABLE test1.a (a INT PRIMARY KEY);
16 CREATE TABLE test1.b (b INT PRIMARY KEY);
17 CREATE TABLE test1.c (c INT PRIMARY KEY);
18
19 CREATE TABLE test.a (a INT PRIMARY KEY);
20 CREATE TABLE test.b (b INT PRIMARY KEY);
21 CREATE TABLE test.c (c INT PRIMARY KEY);
22
23EOF
24
25# This is a workaround to pass --databases='test test1.b test1.c ... ' (containing spaces)
26$IB_BIN $IB_ARGS --no-timestamp --databases='test test1.b test1.c mysql performance_schema' $topdir/backup
27innobackupex --apply-log $topdir/backup
28vlog "Backup taken"
29
30stop_server
31
32# Restore partial backup
33# Remove database
34rm -rf $mysql_datadir/*
35vlog "Original database removed"
36
37# Restore database from backup
38innobackupex --copy-back $topdir/backup
39vlog "database restored from backup"
40
41start_server
42
43OUT=`run_cmd $MYSQL $MYSQL_ARGS -e "USE test; SHOW TABLES; USE test1; SHOW TABLES;" | tr -d '\n'`
44
45if [ $OUT != "Tables_in_testabcTables_in_test1bc" ] ; then
46 vlog "Backed up tables set doesn't match filter" ;
47 exit 1;
48fi
049
=== added file 'test/t/ib_databases_file.sh'
--- test/t/ib_databases_file.sh 1970-01-01 00:00:00 +0000
+++ test/t/ib_databases_file.sh 2014-08-22 16:46:12 +0000
@@ -0,0 +1,57 @@
1########################################################################
2# Bug #569387: innobackupex ignores --databases
3# Testcase covers using --databases option with InnoDB
4# database (list is specified in file which option
5# points to)
6########################################################################
7
8. inc/common.sh
9
10start_server --innodb-file-per-table
11
12cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
13
14 CREATE DATABASE test1;
15
16 CREATE TABLE test1.a (a INT PRIMARY KEY) engine=InnoDB;
17 CREATE TABLE test1.b (b INT PRIMARY KEY) engine=InnoDB;
18 CREATE TABLE test1.c (c INT PRIMARY KEY) engine=InnoDB;
19
20 CREATE TABLE test.a (a INT PRIMARY KEY) engine=InnoDB;
21 CREATE TABLE test.b (b INT PRIMARY KEY) engine=InnoDB;
22 CREATE TABLE test.c (c INT PRIMARY KEY) engine=InnoDB;
23
24EOF
25
26# Take a backup
27# Backup the whole test and b,c from test1
28cat >$topdir/databases_file <<EOF
29mysql
30performance_schema
31test
32test1.b
33test1.c
34EOF
35innobackupex --no-timestamp --databases=$topdir/databases_file $topdir/backup
36innobackupex --apply-log $topdir/backup
37vlog "Backup taken"
38
39stop_server
40
41# Restore partial backup
42# Remove database
43rm -rf $mysql_datadir/*
44vlog "Original database removed"
45
46# Restore database from backup
47innobackupex --copy-back $topdir/backup
48vlog "database restored from backup"
49
50start_server
51
52OUT=`run_cmd $MYSQL $MYSQL_ARGS -e "USE test; SHOW TABLES; USE test1; SHOW TABLES;" | tr -d '\n'`
53
54if [ $OUT != "Tables_in_testabcTables_in_test1bc" ] ; then
55 vlog "Backed up tables set don't match filter" ;
56 exit 1;
57fi
058
=== added file 'test/t/xb_databases_options.sh'
--- test/t/xb_databases_options.sh 1970-01-01 00:00:00 +0000
+++ test/t/xb_databases_options.sh 2014-08-22 16:46:12 +0000
@@ -0,0 +1,218 @@
1########################################################################
2# Bug #569387: xtrabackup ignores --databases, i.e. when --databases
3#
4# Test following xtrabackup options
5# --databases
6# --databases-file
7# --tables
8# --tables-file
9########################################################################
10
11. inc/common.sh
12
13start_server --innodb-file-per-table
14
15function setup_test {
16
17 cat <<EOF | run_cmd $MYSQL $MYSQL_ARGS
18
19 create database database1;
20 create database database2;
21 create database test1;
22 create database test2;
23 create database thisisadatabase;
24 create database testdatabase;
25
26 create table database1.thisisatable (a int primary key) engine=InnoDB;
27 create table database1.t (a int primary key) engine=InnoDB;
28 create table database1.t1 (a int primary key) engine=InnoDB;
29 create table database1.t2 (a int primary key) engine=InnoDB;
30 create table database1.ancor (a int primary key) engine=InnoDB;
31 create table database1.glow (a int primary key) engine=InnoDB;
32
33 create table database2.thisisatable (a int primary key) engine=InnoDB;
34 create table database2.t (a int primary key) engine=InnoDB;
35 create table database2.t1 (a int primary key) engine=InnoDB;
36 create table database2.t2 (a int primary key) engine=InnoDB;
37 create table database2.ancor (a int primary key) engine=InnoDB;
38 create table database2.glow (a int primary key) engine=InnoDB;
39
40 create table test1.thisisatable (a int primary key) engine=InnoDB;
41 create table test1.t (a int primary key) engine=InnoDB;
42 create table test1.t1 (a int primary key) engine=InnoDB;
43 create table test1.t2 (a int primary key) engine=InnoDB;
44 create table test1.ancor (a int primary key) engine=InnoDB;
45 create table test1.glow (a int primary key) engine=InnoDB;
46
47 create table test2.thisisatable (a int primary key) engine=InnoDB;
48 create table test2.t (a int primary key) engine=InnoDB;
49 create table test2.t1 (a int primary key) engine=InnoDB;
50 create table test2.t2 (a int primary key) engine=InnoDB;
51 create table test2.ancor (a int primary key) engine=InnoDB;
52 create table test2.glow (a int primary key) engine=InnoDB;
53
54 create table thisisadatabase.thisisatable (a int primary key) engine=InnoDB;
55 create table thisisadatabase.t (a int primary key) engine=InnoDB;
56 create table thisisadatabase.t1 (a int primary key) engine=InnoDB;
57 create table thisisadatabase.t2 (a int primary key) engine=InnoDB;
58 create table thisisadatabase.ancor (a int primary key) engine=InnoDB;
59 create table thisisadatabase.glow (a int primary key) engine=InnoDB;
60
61 create table testdatabase.thisisatable (a int primary key) engine=InnoDB;
62 create table testdatabase.t (a int primary key) engine=InnoDB;
63 create table testdatabase.t1 (a int primary key) engine=InnoDB;
64 create table testdatabase.t2 (a int primary key) engine=InnoDB;
65 create table testdatabase.ancor (a int primary key) engine=InnoDB;
66 create table testdatabase.glow (a int primary key) engine=InnoDB;
67
68EOF
69
70}
71
72function ls_dir {
73 ( cd $1 ; find . -name '*.ibd' -type f | sort -d )
74}
75
76setup_test
77
78backup_dir=$topdir/backup_dir
79mkdir -p $backup_dir
80
81# invalid characters
82xtrabackup --backup --databases=a/b/c --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is not valid'
83
84# too long name
85xtrabackup --backup --databases=verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
86
87# too long name
88xtrabackup --backup --databases=test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
89
90# not fully qualified name
91xtrabackup --backup --tables-file=<(echo verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow) --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is not fully qualified'
92
93# again too long name
94xtrabackup --backup --tables-file=<(echo test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow) --target-dir=$backup_dir --datadir=$mysql_datadir 2>&1 | grep 'is too long'
95
96# should go fine, we cannot validate regex against length
97xtrabackup --backup --tables=verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir
98rm -rf $backup_dir/*
99
100# should go fine, we cannot validate regex against length
101xtrabackup --backup --tables=test1.verylonglonglongname111111skjhkdjhfkjdhgkjdfh1555555555555511stillnotlongenoughsowilladdmore1111111111111111111114848484848484fkjhdjfhkdjfhd8484848aaaaaaaaaancnvjvifmifjhfkmfkfnbfifnfkfik4848484841111111prettyenoughnow --target-dir=$backup_dir --datadir=$mysql_datadir
102rm -rf $backup_dir/*
103
104vlog "Testing with --databases=..."
105$XB_BIN $XB_ARGS --backup --databases='database database2 test1.t test1.ancor thisisadatabase testdatabase.t testdatabase.t7' --target-dir=$backup_dir --datadir=$mysql_datadir
106diff -u <(ls_dir $backup_dir) - <<EOF
107./database2/ancor.ibd
108./database2/glow.ibd
109./database2/t1.ibd
110./database2/t2.ibd
111./database2/thisisatable.ibd
112./database2/t.ibd
113./test1/ancor.ibd
114./test1/t.ibd
115./testdatabase/t.ibd
116./thisisadatabase/ancor.ibd
117./thisisadatabase/glow.ibd
118./thisisadatabase/t1.ibd
119./thisisadatabase/t2.ibd
120./thisisadatabase/thisisatable.ibd
121./thisisadatabase/t.ibd
122EOF
123rm -rf $backup_dir/*
124
125vlog "Testing with --databases-file=..."
126cat >$topdir/list <<EOF
127database
128database2
129test1.t
130test1.ancor
131thisisadatabase
132testdatabase.t
133testdatabase.t7
134EOF
135xtrabackup --backup --databases-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
136diff -u <(ls_dir $backup_dir) - <<EOF
137./database2/ancor.ibd
138./database2/glow.ibd
139./database2/t1.ibd
140./database2/t2.ibd
141./database2/thisisatable.ibd
142./database2/t.ibd
143./test1/ancor.ibd
144./test1/t.ibd
145./testdatabase/t.ibd
146./thisisadatabase/ancor.ibd
147./thisisadatabase/glow.ibd
148./thisisadatabase/t1.ibd
149./thisisadatabase/t2.ibd
150./thisisadatabase/thisisatable.ibd
151./thisisadatabase/t.ibd
152EOF
153rm -rf $backup_dir/*
154
155vlog "Testing failure with --tables-file=..."
156cat >$topdir/list <<EOF
157database
158database2
159test1.t
160test1.ancor
161thisisadatabase
162testdatabase.t
163testdatabase.t7
164EOF
165run_cmd_expect_failure $XB_BIN $XB_ARGS --backup --tables-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
166diff -u <(ls_dir $backup_dir) - <<EOF
167EOF
168rm -rf $backup_dir/*
169
170vlog "Testing with --tables-file=..."
171cat >$topdir/list <<EOF
172test1.t
173test1.ancor
174testdatabase.t
175testdatabase.t7
176EOF
177xtrabackup --backup --tables-file=$topdir/list --target-dir=$backup_dir --datadir=$mysql_datadir
178diff -u <(ls_dir $backup_dir) - <<EOF
179./test1/ancor.ibd
180./test1/t.ibd
181./testdatabase/t.ibd
182EOF
183rm -rf $backup_dir/*
184
185vlog "Testing with --tables=..."
186xtrabackup --backup --tables='dat.base,test1.t,test1.a' --target-dir=$backup_dir --datadir=$mysql_datadir
187diff -u <(ls_dir $backup_dir) - <<EOF
188./database1/ancor.ibd
189./database1/glow.ibd
190./database1/t1.ibd
191./database1/t2.ibd
192./database1/thisisatable.ibd
193./database1/t.ibd
194./database2/ancor.ibd
195./database2/glow.ibd
196./database2/t1.ibd
197./database2/t2.ibd
198./database2/thisisatable.ibd
199./database2/t.ibd
200./test1/ancor.ibd
201./test1/t1.ibd
202./test1/t2.ibd
203./test1/thisisatable.ibd
204./test1/t.ibd
205./testdatabase/ancor.ibd
206./testdatabase/glow.ibd
207./testdatabase/t1.ibd
208./testdatabase/t2.ibd
209./testdatabase/thisisatable.ibd
210./testdatabase/t.ibd
211./thisisadatabase/ancor.ibd
212./thisisadatabase/glow.ibd
213./thisisadatabase/t1.ibd
214./thisisadatabase/t2.ibd
215./thisisadatabase/thisisatable.ibd
216./thisisadatabase/t.ibd
217EOF
218rm -rf $backup_dir/*

Subscribers

People subscribed via source and target branches