Merge lp:~sergei.glushchenko/percona-xtrabackup/xb2.0-bug483827 into lp:percona-xtrabackup/2.0

[Sergei Glushchenko]

Bug #483827. Support for mysqld_multi.
Fix based on work of Daniël van Eeden (lp:~dveeden/percona-xtrabackup/lp483827)
--section option added to innobackupex, specifying which section of my.cnf to handle.
--mysqld-section option added to xtrabackup for the same purpose.
Tescase does not run mysqld_multi, it just tests correctness of --section option.

http://jenkins.percona.com/view/XtraBackup/job/percona-xtrabackup-2.0-param/197/

[Alexey Kopytov (akopytov)]

Sergei,

   - sections in my.cnf are called groups in MySQL documentation. So
     let's be consistent and call them groups as well, and name the
     options (both the innobackupex and the xtrabackup one)
     --defaults-group, rather than --section and --mysqld-section

   - it looks like get_option is always passed the same value as its
     second argument, so I wonder if it really needs that argument, or
     rather just use $option_mysqld_section (or whatever it will be
     renamed to, i.e. $option_defaults_group) internally.

   - s/laod/load/
   - s/accetps/accepts/
   - s/rstore_args/restore_args/. Though you don't really need that
     function, as tests are executed in a separate shell process, so
     modifications to variables have no effect on other tests anyway.

   - I know that code in main() that scan options for "--mysqld-section"
     was copy-pasted, but please format it according to InnoDB
     style. Because currently it's a terrible mix of all possible
     formatting styles.

   - in the same code, I don't think strstr() is necessary, because you
     already have the pointer to '=' (or terminating zero) in optend.

review: Needs Fixing

[Sergei Glushchenko (sergei.glushchenko)]

--defaults-group option instead of --section and --mysql-section

kept group name argument for has_option.
fixed typos.

reformatted code to be more InnoDB-style (if there are other requirements, I'm ready for the next iteration).

removed unnecessary strstr.

[Alexey Kopytov (akopytov)]

Sergei,

As discussed on IRC, --defaults-group does not necessarily have to be the first option, so docs should be corrected.

And, while you are at it would be good to correct built-in help for --defaults-group in innobackupex.

Help text for other options either have units or some description of possible values after the '=' sign, e.g.:

--ibbackup=IBBACKUP-BINARY
--remote-host=HOSTNAME

So the text for --defaults-group should be something like "--defaults-group=GROUP-NAME" rather thatn "--defaults-group=mysqld".

review: Needs Fixing

[Sergei Glushchenko (sergei.glushchenko)]

Docs fixed

[Alexey Kopytov (akopytov)]

review: Approve

=== modified file 'doc/source/how-tos.rst'
--- doc/source/how-tos.rst	2011-07-07 05:32:50 +0000
+++ doc/source/how-tos.rst	2012-05-18 11:25:23 +0000
@@ -13,6 +13,7 @@
    howtos/recipes_ibkx_local
    howtos/recipes_ibkx_stream
    howtos/recipes_ibkx_inc
+   howtos/recipes_ibkx_multi
 
 
 .. _recipes-xbk:
 
=== modified file 'doc/source/innobackupex/innobackupex_option_reference.rst'
--- doc/source/innobackupex/innobackupex_option_reference.rst	2012-05-15 09:30:50 +0000
+++ doc/source/innobackupex/innobackupex_option_reference.rst	2012-05-18 11:25:23 +0000
@@ -70,6 +70,10 @@
 
    This option accepts a string argument that specifies the port to use when connecting to the database server with TCP/IP. It is passed to the :command:`mysql` child process. It is passed to the :command:`mysql` child process without alteration. See :command:`mysql --help` for details.
 
+.. option:: --defaults-group
+
+   This option accepts a string argument that specifies the group which should be read from the configuration file. This is needed if you use mysqld_multi.
+
 .. option:: --socket
 
    This option accepts a string argument that specifies the socket to use when connecting to the local database server with a UNIX domain socket. It is passed to the mysql child process without alteration. See :command:`mysql --help` for details.
 
=== modified file 'doc/source/xtrabackup_bin/xbk_option_reference.rst'
--- doc/source/xtrabackup_bin/xbk_option_reference.rst	2012-04-21 07:12:28 +0000
+++ doc/source/xtrabackup_bin/xbk_option_reference.rst	2012-05-18 11:25:23 +0000
@@ -101,6 +101,10 @@
     --innodb-read-io-threads
     --innodb-write-io-threads
 
+.. option:: --defaults-group
+
+   This option is to set the group which should be read from the configuration file. This is used by innobackupex if you use the `--defaults-group` option. It is needed for mysqld_multi deployments.
+
 .. option:: --log-stream
 
    Makes xtrabackup not copy data files, and output the contents of the InnoDB log files to STDOUT until the :option:`--suspend-at-end` file is deleted. This option enables :option:`--suspend-at-end` automatically.
 
=== modified file 'innobackupex'
--- innobackupex	2012-05-18 11:25:23 +0000
+++ innobackupex	2012-05-18 11:25:23 +0000
@@ -84,6 +84,7 @@
 my $option_mysql_port = '';
 my $option_mysql_socket = '';
 my $option_mysql_host = '';
+my $option_defaults_group = 'mysqld';
 my $option_no_timestamp = '';
 my $option_slave_info = '';
 my $option_galera_info = '';
@@ -377,7 +378,7 @@
 # process.
 #
 sub backup {
-    my $orig_datadir = get_option(\%config, 'mysqld', 'datadir');
+    my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
 
     # check that we can connect to the database. This done by
     # connecting, issuing a query, and closing the connection.
@@ -626,13 +627,13 @@
 # back to their original locations.
 #
 sub copy_back {
-    my $orig_datadir = get_option(\%config, 'mysqld', 'datadir');
+    my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
     my $orig_ibdata_dir = 
-        get_option(\%config, 'mysqld', 'innodb_data_home_dir');
+        get_option(\%config, $option_defaults_group, 'innodb_data_home_dir');
     my $orig_innodb_data_file_path = 
-        get_option(\%config, 'mysqld', 'innodb_data_file_path');
+        get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
     my $orig_iblog_dir =
-        get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
+        get_option(\%config, $option_defaults_group, 'innodb_log_group_home_dir');
     my $iblog_files = 'ib_logfile.*';
     my $excluded_files = 
         '\.\.?|backup-my\.cnf|xtrabackup_logfile|' .
@@ -740,6 +741,10 @@
         $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
     }
 
+    if ($option_defaults_group) {
+        $options = $options . " --defaults-group=\"$option_defaults_group\" ";
+    }
+
     $options = $options . "--prepare --target-dir=$backup_dir";
 
     if ($option_uncompress) {
@@ -863,6 +868,10 @@
         $options = $options . " --defaults-file=\"$option_defaults_file\" ";
     }
 
+    if ($option_defaults_group) {
+        $options = $options . " --defaults-group=\"$option_defaults_group\" ";
+    }
+
     $options = $options . "--backup --suspend-at-end";
 
     if (!$option_remote_host && !$option_stream) {
@@ -925,13 +934,13 @@
 
             if($option_remote_host) {
                 #direct copy to remote
-                my $orig_datadir = get_option(\%config, 'mysqld', 'datadir');
+                my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
                 my $orig_ibdata_dir =
-                    get_option(\%config, 'mysqld', 'innodb_data_home_dir');
+                    get_option(\%config, $option_defaults_group, 'innodb_data_home_dir');
                 my $orig_innodb_data_file_path =
-                    get_option(\%config, 'mysqld', 'innodb_data_file_path');
+                    get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
                 my $innodb_flush_method = 
-                    get_option(\%config, 'mysqld', 'innodb_flush_method');
+                    get_option(\%config, $option_defaults_group, 'innodb_flush_method');
                 my $innodb_use_odirect;
                 $innodb_use_odirect = 1 if $innodb_flush_method =~ m/^(ALL_)?O_DIRECT$/i;
 
@@ -1582,7 +1591,7 @@
     read_config_file(\%config);
 
     if(!$option_tmpdir) {
-        $option_tmpdir = get_option(\%config, 'mysqld', 'tmpdir');
+        $option_tmpdir = get_option(\%config, $option_defaults_group, 'tmpdir');
     }
 
     # get innodb log home directory from options file
@@ -1658,8 +1667,8 @@
 
     my $option_name;
     foreach $option_name (@option_names) {
-        if (has_option(\%config, 'mysqld', $option_name)) {
-            my $option_value = get_option(\%config, 'mysqld', $option_name);
+        if (has_option(\%config, $option_defaults_group, $option_name)) {
+            my $option_value = get_option(\%config, $option_defaults_group, $option_name);
             $options_dump .= "$option_name=$option_value\n";
         }
     }
@@ -1725,6 +1734,7 @@
                         'user=s' => \$option_mysql_user,
                         'host=s' => \$option_mysql_host,
                         'port=s' => \$option_mysql_port,
+                        'defaults-group=s' => \$option_defaults_group,
                         'slave-info' => \$option_slave_info,
                         'galera-info' => \$option_galera_info,
                         'socket=s' => \$option_mysql_socket,
@@ -1840,7 +1850,7 @@
 sub make_backup_dir {
     my $dir;
     my $innodb_data_file_path = 
-        get_option(\%config, 'mysqld', 'innodb_data_file_path');
+        get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
 
     # create backup directory
     $dir = $backup_root;
@@ -1932,7 +1942,7 @@
 #
 sub backup_files {
     my $prep_mode = shift;
-    my $source_dir = get_option(\%config, 'mysqld', 'datadir');
+    my $source_dir = get_option(\%config, $option_defaults_group, 'datadir');
     my @list;
     my $file;
     my $database;
@@ -2211,6 +2221,10 @@
         $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
     }
 
+    if ($option_defaults_group) {
+        $options = $options . " --defaults-group=\"$option_defaults_group\" ";
+    }
+
     $options = $options . "--print-param";
 
 
@@ -2616,7 +2630,7 @@
              [--no-timestamp] [--ibbackup=IBBACKUP-BINARY]
              [--slave-info] [--stream=tar|xbstream]
              [--scpopt=OPTIONS-FOR-SCP]  [--sshopt=OPTIONS-FOR-SSH]
-             [--defaults-file=MY.CNF]
+             [--defaults-file=MY.CNF] [--defaults-group=GROUP-NAME]
              [--databases=LIST] [--remote-host=HOSTNAME] [--no-lock] 
              [--tmpdir=DIRECTORY] [--tables-file=FILE]
              [--incremental] [--incremental-basedir]
@@ -2628,7 +2642,7 @@
              [--export] [--redo-only] [--ibbackup=IBBACKUP-BINARY]
              BACKUP-DIR
 
-innobackupex --copy-back [--defaults-file=MY.CNF] BACKUP-DIR
+innobackupex --copy-back [--defaults-file=MY.CNF] [--defaults-group=GROUP-NAME] BACKUP-DIR
 
 =head1 DESCRIPTION
 
@@ -2811,6 +2825,10 @@
 
 This option specifies the MySQL username used when connecting to the server, if that's not the current user. The option accepts a string argument.  It is passed to the mysql child process without alteration. See mysql --help for details.
 
+=item --defaults-group=GROUP-NAME
+
+This option specifies the group name in my.cnf which should be used. This is needed for mysqld_multi deployments.
+
 =item --version
 
 This option displays the xtrabackup version and copyright notice and then exits.
 
=== modified file 'src/xtrabackup.c'
--- src/xtrabackup.c	2012-05-18 11:25:23 +0000
+++ src/xtrabackup.c	2012-05-18 11:25:23 +0000
@@ -769,6 +769,8 @@
 char *opt_mysql_tmpdir = NULL;
 MY_TMPDIR mysql_tmpdir_list;
 
+const char *defaults_group = "mysqld";
+
 /* === static parameters in ha_innodb.cc */
 
 #define HA_INNOBASE_ROWS_IN_TABLE 10000 /* to get optimization right */
@@ -994,7 +996,8 @@
   OPT_INNODB_SYNC_SPIN_LOOPS,
   OPT_INNODB_THREAD_CONCURRENCY,
   OPT_INNODB_THREAD_SLEEP_DELAY,
-  OPT_XTRA_DEBUG_SYNC
+  OPT_XTRA_DEBUG_SYNC,
+  OPT_DEFAULTS_GROUP
 };
 
 static struct my_option my_long_options[] =
@@ -1278,6 +1281,9 @@
    0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
 #endif
 
+  {"defaults_group", OPT_DEFAULTS_GROUP, "defaults group in config file (default \"mysqld\").",
+   (G_PTR*) &defaults_group, (G_PTR*) &defaults_group,
+   0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
   { 0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}
 };
 
@@ -1337,7 +1343,7 @@
 #endif
 }
 
-static const char *load_default_groups[]= { "mysqld","xtrabackup",0 };
+static const char *load_default_groups[]= { "mysqld","xtrabackup",0,0 };
 
 static void print_version(void)
 {
@@ -6280,6 +6286,17 @@
 	MY_INIT(argv[0]);
 	xb_regex_init();
 
+	/* scan options for group to load defaults from */
+	{
+		int	i;
+		char*	optend;
+		for (i=1; i < argc; i++) {
+			optend = strcend(argv[i], '=');
+			if (strncmp(argv[i], "--defaults-group", optend - argv[i]) == 0) {
+				load_default_groups[2] = defaults_group = optend + 1;
+			}
+		}
+	}
 	load_defaults("my",load_default_groups,&argc,&argv);
 
 	/* ignore unsupported options */
@@ -6518,7 +6535,7 @@
 			exit(EXIT_FAILURE);
 
 		printf("# This MySQL options file was generated by XtraBackup.\n");
-		printf("[mysqld]\n");
+		printf("[%s]\n", defaults_group);
 		printf("datadir = \"%s\"\n", mysql_data_home);
 		printf("tmpdir = \"%s\"\n", mysql_tmpdir_list.list[0]);
 		printf("innodb_data_home_dir = \"%s\"\n",
 
=== added file 'test/t/bug483827.sh'
--- test/t/bug483827.sh	1970-01-01 00:00:00 +0000
+++ test/t/bug483827.sh	2012-05-18 11:25:23 +0000
@@ -0,0 +1,45 @@
+########################################################################
+# Bug #483827: support for mysqld_multi
+########################################################################
+
+function modify_args()
+{
+  XB_ARGS=`echo $XB_ARGS | sed -e 's/my.cnf/my_multi.cnf/'`
+  IB_ARGS=`echo $IB_ARGS | sed -e 's/my.cnf/my_multi.cnf/'`
+}
+
+. inc/common.sh
+
+init
+mv ${mysql_datadir} ${mysql_datadir}1
+run_mysqld --datadir=${mysql_datadir}1
+
+backup_dir=$topdir/backup
+rm -rf $backup_dir
+
+# change defaults file from my.cnf to my_multi.cnf
+modify_args
+
+# make my_multi.cnf
+echo "
+[mysqld1]
+datadir=${mysql_datadir}1
+tmpdir=$mysql_tmpdir" > $topdir/my_multi.cnf
+
+# Backup
+innobackupex --no-timestamp --defaults-group=mysqld1 $backup_dir
+innobackupex --apply-log $backup_dir
+
+stop_mysqld
+
+# clean datadir
+rm -rf ${mysql_datadir}1/*
+
+# restore backup
+innobackupex --copy-back --defaults-group=mysqld1 $backup_dir
+
+# make sure that data are in correct place
+if [ ! -f ${mysql_datadir}1/ibdata1 ] ; then
+  vlog "Data not found in ${mysql_datadir}1"
+  exit -1
+fi