Merge lp:~akopytov/percona-xtrabackup/read-server-options-with-show-variables into lp:percona-xtrabackup/2.2

Proposed by Alexey Kopytov on 2014-10-06
Status: Merged
Approved by: Sergei Glushchenko on 2014-10-08
Approved revision: 5032
Merged at revision: 5032
Proposed branch: lp:~akopytov/percona-xtrabackup/read-server-options-with-show-variables
Merge into: lp:percona-xtrabackup/2.2
Diff against target: 875 lines (+385/-197)
8 files modified
storage/innobase/xtrabackup/innobackupex.pl (+182/-136)
storage/innobase/xtrabackup/src/xtrabackup.cc (+106/-48)
storage/innobase/xtrabackup/test/run.sh (+2/-6)
storage/innobase/xtrabackup/test/t/bug1062684.sh (+6/-6)
storage/innobase/xtrabackup/test/t/bug1334062.sh (+24/-0)
storage/innobase/xtrabackup/test/t/bug1343722.sh (+18/-0)
storage/innobase/xtrabackup/test/t/bug766305.sh (+1/-1)
storage/innobase/xtrabackup/test/t/xb_print_param.sh (+46/-0)
To merge this branch: bzr merge lp:~akopytov/percona-xtrabackup/read-server-options-with-show-variables
Reviewer Review Type Date Requested Status
Sergei Glushchenko (community) 2014-10-06 Approve on 2014-10-08
Review via email: mp+237249@code.launchpad.net

Description of the change

Implementation of
https://blueprints.launchpad.net/percona-xtrabackup/+spec/read-server-options-with-show-variables

On the backup stage, read options from the server via SHOW VARIABLES
rather than my.cnf. For the ‘datadir’ option, make sure we get the same
value as we would get from my.cnf (if the option is defined there),
because the xtrabackup binary does not have a server connection and uses
my.cnf instead.

To post a comment you must log in.

Hi Alexey,

The patch itself looks good, but

I am concerned of "Bug #1343722: Too easy to backup wrong datadir with
multiple instances". Not only datadir but every other important option
can be different if we read options from my.cnf which doesn't belong
to MySQL instance.

This patch makes following change:

innobackupex reads options from running MySQL.
xtrabackup reads options from my.cnf.

Now instead of single source we have two sources which can be out of
sync. We making sure that xtrabackup and innobackupex agree about
"datadir" but we don't take proper care of other options. Wouldn't it
be better to make both innobackupex and xtrabackup to use runtime
options (innobackupex could simply pass all sensitive options to
xtrabackup or even dump them into tmp-my.cnf).

review: Needs Information
Alexey Kopytov (akopytov) wrote :
Download full text (3.2 KiB)

Hi Sergei,

On Tue, Oct 07 2014 13:05:13 +0400, Sergei Glushchenko wrote:

> Hi Alexey,
>
> The patch itself looks good, but
>
> I am concerned of "Bug #1343722: Too easy to backup wrong datadir with
> multiple instances". Not only datadir but every other important option
> can be different if we read options from my.cnf which doesn't belong
> to MySQL instance.
>
> This patch makes following change:
>
> innobackupex reads options from running MySQL.
> xtrabackup reads options from my.cnf.
>
> Now instead of single source we have two sources which can be out of
> sync. We making sure that xtrabackup and innobackupex agree about
> "datadir" but we don't take proper care of other options. Wouldn't it
> be better to make both innobackupex and xtrabackup to use runtime
> options (innobackupex could simply pass all sensitive options to
> xtrabackup or even dump them into tmp-my.cnf).

Well, the patch went through multiple iterations. In one of them, it did
exactly that: it verified all options rather than just datadir. A few
problems had been revealed in testing:

- boolean values like innodb_fast_checksum can only be to 0/1 (or not
  have a value) in the defaults file. That’s how ‘xtrabackup
  --print-param’ prints them. But their values in SHOW VARIABLES are
  ON/OFF. So this must be taken into account when validating all
  options.

- path options like innodb_data_home_dir, innodb_log_group_home_dir,
  etc. They can be specified in an unnormalized way (with multiple
  slashes, “.”, “..”, etc.) in configuration file, but for some of them
  the server normalizes the paths as shown in SHOW VARIABLES, and for
  some of them no normalization is done. This could be solved with
  realpath() as we currently do with datadir. But on top of that, those
  paths can be relative to datadir, and that would require even more
  complex validation logic than just using realpath()

- sometimes we deliberately want to override some options for xtrabackup
  in the [xtrabackup] section of my.cnf, so that the values would be
  deliberately different between SHOW VARIABLES and the ones printed by
  ‘xtrabackup --print-param’.

After some considerations, I decided to only validate datadir, as
everything else would be a bit too risky for a point release. The patch
is rather invasive for a point release even without those changes.

As to passing all sensitive options from innobackupex to xtrabackup,
that’s what we do now for some of them (namely, datadir,
innodb_log_file_size and innodb_data_file_path), because those variables
have different defaults across major server versions, so using runtime
values is crucial.

Passing other value would work, and would eliminate possibility (which
is rather low in practice) of cases when ‘datadir’ is the same in SHOW
VARIABLES and my.cnf, but other (sensitive for XtraBackup) options are
different. But it comes with its own bunch of drawbacks and further
complications and corner cases in the code.

I decided to keep the patch minimal to solve current issues
for a point release, and scheduled
https://blueprints.launchpad.net/percona-xtrabackup/+spec/rewrite-innobackupex-in-c
for the next major release. Seriously, we should do it ASAP...

Read more...

Hi Alexey,

The patch looks good to me. I agree that getting rid of innobackupex will solve many issues and I believe we should include https://blueprints.launchpad.net/percona-xtrabackup/+spec/rewrite-innobackupex-in-c in the roadmap if it is not there yet.

Approved.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'storage/innobase/xtrabackup/innobackupex.pl'
--- storage/innobase/xtrabackup/innobackupex.pl 2014-10-05 12:02:02 +0000
+++ storage/innobase/xtrabackup/innobackupex.pl 2014-10-06 12:20:08 +0000
@@ -23,6 +23,7 @@
23use English qw(-no_match_vars);23use English qw(-no_match_vars);
24use Time::HiRes qw(usleep);24use Time::HiRes qw(usleep);
25use Carp qw(longmess);25use Carp qw(longmess);
26use Cwd qw(realpath);
2627
27# version of this script28# version of this script
28my $innobackup_version = '1.5.1-xtrabackup';29my $innobackup_version = '1.5.1-xtrabackup';
@@ -1896,10 +1897,10 @@
1896# process.1897# process.
1897#1898#
1898sub backup {1899sub backup {
1899 my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');1900 my $orig_datadir = get_option('datadir');
1900 my $suspend_file;1901 my $suspend_file;
1901 my $buffer_pool_filename = get_option(\%config, $option_defaults_group,1902 my $buffer_pool_filename = get_option_safe('innodb_buffer_pool_filename',
1902 'innodb_buffer_pool_filename');1903 '');
19031904
1904 detect_mysql_capabilities_for_backup(\%mysql);1905 detect_mysql_capabilities_for_backup(\%mysql);
19051906
@@ -2038,7 +2039,8 @@
2038 # copy ib_lru_dump2039 # copy ib_lru_dump
2039 # Copy buffer poll dump and/or LRU dump2040 # Copy buffer poll dump and/or LRU dump
2040 foreach my $dump_name ($buffer_pool_filename, 'ib_lru_dump') {2041 foreach my $dump_name ($buffer_pool_filename, 'ib_lru_dump') {
2041 if (!$option_rsync && -e "$orig_datadir/$dump_name") {2042 if (!$option_rsync && $dump_name ne '' &&
2043 -e "$orig_datadir/$dump_name") {
2042 backup_file("$orig_datadir", "$dump_name", "$backup_dir/$dump_name")2044 backup_file("$orig_datadir", "$dump_name", "$backup_dir/$dump_name")
2043 }2045 }
2044 }2046 }
@@ -2401,14 +2403,6 @@
2401#2403#
2402sub copy_back {2404sub copy_back {
2403 my $move_flag = shift;2405 my $move_flag = shift;
2404 my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
2405 my $orig_ibdata_dir =
2406 get_option(\%config, $option_defaults_group, 'innodb_data_home_dir');
2407 my $orig_innodb_data_file_path =
2408 get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
2409 my $orig_iblog_dir =
2410 get_option(\%config, $option_defaults_group, 'innodb_log_group_home_dir');
2411 my $orig_undo_dir = $orig_ibdata_dir;
2412 my $iblog_files = 'ib_logfile.*';2406 my $iblog_files = 'ib_logfile.*';
2413 my $ibundo_files = 'undo[0-9]{3}';2407 my $ibundo_files = 'undo[0-9]{3}';
2414 my $excluded_files = 2408 my $excluded_files =
@@ -2418,21 +2412,24 @@
2418 '.*\.pmap|.*\.tmp|' .2412 '.*\.pmap|.*\.tmp|' .
2419 $iblog_files . '|'.2413 $iblog_files . '|'.
2420 $ibundo_files;2414 $ibundo_files;
2421 my $compressed_data_file = '.*\.ibz$';
2422 my $file;2415 my $file;
2423 my $backup_innodb_data_file_path;2416
24242417 my $orig_datadir = get_option('datadir');
2425 if (has_option(\%config, $option_defaults_group, 'innodb_doublewrite_file')) {2418 my $orig_ibdata_dir = get_option_safe('innodb_data_home_dir',
2419 $orig_datadir);
2420 my $orig_innodb_data_file_path = get_option_safe('innodb_data_file_path',
2421 '');
2422 my $orig_iblog_dir = get_option_safe('innodb_log_group_home_dir',
2423 $orig_datadir);
2424 my $orig_undo_dir = get_option_safe('innodb_undo_directory',
2425 $orig_datadir);
2426
2427 if (has_option('innodb_doublewrite_file')) {
2426 my $doublewrite_file =2428 my $doublewrite_file =
2427 get_option(\%config, $option_defaults_group,2429 get_option('innodb_doublewrite_file');
2428 'innodb_doublewrite_file');
2429 $excluded_files = $excluded_files . '|' . $doublewrite_file;2430 $excluded_files = $excluded_files . '|' . $doublewrite_file;
2430 }2431 }
24312432
2432 if (has_option(\%config, $option_defaults_group, 'innodb_undo_directory')) {
2433 $orig_undo_dir = get_option(\%config, $option_defaults_group,
2434 'innodb_undo_directory');
2435 }
2436 # check whether files should be copied or moved to dest directory2433 # check whether files should be copied or moved to dest directory
2437 my $move_or_copy_file = $move_flag ? \&move_file : \&copy_file;2434 my $move_or_copy_file = $move_flag ? \&move_file : \&copy_file;
2438 my $move_or_copy_dir = $move_flag ?2435 my $move_or_copy_dir = $move_flag ?
@@ -2568,11 +2565,7 @@
2568 my $cmdline_copy = '';2565 my $cmdline_copy = '';
2569 my $options = '';2566 my $options = '';
25702567
2571 if ($option_defaults_file) {2568 $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
2572 $options = $options . " --defaults-file=\"$option_defaults_file\" ";
2573 } else {
2574 $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
2575 }
25762569
2577 if ($option_defaults_extra_file) {2570 if ($option_defaults_extra_file) {
2578 $options = $options . " --defaults-extra-file=\"$option_defaults_extra_file\" ";2571 $options = $options . " --defaults-extra-file=\"$option_defaults_extra_file\" ";
@@ -2603,7 +2596,7 @@
2603 }2596 }
26042597
2605 my $innodb_data_file_path = 2598 my $innodb_data_file_path =
2606 get_option(\%config, $option_defaults_group, 'innodb_data_file_path');2599 get_option('innodb_data_file_path');
26072600
2608 # run ibbackup as a child process2601 # run ibbackup as a child process
2609 $cmdline = "$option_ibbackup_binary $options";2602 $cmdline = "$option_ibbackup_binary $options";
@@ -2786,6 +2779,21 @@
2786 $options = $options . " --target-dir=$backup_dir";2779 $options = $options . " --target-dir=$backup_dir";
2787 }2780 }
27882781
2782 my $datadir = get_option('datadir');
2783 if (!has_option_in_config('datadir')) {
2784 $options .= " --datadir=\"$datadir\"";
2785 }
2786
2787 my $innodb_log_file_size = get_option('innodb_log_file_size');
2788 if (!has_option_in_config('innodb_log_file_size')) {
2789 $options .= " --innodb_log_file_size=\"$innodb_log_file_size\"";
2790 }
2791
2792 my $innodb_data_file_path = get_option('innodb_data_file_path');
2793 if (!has_option_in_config('innodb_data_file_path')) {
2794 $options .= " --innodb_data_file_path=\"$innodb_data_file_path\"";
2795 }
2796
2789 if ($option_tmpdir) {2797 if ($option_tmpdir) {
2790 $options .= " --tmpdir=$option_tmpdir";2798 $options .= " --tmpdir=$option_tmpdir";
2791 }2799 }
@@ -3579,44 +3587,44 @@
3579 print STDERR " At the end of a successful $run run $innobackup_script\n";3587 print STDERR " At the end of a successful $run run $innobackup_script\n";
3580 print STDERR " prints \"completed OK!\".\n\n";3588 print STDERR " prints \"completed OK!\".\n\n";
35813589
3582 if (!$option_copy_back && !$option_move_back3590 if ($option_apply_log || $option_copy_back || $option_move_back) {
3583 && !$option_decrypt && !$option_decompress) {3591 # read server configuration file
3584 # we are making a backup or applying log to backup3592 read_config_file(\%config);
3585 if (!$option_apply_log) {3593 } elsif ($option_backup) {
3586 # we are making a backup, we need mysql server3594 # we are making a backup, read server configuration from SHOW VARIABLES
3587 get_mysql_vars(\%mysql);3595 get_mysql_vars(\%mysql);
3588 $mysql_server_version = $mysql{vars}->{version}->{Value};3596
3589 print STDERR "$prefix Using mysql server version $mysql_server_version\n";3597 # and make sure datadir value is the same in configuration file
3598 read_config_file(\%config);
3599
3600 my $server_val = $mysql{vars}->{datadir}->{Value};
3601
3602 if (has_option_in_config('datadir')) {
3603 my $config_val = $config{$option_defaults_group}{datadir};
3604
3605 if ($server_val ne $config_val &&
3606 # Try to canonicalize paths
3607 realpath($server_val) ne realpath($config_val)) {
3608
3609 die "option 'datadir' has different values:\n" .
3610 " '$config_val' in defaults file\n" .
3611 " '$server_val' in SHOW VARIABLES\n"
3612 }
3590 }3613 }
3614
3615 $mysql_server_version = $mysql{vars}->{version}->{Value};
3616 print STDERR "$prefix Using server version $mysql_server_version\n";
3591 print STDERR "\n";3617 print STDERR "\n";
3592 3618 }
3593 if ($option_include 3619
3594 && $ibbackup_version 3620 if (!$option_tmpdir && ($option_backup || $option_apply_log) &&
3595 && $ibbackup_version le "2.0") {3621 has_option('tmpdir')) {
3596 # --include option was given, but ibbackup is too3622 $option_tmpdir = get_option('tmpdir');
3597 # old to support it3623 # tmpdir can be a colon-separated list of multiple directories
3598 die "--include option was given, but ibbackup is too old"3624 $option_tmpdir = (split(/:/, $option_tmpdir))[0];
3599 . " to support it. You must upgrade to InnoDB Hot Backup"3625 }
3600 . " v2.0 in order to use --include option.\n";3626
3601 }3627 if ($option_backup) {
3602 }
3603
3604 if (!$option_decrypt && !$option_decompress) {
3605 # read MySQL options file
3606 #read_config_file($config_file, \%config);
3607 read_config_file(\%config);
3608
3609 if(!$option_tmpdir) {
3610 $option_tmpdir = get_option(\%config, $option_defaults_group, 'tmpdir');
3611 }
3612 }
3613
3614 # get innodb log home directory from options file
3615 #$innodb_log_group_home_dir =
3616 # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
3617
3618 if (!$option_apply_log && !$option_copy_back && !$option_move_back
3619 && !$option_decrypt && !$option_decompress) {
3620 # we are making a backup, create a new backup directory3628 # we are making a backup, create a new backup directory
3621 $backup_dir = File::Spec->rel2abs(make_backup_dir());3629 $backup_dir = File::Spec->rel2abs(make_backup_dir());
3622 print STDERR "$prefix Created backup directory $backup_dir\n";3630 print STDERR "$prefix Created backup directory $backup_dir\n";
@@ -3656,10 +3664,6 @@
3656 die "Failed to delete " .3664 die "Failed to delete " .
3657 "'$option_tmpdir/$xtrabackup_pid_file': $!";3665 "'$option_tmpdir/$xtrabackup_pid_file': $!";
3658 }3666 }
3659
3660 } elsif ($option_copy_back || $option_move_back) {
3661 #$backup_config_file = $backup_dir . '/backup-my.cnf';
3662 #read_config_file($backup_config_file, \%backup_config);
3663 }3667 }
3664}3668}
36653669
@@ -3693,16 +3697,18 @@
36933697
3694 my $option_name;3698 my $option_name;
3695 foreach $option_name (@option_names) {3699 foreach $option_name (@option_names) {
3696 if (has_option(\%config, $option_defaults_group, $option_name)) {3700 if (has_option($option_name)) {
3697 my $option_value = get_option(\%config, $option_defaults_group, $option_name);3701 my $option_value = get_option($option_name);
3698 $options_dump .= "$option_name=$option_value\n";3702 $options_dump .= "$option_name=$option_value\n";
3699 }3703 }
3700 }3704 }
3701 if (has_option(\%config,3705 if (has_option('innodb_doublewrite_file')) {
3702 $option_defaults_group, "innodb_doublewrite_file")) {3706 my $option_value = (split(/\/+/,
3703 $options_dump .= "innodb_doublewrite_file=" . (split(/\/+/,3707 get_option('innodb_doublewrite_file')))[-1];
3704 get_option(\%config, $option_defaults_group,3708
3705 'innodb_doublewrite_file')))[-1] . "\n";3709 if (defined($option_value)) {
3710 $options_dump .= "innodb_doublewrite_file=" . $option_value . "\n";
3711 }
3706 }3712 }
37073713
3708 write_to_backup_file("$filename", "$options_dump");3714 write_to_backup_file("$filename", "$options_dump");
@@ -3940,8 +3946,6 @@
3940#3946#
3941sub make_backup_dir {3947sub make_backup_dir {
3942 my $dir;3948 my $dir;
3943 my $innodb_data_file_path =
3944 get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
39453949
3946 # create backup directory3950 # create backup directory
3947 $dir = $backup_root;3951 $dir = $backup_root;
@@ -3953,19 +3957,6 @@
3953 unless $option_no_timestamp;3957 unless $option_no_timestamp;
3954 mkdir($dir, 0777) || die "Failed to create backup directory $dir: $!";3958 mkdir($dir, 0777) || die "Failed to create backup directory $dir: $!";
39553959
3956 # create subdirectories for ibdata files if needed
3957# foreach my $a (split(/;/, $innodb_data_file_path)) {
3958# my $path = (split(/:/,$a))[0];
3959# my @relative_path = split(/\/+/, $path);
3960# pop @relative_path;
3961# if (@relative_path) {
3962# # there is a non-trivial path from the backup directory
3963# # to the directory of this backup ibdata file, check
3964# # that all the directories in the path exist.
3965# create_path_if_needed($dir, \@relative_path);
3966# }
3967# }
3968
3969 return $dir;3960 return $dir;
3970}3961}
39713962
@@ -4023,9 +4014,9 @@
4023#4014#
4024sub backup_files {4015sub backup_files {
4025 my $prep_mode = shift;4016 my $prep_mode = shift;
4026 my $source_dir = get_option(\%config, $option_defaults_group, 'datadir');4017 my $source_dir = get_option('datadir');
4027 my $buffer_pool_filename = get_option(\%config, $option_defaults_group,4018 my $buffer_pool_filename = get_option_safe('innodb_buffer_pool_filename',
4028 'innodb_buffer_pool_filename');4019 '');
4029 my @list;4020 my @list;
4030 my $file;4021 my $file;
4031 my $database;4022 my $database;
@@ -4145,7 +4136,7 @@
41454136
4146 if ($option_rsync) {4137 if ($option_rsync) {
4147 foreach my $dump_name ($buffer_pool_filename, 'ib_lru_dump') {4138 foreach my $dump_name ($buffer_pool_filename, 'ib_lru_dump') {
4148 if (-e "$source_dir/$dump_name") {4139 if ($dump_name ne '' && -e "$source_dir/$dump_name") {
4149 print RSYNC "$dump_name\n";4140 print RSYNC "$dump_name\n";
4150 if (!$prep_mode) {4141 if (!$prep_mode) {
4151 $rsync_files_hash{"$dump_name"} = 1;4142 $rsync_files_hash{"$dump_name"} = 1;
@@ -4289,10 +4280,11 @@
4289 my $options = '';4280 my $options = '';
42904281
42914282
4292 if ($option_defaults_file) {4283 if ($option_apply_log) {
4284 $options = $options .
4285 " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
4286 } elsif ($option_defaults_file) {
4293 $options = $options . " --defaults-file=\"$option_defaults_file\" ";4287 $options = $options . " --defaults-file=\"$option_defaults_file\" ";
4294 } elsif ($option_apply_log) {
4295 $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
4296 }4288 }
42974289
4298 if ($option_defaults_extra_file) {4290 if ($option_defaults_extra_file) {
@@ -4367,60 +4359,114 @@
4367 }4359 }
4368 }4360 }
4369}4361}
4370 4362
43714363
4372# has_option return whether the config has an option with the given name4364# has_option_in_config return true if the configuration file defines an option
4373# Parameters:4365# with the given name.
4374# config_ref a reference to a config data4366#
4375# group option group name4367# Parameters:
4368# option_name name of the option
4369# Return value:
4370# true if option exists, otherwise false
4371#
4372sub has_option_in_config {
4373 my $option_name = shift;
4374 my $group_hash_ref;
4375
4376 if (!exists $config{$option_defaults_group}) {
4377 return 0;
4378 }
4379
4380 $group_hash_ref = $config{$option_defaults_group};
4381
4382 return exists ${$group_hash_ref}{$option_name};
4383}
4384
4385
4386# has_option returns 1 if an option with the given name exists in either
4387# defaults file file as reported by 'xtrabackup --print-param' or (in backup
4388# mode) SHOW VARIABLES. Otherwise returns 0.
4389#
4390# Parameters:
4376# option_name name of the option4391# option_name name of the option
4377# Return value:4392# Return value:
4378# true if option exists, otherwise false4393# true if option exists, otherwise false
4379#4394#
4380sub has_option {4395sub has_option {
4381 my $config_ref = shift;
4382 my $group = shift;
4383 my $option_name = shift;4396 my $option_name = shift;
4384 my $group_hash_ref;4397
43854398 if (has_option_in_config($option_name)) {
4386 if (!exists ${$config_ref}{$group}) {4399 return 1;
4387 # no group4400 }
4388 die "no '$group' group in MySQL options";4401
4389 }4402 if ($option_backup) {
43904403 if (!defined($mysql{vars})) {
4391 $group_hash_ref = ${$config_ref}{$group};4404 get_mysql_vars(\%mysql);
4392 4405 }
4393 return exists ${$group_hash_ref}{$option_name};4406
4407 return defined($mysql{vars}->{$option_name});
4408 }
4409
4410 return 0;
4394}4411}
4395 4412
43964413
4397# get_option subroutine returns the value of given option in the config4414# get_option returns the value of an option with the given name as defined
4398# structure. If option is missing, this subroutine calls exit.4415# either in defaults file as reported by 'xtrabackup --print-param' or (in
4399# Parameters:4416# backup mode) SHOW VARIABLES.
4400# config_ref a reference to a config data4417#
4401# group option group name4418# This subroutine aborts with an error if the option is not defined.
4419#
4420# Parameters:
4402# option_name name of the option4421# option_name name of the option
4403# Return value:4422# Return value:
4404# option value as a string4423# option value as a string
4405#4424#
4406sub get_option {4425sub get_option {
4407 my $config_ref = shift;
4408 my $group = shift;
4409 my $option_name = shift;4426 my $option_name = shift;
4410 my $group_hash_ref;4427 my $group_hash_ref;
44114428
4412 if (!exists $config{$group}) {4429 if (!$option_backup) {
4413 # no group4430 if (!exists $config{$option_defaults_group}) {
4414 die "no '$group' group in MySQL options";4431 # no group
4415 }4432 die "no '$option_defaults_group' group in server configuration " .
4416 4433 "file '$option_defaults_file'";
4417 $group_hash_ref = ${$config_ref}{$group};4434 }
4418 if (!exists ${$group_hash_ref}{$option_name}) {4435
4419 # no option4436 $group_hash_ref = $config{$option_defaults_group};
4420 die "no '$option_name' option in group '$group' in MySQL options";4437 if (!exists ${$group_hash_ref}{$option_name}) {
4421 }4438 # no option
44224439 die "no '$option_name' option in group '$option_defaults_group' " .
4423 return ${$group_hash_ref}{$option_name};4440 "in server configuration file '$option_defaults_file'";
4441 }
4442
4443 return ${$group_hash_ref}{$option_name};
4444 }
4445
4446 if (!defined($mysql{vars})) {
4447 get_mysql_vars(\%mysql);
4448 }
4449
4450 if (!defined($mysql{vars}->{$option_name})) {
4451 die "no '$option_name' option in SHOW VARIABLES";
4452 }
4453
4454 return $mysql{vars}->{$option_name}->{Value};
4455}
4456
4457#
4458# Identical to get_option, except that the second argument is returned on error,
4459# i.e. if the option is not defined.
4460#
4461sub get_option_safe {
4462 my $option_name = shift;
4463 my $fallback_value = shift;
4464
4465 if (has_option($option_name)) {
4466 return get_option($option_name);
4467 }
4468
4469 return $fallback_value;
4424}4470}
44254471
4426# get_table_name subroutine returns table name of specified file.4472# get_table_name subroutine returns table name of specified file.
44274473
=== modified file 'storage/innobase/xtrabackup/src/xtrabackup.cc'
--- storage/innobase/xtrabackup/src/xtrabackup.cc 2014-09-25 07:02:53 +0000
+++ storage/innobase/xtrabackup/src/xtrabackup.cc 2014-10-06 12:20:08 +0000
@@ -66,6 +66,8 @@
66#include <row0quiesce.h>66#include <row0quiesce.h>
67#include <srv0start.h>67#include <srv0start.h>
6868
69#include <sstream>
70
69#define G_PTR uchar*71#define G_PTR uchar*
7072
71#include "common.h"73#include "common.h"
@@ -309,6 +311,10 @@
309ds_ctxt_t *ds_data = NULL;311ds_ctxt_t *ds_data = NULL;
310ds_ctxt_t *ds_meta = NULL;312ds_ctxt_t *ds_meta = NULL;
311313
314/* String buffer used by --print-param to accumulate server options as they are
315parsed from the defaults file */
316static std::ostringstream print_param_str;
317
312extern "C" sig_handler handle_fatal_signal(int sig);318extern "C" sig_handler handle_fatal_signal(int sig);
313319
314/* Simple datasink creation tracking...add datasinks in the reverse order you320/* Simple datasink creation tracking...add datasinks in the reverse order you
@@ -483,7 +489,7 @@
483 OPT_XTRA_REBUILD_THREADS,489 OPT_XTRA_REBUILD_THREADS,
484 OPT_INNODB_CHECKSUM_ALGORITHM,490 OPT_INNODB_CHECKSUM_ALGORITHM,
485 OPT_INNODB_UNDO_DIRECTORY,491 OPT_INNODB_UNDO_DIRECTORY,
486 OPT_UNDO_TABLESPACES,492 OPT_INNODB_UNDO_TABLESPACES,
487 OPT_INNODB_LOG_CHECKSUM_ALGORITHM,493 OPT_INNODB_LOG_CHECKSUM_ALGORITHM,
488 OPT_XTRA_INCREMENTAL_FORCE_SCAN,494 OPT_XTRA_INCREMENTAL_FORCE_SCAN,
489 OPT_DEFAULTS_GROUP,495 OPT_DEFAULTS_GROUP,
@@ -839,7 +845,7 @@
839 (G_PTR*) &srv_undo_dir, (G_PTR*) &srv_undo_dir,845 (G_PTR*) &srv_undo_dir, (G_PTR*) &srv_undo_dir,
840 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},846 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
841847
842 {"innodb_undo_tablespaces", OPT_UNDO_TABLESPACES,848 {"innodb_undo_tablespaces", OPT_INNODB_UNDO_TABLESPACES,
843 "Number of undo tablespaces to use.",849 "Number of undo tablespaces to use.",
844 (G_PTR*)&srv_undo_tablespaces, (G_PTR*)&srv_undo_tablespaces,850 (G_PTR*)&srv_undo_tablespaces, (G_PTR*)&srv_undo_tablespaces,
845 0, GET_ULONG, REQUIRED_ARG, 0, 0, 126, 0, 1, 0},851 0, GET_ULONG, REQUIRED_ARG, 0, 0, 126, 0, 1, 0},
@@ -959,6 +965,9 @@
959 my_print_variables(xb_long_options);965 my_print_variables(xb_long_options);
960}966}
961967
968#define ADD_PRINT_PARAM_OPT(value) \
969 print_param_str << opt->name << "=" << value << "\n";
970
962static my_bool971static my_bool
963get_one_option(int optid, const struct my_option *opt __attribute__((unused)),972get_one_option(int optid, const struct my_option *opt __attribute__((unused)),
964 char *argument)973 char *argument)
@@ -967,7 +976,94 @@
967 case 'h':976 case 'h':
968 strmake(mysql_real_data_home,argument, FN_REFLEN - 1);977 strmake(mysql_real_data_home,argument, FN_REFLEN - 1);
969 mysql_data_home= mysql_real_data_home;978 mysql_data_home= mysql_real_data_home;
970 break;979
980 ADD_PRINT_PARAM_OPT(mysql_real_data_home);
981 break;
982
983 case 't':
984
985 ADD_PRINT_PARAM_OPT(opt_mysql_tmpdir);
986 break;
987
988 case OPT_INNODB_DATA_HOME_DIR:
989
990 ADD_PRINT_PARAM_OPT(innobase_data_home_dir);
991 break;
992
993 case OPT_INNODB_DATA_FILE_PATH:
994
995 ADD_PRINT_PARAM_OPT(innobase_data_file_path);
996 break;
997
998 case OPT_INNODB_LOG_GROUP_HOME_DIR:
999
1000 ADD_PRINT_PARAM_OPT(srv_log_group_home_dir);
1001 break;
1002
1003 case OPT_INNODB_LOG_FILES_IN_GROUP:
1004
1005 ADD_PRINT_PARAM_OPT(innobase_log_files_in_group);
1006 break;
1007
1008 case OPT_INNODB_LOG_FILE_SIZE:
1009
1010 ADD_PRINT_PARAM_OPT(innobase_log_file_size);
1011 break;
1012
1013 case OPT_INNODB_FLUSH_METHOD:
1014
1015 ADD_PRINT_PARAM_OPT(innobase_unix_file_flush_method);
1016 break;
1017
1018 case OPT_INNODB_PAGE_SIZE:
1019
1020 ADD_PRINT_PARAM_OPT(innobase_page_size);
1021 break;
1022
1023 case OPT_INNODB_FAST_CHECKSUM:
1024
1025 ADD_PRINT_PARAM_OPT(!!innobase_fast_checksum);
1026 break;
1027
1028 case OPT_INNODB_LOG_BLOCK_SIZE:
1029
1030 ADD_PRINT_PARAM_OPT(innobase_log_block_size);
1031 break;
1032
1033 case OPT_INNODB_DOUBLEWRITE_FILE:
1034
1035 ADD_PRINT_PARAM_OPT(innobase_doublewrite_file);
1036 break;
1037
1038 case OPT_INNODB_UNDO_DIRECTORY:
1039
1040 ADD_PRINT_PARAM_OPT(srv_undo_dir);
1041 break;
1042
1043 case OPT_INNODB_UNDO_TABLESPACES:
1044
1045 ADD_PRINT_PARAM_OPT(srv_undo_tablespaces);
1046 break;
1047
1048 case OPT_INNODB_CHECKSUM_ALGORITHM:
1049
1050 ut_a(srv_checksum_algorithm <= SRV_CHECKSUM_ALGORITHM_STRICT_NONE);
1051
1052 ADD_PRINT_PARAM_OPT(innodb_checksum_algorithm_names[srv_checksum_algorithm]);
1053 break;
1054
1055 case OPT_INNODB_LOG_CHECKSUM_ALGORITHM:
1056
1057 ut_a(srv_log_checksum_algorithm <= SRV_CHECKSUM_ALGORITHM_STRICT_NONE);
1058
1059 ADD_PRINT_PARAM_OPT(innodb_checksum_algorithm_names[srv_log_checksum_algorithm]);
1060 break;
1061
1062 case OPT_INNODB_BUFFER_POOL_FILENAME:
1063
1064 ADD_PRINT_PARAM_OPT(innobase_buffer_pool_filename);
1065 break;
1066
971 case OPT_XTRA_TARGET_DIR:1067 case OPT_XTRA_TARGET_DIR:
972 strmake(xtrabackup_real_target_dir,argument, sizeof(xtrabackup_real_target_dir)-1);1068 strmake(xtrabackup_real_target_dir,argument, sizeof(xtrabackup_real_target_dir)-1);
973 xtrabackup_target_dir= xtrabackup_real_target_dir;1069 xtrabackup_target_dir= xtrabackup_real_target_dir;
@@ -6045,6 +6141,10 @@
6045 }6141 }
6046 load_defaults("my", xb_load_default_groups, &argc, &argv);6142 load_defaults("my", xb_load_default_groups, &argc, &argv);
60476143
6144 print_param_str <<
6145 "# This MySQL options file was generated by XtraBackup.\n"
6146 "[" << defaults_group << "]\n";
6147
6048 /* ignore unsupported options */6148 /* ignore unsupported options */
6049 {6149 {
6050 int i,j,argc_new,find;6150 int i,j,argc_new,find;
@@ -6166,51 +6266,9 @@
61666266
6167 /* --print-param */6267 /* --print-param */
6168 if (xtrabackup_print_param) {6268 if (xtrabackup_print_param) {
6169 /* === some variables from mysqld === */6269
6170 memset((G_PTR) &mysql_tmpdir_list, 0,6270 printf("%s", print_param_str.str().c_str());
6171 sizeof(mysql_tmpdir_list));6271
6172
6173 if (init_tmpdir(&mysql_tmpdir_list, opt_mysql_tmpdir))
6174 exit(EXIT_FAILURE);
6175
6176 printf("# This MySQL options file was generated by XtraBackup.\n");
6177 printf("[%s]\n", defaults_group);
6178 printf("datadir = \"%s\"\n", mysql_data_home);
6179 printf("tmpdir = \"%s\"\n", mysql_tmpdir_list.list[0]);
6180 printf("innodb_data_home_dir = \"%s\"\n",
6181 innobase_data_home_dir ? innobase_data_home_dir : mysql_data_home);
6182 printf("innodb_data_file_path = \"%s\"\n",
6183 innobase_data_file_path ? innobase_data_file_path : "ibdata1:10M:autoextend");
6184 printf("innodb_log_group_home_dir = \"%s\"\n",
6185 srv_log_group_home_dir ?
6186 srv_log_group_home_dir : mysql_data_home);
6187 printf("innodb_log_files_in_group = %ld\n", innobase_log_files_in_group);
6188 printf("innodb_log_file_size = %lld\n", innobase_log_file_size);
6189 printf("innodb_flush_method = \"%s\"\n",
6190 (innobase_unix_file_flush_method != NULL) ?
6191 innobase_unix_file_flush_method : "");
6192 printf("innodb_page_size = %lld\n", innobase_page_size);
6193 printf("innodb_fast_checksum = %d\n", innobase_fast_checksum);
6194 printf("innodb_log_block_size = %lu\n", innobase_log_block_size);
6195 if (innobase_doublewrite_file != NULL) {
6196 printf("innodb_doublewrite_file = %s\n", innobase_doublewrite_file);
6197 }
6198 if (srv_undo_dir) {
6199
6200 printf("innodb_undo_directory = \"%s\"\n",
6201 srv_undo_dir);
6202 }
6203 printf("innodb_undo_tablespaces = %lu\n", srv_undo_tablespaces);
6204 printf("innodb_checksum_algorithm = %s\n",
6205 innodb_checksum_algorithm_names[srv_checksum_algorithm]
6206 );
6207 printf("innodb_log_checksum_algorithm = %s\n",
6208 innodb_checksum_algorithm_names[srv_log_checksum_algorithm]
6209 );
6210 printf("innodb_buffer_pool_filename = \"%s\"\n",
6211 innobase_buffer_pool_filename ?
6212 innobase_buffer_pool_filename :
6213 "ib_buffer_pool");
6214 exit(EXIT_SUCCESS);6272 exit(EXIT_SUCCESS);
6215 }6273 }
62166274
62176275
=== modified file 'storage/innobase/xtrabackup/test/run.sh'
--- storage/innobase/xtrabackup/test/run.sh 2014-09-10 12:10:01 +0000
+++ storage/innobase/xtrabackup/test/run.sh 2014-10-06 12:20:08 +0000
@@ -373,9 +373,6 @@
373 fi373 fi
374 fi374 fi
375375
376 # Version-specific defaults
377 DEFAULT_IBDATA_SIZE="10M"
378
379 # Determine MySQL flavor376 # Determine MySQL flavor
380 if [[ "$MYSQL_VERSION" =~ "MariaDB" ]]377 if [[ "$MYSQL_VERSION" =~ "MariaDB" ]]
381 then378 then
@@ -410,8 +407,7 @@
410 ;;407 ;;
411 5.5 )408 5.5 )
412 ;;409 ;;
413 5.6 | 10.0 )410 5.6 | 10.0 | 10.1)
414 DEFAULT_IBDATA_SIZE="12M"
415 ;;411 ;;
416 *)412 *)
417 vlog "Unknown MySQL/InnoDB version: $MYSQL_VERSION/$INNODB_VERSION"413 vlog "Unknown MySQL/InnoDB version: $MYSQL_VERSION/$INNODB_VERSION"
@@ -440,7 +436,7 @@
440 export MYSQL_VERSION MYSQL_VERSION_COMMENT MYSQL_FLAVOR \436 export MYSQL_VERSION MYSQL_VERSION_COMMENT MYSQL_FLAVOR \
441 INNODB_VERSION XTRADB_VERSION INNODB_FLAVOR \437 INNODB_VERSION XTRADB_VERSION INNODB_FLAVOR \
442 XB_BIN IB_BIN IB_ARGS XB_ARGS MYSQLD_EXTRA_ARGS \438 XB_BIN IB_BIN IB_ARGS XB_ARGS MYSQLD_EXTRA_ARGS \
443 DEFAULT_IBDATA_SIZE WSREP_READY LIBGALERA_PATH439 WSREP_READY LIBGALERA_PATH
444}440}
445441
446###########################################################################442###########################################################################
447443
=== modified file 'storage/innobase/xtrabackup/test/t/bug1062684.sh'
--- storage/innobase/xtrabackup/test/t/bug1062684.sh 2013-04-27 18:46:54 +0000
+++ storage/innobase/xtrabackup/test/t/bug1062684.sh 2014-10-06 12:20:08 +0000
@@ -7,7 +7,7 @@
7. inc/common.sh7. inc/common.sh
88
9MYSQLD_EXTRA_MY_CNF_OPTS="9MYSQLD_EXTRA_MY_CNF_OPTS="
10innodb-data-file-path=ibdata1:${DEFAULT_IBDATA_SIZE};ibdata2:5M:autoextend10innodb-data-file-path=ibdata1:10M;ibdata2:5M:autoextend
11"11"
1212
13start_server13start_server
@@ -22,8 +22,8 @@
22mkdir -p $topdir/backup22mkdir -p $topdir/backup
2323
24vlog "Starting backup"24vlog "Starting backup"
25innobackupex $topdir/backup25full_backup_dir=$topdir/backup/full
26full_backup_dir=`grep "innobackupex: Backup created in directory" $OUTFILE | awk -F\' '{print $2}'`26innobackupex --no-timestamp $full_backup_dir
27vlog "Full backup done to directory $full_backup_dir"27vlog "Full backup done to directory $full_backup_dir"
2828
29# Changing data29# Changing data
@@ -47,9 +47,9 @@
47vlog "###############"47vlog "###############"
4848
49# Incremental backup49# Incremental backup
50inc_backup_dir=$topdir/backup/inc
50innobackupex --incremental --incremental-basedir=$full_backup_dir \51innobackupex --incremental --incremental-basedir=$full_backup_dir \
51 $topdir/backup52 --no-timestamp $inc_backup_dir
52inc_backup_dir=`grep "innobackupex: Backup created in directory" $OUTFILE | tail -n 1 | awk -F\' '{print $2}'`
53vlog "Incremental backup done to directory $inc_backup_dir"53vlog "Incremental backup done to directory $inc_backup_dir"
5454
55vlog "Preparing backup"55vlog "Preparing backup"
@@ -84,7 +84,7 @@
84innobackupex --copy-back $full_backup_dir84innobackupex --copy-back $full_backup_dir
85vlog "Data restored"85vlog "Data restored"
8686
87start_server --innodb-data-file-path="ibdata1:${DEFAULT_IBDATA_SIZE};ibdata2:5M:autoextend"87start_server
8888
89vlog "Checking checksums"89vlog "Checking checksums"
90checksum_test_b=`checksum_table incremental_sample test`90checksum_test_b=`checksum_table incremental_sample test`
9191
=== added file 'storage/innobase/xtrabackup/test/t/bug1334062.sh'
--- storage/innobase/xtrabackup/test/t/bug1334062.sh 1970-01-01 00:00:00 +0000
+++ storage/innobase/xtrabackup/test/t/bug1334062.sh 2014-10-06 12:20:08 +0000
@@ -0,0 +1,24 @@
1#############################################################################
2# Bug #1334062: Xtrabackup 2.2.3 fails to perform a full backup on PS 5.5 if
3# innodb_log_file_size on the [mysqld] section of my.cnf is not
4# set
5#############################################################################
6
7start_server
8
9sed -i -e 's/innodb_log_file_size=.*//' $MYSQLD_VARDIR/my.cnf
10
11grep innodb_log_file_size $MYSQLD_VARDIR/my.cnf &&
12 die "innodb_log_file_size is present in my.cnf"
13
14innobackupex --no-timestamp $topdir/backup
15
16innobackupex --apply-log $topdir/backup
17
18stop_server
19
20rm -rf $MYSQLD_DATADIR/*
21
22innobackupex --copy-back $topdir/backup
23
24start_server
025
=== added file 'storage/innobase/xtrabackup/test/t/bug1343722.sh'
--- storage/innobase/xtrabackup/test/t/bug1343722.sh 1970-01-01 00:00:00 +0000
+++ storage/innobase/xtrabackup/test/t/bug1343722.sh 2014-10-06 12:20:08 +0000
@@ -0,0 +1,18 @@
1########################################################################
2# Bug #1343722: Too easy to backup wrong datadir with multiple instances
3########################################################################
4
5start_server_with_id 1
6
7socket=$MYSQLD_SOCKET
8
9start_server_with_id 2
10
11# Try to backup server 2, but use server 1's connection socket
12$IB_BIN $IB_ARGS --socket=$socket --no-timestamp $topdir/backup 2>&1 |
13 grep 'has different values'
14
15if [[ ${PIPESTATUS[0]} == 0 ]]
16then
17 die "innobackupex did not fail as expected"
18fi
019
=== modified file 'storage/innobase/xtrabackup/test/t/bug766305.sh'
--- storage/innobase/xtrabackup/test/t/bug766305.sh 2014-09-20 17:05:56 +0000
+++ storage/innobase/xtrabackup/test/t/bug766305.sh 2014-10-06 12:20:08 +0000
@@ -4,7 +4,7 @@
44
5start_server --innodb_file_per_table5start_server --innodb_file_per_table
66
7if [ ${ASAN_OPTIONS:-undefined} = "undefined" ]7if [ ${ASAN_OPTIONS:-undefined} != "undefined" ]
8then8then
9 skip_test "Incompatible with AddressSanitizer"9 skip_test "Incompatible with AddressSanitizer"
10fi10fi
1111
=== added file 'storage/innobase/xtrabackup/test/t/xb_print_param.sh'
--- storage/innobase/xtrabackup/test/t/xb_print_param.sh 1970-01-01 00:00:00 +0000
+++ storage/innobase/xtrabackup/test/t/xb_print_param.sh 2014-10-06 12:20:08 +0000
@@ -0,0 +1,46 @@
1########################################################################
2# Tests for xtrabackup --print-param
3########################################################################
4
5my_cnf="[mysqld]
6datadir=/some/data/dir
7tmpdir=/some/tmp/dir1:/some/tmp/dir2
8innodb_data_home_dir=/some/innodb/dir
9innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend
10innodb_log_group_home_dir=/some/log/dir
11innodb_log_files_in_group=3
12innodb_log_file_size=5M
13innodb_flush_method=O_DIRECT
14innodb_page_size=4K
15innodb_fast_checksum=1
16innodb_log_block_size=4K
17innodb_doublewrite_file=/some/doublewrite/file
18innodb_undo_directory=/some/undo/directory
19innodb_undo_tablespaces=8
20innodb_checksum_algorithm=strict_crc32
21innodb_log_checksum_algorithm=none
22innodb_buffer_pool_filename=/some/buffer/pool/file"
23
24echo "$my_cnf" >$topdir/my.cnf
25
26diff -u <($XB_BIN --defaults-file=$topdir/my.cnf --print-param) - <<EOF
27# This MySQL options file was generated by XtraBackup.
28[mysqld]
29datadir=/some/data/dir
30tmpdir=/some/tmp/dir1:/some/tmp/dir2
31innodb_data_home_dir=/some/innodb/dir
32innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend
33innodb_log_group_home_dir=/some/log/dir
34innodb_log_files_in_group=3
35innodb_log_file_size=5242880
36innodb_flush_method=O_DIRECT
37innodb_page_size=4096
38innodb_fast_checksum=1
39innodb_log_block_size=4096
40innodb_doublewrite_file=/some/doublewrite/file
41innodb_undo_directory=/some/undo/directory
42innodb_undo_tablespaces=8
43innodb_checksum_algorithm=strict_crc32
44innodb_log_checksum_algorithm=none
45innodb_buffer_pool_filename=/some/buffer/pool/file
46EOF

Subscribers

People subscribed via source and target branches

to all changes: