Merge lp:~akopytov/percona-xtrabackup/read-server-options-with-show-variables into lp:percona-xtrabackup/2.2

Proposed by Alexey Kopytov on 2014-10-06
Status: Merged
Approved by: Sergei Glushchenko on 2014-10-08
Approved revision: 5032
Merged at revision: 5032
Proposed branch: lp:~akopytov/percona-xtrabackup/read-server-options-with-show-variables
Merge into: lp:percona-xtrabackup/2.2
Diff against target: 875 lines (+385/-197)
8 files modified
storage/innobase/xtrabackup/innobackupex.pl (+182/-136)
storage/innobase/xtrabackup/src/xtrabackup.cc (+106/-48)
storage/innobase/xtrabackup/test/run.sh (+2/-6)
storage/innobase/xtrabackup/test/t/bug1062684.sh (+6/-6)
storage/innobase/xtrabackup/test/t/bug1334062.sh (+24/-0)
storage/innobase/xtrabackup/test/t/bug1343722.sh (+18/-0)
storage/innobase/xtrabackup/test/t/bug766305.sh (+1/-1)
storage/innobase/xtrabackup/test/t/xb_print_param.sh (+46/-0)
To merge this branch: bzr merge lp:~akopytov/percona-xtrabackup/read-server-options-with-show-variables
Reviewer Review Type Date Requested Status
Sergei Glushchenko 2014-10-06 Approve on 2014-10-08
Review via email: mp+237249@code.launchpad.net

Description of the change

Implementation of
https://blueprints.launchpad.net/percona-xtrabackup/+spec/read-server-options-with-show-variables

On the backup stage, read options from the server via SHOW VARIABLES
rather than my.cnf. For the ‘datadir’ option, make sure we get the same
value as we would get from my.cnf (if the option is defined there),
because the xtrabackup binary does not have a server connection and uses
my.cnf instead.

To post a comment you must log in.

Hi Alexey,

The patch itself looks good, but

I am concerned of "Bug #1343722: Too easy to backup wrong datadir with
multiple instances". Not only datadir but every other important option
can be different if we read options from my.cnf which doesn't belong
to MySQL instance.

This patch makes following change:

innobackupex reads options from running MySQL.
xtrabackup reads options from my.cnf.

Now instead of single source we have two sources which can be out of
sync. We making sure that xtrabackup and innobackupex agree about
"datadir" but we don't take proper care of other options. Wouldn't it
be better to make both innobackupex and xtrabackup to use runtime
options (innobackupex could simply pass all sensitive options to
xtrabackup or even dump them into tmp-my.cnf).

review: Needs Information
Alexey Kopytov (akopytov) wrote :
Download full text (3.2 KiB)

Hi Sergei,

On Tue, Oct 07 2014 13:05:13 +0400, Sergei Glushchenko wrote:

> Hi Alexey,
>
> The patch itself looks good, but
>
> I am concerned of "Bug #1343722: Too easy to backup wrong datadir with
> multiple instances". Not only datadir but every other important option
> can be different if we read options from my.cnf which doesn't belong
> to MySQL instance.
>
> This patch makes following change:
>
> innobackupex reads options from running MySQL.
> xtrabackup reads options from my.cnf.
>
> Now instead of single source we have two sources which can be out of
> sync. We making sure that xtrabackup and innobackupex agree about
> "datadir" but we don't take proper care of other options. Wouldn't it
> be better to make both innobackupex and xtrabackup to use runtime
> options (innobackupex could simply pass all sensitive options to
> xtrabackup or even dump them into tmp-my.cnf).

Well, the patch went through multiple iterations. In one of them, it did
exactly that: it verified all options rather than just datadir. A few
problems had been revealed in testing:

- boolean values like innodb_fast_checksum can only be to 0/1 (or not
  have a value) in the defaults file. That’s how ‘xtrabackup
  --print-param’ prints them. But their values in SHOW VARIABLES are
  ON/OFF. So this must be taken into account when validating all
  options.

- path options like innodb_data_home_dir, innodb_log_group_home_dir,
  etc. They can be specified in an unnormalized way (with multiple
  slashes, “.”, “..”, etc.) in configuration file, but for some of them
  the server normalizes the paths as shown in SHOW VARIABLES, and for
  some of them no normalization is done. This could be solved with
  realpath() as we currently do with datadir. But on top of that, those
  paths can be relative to datadir, and that would require even more
  complex validation logic than just using realpath()

- sometimes we deliberately want to override some options for xtrabackup
  in the [xtrabackup] section of my.cnf, so that the values would be
  deliberately different between SHOW VARIABLES and the ones printed by
  ‘xtrabackup --print-param’.

After some considerations, I decided to only validate datadir, as
everything else would be a bit too risky for a point release. The patch
is rather invasive for a point release even without those changes.

As to passing all sensitive options from innobackupex to xtrabackup,
that’s what we do now for some of them (namely, datadir,
innodb_log_file_size and innodb_data_file_path), because those variables
have different defaults across major server versions, so using runtime
values is crucial.

Passing other value would work, and would eliminate possibility (which
is rather low in practice) of cases when ‘datadir’ is the same in SHOW
VARIABLES and my.cnf, but other (sensitive for XtraBackup) options are
different. But it comes with its own bunch of drawbacks and further
complications and corner cases in the code.

I decided to keep the patch minimal to solve current issues
for a point release, and scheduled
https://blueprints.launchpad.net/percona-xtrabackup/+spec/rewrite-innobackupex-in-c
for the next major release. Seriously, we should do it ASAP...

Read more...

Hi Alexey,

The patch looks good to me. I agree that getting rid of innobackupex will solve many issues and I believe we should include https://blueprints.launchpad.net/percona-xtrabackup/+spec/rewrite-innobackupex-in-c in the roadmap if it is not there yet.

Approved.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'storage/innobase/xtrabackup/innobackupex.pl'
2--- storage/innobase/xtrabackup/innobackupex.pl 2014-10-05 12:02:02 +0000
3+++ storage/innobase/xtrabackup/innobackupex.pl 2014-10-06 12:20:08 +0000
4@@ -23,6 +23,7 @@
5 use English qw(-no_match_vars);
6 use Time::HiRes qw(usleep);
7 use Carp qw(longmess);
8+use Cwd qw(realpath);
9
10 # version of this script
11 my $innobackup_version = '1.5.1-xtrabackup';
12@@ -1896,10 +1897,10 @@
13 # process.
14 #
15 sub backup {
16- my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
17+ my $orig_datadir = get_option('datadir');
18 my $suspend_file;
19- my $buffer_pool_filename = get_option(\%config, $option_defaults_group,
20- 'innodb_buffer_pool_filename');
21+ my $buffer_pool_filename = get_option_safe('innodb_buffer_pool_filename',
22+ '');
23
24 detect_mysql_capabilities_for_backup(\%mysql);
25
26@@ -2038,7 +2039,8 @@
27 # copy ib_lru_dump
28 # Copy buffer poll dump and/or LRU dump
29 foreach my $dump_name ($buffer_pool_filename, 'ib_lru_dump') {
30- if (!$option_rsync && -e "$orig_datadir/$dump_name") {
31+ if (!$option_rsync && $dump_name ne '' &&
32+ -e "$orig_datadir/$dump_name") {
33 backup_file("$orig_datadir", "$dump_name", "$backup_dir/$dump_name")
34 }
35 }
36@@ -2401,14 +2403,6 @@
37 #
38 sub copy_back {
39 my $move_flag = shift;
40- my $orig_datadir = get_option(\%config, $option_defaults_group, 'datadir');
41- my $orig_ibdata_dir =
42- get_option(\%config, $option_defaults_group, 'innodb_data_home_dir');
43- my $orig_innodb_data_file_path =
44- get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
45- my $orig_iblog_dir =
46- get_option(\%config, $option_defaults_group, 'innodb_log_group_home_dir');
47- my $orig_undo_dir = $orig_ibdata_dir;
48 my $iblog_files = 'ib_logfile.*';
49 my $ibundo_files = 'undo[0-9]{3}';
50 my $excluded_files =
51@@ -2418,21 +2412,24 @@
52 '.*\.pmap|.*\.tmp|' .
53 $iblog_files . '|'.
54 $ibundo_files;
55- my $compressed_data_file = '.*\.ibz$';
56 my $file;
57- my $backup_innodb_data_file_path;
58-
59- if (has_option(\%config, $option_defaults_group, 'innodb_doublewrite_file')) {
60+
61+ my $orig_datadir = get_option('datadir');
62+ my $orig_ibdata_dir = get_option_safe('innodb_data_home_dir',
63+ $orig_datadir);
64+ my $orig_innodb_data_file_path = get_option_safe('innodb_data_file_path',
65+ '');
66+ my $orig_iblog_dir = get_option_safe('innodb_log_group_home_dir',
67+ $orig_datadir);
68+ my $orig_undo_dir = get_option_safe('innodb_undo_directory',
69+ $orig_datadir);
70+
71+ if (has_option('innodb_doublewrite_file')) {
72 my $doublewrite_file =
73- get_option(\%config, $option_defaults_group,
74- 'innodb_doublewrite_file');
75+ get_option('innodb_doublewrite_file');
76 $excluded_files = $excluded_files . '|' . $doublewrite_file;
77 }
78
79- if (has_option(\%config, $option_defaults_group, 'innodb_undo_directory')) {
80- $orig_undo_dir = get_option(\%config, $option_defaults_group,
81- 'innodb_undo_directory');
82- }
83 # check whether files should be copied or moved to dest directory
84 my $move_or_copy_file = $move_flag ? \&move_file : \&copy_file;
85 my $move_or_copy_dir = $move_flag ?
86@@ -2568,11 +2565,7 @@
87 my $cmdline_copy = '';
88 my $options = '';
89
90- if ($option_defaults_file) {
91- $options = $options . " --defaults-file=\"$option_defaults_file\" ";
92- } else {
93- $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
94- }
95+ $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
96
97 if ($option_defaults_extra_file) {
98 $options = $options . " --defaults-extra-file=\"$option_defaults_extra_file\" ";
99@@ -2603,7 +2596,7 @@
100 }
101
102 my $innodb_data_file_path =
103- get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
104+ get_option('innodb_data_file_path');
105
106 # run ibbackup as a child process
107 $cmdline = "$option_ibbackup_binary $options";
108@@ -2786,6 +2779,21 @@
109 $options = $options . " --target-dir=$backup_dir";
110 }
111
112+ my $datadir = get_option('datadir');
113+ if (!has_option_in_config('datadir')) {
114+ $options .= " --datadir=\"$datadir\"";
115+ }
116+
117+ my $innodb_log_file_size = get_option('innodb_log_file_size');
118+ if (!has_option_in_config('innodb_log_file_size')) {
119+ $options .= " --innodb_log_file_size=\"$innodb_log_file_size\"";
120+ }
121+
122+ my $innodb_data_file_path = get_option('innodb_data_file_path');
123+ if (!has_option_in_config('innodb_data_file_path')) {
124+ $options .= " --innodb_data_file_path=\"$innodb_data_file_path\"";
125+ }
126+
127 if ($option_tmpdir) {
128 $options .= " --tmpdir=$option_tmpdir";
129 }
130@@ -3579,44 +3587,44 @@
131 print STDERR " At the end of a successful $run run $innobackup_script\n";
132 print STDERR " prints \"completed OK!\".\n\n";
133
134- if (!$option_copy_back && !$option_move_back
135- && !$option_decrypt && !$option_decompress) {
136- # we are making a backup or applying log to backup
137- if (!$option_apply_log) {
138- # we are making a backup, we need mysql server
139- get_mysql_vars(\%mysql);
140- $mysql_server_version = $mysql{vars}->{version}->{Value};
141- print STDERR "$prefix Using mysql server version $mysql_server_version\n";
142+ if ($option_apply_log || $option_copy_back || $option_move_back) {
143+ # read server configuration file
144+ read_config_file(\%config);
145+ } elsif ($option_backup) {
146+ # we are making a backup, read server configuration from SHOW VARIABLES
147+ get_mysql_vars(\%mysql);
148+
149+ # and make sure datadir value is the same in configuration file
150+ read_config_file(\%config);
151+
152+ my $server_val = $mysql{vars}->{datadir}->{Value};
153+
154+ if (has_option_in_config('datadir')) {
155+ my $config_val = $config{$option_defaults_group}{datadir};
156+
157+ if ($server_val ne $config_val &&
158+ # Try to canonicalize paths
159+ realpath($server_val) ne realpath($config_val)) {
160+
161+ die "option 'datadir' has different values:\n" .
162+ " '$config_val' in defaults file\n" .
163+ " '$server_val' in SHOW VARIABLES\n"
164+ }
165 }
166+
167+ $mysql_server_version = $mysql{vars}->{version}->{Value};
168+ print STDERR "$prefix Using server version $mysql_server_version\n";
169 print STDERR "\n";
170-
171- if ($option_include
172- && $ibbackup_version
173- && $ibbackup_version le "2.0") {
174- # --include option was given, but ibbackup is too
175- # old to support it
176- die "--include option was given, but ibbackup is too old"
177- . " to support it. You must upgrade to InnoDB Hot Backup"
178- . " v2.0 in order to use --include option.\n";
179- }
180- }
181-
182- if (!$option_decrypt && !$option_decompress) {
183- # read MySQL options file
184- #read_config_file($config_file, \%config);
185- read_config_file(\%config);
186-
187- if(!$option_tmpdir) {
188- $option_tmpdir = get_option(\%config, $option_defaults_group, 'tmpdir');
189- }
190- }
191-
192- # get innodb log home directory from options file
193- #$innodb_log_group_home_dir =
194- # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
195-
196- if (!$option_apply_log && !$option_copy_back && !$option_move_back
197- && !$option_decrypt && !$option_decompress) {
198+ }
199+
200+ if (!$option_tmpdir && ($option_backup || $option_apply_log) &&
201+ has_option('tmpdir')) {
202+ $option_tmpdir = get_option('tmpdir');
203+ # tmpdir can be a colon-separated list of multiple directories
204+ $option_tmpdir = (split(/:/, $option_tmpdir))[0];
205+ }
206+
207+ if ($option_backup) {
208 # we are making a backup, create a new backup directory
209 $backup_dir = File::Spec->rel2abs(make_backup_dir());
210 print STDERR "$prefix Created backup directory $backup_dir\n";
211@@ -3656,10 +3664,6 @@
212 die "Failed to delete " .
213 "'$option_tmpdir/$xtrabackup_pid_file': $!";
214 }
215-
216- } elsif ($option_copy_back || $option_move_back) {
217- #$backup_config_file = $backup_dir . '/backup-my.cnf';
218- #read_config_file($backup_config_file, \%backup_config);
219 }
220 }
221
222@@ -3693,16 +3697,18 @@
223
224 my $option_name;
225 foreach $option_name (@option_names) {
226- if (has_option(\%config, $option_defaults_group, $option_name)) {
227- my $option_value = get_option(\%config, $option_defaults_group, $option_name);
228+ if (has_option($option_name)) {
229+ my $option_value = get_option($option_name);
230 $options_dump .= "$option_name=$option_value\n";
231 }
232 }
233- if (has_option(\%config,
234- $option_defaults_group, "innodb_doublewrite_file")) {
235- $options_dump .= "innodb_doublewrite_file=" . (split(/\/+/,
236- get_option(\%config, $option_defaults_group,
237- 'innodb_doublewrite_file')))[-1] . "\n";
238+ if (has_option('innodb_doublewrite_file')) {
239+ my $option_value = (split(/\/+/,
240+ get_option('innodb_doublewrite_file')))[-1];
241+
242+ if (defined($option_value)) {
243+ $options_dump .= "innodb_doublewrite_file=" . $option_value . "\n";
244+ }
245 }
246
247 write_to_backup_file("$filename", "$options_dump");
248@@ -3940,8 +3946,6 @@
249 #
250 sub make_backup_dir {
251 my $dir;
252- my $innodb_data_file_path =
253- get_option(\%config, $option_defaults_group, 'innodb_data_file_path');
254
255 # create backup directory
256 $dir = $backup_root;
257@@ -3953,19 +3957,6 @@
258 unless $option_no_timestamp;
259 mkdir($dir, 0777) || die "Failed to create backup directory $dir: $!";
260
261- # create subdirectories for ibdata files if needed
262-# foreach my $a (split(/;/, $innodb_data_file_path)) {
263-# my $path = (split(/:/,$a))[0];
264-# my @relative_path = split(/\/+/, $path);
265-# pop @relative_path;
266-# if (@relative_path) {
267-# # there is a non-trivial path from the backup directory
268-# # to the directory of this backup ibdata file, check
269-# # that all the directories in the path exist.
270-# create_path_if_needed($dir, \@relative_path);
271-# }
272-# }
273-
274 return $dir;
275 }
276
277@@ -4023,9 +4014,9 @@
278 #
279 sub backup_files {
280 my $prep_mode = shift;
281- my $source_dir = get_option(\%config, $option_defaults_group, 'datadir');
282- my $buffer_pool_filename = get_option(\%config, $option_defaults_group,
283- 'innodb_buffer_pool_filename');
284+ my $source_dir = get_option('datadir');
285+ my $buffer_pool_filename = get_option_safe('innodb_buffer_pool_filename',
286+ '');
287 my @list;
288 my $file;
289 my $database;
290@@ -4145,7 +4136,7 @@
291
292 if ($option_rsync) {
293 foreach my $dump_name ($buffer_pool_filename, 'ib_lru_dump') {
294- if (-e "$source_dir/$dump_name") {
295+ if ($dump_name ne '' && -e "$source_dir/$dump_name") {
296 print RSYNC "$dump_name\n";
297 if (!$prep_mode) {
298 $rsync_files_hash{"$dump_name"} = 1;
299@@ -4289,10 +4280,11 @@
300 my $options = '';
301
302
303- if ($option_defaults_file) {
304+ if ($option_apply_log) {
305+ $options = $options .
306+ " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
307+ } elsif ($option_defaults_file) {
308 $options = $options . " --defaults-file=\"$option_defaults_file\" ";
309- } elsif ($option_apply_log) {
310- $options = $options . " --defaults-file=\"${backup_dir}/backup-my.cnf\" ";
311 }
312
313 if ($option_defaults_extra_file) {
314@@ -4367,60 +4359,114 @@
315 }
316 }
317 }
318-
319-
320-# has_option return whether the config has an option with the given name
321-# Parameters:
322-# config_ref a reference to a config data
323-# group option group name
324+
325+
326+# has_option_in_config return true if the configuration file defines an option
327+# with the given name.
328+#
329+# Parameters:
330+# option_name name of the option
331+# Return value:
332+# true if option exists, otherwise false
333+#
334+sub has_option_in_config {
335+ my $option_name = shift;
336+ my $group_hash_ref;
337+
338+ if (!exists $config{$option_defaults_group}) {
339+ return 0;
340+ }
341+
342+ $group_hash_ref = $config{$option_defaults_group};
343+
344+ return exists ${$group_hash_ref}{$option_name};
345+}
346+
347+
348+# has_option returns 1 if an option with the given name exists in either
349+# defaults file file as reported by 'xtrabackup --print-param' or (in backup
350+# mode) SHOW VARIABLES. Otherwise returns 0.
351+#
352+# Parameters:
353 # option_name name of the option
354 # Return value:
355 # true if option exists, otherwise false
356 #
357 sub has_option {
358- my $config_ref = shift;
359- my $group = shift;
360 my $option_name = shift;
361- my $group_hash_ref;
362-
363- if (!exists ${$config_ref}{$group}) {
364- # no group
365- die "no '$group' group in MySQL options";
366- }
367-
368- $group_hash_ref = ${$config_ref}{$group};
369-
370- return exists ${$group_hash_ref}{$option_name};
371+
372+ if (has_option_in_config($option_name)) {
373+ return 1;
374+ }
375+
376+ if ($option_backup) {
377+ if (!defined($mysql{vars})) {
378+ get_mysql_vars(\%mysql);
379+ }
380+
381+ return defined($mysql{vars}->{$option_name});
382+ }
383+
384+ return 0;
385 }
386-
387-
388-# get_option subroutine returns the value of given option in the config
389-# structure. If option is missing, this subroutine calls exit.
390-# Parameters:
391-# config_ref a reference to a config data
392-# group option group name
393+
394+
395+# get_option returns the value of an option with the given name as defined
396+# either in defaults file as reported by 'xtrabackup --print-param' or (in
397+# backup mode) SHOW VARIABLES.
398+#
399+# This subroutine aborts with an error if the option is not defined.
400+#
401+# Parameters:
402 # option_name name of the option
403 # Return value:
404 # option value as a string
405 #
406 sub get_option {
407- my $config_ref = shift;
408- my $group = shift;
409 my $option_name = shift;
410 my $group_hash_ref;
411
412- if (!exists $config{$group}) {
413- # no group
414- die "no '$group' group in MySQL options";
415- }
416-
417- $group_hash_ref = ${$config_ref}{$group};
418- if (!exists ${$group_hash_ref}{$option_name}) {
419- # no option
420- die "no '$option_name' option in group '$group' in MySQL options";
421- }
422-
423- return ${$group_hash_ref}{$option_name};
424+ if (!$option_backup) {
425+ if (!exists $config{$option_defaults_group}) {
426+ # no group
427+ die "no '$option_defaults_group' group in server configuration " .
428+ "file '$option_defaults_file'";
429+ }
430+
431+ $group_hash_ref = $config{$option_defaults_group};
432+ if (!exists ${$group_hash_ref}{$option_name}) {
433+ # no option
434+ die "no '$option_name' option in group '$option_defaults_group' " .
435+ "in server configuration file '$option_defaults_file'";
436+ }
437+
438+ return ${$group_hash_ref}{$option_name};
439+ }
440+
441+ if (!defined($mysql{vars})) {
442+ get_mysql_vars(\%mysql);
443+ }
444+
445+ if (!defined($mysql{vars}->{$option_name})) {
446+ die "no '$option_name' option in SHOW VARIABLES";
447+ }
448+
449+ return $mysql{vars}->{$option_name}->{Value};
450+}
451+
452+#
453+# Identical to get_option, except that the second argument is returned on error,
454+# i.e. if the option is not defined.
455+#
456+sub get_option_safe {
457+ my $option_name = shift;
458+ my $fallback_value = shift;
459+
460+ if (has_option($option_name)) {
461+ return get_option($option_name);
462+ }
463+
464+ return $fallback_value;
465 }
466
467 # get_table_name subroutine returns table name of specified file.
468
469=== modified file 'storage/innobase/xtrabackup/src/xtrabackup.cc'
470--- storage/innobase/xtrabackup/src/xtrabackup.cc 2014-09-25 07:02:53 +0000
471+++ storage/innobase/xtrabackup/src/xtrabackup.cc 2014-10-06 12:20:08 +0000
472@@ -66,6 +66,8 @@
473 #include <row0quiesce.h>
474 #include <srv0start.h>
475
476+#include <sstream>
477+
478 #define G_PTR uchar*
479
480 #include "common.h"
481@@ -309,6 +311,10 @@
482 ds_ctxt_t *ds_data = NULL;
483 ds_ctxt_t *ds_meta = NULL;
484
485+/* String buffer used by --print-param to accumulate server options as they are
486+parsed from the defaults file */
487+static std::ostringstream print_param_str;
488+
489 extern "C" sig_handler handle_fatal_signal(int sig);
490
491 /* Simple datasink creation tracking...add datasinks in the reverse order you
492@@ -483,7 +489,7 @@
493 OPT_XTRA_REBUILD_THREADS,
494 OPT_INNODB_CHECKSUM_ALGORITHM,
495 OPT_INNODB_UNDO_DIRECTORY,
496- OPT_UNDO_TABLESPACES,
497+ OPT_INNODB_UNDO_TABLESPACES,
498 OPT_INNODB_LOG_CHECKSUM_ALGORITHM,
499 OPT_XTRA_INCREMENTAL_FORCE_SCAN,
500 OPT_DEFAULTS_GROUP,
501@@ -839,7 +845,7 @@
502 (G_PTR*) &srv_undo_dir, (G_PTR*) &srv_undo_dir,
503 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
504
505- {"innodb_undo_tablespaces", OPT_UNDO_TABLESPACES,
506+ {"innodb_undo_tablespaces", OPT_INNODB_UNDO_TABLESPACES,
507 "Number of undo tablespaces to use.",
508 (G_PTR*)&srv_undo_tablespaces, (G_PTR*)&srv_undo_tablespaces,
509 0, GET_ULONG, REQUIRED_ARG, 0, 0, 126, 0, 1, 0},
510@@ -959,6 +965,9 @@
511 my_print_variables(xb_long_options);
512 }
513
514+#define ADD_PRINT_PARAM_OPT(value) \
515+ print_param_str << opt->name << "=" << value << "\n";
516+
517 static my_bool
518 get_one_option(int optid, const struct my_option *opt __attribute__((unused)),
519 char *argument)
520@@ -967,7 +976,94 @@
521 case 'h':
522 strmake(mysql_real_data_home,argument, FN_REFLEN - 1);
523 mysql_data_home= mysql_real_data_home;
524- break;
525+
526+ ADD_PRINT_PARAM_OPT(mysql_real_data_home);
527+ break;
528+
529+ case 't':
530+
531+ ADD_PRINT_PARAM_OPT(opt_mysql_tmpdir);
532+ break;
533+
534+ case OPT_INNODB_DATA_HOME_DIR:
535+
536+ ADD_PRINT_PARAM_OPT(innobase_data_home_dir);
537+ break;
538+
539+ case OPT_INNODB_DATA_FILE_PATH:
540+
541+ ADD_PRINT_PARAM_OPT(innobase_data_file_path);
542+ break;
543+
544+ case OPT_INNODB_LOG_GROUP_HOME_DIR:
545+
546+ ADD_PRINT_PARAM_OPT(srv_log_group_home_dir);
547+ break;
548+
549+ case OPT_INNODB_LOG_FILES_IN_GROUP:
550+
551+ ADD_PRINT_PARAM_OPT(innobase_log_files_in_group);
552+ break;
553+
554+ case OPT_INNODB_LOG_FILE_SIZE:
555+
556+ ADD_PRINT_PARAM_OPT(innobase_log_file_size);
557+ break;
558+
559+ case OPT_INNODB_FLUSH_METHOD:
560+
561+ ADD_PRINT_PARAM_OPT(innobase_unix_file_flush_method);
562+ break;
563+
564+ case OPT_INNODB_PAGE_SIZE:
565+
566+ ADD_PRINT_PARAM_OPT(innobase_page_size);
567+ break;
568+
569+ case OPT_INNODB_FAST_CHECKSUM:
570+
571+ ADD_PRINT_PARAM_OPT(!!innobase_fast_checksum);
572+ break;
573+
574+ case OPT_INNODB_LOG_BLOCK_SIZE:
575+
576+ ADD_PRINT_PARAM_OPT(innobase_log_block_size);
577+ break;
578+
579+ case OPT_INNODB_DOUBLEWRITE_FILE:
580+
581+ ADD_PRINT_PARAM_OPT(innobase_doublewrite_file);
582+ break;
583+
584+ case OPT_INNODB_UNDO_DIRECTORY:
585+
586+ ADD_PRINT_PARAM_OPT(srv_undo_dir);
587+ break;
588+
589+ case OPT_INNODB_UNDO_TABLESPACES:
590+
591+ ADD_PRINT_PARAM_OPT(srv_undo_tablespaces);
592+ break;
593+
594+ case OPT_INNODB_CHECKSUM_ALGORITHM:
595+
596+ ut_a(srv_checksum_algorithm <= SRV_CHECKSUM_ALGORITHM_STRICT_NONE);
597+
598+ ADD_PRINT_PARAM_OPT(innodb_checksum_algorithm_names[srv_checksum_algorithm]);
599+ break;
600+
601+ case OPT_INNODB_LOG_CHECKSUM_ALGORITHM:
602+
603+ ut_a(srv_log_checksum_algorithm <= SRV_CHECKSUM_ALGORITHM_STRICT_NONE);
604+
605+ ADD_PRINT_PARAM_OPT(innodb_checksum_algorithm_names[srv_log_checksum_algorithm]);
606+ break;
607+
608+ case OPT_INNODB_BUFFER_POOL_FILENAME:
609+
610+ ADD_PRINT_PARAM_OPT(innobase_buffer_pool_filename);
611+ break;
612+
613 case OPT_XTRA_TARGET_DIR:
614 strmake(xtrabackup_real_target_dir,argument, sizeof(xtrabackup_real_target_dir)-1);
615 xtrabackup_target_dir= xtrabackup_real_target_dir;
616@@ -6045,6 +6141,10 @@
617 }
618 load_defaults("my", xb_load_default_groups, &argc, &argv);
619
620+ print_param_str <<
621+ "# This MySQL options file was generated by XtraBackup.\n"
622+ "[" << defaults_group << "]\n";
623+
624 /* ignore unsupported options */
625 {
626 int i,j,argc_new,find;
627@@ -6166,51 +6266,9 @@
628
629 /* --print-param */
630 if (xtrabackup_print_param) {
631- /* === some variables from mysqld === */
632- memset((G_PTR) &mysql_tmpdir_list, 0,
633- sizeof(mysql_tmpdir_list));
634-
635- if (init_tmpdir(&mysql_tmpdir_list, opt_mysql_tmpdir))
636- exit(EXIT_FAILURE);
637-
638- printf("# This MySQL options file was generated by XtraBackup.\n");
639- printf("[%s]\n", defaults_group);
640- printf("datadir = \"%s\"\n", mysql_data_home);
641- printf("tmpdir = \"%s\"\n", mysql_tmpdir_list.list[0]);
642- printf("innodb_data_home_dir = \"%s\"\n",
643- innobase_data_home_dir ? innobase_data_home_dir : mysql_data_home);
644- printf("innodb_data_file_path = \"%s\"\n",
645- innobase_data_file_path ? innobase_data_file_path : "ibdata1:10M:autoextend");
646- printf("innodb_log_group_home_dir = \"%s\"\n",
647- srv_log_group_home_dir ?
648- srv_log_group_home_dir : mysql_data_home);
649- printf("innodb_log_files_in_group = %ld\n", innobase_log_files_in_group);
650- printf("innodb_log_file_size = %lld\n", innobase_log_file_size);
651- printf("innodb_flush_method = \"%s\"\n",
652- (innobase_unix_file_flush_method != NULL) ?
653- innobase_unix_file_flush_method : "");
654- printf("innodb_page_size = %lld\n", innobase_page_size);
655- printf("innodb_fast_checksum = %d\n", innobase_fast_checksum);
656- printf("innodb_log_block_size = %lu\n", innobase_log_block_size);
657- if (innobase_doublewrite_file != NULL) {
658- printf("innodb_doublewrite_file = %s\n", innobase_doublewrite_file);
659- }
660- if (srv_undo_dir) {
661-
662- printf("innodb_undo_directory = \"%s\"\n",
663- srv_undo_dir);
664- }
665- printf("innodb_undo_tablespaces = %lu\n", srv_undo_tablespaces);
666- printf("innodb_checksum_algorithm = %s\n",
667- innodb_checksum_algorithm_names[srv_checksum_algorithm]
668- );
669- printf("innodb_log_checksum_algorithm = %s\n",
670- innodb_checksum_algorithm_names[srv_log_checksum_algorithm]
671- );
672- printf("innodb_buffer_pool_filename = \"%s\"\n",
673- innobase_buffer_pool_filename ?
674- innobase_buffer_pool_filename :
675- "ib_buffer_pool");
676+
677+ printf("%s", print_param_str.str().c_str());
678+
679 exit(EXIT_SUCCESS);
680 }
681
682
683=== modified file 'storage/innobase/xtrabackup/test/run.sh'
684--- storage/innobase/xtrabackup/test/run.sh 2014-09-10 12:10:01 +0000
685+++ storage/innobase/xtrabackup/test/run.sh 2014-10-06 12:20:08 +0000
686@@ -373,9 +373,6 @@
687 fi
688 fi
689
690- # Version-specific defaults
691- DEFAULT_IBDATA_SIZE="10M"
692-
693 # Determine MySQL flavor
694 if [[ "$MYSQL_VERSION" =~ "MariaDB" ]]
695 then
696@@ -410,8 +407,7 @@
697 ;;
698 5.5 )
699 ;;
700- 5.6 | 10.0 )
701- DEFAULT_IBDATA_SIZE="12M"
702+ 5.6 | 10.0 | 10.1)
703 ;;
704 *)
705 vlog "Unknown MySQL/InnoDB version: $MYSQL_VERSION/$INNODB_VERSION"
706@@ -440,7 +436,7 @@
707 export MYSQL_VERSION MYSQL_VERSION_COMMENT MYSQL_FLAVOR \
708 INNODB_VERSION XTRADB_VERSION INNODB_FLAVOR \
709 XB_BIN IB_BIN IB_ARGS XB_ARGS MYSQLD_EXTRA_ARGS \
710- DEFAULT_IBDATA_SIZE WSREP_READY LIBGALERA_PATH
711+ WSREP_READY LIBGALERA_PATH
712 }
713
714 ###########################################################################
715
716=== modified file 'storage/innobase/xtrabackup/test/t/bug1062684.sh'
717--- storage/innobase/xtrabackup/test/t/bug1062684.sh 2013-04-27 18:46:54 +0000
718+++ storage/innobase/xtrabackup/test/t/bug1062684.sh 2014-10-06 12:20:08 +0000
719@@ -7,7 +7,7 @@
720 . inc/common.sh
721
722 MYSQLD_EXTRA_MY_CNF_OPTS="
723-innodb-data-file-path=ibdata1:${DEFAULT_IBDATA_SIZE};ibdata2:5M:autoextend
724+innodb-data-file-path=ibdata1:10M;ibdata2:5M:autoextend
725 "
726
727 start_server
728@@ -22,8 +22,8 @@
729 mkdir -p $topdir/backup
730
731 vlog "Starting backup"
732-innobackupex $topdir/backup
733-full_backup_dir=`grep "innobackupex: Backup created in directory" $OUTFILE | awk -F\' '{print $2}'`
734+full_backup_dir=$topdir/backup/full
735+innobackupex --no-timestamp $full_backup_dir
736 vlog "Full backup done to directory $full_backup_dir"
737
738 # Changing data
739@@ -47,9 +47,9 @@
740 vlog "###############"
741
742 # Incremental backup
743+inc_backup_dir=$topdir/backup/inc
744 innobackupex --incremental --incremental-basedir=$full_backup_dir \
745- $topdir/backup
746-inc_backup_dir=`grep "innobackupex: Backup created in directory" $OUTFILE | tail -n 1 | awk -F\' '{print $2}'`
747+ --no-timestamp $inc_backup_dir
748 vlog "Incremental backup done to directory $inc_backup_dir"
749
750 vlog "Preparing backup"
751@@ -84,7 +84,7 @@
752 innobackupex --copy-back $full_backup_dir
753 vlog "Data restored"
754
755-start_server --innodb-data-file-path="ibdata1:${DEFAULT_IBDATA_SIZE};ibdata2:5M:autoextend"
756+start_server
757
758 vlog "Checking checksums"
759 checksum_test_b=`checksum_table incremental_sample test`
760
761=== added file 'storage/innobase/xtrabackup/test/t/bug1334062.sh'
762--- storage/innobase/xtrabackup/test/t/bug1334062.sh 1970-01-01 00:00:00 +0000
763+++ storage/innobase/xtrabackup/test/t/bug1334062.sh 2014-10-06 12:20:08 +0000
764@@ -0,0 +1,24 @@
765+#############################################################################
766+# Bug #1334062: Xtrabackup 2.2.3 fails to perform a full backup on PS 5.5 if
767+# innodb_log_file_size on the [mysqld] section of my.cnf is not
768+# set
769+#############################################################################
770+
771+start_server
772+
773+sed -i -e 's/innodb_log_file_size=.*//' $MYSQLD_VARDIR/my.cnf
774+
775+grep innodb_log_file_size $MYSQLD_VARDIR/my.cnf &&
776+ die "innodb_log_file_size is present in my.cnf"
777+
778+innobackupex --no-timestamp $topdir/backup
779+
780+innobackupex --apply-log $topdir/backup
781+
782+stop_server
783+
784+rm -rf $MYSQLD_DATADIR/*
785+
786+innobackupex --copy-back $topdir/backup
787+
788+start_server
789
790=== added file 'storage/innobase/xtrabackup/test/t/bug1343722.sh'
791--- storage/innobase/xtrabackup/test/t/bug1343722.sh 1970-01-01 00:00:00 +0000
792+++ storage/innobase/xtrabackup/test/t/bug1343722.sh 2014-10-06 12:20:08 +0000
793@@ -0,0 +1,18 @@
794+########################################################################
795+# Bug #1343722: Too easy to backup wrong datadir with multiple instances
796+########################################################################
797+
798+start_server_with_id 1
799+
800+socket=$MYSQLD_SOCKET
801+
802+start_server_with_id 2
803+
804+# Try to backup server 2, but use server 1's connection socket
805+$IB_BIN $IB_ARGS --socket=$socket --no-timestamp $topdir/backup 2>&1 |
806+ grep 'has different values'
807+
808+if [[ ${PIPESTATUS[0]} == 0 ]]
809+then
810+ die "innobackupex did not fail as expected"
811+fi
812
813=== modified file 'storage/innobase/xtrabackup/test/t/bug766305.sh'
814--- storage/innobase/xtrabackup/test/t/bug766305.sh 2014-09-20 17:05:56 +0000
815+++ storage/innobase/xtrabackup/test/t/bug766305.sh 2014-10-06 12:20:08 +0000
816@@ -4,7 +4,7 @@
817
818 start_server --innodb_file_per_table
819
820-if [ ${ASAN_OPTIONS:-undefined} = "undefined" ]
821+if [ ${ASAN_OPTIONS:-undefined} != "undefined" ]
822 then
823 skip_test "Incompatible with AddressSanitizer"
824 fi
825
826=== added file 'storage/innobase/xtrabackup/test/t/xb_print_param.sh'
827--- storage/innobase/xtrabackup/test/t/xb_print_param.sh 1970-01-01 00:00:00 +0000
828+++ storage/innobase/xtrabackup/test/t/xb_print_param.sh 2014-10-06 12:20:08 +0000
829@@ -0,0 +1,46 @@
830+########################################################################
831+# Tests for xtrabackup --print-param
832+########################################################################
833+
834+my_cnf="[mysqld]
835+datadir=/some/data/dir
836+tmpdir=/some/tmp/dir1:/some/tmp/dir2
837+innodb_data_home_dir=/some/innodb/dir
838+innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend
839+innodb_log_group_home_dir=/some/log/dir
840+innodb_log_files_in_group=3
841+innodb_log_file_size=5M
842+innodb_flush_method=O_DIRECT
843+innodb_page_size=4K
844+innodb_fast_checksum=1
845+innodb_log_block_size=4K
846+innodb_doublewrite_file=/some/doublewrite/file
847+innodb_undo_directory=/some/undo/directory
848+innodb_undo_tablespaces=8
849+innodb_checksum_algorithm=strict_crc32
850+innodb_log_checksum_algorithm=none
851+innodb_buffer_pool_filename=/some/buffer/pool/file"
852+
853+echo "$my_cnf" >$topdir/my.cnf
854+
855+diff -u <($XB_BIN --defaults-file=$topdir/my.cnf --print-param) - <<EOF
856+# This MySQL options file was generated by XtraBackup.
857+[mysqld]
858+datadir=/some/data/dir
859+tmpdir=/some/tmp/dir1:/some/tmp/dir2
860+innodb_data_home_dir=/some/innodb/dir
861+innodb_data_file_path=ibdata1:10M;ibdata2:5M:autoextend
862+innodb_log_group_home_dir=/some/log/dir
863+innodb_log_files_in_group=3
864+innodb_log_file_size=5242880
865+innodb_flush_method=O_DIRECT
866+innodb_page_size=4096
867+innodb_fast_checksum=1
868+innodb_log_block_size=4096
869+innodb_doublewrite_file=/some/doublewrite/file
870+innodb_undo_directory=/some/undo/directory
871+innodb_undo_tablespaces=8
872+innodb_checksum_algorithm=strict_crc32
873+innodb_log_checksum_algorithm=none
874+innodb_buffer_pool_filename=/some/buffer/pool/file
875+EOF

Subscribers

People subscribed via source and target branches

to all changes: