Merge lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite into lp:percona-xtrabackup/2.0

Proposed by Daniel Nichter
Status: Rejected
Rejected by: Vadim Tkachenko
Proposed branch: lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite
Merge into: lp:percona-xtrabackup/2.0
Diff against target: 6180 lines (+6137/-0)
8 files modified
innobackupex/common/DSNParser.pm (+417/-0)
innobackupex/common/IbexTest.pm (+133/-0)
innobackupex/common/Sandbox.pm (+168/-0)
innobackupex/innobackupex (+2400/-0)
innobackupex/t/001_get_mysql_options.t (+61/-0)
innobackupex/t/100_backup.t (+87/-0)
innobackupex/t/samples/basic-tables.sql (+17/-0)
innobackupex/test-coverage.txt (+2854/-0)
To merge this branch: bzr merge lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite
Reviewer Review Type Date Requested Status
Vadim Tkachenko Needs Fixing
Review via email: mp+17861@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Daniel Nichter (daniel-nichter) wrote :

I didn't want to make these kinds of extensive changes to innobackupex-1.5.1 so I made a new dir, innobackupex, in which innobackupex (the script) has been substantially altered. Changes include:

1) Debugging w/ IBEXDEBUG=1 (like MKDEBUG=1) to expose stuff that was hidden before.

2) All system() calls are now ran through system_call() so that every command can be seen when IBEXDEBUG=1.

3) Removed all $option_ vars, replaced with $o hashref.

4) Tweaked help/usage output a little, added to its end a list of the current option values after cmd line parsing.

5) Dogged it, which means I ran it through perltidy to format it according to the standards explained in the book Perl Best Practices.

6) innobackupex/t/ has the first incarnation of a test suite. It requires that the environment variable XTRABACKUP_TRUNK be set.

7) Made code a runable module so it can be tested and test coverage can be established. innobackupex/test-coverage.txt shows what we have so far, about 50% coverage.

It doesn't seem like this script was designed from the start to be testable. Testing it will be difficult and require a good amount of work. xtrabackup is also very talkative which means that innobackupex's tests will be noisy; xtrabackup should have a --quiet option. Since innobackupex forks children and runs shell commands capturing STDOUT and STDERR is difficult to impossible.

All in all, my branch should be as reliable, if not more, than the original code.

Revision history for this message
Vadim Tkachenko (vadim-tk) wrote :

Daniel,

Could you re-allocate bugfix510960 and test suite into two different branches ?

bugfix should be included in main trunk, while I agree script with test suite could be in different dir.

review: Needs Fixing
Revision history for this message
Daniel Nichter (daniel-nichter) wrote :

Vadim,

Vadim Tkachenko wrote:
> Review: Needs Fixing
> Daniel,
>
> Could you re-allocate bugfix510960 and test suite into two different branches ?
>
> bugfix should be included in main trunk, while I agree script with test suite could be in different dir.

So,

branch1 = apply fix(es) to original code in trunk
branch2 = separate innobackupex dir with my modified/fixed code and test suite

or,

branch1 = separate innodbackupex dir w/ my modified/fixed code
branch2 = add test suite in separate innobackupex dir

?

Revision history for this message
Vadim Tkachenko (vadim-tk) wrote :

Daniel,

branch1 = apply fix(es) to original code in trunk
branch2 = separate innobackupex dir with my modified/fixed code and test suite

On Thu, Jan 21, 2010 at 9:00 PM, Daniel Nichter <email address hidden> wrote:
> Vadim,
>
> Vadim Tkachenko wrote:
>> Review: Needs Fixing
>> Daniel,
>>
>> Could you re-allocate bugfix510960 and test suite into two different branches ?
>>
>> bugfix should be included in main trunk, while I agree script with test suite could be in different dir.
>
> So,
>
> branch1 = apply fix(es) to original code in trunk
> branch2 = separate innobackupex dir with my modified/fixed code and test suite
>
> or,
>
> branch1 = separate innodbackupex dir w/ my modified/fixed code
> branch2 = add test suite in separate innobackupex dir
>
> ?
> --
> https://code.launchpad.net/~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite/+merge/17861
> You are reviewing the proposed merge of lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite into lp:percona-xtrabackup.
>

--
Vadim Tkachenko, CTO, Percona Inc.
Phone +1-888-401-3403, Skype: vadimtk153
Schedule meeting: http://tungle.me/VadimTkachenko

Unmerged revisions

115. By Daniel Nichter

Fix bugs 510960 510964 510965 510966 510969 510970. -- Add innobackupex/ with new, enhanced copy of innobackupex-1.5.1. First incarnation of a test suite.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added directory 'innobackupex'
2=== added directory 'innobackupex/common'
3=== added file 'innobackupex/common/DSNParser.pm'
4--- innobackupex/common/DSNParser.pm 1970-01-01 00:00:00 +0000
5+++ innobackupex/common/DSNParser.pm 2010-01-22 01:31:15 +0000
6@@ -0,0 +1,417 @@
7+# This program is copyright 2007-2009 Baron Schwartz.
8+# Feedback and improvements are welcome.
9+#
10+# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
11+# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
12+# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.
13+#
14+# This program is free software; you can redistribute it and/or modify it under
15+# the terms of the GNU General Public License as published by the Free Software
16+# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar
17+# systems, you can issue `man perlgpl' or `man perlartistic' to read these
18+# licenses.
19+#
20+# You should have received a copy of the GNU General Public License along with
21+# this program; if not, write to the Free Software Foundation, Inc., 59 Temple
22+# Place, Suite 330, Boston, MA 02111-1307 USA.
23+# ###########################################################################
24+# DSNParser package $Revision: 5266 $
25+# ###########################################################################
26+package DSNParser;
27+
28+use strict;
29+use warnings FATAL => 'all';
30+use English qw(-no_match_vars);
31+use Data::Dumper;
32+$Data::Dumper::Indent = 0;
33+$Data::Dumper::Quotekeys = 0;
34+
35+eval {
36+ require DBI;
37+};
38+my $have_dbi = $EVAL_ERROR ? 0 : 1;
39+
40+use constant MKDEBUG => $ENV{MKDEBUG} || 0;
41+
42+# Defaults are built-in, but you can add/replace items by passing them as
43+# hashrefs of {key, desc, copy, dsn}. The desc and dsn items are optional.
44+# You can set properties with the prop() sub. Don't set the 'opts' property.
45+sub new {
46+ my ( $class, @opts ) = @_;
47+ my $self = {
48+ opts => {
49+ A => {
50+ desc => 'Default character set',
51+ dsn => 'charset',
52+ copy => 1,
53+ },
54+ D => {
55+ desc => 'Database to use',
56+ dsn => 'database',
57+ copy => 1,
58+ },
59+ F => {
60+ desc => 'Only read default options from the given file',
61+ dsn => 'mysql_read_default_file',
62+ copy => 1,
63+ },
64+ h => {
65+ desc => 'Connect to host',
66+ dsn => 'host',
67+ copy => 1,
68+ },
69+ p => {
70+ desc => 'Password to use when connecting',
71+ dsn => 'password',
72+ copy => 1,
73+ },
74+ P => {
75+ desc => 'Port number to use for connection',
76+ dsn => 'port',
77+ copy => 1,
78+ },
79+ S => {
80+ desc => 'Socket file to use for connection',
81+ dsn => 'mysql_socket',
82+ copy => 1,
83+ },
84+ u => {
85+ desc => 'User for login if not current user',
86+ dsn => 'user',
87+ copy => 1,
88+ },
89+ },
90+ };
91+ foreach my $opt ( @opts ) {
92+ MKDEBUG && _d('Adding extra property', $opt->{key});
93+ $self->{opts}->{$opt->{key}} = { desc => $opt->{desc}, copy => $opt->{copy} };
94+ }
95+ return bless $self, $class;
96+}
97+
98+# Recognized properties:
99+# * dbidriver: which DBI driver to use; assumes mysql, supports Pg.
100+# * required: which parts are required (hashref).
101+# * set-vars: a list of variables to set after connecting
102+sub prop {
103+ my ( $self, $prop, $value ) = @_;
104+ if ( @_ > 2 ) {
105+ MKDEBUG && _d('Setting', $prop, 'property');
106+ $self->{$prop} = $value;
107+ }
108+ return $self->{$prop};
109+}
110+
111+# Parse DSN string, like "h=host,P=3306", and return hashref with
112+# all DSN values, like:
113+# {
114+# D => undef,
115+# F => undef,
116+# h => 'host',
117+# p => undef,
118+# P => 3306,
119+# S => undef,
120+# t => undef,
121+# u => undef,
122+# A => undef,
123+# }
124+sub parse {
125+ my ( $self, $dsn, $prev, $defaults ) = @_;
126+ if ( !$dsn ) {
127+ MKDEBUG && _d('No DSN to parse');
128+ return;
129+ }
130+ MKDEBUG && _d('Parsing', $dsn);
131+ $prev ||= {};
132+ $defaults ||= {};
133+ my %given_props;
134+ my %final_props;
135+ my %opts = %{$self->{opts}};
136+
137+ # Parse given props
138+ foreach my $dsn_part ( split(/,/, $dsn) ) {
139+ if ( my ($prop_key, $prop_val) = $dsn_part =~ m/^(.)=(.*)$/ ) {
140+ # Handle the typical DSN parts like h=host, P=3306, etc.
141+ $given_props{$prop_key} = $prop_val;
142+ }
143+ else {
144+ # Handle barewords
145+ MKDEBUG && _d('Interpreting', $dsn_part, 'as h=', $dsn_part);
146+ $given_props{h} = $dsn_part;
147+ }
148+ }
149+
150+ # Fill in final props from given, previous, and/or default props
151+ foreach my $key ( keys %opts ) {
152+ MKDEBUG && _d('Finding value for', $key);
153+ $final_props{$key} = $given_props{$key};
154+ if ( !defined $final_props{$key}
155+ && defined $prev->{$key} && $opts{$key}->{copy} )
156+ {
157+ $final_props{$key} = $prev->{$key};
158+ MKDEBUG && _d('Copying value for', $key, 'from previous DSN');
159+ }
160+ if ( !defined $final_props{$key} ) {
161+ $final_props{$key} = $defaults->{$key};
162+ MKDEBUG && _d('Copying value for', $key, 'from defaults');
163+ }
164+ }
165+
166+ # Sanity check props
167+ foreach my $key ( keys %given_props ) {
168+ die "Unrecognized DSN part '$key' in '$dsn'\n"
169+ unless exists $opts{$key};
170+ }
171+ if ( (my $required = $self->prop('required')) ) {
172+ foreach my $key ( keys %$required ) {
173+ die "Missing DSN part '$key' in '$dsn'\n" unless $final_props{$key};
174+ }
175+ }
176+
177+ return \%final_props;
178+}
179+
180+# Like parse() above but takes an OptionParser object instead of
181+# a DSN string.
182+sub parse_options {
183+ my ( $self, $o ) = @_;
184+ die 'I need an OptionParser object' unless ref $o eq 'OptionParser';
185+ my $dsn_string
186+ = join(',',
187+ map { "$_=".$o->get($_); }
188+ grep { $o->has($_) && $o->get($_) }
189+ keys %{$self->{opts}}
190+ );
191+ MKDEBUG && _d('DSN string made from options:', $dsn_string);
192+ return $self->parse($dsn_string);
193+}
194+
195+sub as_string {
196+ my ( $self, $dsn ) = @_;
197+ return $dsn unless ref $dsn;
198+ return join(',',
199+ map { "$_=" . ($_ eq 'p' ? '...' : $dsn->{$_}) }
200+ grep { defined $dsn->{$_} && $self->{opts}->{$_} }
201+ sort keys %$dsn );
202+}
203+
204+sub usage {
205+ my ( $self ) = @_;
206+ my $usage
207+ = "DSN syntax is key=value[,key=value...] Allowable DSN keys:\n\n"
208+ . " KEY COPY MEANING\n"
209+ . " === ==== =============================================\n";
210+ my %opts = %{$self->{opts}};
211+ foreach my $key ( sort keys %opts ) {
212+ $usage .= " $key "
213+ . ($opts{$key}->{copy} ? 'yes ' : 'no ')
214+ . ($opts{$key}->{desc} || '[No description]')
215+ . "\n";
216+ }
217+ $usage .= "\n If the DSN is a bareword, the word is treated as the 'h' key.\n";
218+ return $usage;
219+}
220+
221+# Supports PostgreSQL via the dbidriver element of $info, but assumes MySQL by
222+# default.
223+sub get_cxn_params {
224+ my ( $self, $info ) = @_;
225+ my $dsn;
226+ my %opts = %{$self->{opts}};
227+ my $driver = $self->prop('dbidriver') || '';
228+ if ( $driver eq 'Pg' ) {
229+ $dsn = 'DBI:Pg:dbname=' . ( $info->{D} || '' ) . ';'
230+ . join(';', map { "$opts{$_}->{dsn}=$info->{$_}" }
231+ grep { defined $info->{$_} }
232+ qw(h P));
233+ }
234+ else {
235+ $dsn = 'DBI:mysql:' . ( $info->{D} || '' ) . ';'
236+ . join(';', map { "$opts{$_}->{dsn}=$info->{$_}" }
237+ grep { defined $info->{$_} }
238+ qw(F h P S A))
239+ . ';mysql_read_default_group=client';
240+ }
241+ MKDEBUG && _d($dsn);
242+ return ($dsn, $info->{u}, $info->{p});
243+}
244+
245+# Fills in missing info from a DSN after successfully connecting to the server.
246+sub fill_in_dsn {
247+ my ( $self, $dbh, $dsn ) = @_;
248+ my $vars = $dbh->selectall_hashref('SHOW VARIABLES', 'Variable_name');
249+ my ($user, $db) = $dbh->selectrow_array('SELECT USER(), DATABASE()');
250+ $user =~ s/@.*//;
251+ $dsn->{h} ||= $vars->{hostname}->{Value};
252+ $dsn->{S} ||= $vars->{'socket'}->{Value};
253+ $dsn->{P} ||= $vars->{port}->{Value};
254+ $dsn->{u} ||= $user;
255+ $dsn->{D} ||= $db;
256+}
257+
258+# Actually opens a connection, then sets some things on the connection so it is
259+# the way the Maatkit tools will expect. Tools should NEVER open their own
260+# connection or use $dbh->reconnect, or these things will not take place!
261+sub get_dbh {
262+ my ( $self, $cxn_string, $user, $pass, $opts ) = @_;
263+ $opts ||= {};
264+ my $defaults = {
265+ AutoCommit => 0,
266+ RaiseError => 1,
267+ PrintError => 0,
268+ ShowErrorStatement => 1,
269+ mysql_enable_utf8 => ($cxn_string =~ m/charset=utf8/ ? 1 : 0),
270+ };
271+ @{$defaults}{ keys %$opts } = values %$opts;
272+
273+ if ( !$have_dbi ) {
274+ die "Cannot connect to MySQL because the Perl DBI module is not "
275+ . "installed or not found. Run 'perl -MDBI' to see the directories "
276+ . "that Perl searches for DBI. If DBI is not installed, try:\n"
277+ . " Debian/Ubuntu apt-get install libdbi-perl\n"
278+ . " RHEL/CentOS yum install perl-DBI\n"
279+ . " OpenSolaris pgk install pkg:/SUNWpmdbi\n";
280+
281+ }
282+
283+ # Try twice to open the $dbh and set it up as desired.
284+ my $dbh;
285+ my $tries = 2;
286+ while ( !$dbh && $tries-- ) {
287+ MKDEBUG && _d($cxn_string, ' ', $user, ' ', $pass, ' {',
288+ join(', ', map { "$_=>$defaults->{$_}" } keys %$defaults ), '}');
289+
290+ eval {
291+ $dbh = DBI->connect($cxn_string, $user, $pass, $defaults);
292+
293+ # If it's a MySQL connection, set some options.
294+ if ( $cxn_string =~ m/mysql/i ) {
295+ my $sql;
296+
297+ # Set SQL_MODE and options for SHOW CREATE TABLE.
298+ $sql = q{SET @@SQL_QUOTE_SHOW_CREATE = 1}
299+ . q{/*!40101, @@SQL_MODE='NO_AUTO_VALUE_ON_ZERO'*/};
300+ MKDEBUG && _d($dbh, ':', $sql);
301+ $dbh->do($sql);
302+
303+ # Set character set and binmode on STDOUT.
304+ if ( my ($charset) = $cxn_string =~ m/charset=(\w+)/ ) {
305+ $sql = "/*!40101 SET NAMES $charset*/";
306+ MKDEBUG && _d($dbh, ':', $sql);
307+ $dbh->do($sql);
308+ MKDEBUG && _d('Enabling charset for STDOUT');
309+ if ( $charset eq 'utf8' ) {
310+ binmode(STDOUT, ':utf8')
311+ or die "Can't binmode(STDOUT, ':utf8'): $OS_ERROR";
312+ }
313+ else {
314+ binmode(STDOUT) or die "Can't binmode(STDOUT): $OS_ERROR";
315+ }
316+ }
317+
318+ if ( $self->prop('set-vars') ) {
319+ $sql = "SET " . $self->prop('set-vars');
320+ MKDEBUG && _d($dbh, ':', $sql);
321+ $dbh->do($sql);
322+ }
323+ }
324+ };
325+ if ( !$dbh && $EVAL_ERROR ) {
326+ MKDEBUG && _d($EVAL_ERROR);
327+ if ( $EVAL_ERROR =~ m/not a compiled character set|character set utf8/ ) {
328+ MKDEBUG && _d('Going to try again without utf8 support');
329+ delete $defaults->{mysql_enable_utf8};
330+ }
331+ elsif ( $EVAL_ERROR =~ m/locate DBD\/mysql/i ) {
332+ die "Cannot connect to MySQL because the Perl DBD::mysql module is "
333+ . "not installed or not found. Run 'perl -MDBD::mysql' to see "
334+ . "the directories that Perl searches for DBD::mysql. If "
335+ . "DBD::mysql is not installed, try:\n"
336+ . " Debian/Ubuntu apt-get install libdbd-mysql-perl\n"
337+ . " RHEL/CentOS yum install perl-DBD-MySQL\n"
338+ . " OpenSolaris pgk install pkg:/SUNWapu13dbd-mysql\n";
339+ }
340+ if ( !$tries ) {
341+ die $EVAL_ERROR;
342+ }
343+ }
344+ }
345+
346+ MKDEBUG && _d('DBH info: ',
347+ $dbh,
348+ Dumper($dbh->selectrow_hashref(
349+ 'SELECT DATABASE(), CONNECTION_ID(), VERSION()/*!50038 , @@hostname*/')),
350+ 'Connection info:', $dbh->{mysql_hostinfo},
351+ 'Character set info:', Dumper($dbh->selectall_arrayref(
352+ 'SHOW VARIABLES LIKE "character_set%"', { Slice => {}})),
353+ '$DBD::mysql::VERSION:', $DBD::mysql::VERSION,
354+ '$DBI::VERSION:', $DBI::VERSION,
355+ );
356+
357+ return $dbh;
358+}
359+
360+# Tries to figure out a hostname for the connection.
361+sub get_hostname {
362+ my ( $self, $dbh ) = @_;
363+ if ( my ($host) = ($dbh->{mysql_hostinfo} || '') =~ m/^(\w+) via/ ) {
364+ return $host;
365+ }
366+ my ( $hostname, $one ) = $dbh->selectrow_array(
367+ 'SELECT /*!50038 @@hostname, */ 1');
368+ return $hostname;
369+}
370+
371+# Disconnects a database handle, but complains verbosely if there are any active
372+# children. These are usually $sth handles that haven't been finish()ed.
373+sub disconnect {
374+ my ( $self, $dbh ) = @_;
375+ MKDEBUG && $self->print_active_handles($dbh);
376+ $dbh->disconnect;
377+}
378+
379+sub print_active_handles {
380+ my ( $self, $thing, $level ) = @_;
381+ $level ||= 0;
382+ printf("# Active %sh: %s %s %s\n", ($thing->{Type} || 'undef'), "\t" x $level,
383+ $thing, (($thing->{Type} || '') eq 'st' ? $thing->{Statement} || '' : ''))
384+ or die "Cannot print: $OS_ERROR";
385+ foreach my $handle ( grep {defined} @{ $thing->{ChildHandles} } ) {
386+ $self->print_active_handles( $handle, $level + 1 );
387+ }
388+}
389+
390+# Copy all set vals in dsn_1 to dsn_2. Existing val in dsn_2 are not
391+# overwritten unless overwrite=>1 is given, but undef never overwrites a
392+# val.
393+sub copy {
394+ my ( $self, $dsn_1, $dsn_2, %args ) = @_;
395+ die 'I need a dsn_1 argument' unless $dsn_1;
396+ die 'I need a dsn_2 argument' unless $dsn_2;
397+ my %new_dsn = map {
398+ my $key = $_;
399+ my $val;
400+ if ( $args{overwrite} ) {
401+ $val = defined $dsn_1->{$key} ? $dsn_1->{$key} : $dsn_2->{$key};
402+ }
403+ else {
404+ $val = defined $dsn_2->{$key} ? $dsn_2->{$key} : $dsn_1->{$key};
405+ }
406+ $key => $val;
407+ } keys %{$self->{opts}};
408+ return \%new_dsn;
409+}
410+
411+sub _d {
412+ my ($package, undef, $line) = caller 0;
413+ @_ = map { (my $temp = $_) =~ s/\n/\n# /g; $temp; }
414+ map { defined $_ ? $_ : 'undef' }
415+ @_;
416+ print STDERR "# $package:$line $PID ", join(' ', @_), "\n";
417+}
418+
419+1;
420+
421+# ###########################################################################
422+# End DSNParser package
423+# ###########################################################################
424
425=== added file 'innobackupex/common/IbexTest.pm'
426--- innobackupex/common/IbexTest.pm 1970-01-01 00:00:00 +0000
427+++ innobackupex/common/IbexTest.pm 2010-01-22 01:31:15 +0000
428@@ -0,0 +1,133 @@
429+# This program is copyright 2010 Percona Inc.
430+# Feedback and improvements are welcome.
431+#
432+# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
433+# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
434+# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.
435+#
436+# This program is free software; you can redistribute it and/or modify it under
437+# the terms of the GNU General Public License as published by the Free Software
438+# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar
439+# systems, you can issue `man perlgpl' or `man perlartistic' to read these
440+# licenses.
441+#
442+# You should have received a copy of the GNU General Public License along with
443+# this program; if not, write to the Free Software Foundation, Inc., 59 Temple
444+# Place, Suite 330, Boston, MA 02111-1307 USA.
445+package IbexTest;
446+
447+BEGIN {
448+ die "The XTRABACKUP_TRUNK environment variable is not set."
449+ unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK};
450+ unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common";
451+};
452+our $trunk = $ENV{XTRABACKUP_TRUNK};
453+
454+use strict;
455+use warnings FATAL => 'all';
456+use English qw(-no_match_vars);
457+use Test::More;
458+use Data::Dumper;
459+$Data::Dumper::Indent = 1;
460+$Data::Dumper::Sortkeys = 1;
461+$Data::Dumper::Quotekeys = 0;
462+
463+require Exporter;
464+our @ISA = qw(Exporter);
465+our %EXPORT_TAGS = ();
466+our @EXPORT = qw(
467+ output
468+ load_file
469+ wait_until
470+ wait_for
471+ no_diff
472+ $trunk
473+);
474+our @EXPORT_OK = qw();
475+
476+sub output {
477+ my ( $code ) = @_;
478+ die "I need a code argument" unless $code;
479+ my $output = '';
480+ open my $output_fh, '>', \$output
481+ or die "Cannot capture output to variable: $OS_ERROR";
482+ select $output_fh;
483+ eval { $code->() };
484+ close $output_fh;
485+ select STDOUT;
486+ die $EVAL_ERROR if $EVAL_ERROR;
487+ return $output;
488+}
489+
490+# Slurp file and return its entire contents.
491+sub load_file {
492+ my ( $file, %args ) = @_;
493+ $file = "$trunk/$file";
494+ open my $fh, "<", $file or die "Cannot open $file: $OS_ERROR";
495+ my $contents = do { local $/ = undef; <$fh> };
496+ close $fh;
497+ chomp $contents if $args{chomp_contents};
498+ return $contents;
499+}
500+
501+# Wait until code returns true.
502+sub wait_until {
503+ my ( $code, $t, $max_t ) = @_;
504+ my $slept = 0;
505+ my $sleep_int = $t || .5;
506+ $t ||= .5;
507+ $max_t ||= 5;
508+ $t *= 1_000_000;
509+ while ( $slept <= $max_t ) {
510+ return if $code->();
511+ usleep($t);
512+ $slept += $sleep_int;
513+ }
514+ return;
515+}
516+
517+# Wait t seconds for code to return.
518+sub wait_for {
519+ my ( $code, $t ) = @_;
520+ $t ||= 0;
521+ my $mask = POSIX::SigSet->new(&POSIX::SIGALRM);
522+ my $action = POSIX::SigAction->new(
523+ sub { die },
524+ $mask,
525+ );
526+ my $oldaction = POSIX::SigAction->new();
527+ sigaction(&POSIX::SIGALRM, $action, $oldaction);
528+ eval {
529+ alarm $t;
530+ $code->();
531+ alarm 0;
532+ };
533+ if ( $EVAL_ERROR ) {
534+ # alarm was raised
535+ return 1;
536+ }
537+ return 0;
538+}
539+
540+# Returns true (1) if there's no difference between the
541+# cmd's output and the expected output. If $update_sample
542+# or the env var UPDATE_SAMPLES is true then the $cmd
543+# output is written to the $expected_output file.
544+sub no_diff {
545+ my ( $cmd, $expected_output, $update_sample ) = @_;
546+ die "I need a cmd argument" unless $cmd;
547+ die "I need an expected_output argument" unless $expected_output;
548+ my $output_file = '/tmp/ibex-test-output.txt';
549+ $expected_output = "$trunk/$expected_output";
550+ `$cmd > $output_file`;
551+ if ( $ENV{UPDATE_SAMPLES} || $update_sample ) {
552+ `cat $output_file > $expected_output`;
553+ print STDERR "Updated $expected_output\n";
554+ }
555+ my $retval = system("diff $expected_output $output_file");
556+ `rm -rf $output_file`;
557+ $retval = $retval >> 8;
558+ return !$retval;
559+}
560+
561+1;
562
563=== added file 'innobackupex/common/Sandbox.pm'
564--- innobackupex/common/Sandbox.pm 1970-01-01 00:00:00 +0000
565+++ innobackupex/common/Sandbox.pm 2010-01-22 01:31:15 +0000
566@@ -0,0 +1,168 @@
567+# This program is copyright 2010 Percona Inc.
568+# Feedback and improvements are welcome.
569+#
570+# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
571+# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
572+# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.
573+#
574+# This program is free software; you can redistribute it and/or modify it under
575+# the terms of the GNU General Public License as published by the Free Software
576+# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar
577+# systems, you can issue `man perlgpl' or `man perlartistic' to read these
578+# licenses.
579+#
580+# You should have received a copy of the GNU General Public License along with
581+# this program; if not, write to the Free Software Foundation, Inc., 59 Temple
582+# Place, Suite 330, Boston, MA 02111-1307 USA.
583+package Sandbox;
584+
585+BEGIN {
586+ die "The XTRABACKUP_TRUNK environment variable is not set."
587+ unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK};
588+ unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common";
589+};
590+
591+use strict;
592+use warnings FATAL => 'all';
593+use English qw(-no_match_vars);
594+
595+my $trunk = $ENV{XTRABACKUP_TRUNK};
596+
597+use constant IBEXDEBUG => $ENV{IBEXDEBUG} || 0;
598+
599+my %port_for = (
600+ master => 12345,
601+ slave1 => 12346,
602+ slave2 => 12347,
603+ master1 => 12348, # master-master
604+ master2 => 12349, # master-master
605+);
606+
607+sub new {
608+ my ( $class, %args ) = @_;
609+ foreach my $arg ( qw(basedir DSNParser) ) {
610+ die "I need a $arg argument" unless defined $args{$arg};
611+ }
612+
613+ if ( !-d $args{basedir} ) {
614+ die "$args{basedir} is not a directory";
615+ }
616+
617+ return bless { %args }, $class;
618+}
619+
620+sub use {
621+ my ( $self, $server, $cmd ) = @_;
622+ _check_server($server);
623+ return if !defined $cmd || !$cmd;
624+ my $use = $self->_use_for($server) . " $cmd";
625+ IBEXDEBUG && _d('"Executing', $use, 'on', $server);
626+ eval {
627+ `$use`;
628+ };
629+ if ( $EVAL_ERROR ) {
630+ die "Failed to execute $cmd on $server: $EVAL_ERROR";
631+ }
632+ return;
633+}
634+
635+sub create_dbs {
636+ my ( $self, $dbh, $dbs, %args ) = @_;
637+ die 'I need a dbh' if !$dbh;
638+ return if ( !ref $dbs || scalar @$dbs == 0 );
639+ my %default_args = (
640+ repl => 1,
641+ drop => 1,
642+ );
643+ %args = ( %default_args, %args );
644+
645+ $dbh->do('SET SQL_LOG_BIN=0') unless $args{repl};
646+
647+ foreach my $db ( @$dbs ) {
648+ $dbh->do("DROP DATABASE IF EXISTS `$db`") if $args{drop};
649+
650+ my $sql = "CREATE DATABASE `$db`";
651+ eval {
652+ $dbh->do($sql);
653+ };
654+ die $EVAL_ERROR if $EVAL_ERROR;
655+ }
656+
657+ $dbh->do('SET SQL_LOG_BIN=1');
658+
659+ return;
660+}
661+
662+sub get_dbh_for {
663+ my ( $self, $server, $cxn_ops ) = @_;
664+ _check_server($server);
665+ $cxn_ops ||= { AutoCommit => 1 };
666+ IBEXDEBUG && _d('dbh for', $server, 'on port', $port_for{$server});
667+ my $dp = $self->{DSNParser};
668+ my $dsn = $dp->parse('h=127.0.0.1,u=msandbox,p=msandbox,P=' . $port_for{$server});
669+ my $dbh;
670+ eval { $dbh = $dp->get_dbh($dp->get_cxn_params($dsn), $cxn_ops) };
671+ if ( $EVAL_ERROR ) {
672+ IBEXDEBUG && _d('Failed to get dbh for', $server, ':', $EVAL_ERROR);
673+ return 0;
674+ }
675+ return $dbh;
676+}
677+
678+sub load_file {
679+ my ( $self, $server, $file, $use_db ) = @_;
680+ _check_server($server);
681+ $file = "$trunk/$file";
682+ if ( !-f $file ) {
683+ die "$file is not a file";
684+ }
685+
686+ my $d = $use_db ? "-D $use_db" : '';
687+
688+ my $use = $self->_use_for($server) . " $d < $file";
689+ IBEXDEBUG && _d('Loading', $file, 'on', $server, ':', $use);
690+ eval { `$use` };
691+ if ( $EVAL_ERROR ) {
692+ die "Failed to execute $file on $server: $EVAL_ERROR";
693+ }
694+
695+ return;
696+}
697+
698+sub _use_for {
699+ my ( $self, $server ) = @_;
700+ return "$self->{basedir}/$port_for{$server}/use";
701+}
702+
703+sub _check_server {
704+ my ( $server ) = @_;
705+ if ( !exists $port_for{$server} ) {
706+ die "Unknown server $server";
707+ }
708+ return;
709+}
710+
711+sub wipe_clean {
712+ my ( $self, $dbh ) = @_;
713+ foreach my $db ( @{$dbh->selectcol_arrayref('SHOW DATABASES')} ) {
714+ next if $db eq 'mysql';
715+ next if $db eq 'information_schema';
716+ next if $db eq 'sakila';
717+ $dbh->do("DROP DATABASE IF EXISTS `$db`");
718+ }
719+ return;
720+}
721+
722+sub _d {
723+ my ($package, undef, $line) = caller 0;
724+ @_ = map { (my $temp = $_) =~ s/\n/\n# /g; $temp; }
725+ map { defined $_ ? $_ : 'undef' }
726+ @_;
727+ print STDERR "# $package:$line $PID ", join(' ', @_), "\n";
728+}
729+
730+1;
731+
732+# ###########################################################################
733+# End Sandbox package
734+# ###########################################################################
735
736=== added file 'innobackupex/innobackupex'
737--- innobackupex/innobackupex 1970-01-01 00:00:00 +0000
738+++ innobackupex/innobackupex 2010-01-22 01:31:15 +0000
739@@ -0,0 +1,2400 @@
740+#!/usr/bin/env perl
741+
742+# A script for making backups of InnoDB and MyISAM tables, indexes and .frm
743+# files.
744+#
745+# Orginal code copyright 2003, 2009 Innobase Oy.
746+# http://www.innodb.com/download/
747+#
748+# Modified code copyright 2010 Percona Inc.
749+# https://launchpad.net/percona-xtrabackup/trunk
750+
751+# ############################################################################
752+# This is a a runnable module. Check at the end of this package for the call
753+# to main() which actually runs the program.
754+# ############################################################################
755+package innobackupex;
756+
757+use strict;
758+use warnings FATAL => 'all';
759+use English qw(-no_match_vars);
760+use Getopt::Long qw(:config default);
761+use File::Spec;
762+use POSIX "strftime";
763+use POSIX ":sys_wait_h";
764+use POSIX "tmpnam";
765+use FileHandle;
766+use File::Basename;
767+
768+use constant IBEXDEBUG => $ENV{IBEXDEBUG} || 0;
769+
770+# version of this script
771+my $innobackup_version = '1.5.1-xtrabackup';
772+my $innobackup_script = basename($0);
773+
774+# copyright notice
775+my $copyright_notice = "InnoDB Backup Utility v${innobackup_version}; Copyright 2003, 2009 Innobase Oy.
776+All Rights Reserved.
777+
778+This software is published under
779+the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
780+
781+";
782+
783+# required Perl version (5.005)
784+my @required_perl_version = ( 5, 0, 5 );
785+my $required_perl_version_old_style = 5.005;
786+
787+# force flush after every write and print
788+$OUTPUT_AUTOFLUSH = 1;
789+
790+######################################################################
791+# modifiable parameters
792+######################################################################
793+
794+# maximum number of files in a database directory which are
795+# separately printed when a backup is made
796+my $backup_file_print_limit = 9;
797+
798+# timeout in seconds for a reply from mysql
799+my $mysql_response_timeout = 900;
800+
801+# default compression level (this is an argument to ibbackup)
802+my $default_compression_level = 1;
803+
804+# time in seconds after which a dummy query is sent to mysql server
805+# in order to keep the database connection alive
806+my $mysql_keep_alive_timeout = 1800;
807+
808+######################################################################
809+# end of modifiable parameters
810+######################################################################
811+
812+# command line options
813+my $o = {
814+ 'compress' => 999,
815+ 'databases' => '',
816+ 'ibbackup' => 'xtrabackup',
817+ 'scp-opt' => '-Cp -c arcfour',
818+ 'stream' => '',
819+};
820+
821+# root of the backup directory
822+my $backup_root = '';
823+
824+# backup directory pathname
825+my $backup_dir = '';
826+
827+# name of the ibbackup suspend-at-end file
828+my $suspend_file = '';
829+
830+# name of the temporary transaction log file during the backup
831+my $tmp_logfile = '';
832+
833+# home directory of innoDB log files
834+my $innodb_log_group_home_dir = '';
835+
836+# backup my.cnf file
837+my $backup_config_file = '';
838+
839+# options from the options file
840+my %config;
841+
842+# options from the backup options file
843+#my %backup_config;
844+
845+# list of databases to be included in a backup
846+my %databases_list;
847+
848+# prefix for output lines
849+my $prefix = "$innobackup_script:";
850+
851+# process id of mysql client program (runs as a child process of this script)
852+my $mysql_pid = '';
853+
854+# mysql server version string
855+my $mysql_server_version = '';
856+
857+# name of the file where stderr of mysql process is directed
858+my $mysql_stderr;
859+
860+# name of the file where stdout of mysql process is directed
861+my $mysql_stdout;
862+
863+# name of the file where binlog position info is written
864+my $binlog_info;
865+
866+# name of the file where slave info is written
867+my $slave_info;
868+
869+# mysql binlog position as given by "SHOW MASTER STATUS" command
870+my $mysql_binlog_position = '';
871+
872+# mysql master's binlog position as given by "SHOW SLAVE STATUS" command
873+# run on a slave server
874+my $mysql_slave_position = '';
875+
876+# time of the most recent mysql_check call. (value returned by time() function)
877+my $mysql_last_access_time = 0;
878+
879+# process id of ibbackup program (runs as a child process of this script)
880+my $ibbackup_pid = '';
881+
882+# a counter for numbering mysql connection checks
883+my $hello_id = 0;
884+
885+# the request which has been sent to mysqld, but to which
886+# mysqld has not yet replied. Empty string denotes that no
887+# request has been sent to mysqld or that mysqld has replied
888+# to all requests.
889+my $current_mysql_request = '';
890+
891+# escape sequences for options files
892+my %option_value_escapes = (
893+ 'b' => "\b",
894+ 't' => "\t",
895+ 'n' => "\n",
896+ 'r' => "\r",
897+ "\\" => "\\",
898+ 's' => ' '
899+);
900+
901+# signal that is sent to child processes when they are killed
902+my $kill_signal = 15;
903+
904+# current local time
905+my $now;
906+
907+# incremental backup base directory
908+my $incremental_basedir = '';
909+
910+my $src_name;
911+my $dst_name;
912+my $win = ( $^O eq 'MSWin32' ? 1 : 0 );
913+my $CP_CMD = ( $win eq 1 ? "copy /Y" : "cp -p" );
914+
915+######################################################################
916+# program execution begins here
917+######################################################################
918+sub main {
919+ @ARGV = @_; # set global ARGV for this package
920+
921+ # check command-line args
922+ check_args();
923+
924+ # print program version and copyright
925+ print_version();
926+
927+ # initialize global variables and perform some checks
928+ init();
929+
930+ if ( $o->{'copy-back'} ) {
931+ # copy files from backup directory back to their original locations
932+ copy_back();
933+ }
934+ elsif ( $o->{'apply-log'} ) {
935+ # expand data files in backup directory by applying log files to them
936+ apply_log();
937+ }
938+ else {
939+ # make a backup of InnoDB and MyISAM tables, indexes and .frm files.
940+ backup();
941+ }
942+
943+ # program has completed successfully
944+ $now = current_time();
945+ print STDERR "$now $prefix completed OK!\n";
946+
947+ if ( $o->{'stream'} eq 'tar' ) {
948+ print STDERR "$prefix You must use -i (--ignore-zeros) option for extraction of the tar stream.\n";
949+ }
950+
951+ return 0;
952+}
953+
954+##############################################################################
955+# Subroutines
956+##############################################################################
957+
958+#
959+# print_version subroutine prints program version and copyright.
960+#
961+sub print_version {
962+ printf( STDERR $copyright_notice );
963+}
964+
965+#
966+# usage subroutine prints instructions of how to use this program to stdout.
967+#
968+sub usage {
969+ print STDOUT <<EOF;
970+
971+Usage:
972+
973+ $innobackup_script BACKUP-ROOT-DIR [OPTIONS]
974+
975+ $innobackup_script --apply-log BACKUP-DIR [OPTIONS]
976+
977+ $innobackup_script --copy-back BACKUP-DIR [OPTIONS]
978+
979+The first command line above makes a hot backup of a MySQL database.
980+By default it creates a backup directory (named by the current date
981+and time) in the given BACKUP-ROOT-DIR directory. With the --no-timestamp
982+option it does not create a time-stamped backup directory, but it puts
983+the backup in the given directory (which must not exist). This
984+command makes a complete backup of all MyISAM and InnoDB tables and
985+indexes in all databases or in all of the databases specified with the
986+--databases option. The created backup contains .frm, .MRG, .MYD,
987+.MYI, .TRG, .TRN, .ARM, .ARZ, .opt, .par, and InnoDB data and log files.
988+The MY.CNF options file defines the location of the database. This command
989+connects to the MySQL server using mysql client program, and runs
990+ibbackup (InnoDB Hot Backup program) as a child process.
991+
992+The command with --apply-log option prepares a backup for starting a MySQL
993+server on the backup. This command expands InnoDB data files as specified
994+in BACKUP-DIR/backup-my.cnf using BACKUP-DIR/ibbackup_logfile,
995+and creates new InnoDB log files as specified in BACKUP-DIR/backup-my.cnf.
996+The BACKUP-DIR should be a path name of a backup directory created by
997+innobackup. This command runs ibbackup as a child process, but it does not
998+connect to the database server.
999+
1000+The command with --copy-back option copies data, index, and log files
1001+from backup directory back to their original locations.
1002+The MY.CNF options file defines the original location of the database.
1003+The BACKUP-DIR is a path name of a backup directory created by innobackup.
1004+
1005+On success the exit code of innobackup process is 0. A non-zero exit code
1006+indicates an error.
1007+
1008+Options:
1009+
1010+ --help Display this helpscreen and exit.
1011+
1012+ --version Print version information and exit.
1013+
1014+ --defaults-file=[MY.CNF]
1015+ This option is passed to the xtrabackup child process
1016+ and the mysql child process. It specifies which file
1017+ to read default options from.
1018+
1019+ --apply-log
1020+ Prepare a backup for starting mysql server on the backup.
1021+ Expand InnoDB data files as specified in
1022+ backup-dir/backup-my.cnf, using backup-dir/ibbackup_logfile,
1023+ and create new log files as specified in
1024+ backup-dir/backup-my.cnf.
1025+
1026+ --copy-back
1027+ Copy data and index files from backup directory back to
1028+ their original locations.
1029+
1030+ --remote-host=[USER@]HOSTNAME
1031+ If this option is specified, backup files will be created
1032+ at the remote host by using ssh and scp. BACKUP-ROOT-DIR
1033+ is relative to this host.
1034+
1035+ --stream=[tar|cpio(not implemented)]
1036+ If this option is specified, backup to STDOUT as specified
1037+ format.
1038+
1039+ --tmpdir=DIRECTORY
1040+ When --remote-host or --stream specified, transaction log
1041+ should be stored temporary. The default value is same to
1042+ mysqld setting.
1043+
1044+ --use-memory=MB
1045+ This option is passed to the ibbackup child process.
1046+ It tells ibbackup that it can use MB megabytes of
1047+ memory in restoration.
1048+ Try 'ibbackup --help' for more details on this option.
1049+
1050+ --throttle=IOS
1051+ This option is passed to the xtrabackup child process.
1052+ It limits count of IO operations (pairs of read&write) per
1053+ second to IOS values (for '--backup')
1054+
1055+ --sleep=MS
1056+ This option is passed to the ibbackup child process.
1057+ It instructs the ibbackup program to sleep
1058+ MS milliseconds after each 1 MB of copied data.
1059+ You can use this parameter to tune the additional
1060+ disk i/o load the ibbackup program causes on the computer.
1061+ Try 'ibbackup --help' for more details on this option.
1062+ ** it is not supported by xtrabackup **
1063+
1064+ --compress[=LEVEL]
1065+ This option is passed to the ibbackup child process.
1066+ It instructs ibbackup to compress the backup copies of
1067+ InnoDB data files. Compression level can be
1068+ specified as an optional argument. Compression level is
1069+ an integer between 0 and 9: 1 gives fastest compression,
1070+ 9 gives best compression, and 0 means no compression.
1071+ If compression level is not given, the default level 1 is used.
1072+ Try 'ibbackup --help' for more details on this option.
1073+ ** it is not supported by xtrabackup yet **
1074+
1075+ --include=REGEXP
1076+ This option is passed to the ibbackup child process.
1077+ It tells ibbackup to backup only those per-table data
1078+ files which match the given regular expression. For
1079+ each table with a per-table data file a string of the
1080+ form db_name.table_name is checked against the regular
1081+ expression. If the regular expression matches the
1082+ complete string db_name.table_name, the table is
1083+ included in the backup. The regular expression should
1084+ be of the POSIX 1003.2 "extended" form.
1085+ Try 'ibbackup --help' for more details on this option.
1086+
1087+ --databases=LIST
1088+ This option is used to specify the list of databases that
1089+ innobackup should backup. The list is of the form
1090+ "db_name[.table_name] db_name1[.table_name1] ...".
1091+ If this option is not specified all databases containing
1092+ MyISAM and InnoDB tables will be backed up.
1093+ Please make sure that --databases contains all of the
1094+ innodb databases & tables so that all of the innodb .frm
1095+ files are also backed up. In case the list is very long,
1096+ this can be specified in a file and the full path of the
1097+ file can be specified instead of the list.
1098+
1099+ --uncompress
1100+ This option is passed to the ibbackup child process.
1101+ It tells ibbackup to uncompress compressed InnoDB data files.
1102+ Try 'ibbackup --help' for more details on this option.
1103+ ** it is not supported by xtrabackup yet **
1104+
1105+ --export
1106+ This option is passed to the xtrabackup child process.
1107+ It is effective with '--prepare' option.
1108+ It tells xtrabackup to output "clean" .ibd files and .exp
1109+ files for 'ALTER TABLE ... IMPORT TABLESPACE' command
1110+ at innodb_expand_import option enabled XtraDB.
1111+
1112+ --user=NAME
1113+ This option is passed to the mysql child process.
1114+ It defines the user for database login if not current user.
1115+ Try 'mysql --help' for more details on this option.
1116+
1117+ --password=WORD
1118+ This option is passed to the mysql child process.
1119+ It defines the password to use when connecting to database.
1120+ Try 'mysql --help' for more details on this option.
1121+
1122+ --host=HOST
1123+ This option is passed to the mysql child process.
1124+ It defines the hostname or IP address to use when connecting
1125+ to the database server.
1126+ Try 'mysql --help' for more details on this option.
1127+
1128+ --port=PORT
1129+ This option is passed to the mysql child process.
1130+ It defines the port to use when connecting to local database
1131+ server with TCP/IP.
1132+ Try 'mysql --help' for more details on this option.
1133+
1134+ --slave-info
1135+ This option is useful when backing up a replication
1136+ slave server. It prints the binary log position and
1137+ name of the binary log file of the master server.
1138+ It also writes this information to the 'ibbackup_slave_info'
1139+ file as a 'CHANGE MASTER' command. A new slave for this
1140+ master can be set up by starting a slave server on this
1141+ backup and issuing a 'CHANGE MASTER' command with the binary
1142+ log position saved in the 'ibbackup_slave_info' file.
1143+
1144+ --socket=SOCKET
1145+ This option is passed to the mysql child process.
1146+ It defines the socket to use when connecting to local database
1147+ server with UNIX domain socket.
1148+ Try 'mysql --help' for more details on this option.
1149+
1150+ --no-timestamp
1151+ This option prevents the creation of a new backup
1152+ directory (named by the current date and time) under
1153+ the backup root directory. Instead, the backup is put
1154+ in the directory given on the command-line (in the
1155+ place of BACKUP-ROOT-DIR argument). The directory must not
1156+ exist, because innobackup creates it while making a backup.
1157+
1158+ --ibbackup=IBBACKUP-BINARY
1159+ Use this option to specify which ibbackup (InnoDB Hot
1160+ Backup) binary should be used. IBBACKUP-BINARY
1161+ should be the command used to run ibbackup. This can
1162+ be useful if ibbackup is not in your search path or
1163+ working directory. If this option is not given,
1164+ ibbackup is run with command "ibbackup".
1165+ --no-lock
1166+ Use this option to disable table lock with
1167+ FLUSH TABLES WITH READ LOCK. use it only while ALL your
1168+ tables are InnoDB and you DO NOT CARE about binary log
1169+ position of backup.
1170+
1171+ --scpopt=SCP-OPTIONS
1172+ Use this option to specify the command line options
1173+ to pass to scp. The default options are '-Cp -c arcfour'.
1174+
1175+EOF
1176+
1177+ print STDOUT "Option values after processing arguments:\n\n";
1178+ print STDOUT
1179+ join( "\n", sort map { " --$_=" . $o->{$_} || '' } keys %$o ),
1180+ "\n\n";
1181+
1182+ return;
1183+}
1184+
1185+#
1186+# return current local time as string in form "070816 12:23:15"
1187+#
1188+sub current_time {
1189+ return strftime( "%y%m%d %H:%M:%S", localtime() );
1190+}
1191+
1192+#
1193+# Die subroutine kills all child processes and exits this process.
1194+# This subroutine takes the same argument as the built-in die function.
1195+# Parameters:
1196+# message string which is printed to stdout
1197+#
1198+sub Die {
1199+ my $message = shift;
1200+ my $extra_info = '';
1201+
1202+ # kill all child processes of this process
1203+ kill_child_processes();
1204+
1205+ if ($current_mysql_request) {
1206+ $extra_info = " while waiting for reply to MySQL request:"
1207+ . " '$current_mysql_request'";
1208+ }
1209+
1210+ IBEXDEBUG && _d('Die', $prefix, $message, $extra_info);
1211+ die "$prefix Error: $message$extra_info";
1212+}
1213+
1214+#
1215+# backup subroutine makes a backup of InnoDB and MyISAM tables, indexes and
1216+# .frm files. It connects to the database server and runs ibbackup as a child
1217+# process.
1218+#
1219+sub backup {
1220+ my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' );
1221+
1222+ # check that we can connect to the database. This done by
1223+ # connecting, issuing a query, and closing the connection.
1224+ mysql_open();
1225+ mysql_check();
1226+ mysql_close();
1227+
1228+ # start ibbackup as a child process
1229+ start_ibbackup();
1230+
1231+ # wait for ibbackup to suspend itself
1232+ if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
1233+ wait_for_ibbackup_suspend();
1234+ }
1235+
1236+ if ( !$o->{'incremental'} ) {
1237+ mysql_open();
1238+ mysql_check();
1239+ mysql_lockall() unless $o->{'no-lock'};
1240+ backup_files();
1241+ }
1242+
1243+ # resume ibbackup and wait till it has finished
1244+ resume_ibbackup();
1245+
1246+ if ( !$o->{'incremental'} ) {
1247+ mysql_unlockall() unless $o->{'no-lock'};
1248+ mysql_close();
1249+ }
1250+
1251+ if ( $o->{'remote-host'} ) {
1252+ system_call("scp $o->{'scp-opt'} '$tmp_logfile' "
1253+ . "'$o->{'remote-host'}:$backup_dir/xtrabackup_logfile'");
1254+
1255+ system_call("rm -f '$tmp_logfile'");
1256+
1257+ system_call("scp $o->{'scp-opt'} '$orig_datadir/xtrabackup_checkpoints' "
1258+ . "'$o->{'remote-host'}:$backup_dir/xtrabackup_checkpoints'");
1259+
1260+ system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'");
1261+ }
1262+ elsif ( $o->{'stream'} eq 'tar' ) {
1263+ system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_logfile");
1264+
1265+ system_call("rm -f '$tmp_logfile'");
1266+
1267+ system_call("cd $orig_datadir; tar chf - xtrabackup_checkpoints");
1268+
1269+ system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'");
1270+ }
1271+
1272+ print STDERR "\n$prefix Backup created in directory "
1273+ . ($o->{'remote-host'} ? "$o->{'remote-host'}:" : '')
1274+ . "'$backup_dir'\n";
1275+
1276+ if ($mysql_binlog_position) {
1277+ print STDERR "$prefix MySQL binlog position: $mysql_binlog_position\n";
1278+ }
1279+ if ( $mysql_slave_position && $o->{'slave-info'} ) {
1280+ print STDERR "$prefix MySQL slave binlog position: "
1281+ . "$mysql_slave_position\n";
1282+ }
1283+
1284+ return;
1285+}
1286+
1287+#
1288+# are_equal_innodb_data_file_paths subroutine checks if the given
1289+# InnoDB data file option values are equal.
1290+# Parameters:
1291+# str1 InnoDB data file path option value
1292+# str2 InnoDB data file path option value
1293+# Return value:
1294+# 1 if values are equal
1295+# 0 otherwise
1296+#
1297+sub are_equal_innodb_data_file_paths {
1298+ my $str1 = shift;
1299+ my $str2 = shift;
1300+ my @array1 = split( /;/, $str1 );
1301+ my @array2 = split( /;/, $str2 );
1302+
1303+ if ( $#array1 != $#array2 ) { return 0; }
1304+
1305+ for ( my $i = 0; $i <= $#array1; $i++ ) {
1306+ my @def1 = split( /:/, $array1[$i] );
1307+ my @def2 = split( /:/, $array2[$i] );
1308+
1309+ if ( $#def1 != $#def2 ) { return 0; }
1310+
1311+ for ( my $j = 0; $j <= $#def1; $j++ ) {
1312+ if ( $def1[$j] ne $def2[$j] ) { return 0; }
1313+ }
1314+ }
1315+ return 1;
1316+}
1317+
1318+#
1319+# is_in_array subroutine checks if the given string is in the array.
1320+# Parameters:
1321+# str a string
1322+# array_ref a reference to an array of strings
1323+# Return value:
1324+# 1 if string is in the array
1325+# 0 otherwise
1326+#
1327+sub is_in_array {
1328+ my $str = shift;
1329+ my $array_ref = shift;
1330+
1331+ if ( grep { $str eq $_ } @{$array_ref} ) {
1332+ return 1;
1333+ }
1334+ return 0;
1335+}
1336+
1337+#
1338+# copy_back subroutine copies data and index files from backup directory
1339+# back to their original locations.
1340+#
1341+sub copy_back {
1342+ my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' );
1343+ my $orig_ibdata_dir =
1344+ get_option( \%config, 'mysqld', 'innodb_data_home_dir' );
1345+ my $orig_innodb_data_file_path =
1346+ get_option( \%config, 'mysqld', 'innodb_data_file_path' );
1347+ my $orig_iblog_dir =
1348+ get_option( \%config, 'mysqld', 'innodb_log_group_home_dir' );
1349+ my $excluded_files =
1350+ '^(\.\.?|backup-my\.cnf|xtrabackup_logfile|mysql-std(err|out)|.*\.ibz)$';
1351+ my @ibdata_files;
1352+ my $iblog_files = '^ib_logfile.*$';
1353+ my $compressed_data_file = '.*\.ibz$';
1354+ my $file;
1355+ my $backup_innodb_data_file_path;
1356+
1357+ # check that original data directory exists
1358+ if ( !-d $orig_datadir ) {
1359+ Die "Original data directory '$orig_datadir' does not exist!";
1360+ }
1361+
1362+ # check that original InnoDB data directory exists
1363+ if ( !-d $orig_ibdata_dir ) {
1364+ Die "Original InnoDB data directory '$orig_ibdata_dir' does not exist!";
1365+ }
1366+
1367+ # check that original InnoDB log directory exists
1368+ if ( !-d $orig_iblog_dir ) {
1369+ Die "Original InnoDB log directory '$orig_iblog_dir' does not exist!";
1370+ }
1371+
1372+ # check that the original options file and the backup options file have
1373+ # the same value for "innodb_data_file_path" option
1374+ #$backup_innodb_data_file_path =
1375+ # get_option(\%backup_config, 'mysqld', 'innodb_data_file_path');
1376+ #if (!are_equal_innodb_data_file_paths($orig_innodb_data_file_path,
1377+ # $backup_innodb_data_file_path)
1378+ #) {
1379+ # Die "The value of 'innodb_data_file_path' option in the original "
1380+ # . "my.cnf file '$config_file' is different from the value "
1381+ # . "in the backup my.cnf file '$backup_config_file'.\n(original: "
1382+ # . "'$orig_innodb_data_file_path')\n"
1383+ # . "(backup: '$backup_innodb_data_file_path')";
1384+ #}
1385+
1386+ # make a list of all ibdata files in the backup directory and all
1387+ # directories in the backup directory under which there are ibdata files
1388+ foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) {
1389+ my $path = ( split( /:/, $a ) )[0];
1390+ my $filename = ( split( /\/+/, $path ) )[-1];
1391+
1392+ # check that the backup data file exists
1393+ if ( !-e "$backup_dir/$filename" ) {
1394+ if ( -e "$backup_dir/${filename}.ibz" ) {
1395+ Die
1396+ "Backup data file '$backup_dir/$filename' does not exist, but "
1397+ . "its compressed copy '${path}.ibz' exists. Check that you "
1398+ . "have run '$innobackup_script --apply-log --uncompress "
1399+ . "...' before attempting '$innobackup_script --copy-back ...' !";
1400+ }
1401+ else {
1402+ Die "Backup data file '$backup_dir/$filename' does not exist.";
1403+ }
1404+ }
1405+
1406+ if ( !is_in_array( $filename, \@ibdata_files ) ) {
1407+ push( @ibdata_files, $filename );
1408+ }
1409+ }
1410+
1411+ # copy files from backup dir to their original locations
1412+
1413+ # copy files to original data directory
1414+ opendir( DIR, $backup_dir )
1415+ || Die "Can't open directory '$backup_dir': $OS_ERROR\n";
1416+ print STDERR "$prefix Starting to copy MyISAM tables, indexes,\n";
1417+ print STDERR
1418+ "$prefix .MRG, .TRG, .TRN, .ARM, .ARZ, .opt, and .frm files\n";
1419+ print STDERR "$prefix in '$backup_dir'\n";
1420+ print STDERR "$prefix back to original data directory '$orig_datadir'\n";
1421+ while ( defined( $file = readdir(DIR) ) ) {
1422+ if ( $file =~ /$excluded_files/ ) { next; }
1423+ if ( is_in_array( $file, \@ibdata_files ) ) { next; }
1424+ if ( $file =~ /$iblog_files/ ) { next; }
1425+ if ( -d "$backup_dir/$file" ) {
1426+ my $subdir = "$backup_dir/$file";
1427+ my $file2;
1428+
1429+ print STDERR "$prefix Copying directory '$subdir'\n";
1430+ if ( !-x "\"" . escape_path("$orig_datadir/$file") . "\"" ) {
1431+ system_call("mkdir \"" . escape_path("$orig_datadir/$file") . "\"");
1432+ }
1433+ opendir( SUBDIR, "$subdir" )
1434+ || Die "Can't open directory '$subdir': $OS_ERROR\n";
1435+ while ( defined( $file2 = readdir(SUBDIR) ) ) {
1436+ if ( -d "$subdir/$file2" ) { next; }
1437+ if ( $file2 =~ /$compressed_data_file/ ) { next; }
1438+ $src_name = escape_path("$subdir/$file2");
1439+ $dst_name = escape_path("$orig_datadir/$file");
1440+ system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
1441+ }
1442+ closedir(SUBDIR);
1443+ }
1444+ else {
1445+ print STDERR "$prefix Copying file " . "'$backup_dir/$file'\n";
1446+ $src_name = escape_path("$backup_dir/$file");
1447+ $dst_name = escape_path("$orig_datadir");
1448+ system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
1449+ }
1450+ }
1451+ closedir(DIR);
1452+
1453+ # copy InnoDB data files to original InnoDB data directory
1454+ print STDERR "\n$prefix Starting to copy InnoDB tables and indexes\n";
1455+ print STDERR "$prefix in '$backup_dir'\n";
1456+ print STDERR
1457+ "$prefix back to original InnoDB data directory '$orig_ibdata_dir'\n";
1458+ foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) {
1459+
1460+ # get the relative pathname of a data file
1461+ my $path = ( split( /:/, $a ) )[0];
1462+ my $filename = ( split( /\/+/, $path ) )[-1];
1463+ print STDERR "$prefix Copying file '$backup_dir/$filename'\n";
1464+ $src_name = escape_path("$backup_dir/$filename");
1465+ $dst_name = escape_path("$orig_ibdata_dir/$path");
1466+ system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
1467+ }
1468+
1469+ # copy InnoDB log files to original InnoDB log directory
1470+ opendir( DIR, $backup_dir )
1471+ || Die "Can't open directory '$backup_dir': $OS_ERROR\n";
1472+ print STDERR "\n$prefix Starting to copy InnoDB log files\n";
1473+ print STDERR "$prefix in '$backup_dir'\n";
1474+ print STDERR
1475+ "$prefix back to original InnoDB log directory '$orig_iblog_dir'\n";
1476+ while ( defined( $file = readdir(DIR) ) ) {
1477+ if ( $file =~ /$iblog_files/ && -f "$backup_dir/$file" ) {
1478+ print STDERR "$prefix Copying file '$backup_dir/$file'\n";
1479+ $src_name = escape_path("$backup_dir/$file");
1480+ $dst_name = escape_path("$orig_iblog_dir");
1481+ system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
1482+ }
1483+ }
1484+ closedir(DIR);
1485+
1486+ print STDERR "$prefix Finished copying back files.\n\n";
1487+
1488+ return;
1489+}
1490+
1491+#
1492+# apply_log subroutine prepares a backup for starting database server
1493+# on the backup. It applies InnoDB log files to the InnoDB data files.
1494+#
1495+sub apply_log {
1496+ my $rcode;
1497+ my $cmdline = '';
1498+ my $options = '';
1499+
1500+ if ( $o->{'defaults-file'} ) {
1501+ $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" ";
1502+ }
1503+
1504+ $options = $options . "--prepare --target-dir=$backup_dir";
1505+
1506+ if ( $o->{'uncompress'} ) {
1507+ $options = $options . ' --uncompress';
1508+ }
1509+ if ( $o->{'export'} ) {
1510+ $options = $options . ' --export';
1511+ }
1512+ if ( $o->{'use-memory'} ) {
1513+ $options = $options . " --use-memory=$o->{'use-memory'}";
1514+ }
1515+
1516+ # run ibbackup as a child process
1517+ $cmdline = "$o->{'ibbackup'} $options";
1518+ $now = current_time();
1519+ print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n\n";
1520+ $rcode = system_call($cmdline, 1);
1521+ Die "\n$prefix ibbackup failed" if $rcode;
1522+
1523+ $now = current_time();
1524+ print STDERR
1525+ "\n$now $prefix Restarting xtrabackup with command: $cmdline\nfor creating ib_logfile*\n\n";
1526+ $rcode = system_call($cmdline, 1);
1527+ Die "\n$prefix xtrabackup (2nd execution) failed" if $rcode;
1528+
1529+ return;
1530+}
1531+
1532+#
1533+# wait_for_ibbackup_suspend subroutine waits until ibbackup has suspended
1534+# itself.
1535+#
1536+sub wait_for_ibbackup_suspend {
1537+ print STDERR
1538+ "$prefix Waiting for ibbackup (pid=$ibbackup_pid) to suspend\n";
1539+ print STDERR "$prefix Suspend file '$suspend_file'\n\n";
1540+ for ( ;; ) {
1541+ sleep 2;
1542+ last if -e $suspend_file;
1543+
1544+ # check that ibbackup child process is still alive
1545+ if ( $ibbackup_pid == waitpid( $ibbackup_pid, &WNOHANG ) ) {
1546+ $ibbackup_pid = '';
1547+ Die "ibbackup child process has died";
1548+ }
1549+ }
1550+ $now = current_time();
1551+ print STDERR "\n$now $prefix Continuing after ibbackup has suspended\n";
1552+}
1553+
1554+#
1555+# resume_ibbackup subroutine signals ibbackup to complete its execution
1556+# by deleting the 'ibbackup_suspended' file.
1557+#
1558+sub resume_ibbackup {
1559+ print STDERR "$prefix Resuming ibbackup\n\n";
1560+ unlink $suspend_file || Die "Failed to delete '$suspend_file': $OS_ERROR";
1561+
1562+ # wait for ibbackup to finish
1563+ waitpid( $ibbackup_pid, 0 );
1564+ $ibbackup_pid = '';
1565+}
1566+
1567+#
1568+# start_ibbackup subroutine spawns a child process running ibbackup
1569+# program for backing up InnoDB tables and indexes.
1570+#
1571+sub start_ibbackup {
1572+ my $options = '';
1573+ my $cmdline = '';
1574+ my $pid = undef;
1575+
1576+ if ( $o->{'defaults-file'} ) {
1577+ $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" ";
1578+ }
1579+
1580+ $options = $options . "--backup --suspend-at-end";
1581+
1582+ if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
1583+ $options = $options . " --target-dir=$backup_dir";
1584+ }
1585+ else {
1586+
1587+ #(datadir) for 'xtrabackup_suspended' and 'xtrabackup_checkpoints'
1588+ $options = $options . " --log-stream --target-dir=./";
1589+ }
1590+
1591+ # prepare command line for running ibbackup
1592+ if ( $o->{'throttle'} ) {
1593+ $options = $options . " --throttle=$o->{'throttle'}";
1594+ }
1595+ if ( $o->{'sleep'} ) {
1596+ $options = $options . " --sleep=$o->{'sleep'}";
1597+ }
1598+ if ( $o->{'compress'} ) {
1599+ $options = $options . " --compress=$o->{'compress'}";
1600+ }
1601+ if ( $o->{'use-memory'} ) {
1602+ $options = $options . " --use-memory=$o->{'use-memory'}";
1603+ }
1604+ if ( $o->{'include'} ) {
1605+ $options = $options . " --tables='$o->{'include'}'";
1606+ }
1607+ if ( $o->{'incremental'} ) {
1608+ $options = $options . " --incremental-basedir='$incremental_basedir'";
1609+ }
1610+ $cmdline = "$o->{'ibbackup'} $options";
1611+
1612+ # run ibbackup as a child process
1613+ $now = current_time();
1614+ print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n";
1615+ if ( defined($pid = fork) ) {
1616+ if ($pid) {
1617+ # parent process
1618+ $ibbackup_pid = $pid;
1619+
1620+ # direct copy to remote
1621+ if ( $o->{'remote-host'} || $o->{'stream'} ) {
1622+ my $orig_datadir
1623+ = get_option( \%config, 'mysqld', 'datadir' );
1624+ my $orig_ibdata_dir
1625+ = get_option( \%config, 'mysqld', 'innodb_data_home_dir' );
1626+ my $orig_innodb_data_file_path
1627+ = get_option( \%config, 'mysqld', 'innodb_data_file_path' );
1628+ my $subdir;
1629+ my @list;
1630+
1631+ if ( $o->{'remote-host'} ) {
1632+ if ( system_call("ssh $o->{'remote-host'} test -e "
1633+ . "$backup_dir/ib_logfile0", 1) == 0 ) {
1634+ print STDERR "$prefix Remove "
1635+ . "$o->{'remote-host'}:$backup_dir/ib_logfile*\n";
1636+ system_call( "ssh $o->{'remote-host'} "
1637+ . "rm $backup_dir/ib_logfile\*"
1638+ )
1639+ }
1640+ }
1641+
1642+ wait_for_ibbackup_suspend();
1643+
1644+ #InnoDB data files from original InnoDB data directory
1645+ print STDERR
1646+ "\n$prefix Starting to backup InnoDB tables and indexes\n";
1647+ if ( $o->{'remote-host'} ) {
1648+ print STDERR "$prefix to '$backup_dir'\n";
1649+ }
1650+ print STDERR
1651+ "$prefix from original InnoDB data directory '$orig_ibdata_dir'\n";
1652+ foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) {
1653+ my $path = ( split( /:/, $a ) )[0];
1654+ $path =~ s/([\$\\\" ])/\\$1/g;
1655+ if ( $o->{'remote-host'} ) {
1656+ print STDERR
1657+ "$prefix Backing up file '$orig_ibdata_dir/$path'\n";
1658+ system_call("scp $o->{'scp-opt'} '$orig_ibdata_dir/$path' "
1659+ . "'$o->{'remote-host'}:$backup_dir/$path'");
1660+ }
1661+ elsif ( $o->{'stream'} eq 'tar' ) {
1662+ my $ret = 0;
1663+ print STDERR "$prefix Backing up as tar stream '$path'\n";
1664+ if ( !$o->{'tar4ibd'} ) {
1665+ $ret = system_call(
1666+ "cd $orig_ibdata_dir; tar chf - -b 32 $path", 1);
1667+ }
1668+ else {
1669+ $ret = system_call(
1670+ "cd $orig_ibdata_dir; tar4ibd -c $path", 1);
1671+ }
1672+ if ( $ret == 1 ) {
1673+ print STDERR "$prefix If you use GNU tar, this warning "
1674+ . "can be ignored.\n";
1675+ }
1676+ elsif ( $ret != 0 ) {
1677+ print STDERR "$prefix tar returned with exit code $ret.\n";
1678+ Die "Failed to stream '$path': $OS_ERROR";
1679+ }
1680+ }
1681+ }
1682+
1683+ #copy *.ibd files
1684+ opendir( DIR, $orig_datadir )
1685+ || Die "Can't open directory '$orig_datadir': $OS_ERROR\n";
1686+ while ( defined( $subdir = readdir(DIR) ) ) {
1687+ my $print_each_file = 0;
1688+ my $file_c;
1689+ my $file;
1690+ if ( $subdir eq '.' || $subdir eq '..' ) { next; }
1691+ next unless -d "$orig_datadir/$subdir";
1692+ next unless check_if_required($subdir);
1693+
1694+ @list = glob( "$orig_datadir/$subdir/" . '*.ibd' );
1695+
1696+ $file_c = @list;
1697+ if ( $file_c <= $backup_file_print_limit ) {
1698+ $print_each_file = 1;
1699+ }
1700+ else {
1701+ print STDERR "$prefix Backing up files "
1702+ . "'$orig_datadir/$subdir/*.ibd' ($file_c files)\n";
1703+ }
1704+
1705+ foreach $file ( @list ) {
1706+ if ( $o->{'include'} ) {
1707+ my $table_name;
1708+ $table_name = substr($file, rindex( $file, '/' ));
1709+ $table_name = substr( $table_name, 1,
1710+ rindex( $table_name, '.' ) - 1);
1711+ $table_name = $subdir . "." . $table_name;
1712+ if ( $table_name !~ m/$o->{'include'}/o ) {
1713+ print STDERR "'$file' is skipped.\n";
1714+ next;
1715+ }
1716+ }
1717+
1718+ if ($print_each_file) {
1719+ print STDERR "$prefix Backing up file '$file'\n";
1720+ }
1721+
1722+ if ( $o->{'remote-host'} ) {
1723+ system_call(
1724+ "ssh $o->{'remote-host'} mkdir '$backup_dir/$subdir'",
1725+ 1 # don't die on failure
1726+ );
1727+ system_call("scp $o->{'scp-opt'} '$file' "
1728+ . "'$o->{'remote-host'}:$backup_dir/$subdir/'");
1729+ }
1730+ elsif ( $o->{'stream'} eq 'tar' ) {
1731+ my $ret = 0;
1732+ my $file_name = substr($file, rindex( $file, '/' ) + 1);
1733+ $file_name =~ s/([\$\\\" ])/\\$1/g;
1734+
1735+ if ( !$o->{'tar4ibd'} ) {
1736+ $ret = system_call("cd $orig_datadir; tar chf - -b 32 "
1737+ . "$subdir/$file_name", 1);
1738+ }
1739+ else {
1740+ $ret = system_call("cd $orig_datadir; tar4ibd -c "
1741+ . "$subdir/$file_name", 1);
1742+ }
1743+
1744+ if ( $ret == 1 ) {
1745+ print STDERR "$prefix If you use GNU tar, this warning "
1746+ . "can be ignored.\n";
1747+ }
1748+ elsif ( $ret != 0 ) {
1749+ print STDERR "$prefix tar returned with exit code "
1750+ . "$ret.\n";
1751+ Die "Failed to stream '$subdir/$file_name': $OS_ERROR";
1752+ }
1753+ }
1754+ } # each file
1755+ } # each subdir
1756+ closedir(DIR);
1757+ }
1758+ } # parent process
1759+ else { # child process
1760+ if ( $o->{'remote-host'} || $o->{'stream'} ) {
1761+ open STDOUT, "> $tmp_logfile"
1762+ or Die "Failed to open file '$tmp_logfile': $OS_ERROR";
1763+ }
1764+ IBEXDEBUG && _d($cmdline);
1765+ exec($cmdline) or Die "Failed to exec ibbackup: $OS_ERROR";
1766+ }
1767+ }
1768+ else {
1769+ Die "failed to fork ibbackup child process: $OS_ERROR";
1770+ }
1771+}
1772+
1773+#
1774+# get_mysql_options subroutine returns the options to mysql client program
1775+# as a string. The options are determined from the options given by the
1776+# user to innobackup.
1777+#
1778+sub get_mysql_options {
1779+ my @options;
1780+
1781+ # --defaults-file needs to be first.
1782+ if ( $o->{'defaults-file'} ) {
1783+ push @options, "--defaults-file=$o->{'defaults-file'}";
1784+ }
1785+
1786+ if ( $o->{'host'} ) {
1787+ push @options, "--host=$o->{'host'}";
1788+ }
1789+ if ( $o->{'port'} ) {
1790+ push @options, "--port=$o->{'port'}";
1791+ }
1792+ if ( $o->{'user'} ) {
1793+ push @options, "--user=$o->{'user'}";
1794+ }
1795+ if ( $o->{'password'} ) {
1796+ push @options, "--password=$o->{'password'}";
1797+ }
1798+ if ( $o->{'socket'} ) {
1799+ push @options, "--socket=$o->{'socket'}";
1800+ }
1801+
1802+ push @options, '--unbuffered --';
1803+
1804+ my $options = join( ' ', @options );
1805+ IBEXDEBUG && _d('mysql options:', $options);
1806+ return $options;
1807+}
1808+
1809+#
1810+# mysql_open subroutine starts mysql as a child process with
1811+# a pipe connection.
1812+#
1813+sub mysql_open {
1814+ my $options = get_mysql_options();
1815+
1816+ # run mysql as a child process with a pipe connection
1817+ $now = current_time();
1818+ print STDERR "$now $prefix Starting mysql with options: $options\n";
1819+ $mysql_pid = open(*MYSQL_WRITER,
1820+ "| mysql $options >$mysql_stdout 2>$mysql_stderr ")
1821+ or Die "Failed to spawn mysql child process: $OS_ERROR";
1822+ MYSQL_WRITER->autoflush(1);
1823+ $now = current_time();
1824+ print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n";
1825+
1826+ return;
1827+}
1828+
1829+#
1830+# mysql_check subroutine checks that the connection to mysql child process
1831+# is ok.
1832+#
1833+sub mysql_check {
1834+ my $mysql_pid_copy = $mysql_pid;
1835+
1836+ # send a dummy query to mysql child process
1837+ $hello_id++;
1838+ my $hello_message = "innobackup hello $hello_id";
1839+ IBEXDEBUG && _d('mysql check:', $hello_message);
1840+ print MYSQL_WRITER "select '$hello_message';\n"
1841+ or Die "Connection to mysql child process failed: $OS_ERROR";
1842+
1843+ # wait for reply
1844+ eval {
1845+ local $SIG{ALRM} = sub { die "alarm clock restart" };
1846+ my $stdout = '';
1847+ my $stderr = '';
1848+ alarm $mysql_response_timeout;
1849+ while ( index( $stdout, $hello_message ) < 0 ) {
1850+ sleep 2;
1851+ if ( $mysql_pid && $mysql_pid == waitpid( $mysql_pid, &WNOHANG ) ) {
1852+ my $reason = `cat $mysql_stderr`;
1853+ $mysql_pid = '';
1854+ die "mysql child process has died: $reason";
1855+ }
1856+ $stdout = `cat $mysql_stdout`;
1857+ $stderr = `cat $mysql_stderr`;
1858+ if ($stderr) {
1859+ # mysql has reported an error, do exit
1860+ die "mysql error: $stderr";
1861+ }
1862+ }
1863+ alarm 0;
1864+ };
1865+ if ( $EVAL_ERROR =~ /alarm clock restart/ ) {
1866+ Die "Connection to mysql child process (pid=$mysql_pid_copy) timedout."
1867+ . " (Time limit of $mysql_response_timeout seconds exceeded."
1868+ . " You may adjust time limit by editing the value of parameter"
1869+ . " \"\$mysql_response_timeout\" in this script.)";
1870+ }
1871+ elsif ( $EVAL_ERROR ) {
1872+ Die $EVAL_ERROR;
1873+ }
1874+
1875+ $mysql_last_access_time = time();
1876+
1877+ return;
1878+}
1879+
1880+#
1881+# mysql_keep_alive subroutine tries to keep connection to the mysqld database
1882+# server alive by sending a dummy query when the connection has been idle
1883+# for the specified time.
1884+#
1885+sub mysql_keep_alive {
1886+ if ( ( time() - $mysql_last_access_time ) > $mysql_keep_alive_timeout ) {
1887+ # too long idle, send a dummy query
1888+ mysql_check();
1889+ }
1890+ return;
1891+}
1892+
1893+#
1894+# mysql_send subroutine send a request string to mysql child process.
1895+# This subroutine appends a newline character to the request and checks
1896+# that mysqld receives the query.
1897+# Parameters:
1898+# request request string
1899+#
1900+sub mysql_send {
1901+ my $request = shift;
1902+ $current_mysql_request = $request;
1903+ print MYSQL_WRITER "$request\n";
1904+ mysql_check();
1905+ $current_mysql_request = '';
1906+ return;
1907+}
1908+
1909+#
1910+# mysql_close subroutine terminates mysql child process gracefully.
1911+#
1912+sub mysql_close {
1913+ print MYSQL_WRITER "quit\n";
1914+ $now = current_time();
1915+ print STDERR "$now $prefix Connection to database server closed\n";
1916+ $mysql_pid = '';
1917+ return;
1918+}
1919+
1920+#
1921+# write_binlog_info subroutine retrieves MySQL binlog position and
1922+# saves it in a file. It also prints it to stdout.
1923+#
1924+sub write_binlog_info {
1925+ my @lines;
1926+ my @info_lines = ();
1927+ my $position = '';
1928+ my $filename = '';
1929+
1930+ # get binlog position
1931+ mysql_send "SHOW MASTER STATUS;";
1932+
1933+ # get "show master status" output lines (2) from mysql output
1934+ file_to_array( $mysql_stdout, \@lines );
1935+ foreach my $line (@lines) {
1936+ if ( $line =~ m/innobackup hello/ ) {
1937+
1938+ # this is a hello message, ignore it
1939+ }
1940+ else {
1941+
1942+ # this is output line from "show master status"
1943+ push( @info_lines, $line );
1944+ }
1945+ }
1946+
1947+ # write binlog info file
1948+ if ( !defined $info_lines[1] ) {
1949+ $info_lines[1] = "";
1950+ }
1951+ if ( !$o->{'remote-host'} ) {
1952+ open( FILE, ">$binlog_info" )
1953+ || Die "Failed to open file '$binlog_info': $OS_ERROR";
1954+ }
1955+ else {
1956+ open( FILE, "| ssh $o->{'remote-host'} 'cat > $binlog_info'" )
1957+ || Die
1958+ "Failed to open file '$o->{'remote-host'}:$binlog_info': $OS_ERROR";
1959+ }
1960+ print FILE "$info_lines[1]\n";
1961+ close(FILE);
1962+
1963+ if ( $o->{'stream'} eq 'tar' ) {
1964+ system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_binlog_info");
1965+ unlink $binlog_info or Die "Failed to delete '$binlog_info': $OS_ERROR";
1966+ }
1967+
1968+ # get the name of the last binlog file and position in it
1969+ ( $filename, $position ) = $info_lines[1] =~ /^\s*([^\s]+)\s+(.*)$/;
1970+
1971+ if ( defined $filename && defined $position ) {
1972+ $mysql_binlog_position = "filename '$filename', position $position";
1973+ }
1974+ else {
1975+ $mysql_binlog_position = "filename '', position ";
1976+ }
1977+
1978+ return;
1979+}
1980+
1981+#
1982+# write_slave_info subroutine retrieves MySQL binlog position of the
1983+# master server in a replication setup and saves it in a file. It
1984+# also saves it in $msql_slave_position variable.
1985+#
1986+sub write_slave_info {
1987+ my @lines;
1988+ my @info_lines;
1989+ my $position = '';
1990+ my $filename = '';
1991+ my $master = '';
1992+
1993+ # get slave status. Use single quotes here, otherwise
1994+ # \G is evaluated as a control character.
1995+ mysql_send 'SHOW SLAVE STATUS\G;';
1996+
1997+ # get output of the "show slave status" command from mysql output
1998+ # and extract binlog position of the master server
1999+ file_to_array( $mysql_stdout, \@lines );
2000+ for (@lines) {
2001+ $master = $1 if /Master_Host:\s*(\S*)\s*$/;
2002+ $filename = $1 if /Master_Log_File:\s*(\S*)\s*$/;
2003+ $position = $1 if /Master_Log_Pos:\s*(\S*)\s*$/;
2004+ }
2005+
2006+ # print slave status to a file
2007+ if ( !$o->{'remote-host'} ) {
2008+ open( FILE, ">$slave_info" )
2009+ || Die "Failed to open file '$slave_info': $OS_ERROR";
2010+ }
2011+ else {
2012+ open( FILE, "| ssh $o->{'remote-host'} 'cat > $slave_info'" )
2013+ || Die
2014+ "Failed to open file '$o->{'remote-host'}:$slave_info': $OS_ERROR";
2015+ }
2016+ print FILE
2017+ "CHANGE MASTER TO MASTER_LOG_FILE='$filename', MASTER_LOG_POS=$position\n";
2018+ close(FILE);
2019+
2020+ if ( $o->{'stream'} eq 'tar' ) {
2021+ system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_slave_info");
2022+ unlink $slave_info or Die "Failed to delete '$slave_info': $OS_ERROR";
2023+ }
2024+
2025+ $mysql_slave_position =
2026+ "master host '$master', filename '$filename', position $position";
2027+
2028+ return;
2029+}
2030+
2031+#
2032+# mysql_lockall subroutine puts a read lock on all tables in all databases.
2033+#
2034+sub mysql_lockall {
2035+ $now = current_time();
2036+ print STDERR "$now $prefix Starting to lock all tables...\n";
2037+
2038+ mysql_send "USE mysql;";
2039+
2040+# mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;";
2041+# if (compare_versions($mysql_server_version, '4.1.0') == -1) {
2042+# # MySQL server version is 4.0 or older, ENGINE keyword not supported
2043+# mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) TYPE=INNODB;";
2044+# } else {
2045+# # MySQL server version is 4.1 or newer, use ENGINE keyword
2046+# mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) ENGINE=INNODB;";
2047+# }
2048+ mysql_send "SET AUTOCOMMIT=0;";
2049+
2050+ # mysql_send "INSERT INTO ibbackup_binlog_marker VALUES (1);";
2051+ if ( compare_versions( $mysql_server_version, '4.0.22' ) == 0
2052+ || compare_versions( $mysql_server_version, '4.1.7' ) == 0 )
2053+ {
2054+
2055+ # MySQL server version is 4.0.22 or 4.1.7
2056+ mysql_send "COMMIT;";
2057+ mysql_send "FLUSH TABLES WITH READ LOCK;";
2058+ }
2059+ else {
2060+
2061+ # MySQL server version is other than 4.0.22 or 4.1.7
2062+ mysql_send "FLUSH TABLES WITH READ LOCK;";
2063+ mysql_send "COMMIT;";
2064+ }
2065+ write_binlog_info;
2066+ write_slave_info if $o->{'slave-info'};
2067+
2068+ $now = current_time();
2069+ print STDERR "$now $prefix All tables locked and flushed to disk\n";
2070+
2071+ return;
2072+}
2073+
2074+#
2075+# mysql_unlockall subroutine releases read locks on all tables in all
2076+# databases.
2077+#
2078+sub mysql_unlockall {
2079+ mysql_send "UNLOCK TABLES;";
2080+
2081+ # mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;";
2082+
2083+ $now = current_time();
2084+ print STDERR "$now $prefix All tables unlocked\n";
2085+}
2086+
2087+#
2088+# catch_sigpipe subroutine is a signal handler for SIGPIPE.
2089+#
2090+sub catch_sigpipe {
2091+ my $rcode;
2092+
2093+ if (
2094+ $mysql_pid
2095+ && ( -1 == ( $rcode = waitpid( $mysql_pid, &WNOHANG ) )
2096+ || $rcode == $mysql_pid )
2097+ )
2098+ {
2099+ my $reason = `cat $mysql_stderr`;
2100+ print STDERR "Pipe to mysql child process broken: $reason at";
2101+ system("date +'%H:%M:%S'");
2102+ exit(1);
2103+ }
2104+ else {
2105+ Die "Broken pipe";
2106+ }
2107+}
2108+
2109+#
2110+# kill_child_processes subroutine kills all child processes of this process.
2111+#
2112+sub kill_child_processes {
2113+ if ($ibbackup_pid) {
2114+ kill( $kill_signal, $ibbackup_pid );
2115+ $ibbackup_pid = '';
2116+ }
2117+
2118+ if ($mysql_pid) {
2119+ kill( $kill_signal, $mysql_pid );
2120+ $mysql_pid = '';
2121+ }
2122+}
2123+
2124+#
2125+# require_external subroutine checks that an external program is runnable
2126+# via the shell. This is tested by calling the program with the
2127+# given arguments. It is checked that the program returns 0 and does
2128+# not print anything to stderr. If this check fails, this subroutine exits.
2129+# Parameters:
2130+# program pathname of the program file
2131+# args arguments to the program
2132+# pattern a string containing a regular expression for finding
2133+# the program version.
2134+# this pattern should contain a subpattern enclosed
2135+# in parentheses which is matched with the version.
2136+# version_ref a reference to a variable where the program version
2137+# string is returned. Example "2.0-beta2".
2138+#
2139+sub require_external {
2140+ my $program = shift;
2141+ my $args = shift;
2142+ my $pattern = shift;
2143+ my $version_ref = shift;
2144+ my @lines;
2145+ my $tmp_stdout = tmpnam();
2146+ my $tmp_stderr = tmpnam();
2147+ my $rcode;
2148+ my $error;
2149+
2150+ $rcode = system_call("$program $args >$tmp_stdout 2>$tmp_stderr", 1);
2151+ if ($rcode) {
2152+ $error = $OS_ERROR;
2153+ }
2154+ my $stderr = `cat $tmp_stderr`;
2155+ unlink $tmp_stderr;
2156+ if ( $stderr ne '' ) {
2157+
2158+ # failure
2159+ unlink $tmp_stdout;
2160+ Die "Couldn't run $program: $stderr";
2161+ }
2162+ elsif ($rcode) {
2163+
2164+ # failure
2165+ unlink $tmp_stdout;
2166+ Die "Couldn't run $program: $error";
2167+ }
2168+
2169+ # success
2170+ my $stdout = `cat $tmp_stdout`;
2171+ unlink $tmp_stdout;
2172+ @lines = split( /\n|;/, $stdout );
2173+ print STDERR "$prefix Using $lines[0]\n";
2174+
2175+ # get version string from the first output line of the program
2176+ ${$version_ref} = '';
2177+ if ( $lines[0] =~ /$pattern/ ) {
2178+ ${$version_ref} = $1;
2179+ }
2180+}
2181+
2182+# compare_versions subroutine compares two GNU-style version strings.
2183+# A GNU-style version string consists of three decimal numbers delimitted
2184+# by dots, and optionally followed by extra attributes.
2185+# Examples: "4.0.1", "4.1.1-alpha-debug".
2186+# Parameters:
2187+# str1 a GNU-style version string
2188+# str2 a GNU-style version string
2189+# Return value:
2190+# -1 if str1 < str2
2191+# 0 if str1 == str2
2192+# 1 is str1 > str2
2193+sub compare_versions {
2194+ my $str1 = shift;
2195+ my $str2 = shift;
2196+ my $extra1 = '';
2197+ my $extra2 = '';
2198+ my @array1 = ();
2199+ my @array2 = ();
2200+ my $i;
2201+
2202+ # remove possible extra attributes
2203+ ( $str1, $extra1 ) = $str1 =~ /^([0-9.]*)(.*)/;
2204+ ( $str2, $extra2 ) = $str2 =~ /^([0-9.]*)(.*)/;
2205+
2206+ # split "dotted" decimal number string into an array
2207+ @array1 = split( '\.', $str1 );
2208+ @array2 = split( '\.', $str2 );
2209+
2210+ # compare in lexicographic order
2211+ for ( $i = 0; $i <= $#array1 && $i <= $#array2; $i++ ) {
2212+ if ( $array1[$i] < $array2[$i] ) {
2213+ return -1;
2214+ }
2215+ elsif ( $array1[$i] > $array2[$i] ) {
2216+ return 1;
2217+ }
2218+ }
2219+ if ( $#array1 < $#array2 ) {
2220+ return -1;
2221+ }
2222+ elsif ( $#array1 > $#array2 ) {
2223+ return 1;
2224+ }
2225+ else {
2226+ return 0;
2227+ }
2228+}
2229+
2230+#
2231+# init subroutine initializes global variables and performs some checks on the
2232+# system we are running on.
2233+#
2234+sub init {
2235+ my $mysql_version = '';
2236+ my $ibbackup_version = '';
2237+ my $run = '';
2238+
2239+ # print some instructions to the user
2240+ if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) {
2241+ $run = 'backup';
2242+ }
2243+ elsif ( $o->{'copy-back'} ) {
2244+ $run = 'copy-back';
2245+ }
2246+ else {
2247+ $run = 'apply-log';
2248+ }
2249+ print STDERR
2250+ "IMPORTANT: Please check that the $run run completes successfully.\n";
2251+ print STDERR
2252+ " At the end of a successful $run run $innobackup_script\n";
2253+ print STDERR " prints \"completed OK!\".\n\n";
2254+
2255+ # check that MySQL client program and InnoDB Hot Backup program
2256+ # are runnable via shell
2257+ if ( !$o->{'copy-back'} ) {
2258+
2259+ # we are making a backup or applying log to backup
2260+ if ( !$o->{'apply-log'} ) {
2261+
2262+ # we are making a backup, we need mysql server
2263+ my $output = '';
2264+ my @lines = ();
2265+
2266+ # check that we have mysql client program
2267+ require_external( 'mysql', '--version', 'Ver ([^,]+)',
2268+ \$mysql_version );
2269+
2270+ # get mysql server version
2271+ my $options = get_mysql_options();
2272+ @lines = split( '\n', `mysql $options -e "select \@\@version"` );
2273+ $mysql_server_version = $lines[1];
2274+ print STDERR
2275+ "$prefix Using mysql server version $mysql_server_version\n";
2276+ }
2277+
2278+ #require_external($o->{'ibbackup-binary'}, '--license',
2279+ # 'version (\S+)', \$ibbackup_version);
2280+ print STDERR "\n";
2281+
2282+ if ( $o->{'include'}
2283+ && $ibbackup_version
2284+ && $ibbackup_version le "2.0" )
2285+ {
2286+
2287+ # --include option was given, but ibbackup is too
2288+ # old to support it
2289+ Die "--include option was given, but ibbackup is too old"
2290+ . " to support it. You must upgrade to InnoDB Hot Backup"
2291+ . " v2.0 in order to use --include option.\n";
2292+ }
2293+ }
2294+
2295+ # set signal handlers
2296+ $SIG{PIPE} = \&catch_sigpipe;
2297+
2298+ # read MySQL options file
2299+ #read_config_file($config_file, \%config);
2300+ read_config_file( \%config );
2301+
2302+ if ( !$o->{'tmpdir'} ) {
2303+ $o->{'tmpdir'} = get_option( \%config, 'mysqld', 'tmpdir' );
2304+ }
2305+
2306+ # get innodb log home directory from options file
2307+ #$innodb_log_group_home_dir =
2308+ # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
2309+
2310+ if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) {
2311+
2312+ # we are making a backup, create a new backup directory
2313+ if ( !$o->{'remote-host'} ) {
2314+ $backup_dir = File::Spec->rel2abs( make_backup_dir() );
2315+ }
2316+ else {
2317+ $backup_dir = make_backup_dir();
2318+ }
2319+ print STDERR "$prefix Created backup directory $backup_dir\n";
2320+ if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
2321+ $backup_config_file = $backup_dir . '/backup-my.cnf';
2322+ $suspend_file = $backup_dir . '/xtrabackup_suspended';
2323+ $mysql_stdout = $backup_dir . '/mysql-stdout';
2324+ $mysql_stderr = $backup_dir . '/mysql-stderr';
2325+ $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
2326+ $slave_info = $backup_dir . '/xtrabackup_slave_info';
2327+ }
2328+ else {
2329+ $suspend_file = get_option( \%config, 'mysqld', 'datadir' )
2330+ . '/xtrabackup_suspended';
2331+ $tmp_logfile = $o->{'tmpdir'} . '/xtrabackup_logfile';
2332+ $mysql_stdout = $o->{'tmpdir'} . '/mysql-stdout';
2333+ $mysql_stderr = $o->{'tmpdir'} . '/mysql-stderr';
2334+ if ( $o->{'stream'} ) {
2335+ $backup_config_file = $o->{'tmpdir'} . '/backup-my.cnf';
2336+ $binlog_info = $o->{'tmpdir'} . '/xtrabackup_binlog_info';
2337+ $slave_info = $o->{'tmpdir'} . '/xtrabackup_slave_info';
2338+ }
2339+ else {
2340+ $backup_config_file = $backup_dir . '/backup-my.cnf';
2341+ $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
2342+ $slave_info = $backup_dir . '/xtrabackup_slave_info';
2343+ }
2344+ }
2345+ write_backup_config_file($backup_config_file);
2346+ }
2347+ elsif ( $o->{'copy-back'} ) {
2348+ #$backup_config_file = $backup_dir . '/backup-my.cnf';
2349+ #read_config_file($backup_config_file, \%backup_config);
2350+ }
2351+
2352+ return;
2353+}
2354+
2355+#
2356+# write_backup_config_file subroutine creates a backup options file for
2357+# ibbackup program. It writes to the file only those options that
2358+# are required by ibbackup.
2359+# Parameters:
2360+# filename name for the created options file
2361+#
2362+sub write_backup_config_file {
2363+ my $filename = shift;
2364+ my $innodb_data_file_path =
2365+ get_option( \%config, 'mysqld', 'innodb_data_file_path' );
2366+ my $innodb_log_files_in_group =
2367+ get_option( \%config, 'mysqld', 'innodb_log_files_in_group' );
2368+ my $innodb_log_file_size =
2369+ get_option( \%config, 'mysqld', 'innodb_log_file_size' );
2370+ my $root;
2371+
2372+ my @array = split( /;/, $innodb_data_file_path );
2373+ for ( my $i = 0; $i <= $#array; $i++ ) {
2374+ my @tmp = split( /\/+/, $array[$i] );
2375+ $array[$i] = $tmp[-1];
2376+ }
2377+ $innodb_data_file_path = join( ";", @array );
2378+
2379+ if ( !$o->{'remote-host'} ) {
2380+ $root = $backup_dir;
2381+ open( FILE, "> $filename" )
2382+ || Die "Failed to open file '$filename': $OS_ERROR";
2383+ }
2384+ else {
2385+ $root = `ssh $o->{'remote-host'} 'cd $backup_dir; pwd'`;
2386+ open( FILE, "| ssh $o->{'remote-host'} 'cat > $filename'" )
2387+ || Die
2388+ "Failed to open file '$o->{'remote-host'}:$filename': $OS_ERROR";
2389+ }
2390+
2391+ print FILE
2392+ "# This MySQL options file was generated by $innobackup_script.\n\n"
2393+ . "# The MySQL server\n"
2394+ . "[mysqld]\n"
2395+ . "datadir=$root\n"
2396+ . "innodb_data_home_dir=$root\n"
2397+ . "innodb_data_file_path=$innodb_data_file_path\n"
2398+ . "innodb_log_group_home_dir=$root\n"
2399+ . "innodb_log_files_in_group=$innodb_log_files_in_group\n"
2400+ . "innodb_log_file_size=$innodb_log_file_size\n";
2401+ close(FILE);
2402+
2403+ if ( $o->{'stream'} ) {
2404+ my $filename_dir = dirname($filename);
2405+ my $filename_name = basename($filename);
2406+ if ( $o->{'stream'} eq 'tar' ) {
2407+ system_call("cd $filename_dir; tar chf - $filename_name");
2408+ }
2409+ unlink $filename or Die "Failed to delete '$filename': $OS_ERROR";
2410+ }
2411+}
2412+
2413+#
2414+# check_args subroutine checks command line arguments. If there is a problem,
2415+# this subroutine prints error message and exits.
2416+#
2417+sub check_args {
2418+ my $i;
2419+ my $rcode;
2420+ my $buf;
2421+ my $perl_version;
2422+
2423+ # check the version of the perl we are running
2424+ if ( !defined $PERL_VERSION ) {
2425+
2426+ # this perl is prior to 5.6.0 and uses old style version string
2427+ my $required_version = $required_perl_version_old_style;
2428+ if ( $] lt $required_version ) {
2429+ print STDERR "$prefix Warning: "
2430+ . "Your perl is too old! Innobackup requires\n";
2431+ print STDERR "$prefix Warning: perl $required_version or newer!\n";
2432+ }
2433+ }
2434+ else {
2435+ $perl_version = 'v' . join( '.', @required_perl_version );
2436+ if ( $PERL_VERSION lt $perl_version ) {
2437+ my $version =
2438+ chr( 48 + $required_perl_version[0] ) . "."
2439+ . chr( 48 + $required_perl_version[1] ) . "."
2440+ . chr( 48 + $required_perl_version[2] );
2441+ print STDERR "$prefix Warning: "
2442+ . "Your perl is too old! Innobackup requires\n";
2443+ print STDERR "$prefix Warning: perl $version or newer!\n";
2444+ }
2445+ }
2446+
2447+ if ( @ARGV == 0 ) {
2448+
2449+ # no command line arguments
2450+ usage();
2451+ exit(1);
2452+ }
2453+
2454+ # read command line options
2455+ $rcode = GetOptions(
2456+ $o,
2457+ 'compress:i',
2458+ 'help',
2459+ 'version',
2460+ 'throttle=i',
2461+ 'sleep=i',
2462+ 'apply-log',
2463+ 'copy-back',
2464+ 'include=s',
2465+ 'databases=s',
2466+ 'use-memory=s',
2467+ 'uncompress',
2468+ 'export',
2469+ 'password=s',
2470+ 'host=s',
2471+ 'user=s',
2472+ 'port=s',
2473+ 'slave-info',
2474+ 'socket=s',
2475+ 'no-timestamp',
2476+ 'defaults-file=s',
2477+ 'incremental',
2478+ 'remote-host=s',
2479+ 'stream=s',
2480+ 'tmpdir=s',
2481+ 'no-lock',
2482+ 'ibbackup=s',
2483+ 'scpopt=s',
2484+ );
2485+
2486+ if ( !$rcode ) {
2487+ print STDERR "$prefix Bad command line arguments\n";
2488+ exit 1;
2489+ }
2490+
2491+ if ( $o->{'help'} ) {
2492+ # print help text and exit
2493+ usage();
2494+ exit 1;
2495+ }
2496+
2497+ if ( $o->{'version'} ) {
2498+ # print program version and copyright
2499+ print_version();
2500+ exit 0;
2501+ }
2502+
2503+ if ( $o->{'compress'} == 0 ) {
2504+ # compression level no specified, use default level
2505+ $o->{'compress'} = $default_compression_level;
2506+ }
2507+
2508+ if ( $o->{'compress'} == 999 ) {
2509+ # compress option not given in the command line
2510+ $o->{'compress'} = 0;
2511+ }
2512+
2513+ if ( @ARGV < 1 ) {
2514+ print STDERR "$prefix Missing command line argument\n";
2515+ usage();
2516+ exit(1);
2517+ }
2518+ elsif ( @ARGV > 1 ) {
2519+ print STDERR "$prefix Too many command line arguments\n";
2520+ usage();
2521+ exit(1);
2522+ }
2523+
2524+ if ( $o->{'stream'} ) {
2525+ if ( $o->{'stream'} eq 'tar' ) {
2526+ }
2527+ elsif ( $o->{'stream'} eq 'tar4ibd' ) {
2528+ $o->{'stream'} = 'tar';
2529+ $o->{'tar4ibd'} = 'tar4ibd';
2530+ }
2531+ elsif ( $o->{'stream'} eq 'cpio' ) {
2532+ print STDERR "$prefix --stream=cpio is not supported yet\n";
2533+ exit 1;
2534+ }
2535+ else {
2536+ print STDERR "$prefix Unknown option --stream=$o->{'stream'}\n";
2537+ exit 1;
2538+ }
2539+ }
2540+
2541+ # get options file name
2542+ #$config_file = $ARGV[0];
2543+
2544+ if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) {
2545+ # we are making a backup, get backup root directory
2546+ $backup_root = $ARGV[0];
2547+ if ( $o->{'incremental'} ) {
2548+ if ( $o->{'remote-host'} ) {
2549+ print STDERR "--incremental does not work with --remote-host.\n";
2550+ exit 1;
2551+ }
2552+ my @dirs = `ls -1 -t $backup_root`;
2553+ my $inc_dir = $dirs[0];
2554+ chomp($inc_dir);
2555+ $incremental_basedir = File::Spec->catfile( $backup_root, $inc_dir );
2556+
2557+ #print STDERR "--incremental_basedir=$incremental_basedir\n";
2558+ #print STDERR "incremental backup is not supported for now.\n";
2559+ #exit(1);
2560+ }
2561+ if( $o->{'remote-host'} ) {
2562+ my ($host, $dir) = split( ':', $o->{'remote-host'} );
2563+ if ( $dir ) {
2564+ print STDERR "--remote-host should not have a ':directory' part.\n"
2565+ . "Specify the remote host directory asBACK-ROOT-DIR.";
2566+ exit 1;
2567+ }
2568+ }
2569+ }
2570+ else {
2571+ # get backup directory
2572+ $backup_dir = File::Spec->rel2abs( $ARGV[0] );
2573+ }
2574+
2575+ parse_databases_option_value();
2576+
2577+ return;
2578+}
2579+
2580+#
2581+# make_backup_dir subroutine creates a new backup directory and returns
2582+# its name.
2583+#
2584+sub make_backup_dir {
2585+ my $dir;
2586+ my $innodb_data_file_path =
2587+ get_option( \%config, 'mysqld', 'innodb_data_file_path' );
2588+
2589+ # create backup directory
2590+ $dir = $backup_root;
2591+ if ( $o->{'stream'} ) {
2592+ return $dir;
2593+ }
2594+
2595+ $dir .= '/' . strftime( "%Y-%m-%d_%H-%M-%S", localtime() )
2596+ unless $o->{'no-timestamp'};
2597+ if ( !$o->{'remote-host'} ) {
2598+ mkdir( $dir, 0777 )
2599+ || Die "Failed to create backup directory $dir: $OS_ERROR";
2600+ }
2601+ else {
2602+ system_call("ssh $o->{'remote-host'} mkdir '$dir'");
2603+ }
2604+
2605+ # create subdirectories for ibdata files if needed
2606+ # foreach my $a (split(/;/, $innodb_data_file_path)) {
2607+ # my $path = (split(/:/,$a))[0];
2608+ # my @relative_path = split(/\/+/, $path);
2609+ # pop @relative_path;
2610+ # if (@relative_path) {
2611+ # # there is a non-trivial path from the backup directory
2612+ # # to the directory of this backup ibdata file, check
2613+ # # that all the directories in the path exist.
2614+ # create_path_if_needed($dir, \@relative_path);
2615+ # }
2616+ # }
2617+
2618+ return $dir;
2619+}
2620+
2621+#
2622+# create_path_if_needed subroutine checks that all components
2623+# in the given relative path are directories. If the
2624+# directories do not exist, they are created.
2625+# Parameters:
2626+# root a path to the root directory of the relative pathname
2627+# relative_path a relative pathname (a reference to an array of
2628+# pathname components)
2629+#
2630+sub create_path_if_needed {
2631+ my $root = shift;
2632+ my $relative_path = shift;
2633+ my $path;
2634+
2635+ $path = $root;
2636+ foreach $a ( @{$relative_path} ) {
2637+ $path = $path . "/" . $a;
2638+ if ( !$o->{'remote-host'} ) {
2639+ if ( !-d $path ) {
2640+
2641+ # this directory does not exist, create it !
2642+ mkdir( $path, 0777 )
2643+ || Die "Failed to create backup directory: $OS_ERROR";
2644+ }
2645+ }
2646+ else {
2647+ if ( system_call("ssh $o->{'remote-host'} test -d '$path'", 1) != 0 ) {
2648+ system_call("ssh $o->{'remote-host'} mkdir $path");
2649+ }
2650+ }
2651+ }
2652+}
2653+
2654+#
2655+# remove_from_array subroutine removes excluded element from the array.
2656+# Parameters:
2657+# array_ref a reference to an array of strings
2658+# excluded a string to be excluded from the copy
2659+#
2660+sub remove_from_array {
2661+ my $array_ref = shift;
2662+ my $excluded = shift;
2663+ my @copy = ();
2664+ my $size = 0;
2665+
2666+ foreach my $str ( @{$array_ref} ) {
2667+ if ( $str ne $excluded ) {
2668+ $copy[$size] = $str;
2669+ $size = $size + 1;
2670+ }
2671+ }
2672+ @{$array_ref} = @copy;
2673+}
2674+
2675+#
2676+# backup_files subroutine copies .frm, .MRG, .MYD and .MYI files to
2677+# backup directory.
2678+#
2679+sub backup_files {
2680+ my $source_dir = get_option( \%config, 'mysqld', 'datadir' );
2681+ my @list;
2682+ my $file;
2683+ my $database;
2684+ my $wildcard = '*.{frm,MYD,MYI,MRG,TRG,TRN,ARM,ARZ,opt,par}';
2685+
2686+ opendir( DIR, $source_dir )
2687+ or Die "Cannot open directory '$source_dir': $OS_ERROR\n";
2688+ $now = current_time();
2689+ print STDERR
2690+ "\n$now $prefix Starting to backup .frm, .MRG, .MYD, .MYI,\n";
2691+ print STDERR "$prefix .TRG, .TRN, .ARM, .ARZ and .opt files in\n";
2692+ print STDERR "$prefix subdirectories of '$source_dir'\n";
2693+
2694+ # loop through all database directories
2695+ while ( defined( $database = readdir(DIR) ) ) {
2696+ my $print_each_file = 0;
2697+ my $file_c;
2698+ my @scp_files;
2699+
2700+ # skip files that are not database directories
2701+ next if $database eq '.' || $database eq '..';
2702+ next unless -d "$source_dir/$database";
2703+ next unless check_if_required($database);
2704+
2705+ IBEXDEBUG && _d('Working in database', "$source_dir/$database");
2706+
2707+ if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
2708+ if ( !-e "$backup_dir/$database" ) {
2709+
2710+ # create database directory for the backup
2711+ mkdir( "$backup_dir/$database", 0777 )
2712+ or Die "Cannot create directory '$backup_dir/$database': "
2713+ . $OS_ERROR;
2714+ }
2715+ }
2716+ elsif ( $o->{'remote-host'} ) {
2717+ system_call("ssh $o->{'remote-host'} mkdir '$backup_dir/$database'");
2718+ }
2719+
2720+ # copy files of this database
2721+ opendir( DBDIR, "$source_dir/$database" );
2722+ @list = grep(
2723+ /\.(frm)|(MYD)|(MYI)|(MRG)|(TRG)|(TRN)|(ARM)|(ARZ)|(opt)|(par)$/,
2724+ readdir(DBDIR) );
2725+ closedir DBDIR;
2726+ $file_c = @list;
2727+ if ( $file_c <= $backup_file_print_limit ) {
2728+ $print_each_file = 1;
2729+ }
2730+ else {
2731+ print STDERR "$prefix Backing up files "
2732+ . "'$source_dir/$database/$wildcard' ($file_c files)\n";
2733+ }
2734+ foreach $file (@list) {
2735+
2736+ # copying may take a long time, so we have to prevent
2737+ # mysql connection from timing out
2738+ mysql_keep_alive();
2739+ next unless check_if_required( $database, $file );
2740+
2741+ if ( $print_each_file ) {
2742+ print STDERR "$prefix Backing up file "
2743+ . "'$source_dir/$database/$file'\n";
2744+ }
2745+
2746+ if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
2747+ $src_name = escape_path("$source_dir/$database/$file");
2748+ $dst_name = escape_path("$backup_dir/$database");
2749+ system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
2750+ }
2751+ elsif ( $o->{'remote-host'} ) {
2752+ # Queue up files for one single scp per database.
2753+ push @scp_files, "'$file'";
2754+ }
2755+ elsif ( $o->{'stream'} eq 'tar' ) {
2756+ my $ret = 0;
2757+ my $file_name = substr( $file, rindex( $file, '/' ) + 1 );
2758+ $file_name =~ s/([\$\\\" ])/\\$1/g;
2759+ $ret = system_call(
2760+ "cd $source_dir; tar cf - $database/$file_name", 1);
2761+ if ( $ret == 1 ) {
2762+ print STDERR "$prefix If you use GNU tar, "
2763+ . "this warning can be ignored.\n";
2764+ }
2765+ elsif ( $ret != 0 ) {
2766+ print STDERR "$prefix tar returned with exit code $ret.\n";
2767+ Die "Failed to stream '$database/$file_name': $OS_ERROR";
2768+ }
2769+ }
2770+ }
2771+ if ( $o->{'remote-host'} ) {
2772+ my $scp_file_list =
2773+ join( " ", map {"$source_dir/$database/$_"} @scp_files );
2774+ system_call("scp $o->{'scp-opt'} $scp_file_list "
2775+ . "'$o->{'remote-host'}:$backup_dir/$database/'");
2776+ }
2777+ }
2778+ closedir(DIR);
2779+
2780+ $now = current_time();
2781+ print STDERR
2782+ "$now $prefix Finished backing up .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ and .opt files\n\n";
2783+
2784+ return;
2785+}
2786+
2787+#
2788+# file_to_array subroutine reads the given text file into an array and
2789+# stores each line as an element of the array. The end-of-line
2790+# character(s) are removed from the lines stored in the array.
2791+# Parameters:
2792+# filename name of a text file
2793+# lines_ref a reference to an array
2794+#
2795+sub file_to_array {
2796+ my $filename = shift;
2797+ my $lines_ref = shift;
2798+
2799+ open( FILE, $filename ) || Die "can't open file '$filename': $OS_ERROR";
2800+ @{$lines_ref} = <FILE>;
2801+ close(FILE) || Die "can't close file '$filename': $OS_ERROR";
2802+
2803+ foreach my $a ( @{$lines_ref} ) {
2804+ chomp($a);
2805+ }
2806+}
2807+
2808+#
2809+# unescape_string subroutine expands escape sequences found in the string and
2810+# returns the expanded string. It also removes possible single or double quotes
2811+# around the value.
2812+# Parameters:
2813+# value a string
2814+# Return value:
2815+# a string with expanded escape sequences
2816+#
2817+sub unescape_string {
2818+ my $value = shift;
2819+ my $result = '';
2820+ my $offset = 0;
2821+
2822+ # remove quotes around the value if they exist
2823+ if ( length($value) >= 2 ) {
2824+ if ( ( substr( $value, 0, 1 ) eq "'" && substr( $value, -1, 1 ) eq "'" )
2825+ || ( substr( $value, 0, 1 ) eq '"'
2826+ && substr( $value, -1, 1 ) eq '"' ) )
2827+ {
2828+ $value = substr( $value, 1, -1 );
2829+ }
2830+ }
2831+
2832+ # expand escape sequences
2833+ while ( $offset < length($value) ) {
2834+ my $pos = index( $value, "\\", $offset );
2835+ if ( $pos < 0 ) {
2836+ $pos = length($value);
2837+ $result = $result . substr( $value, $offset, $pos - $offset );
2838+ $offset = $pos;
2839+ }
2840+ else {
2841+ my $replacement = substr( $value, $pos, 2 );
2842+ my $escape_code = substr( $value, $pos + 1, 1 );
2843+ if ( exists $o->{'value-escapes'}{$escape_code} ) {
2844+ $replacement = $o->{'value-escapes'}{$escape_code};
2845+ }
2846+ $result =
2847+ $result
2848+ . substr( $value, $offset, $pos - $offset )
2849+ . $replacement;
2850+ $offset = $pos + 2;
2851+ }
2852+ }
2853+
2854+ return $result;
2855+}
2856+
2857+#
2858+# read_config_file subroutine reads MySQL options file and
2859+# returns the options in a hash containing one hash per group.
2860+# Parameters:
2861+# filename name of a MySQL options file
2862+# groups_ref a reference to hash variable where the read
2863+# options are returned
2864+#
2865+sub read_config_file {
2866+
2867+ #my $filename = shift;
2868+ my $groups_ref = shift;
2869+ my @lines;
2870+ my $i;
2871+ my $group;
2872+ my $group_hash_ref;
2873+
2874+ my $cmdline = '';
2875+ my $options = '';
2876+
2877+ if ( $o->{'defaults-file'} ) {
2878+ $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" ";
2879+ }
2880+
2881+ $options = $options . "--print-param";
2882+
2883+ # read file to an array, one line per element
2884+ #file_to_array($filename, \@lines);
2885+ $cmdline = "$o->{'ibbackup'} $options";
2886+ @lines = `$cmdline`;
2887+
2888+ # classify lines and save option values
2889+ $group = 'default';
2890+ $group_hash_ref = {};
2891+ ${$groups_ref}{$group} = $group_hash_ref;
2892+
2893+ # this pattern described an option value which may be
2894+ # quoted with single or double quotes. This pattern
2895+ # does not work by its own. It assumes that the first
2896+ # opening parenthesis in this string is the second opening
2897+ # parenthesis in the full pattern.
2898+ my $value_pattern = q/((["'])([^\\\4]|(\\[^\4]))*\4)|([^\s]+)/;
2899+ for ( $i = 0; $i < @lines; $i++ ) {
2900+ SWITCH: for ( $lines[$i] ) {
2901+
2902+ # comment
2903+ /^\s*(#|;)/
2904+ && do { last; };
2905+
2906+ # group
2907+ /^\s*\[(.*)\]/
2908+ && do {
2909+ $group = $1;
2910+ if ( !exists ${$groups_ref}{$group} ) {
2911+ $group_hash_ref = {};
2912+ ${$groups_ref}{$group} = $group_hash_ref;
2913+ }
2914+ else {
2915+ $group_hash_ref = ${$groups_ref}{$group};
2916+ }
2917+ last;
2918+ };
2919+
2920+ # option
2921+ /^\s*([^\s=]+)\s*(#.*)?$/
2922+ && do {
2923+ ${$group_hash_ref}{$1} = '';
2924+ last;
2925+ };
2926+
2927+ # set-variable = option = value
2928+ /^\s*set-variable\s*=\s*([^\s=]+)\s*=\s*($value_pattern)\s*(#.*)?$/
2929+ && do { ${$group_hash_ref}{$1} = unescape_string($2); last; };
2930+
2931+ # option = value
2932+ /^\s*([^\s=]+)\s*=\s*($value_pattern)\s*(#.*)?$/
2933+ && do { ${$group_hash_ref}{$1} = unescape_string($2); last; };
2934+
2935+ # empty line
2936+ /^\s*$/
2937+ && do { last; };
2938+
2939+ # unknown
2940+ print( "$prefix: Warning: Ignored unrecognized line ",
2941+ $i + 1, " in options : '${lines[$i]}'\n" );
2942+ }
2943+ }
2944+}
2945+
2946+#
2947+# get_option subroutine returns the value of given option in the config
2948+# structure. If option is missing, this subroutine calls exit.
2949+# Parameters:
2950+# config_ref a reference to a config data
2951+# group option group name
2952+# option_name name of the option
2953+# Return value:
2954+# option value as a string
2955+#
2956+sub get_option {
2957+ my $config_ref = shift;
2958+ my $group = shift;
2959+ my $o->{'name'} = shift;
2960+ my $group_hash_ref;
2961+
2962+ if ( !exists $config{$group} ) {
2963+ # no group
2964+ print STDERR "$prefix fatal error: no '$group' group in MySQL options\n";
2965+ print STDERR "$prefix fatal error: OR no 'datadir' option in group "
2966+ . "'$group' in MySQL options\n";
2967+ exit(1);
2968+ }
2969+
2970+ $group_hash_ref = ${$config_ref}{$group};
2971+ if ( !exists ${$group_hash_ref}{ $o->{'name'} } ) {
2972+ # no option
2973+ print STDERR "$prefix fatal error: no '$o->{'name'}' option in group "
2974+ . "'$group' in MySQL options\n";
2975+ exit(1);
2976+ }
2977+
2978+ return ${$group_hash_ref}{ $o->{'name'} };
2979+}
2980+
2981+# check_if_required subroutine returns 1 if the specified database and
2982+# table needs to be backed up.
2983+# Parameters:
2984+# $_[0] name of database to be checked
2985+# $_[1] full path of table file (This argument is optional)
2986+# Return value:
2987+# 1 if backup should be done and 0 if not
2988+#
2989+sub check_if_required {
2990+ my $db = $_[0];
2991+ my $db_count = keys %databases_list;
2992+ my $table_path;
2993+ if ( defined $_[1] ) {
2994+ $table_path = $_[1];
2995+ }
2996+ else {
2997+ undef $table_path;
2998+ }
2999+
3000+ if ( $db_count == 0 ) {
3001+ # no databases defined with --databases option, include all databases
3002+ return 1;
3003+ }
3004+ if ( defined $databases_list{$db} ) {
3005+ if ( defined $table_path ) {
3006+
3007+ # get the last component in the table pathname
3008+ my $filename = ( reverse( split( /\//, $table_path ) ) )[0];
3009+
3010+ # get name of the table by removing file suffix
3011+ my $table = ( split( /\./, $filename ) )[0];
3012+
3013+ my $db_hash = $databases_list{$db};
3014+ $db_count = keys %$db_hash;
3015+ if ( $db_count > 0 && !defined $databases_list{$db}->{$table} ) {
3016+
3017+ # --databases option specified, but table is not included
3018+ return 0;
3019+ }
3020+ }
3021+ # include this database and table
3022+ return 1;
3023+ }
3024+ else {
3025+ # --databases option given, but database is not included
3026+ return 0;
3027+ }
3028+}
3029+
3030+# parse_databases_option_value subroutine parses the value of
3031+# --databases option. If the option value begins with a slash
3032+# it is considered a pathname and the option value is read
3033+# from the file.
3034+#
3035+# This subroutine sets the global "databases_list" variable.
3036+#
3037+sub parse_databases_option_value {
3038+ my $item;
3039+
3040+ if ( $o->{'databases'} =~ /^\// ) {
3041+
3042+ # the value of the --databases option begins with a slash,
3043+ # the option value is pathname of the file containing
3044+ # list of databases
3045+ if ( !-f $o->{'databases'} ) {
3046+ Die "can't find file '$o->{'databases'}'";
3047+ }
3048+
3049+ # read from file the value of --databases option
3050+ my @lines;
3051+ file_to_array( $o->{'databases'}, \@lines );
3052+ $o->{'databases'} = join( " ", @lines );
3053+ }
3054+
3055+ # mark each database or database.table definition in the
3056+ # global databases_list.
3057+ foreach $item ( split( /\s/, $o->{'databases'} ) ) {
3058+ my $db = "";
3059+ my $table = "";
3060+ my %hash;
3061+
3062+ if ( $item eq "" ) {
3063+
3064+ # ignore empty strings
3065+ next;
3066+ }
3067+
3068+ # get database and table names
3069+ if ( $item =~ /(\S*)\.(\S*)/ ) {
3070+
3071+ # item is of the form DATABASE.TABLE
3072+ $db = $1;
3073+ $table = $2;
3074+ }
3075+ else {
3076+
3077+ # item is database name, table is undefined
3078+ $db = $item;
3079+ }
3080+
3081+ if ( !defined $databases_list{$db} ) {
3082+
3083+ # create empty hash for the database
3084+ $databases_list{$db} = \%hash;
3085+ }
3086+ if ( $table ne "" ) {
3087+
3088+ # add mapping table --> 1 to the database hash
3089+ my $h = $databases_list{$db};
3090+ $h->{$table} = 1;
3091+ }
3092+ }
3093+}
3094+
3095+sub escape_path {
3096+ my $str = shift;
3097+ if ( $win eq 1 ) {
3098+ $str =~ s/\//\\/g;
3099+ $str =~ s/\\\\/\\/g;
3100+ }
3101+ else {
3102+ $str =~ s/\/\//\//g;
3103+ }
3104+ return $str;
3105+
3106+}
3107+
3108+sub system_call {
3109+ my ( $cmd, $dont_check_retval ) = @_;
3110+ IBEXDEBUG && _d($cmd);
3111+ Die "I need a cmd argument" unless $cmd;
3112+
3113+ my $retval = system($cmd);
3114+ IBEXDEBUG && _d('Exit status:', $retval);
3115+
3116+ return $retval if $dont_check_retval;
3117+
3118+ # A non-zero retval (exit status) means that cmd caused an error.
3119+ if ( $retval ) {
3120+ Die "Failed system call: $cmd\n"
3121+ . "Exit status: $retval\n"
3122+ }
3123+
3124+ return $retval;
3125+}
3126+
3127+sub _d {
3128+ my ( $package, undef, $line ) = caller 0;
3129+ @_ = map { ( my $temp = $_ ) =~ s/\n/\n# /g; $temp; }
3130+ map { defined $_ ? $_ : 'undef' } @_;
3131+ print STDERR "# $package:$line $PID ", join( ' ', @_ ), "\n";
3132+}
3133+
3134+# ############################################################################
3135+# Run the program.
3136+# ############################################################################
3137+if ( !caller ) { exit main(@ARGV); }
3138+
3139+1; # Because this is a module as well as a script.
3140
3141=== added directory 'innobackupex/t'
3142=== added file 'innobackupex/t/001_get_mysql_options.t'
3143--- innobackupex/t/001_get_mysql_options.t 1970-01-01 00:00:00 +0000
3144+++ innobackupex/t/001_get_mysql_options.t 2010-01-22 01:31:15 +0000
3145@@ -0,0 +1,61 @@
3146+#!/usr/bin/perl
3147+
3148+BEGIN {
3149+ die "The XTRABACKUP_TRUNK environment variable is not set."
3150+ unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK};
3151+ unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common";
3152+};
3153+
3154+use strict;
3155+use warnings FATAL => 'all';
3156+use English qw(-no_match_vars);
3157+use Test::More tests => 5;
3158+
3159+use IbexTest;
3160+require "$trunk/innobackupex/innobackupex";
3161+
3162+is(
3163+ innobackupex::get_mysql_options(),
3164+ '--unbuffered --',
3165+ 'No options'
3166+);
3167+
3168+@ARGV = qw(--password foo /tmp);
3169+innobackupex::check_args();
3170+is(
3171+ innobackupex::get_mysql_options(),
3172+ '--password=foo --unbuffered --',
3173+ '--password'
3174+);
3175+
3176+# check_args() doesn't clear opts from previous runs so this
3177+# test inherits --password=foo.
3178+@ARGV = qw(--defaults-file=/etc/my.cnf /tmp);
3179+innobackupex::check_args();
3180+is(
3181+ innobackupex::get_mysql_options(),
3182+ '--defaults-file=/etc/my.cnf --password=foo --unbuffered --',
3183+ '--defaults-file'
3184+);
3185+
3186+@ARGV = qw(--port=12345 --user=bob --password=bar /tmp);
3187+innobackupex::check_args();
3188+is(
3189+ innobackupex::get_mysql_options(),
3190+ '--defaults-file=/etc/my.cnf --port=12345 --user=bob --password=bar --unbuffered --',
3191+ '--port and --user'
3192+);
3193+
3194+# The whole enchilada.
3195+@ARGV = qw(--host=10.0.0.1 /tmp);
3196+innobackupex::check_args();
3197+is(
3198+ innobackupex::get_mysql_options(),
3199+ '--defaults-file=/etc/my.cnf --host=10.0.0.1 --port=12345 --user=bob --password=bar --unbuffered --',
3200+ '--host'
3201+);
3202+
3203+# #############################################################################
3204+# Done.
3205+# #############################################################################
3206+exit;
3207
3208=== added file 'innobackupex/t/100_backup.t'
3209--- innobackupex/t/100_backup.t 1970-01-01 00:00:00 +0000
3210+++ innobackupex/t/100_backup.t 2010-01-22 01:31:15 +0000
3211@@ -0,0 +1,87 @@
3212+#!/usr/bin/perl
3213+
3214+BEGIN {
3215+ die "The XTRABACKUP_TRUNK environment variable is not set."
3216+ unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK};
3217+ unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common";
3218+};
3219+
3220+use strict;
3221+use warnings FATAL => 'all';
3222+use English qw(-no_match_vars);
3223+use Test::More;
3224+
3225+use IbexTest;
3226+use DSNParser;
3227+use Sandbox;
3228+require "$trunk/innobackupex/innobackupex";
3229+
3230+my $dp = new DSNParser();
3231+my $sb = new Sandbox(basedir => '/tmp', DSNParser => $dp);
3232+my $dbh = $sb->get_dbh_for('master');
3233+
3234+if ( !$dbh ) {
3235+ plan skip_all => 'Cannot connect to sandbox master';
3236+}
3237+else {
3238+ plan tests => 9;
3239+}
3240+
3241+my $cnf = '/tmp/12345/my.sandbox.cnf';
3242+my $backup_root_dir = '/tmp/ibex';
3243+
3244+diag(`rm -rf $backup_root_dir >/dev/null`);
3245+
3246+$sb->wipe_clean($dbh);
3247+$sb->load_file('master', "innobackupex/t/samples/basic-tables.sql");
3248+
3249+output(
3250+ sub { innobackupex::main('--defaults-file', $cnf, $backup_root_dir,
3251+ qw(--databases ibex --no-timestamp)) }
3252+);
3253+
3254+ok(
3255+ -d $backup_root_dir,
3256+ "Created backup root dir $backup_root_dir"
3257+);
3258+
3259+ok(
3260+ -d "$backup_root_dir/ibex",
3261+ "Created $backup_root_dir/ibex"
3262+);
3263+
3264+ok(
3265+ !-d "$backup_root_dir/mysql",
3266+ "Did not backup mysql database"
3267+);
3268+
3269+my @files = </tmp/12345/data/ibex/*>;
3270+
3271+is_deeply(
3272+ \@files,
3273+ [qw(
3274+ /tmp/12345/data/ibex/db.opt
3275+ /tmp/12345/data/ibex/t1.frm
3276+ /tmp/12345/data/ibex/t2.frm
3277+ /tmp/12345/data/ibex/t2.MYD
3278+ /tmp/12345/data/ibex/t2.MYI
3279+ )],
3280+ 'Copied all files from ibex db'
3281+);
3282+
3283+foreach my $file ( @files ) {
3284+ my ($file2) = $file =~ m/([\w\.]+)$/;
3285+ my $output = `diff $file $backup_root_dir/ibex/$file2`;
3286+ is(
3287+ $output,
3288+ '',
3289+ "$file2 does not differ"
3290+ );
3291+}
3292+
3293+# #############################################################################
3294+# Done.
3295+# #############################################################################
3296+diag(`rm -rf $backup_root_dir >/dev/null`);
3297+$sb->wipe_clean($dbh);
3298+exit;
3299
3300=== added directory 'innobackupex/t/samples'
3301=== added file 'innobackupex/t/samples/basic-tables.sql'
3302--- innobackupex/t/samples/basic-tables.sql 1970-01-01 00:00:00 +0000
3303+++ innobackupex/t/samples/basic-tables.sql 2010-01-22 01:31:15 +0000
3304@@ -0,0 +1,17 @@
3305+drop database if exists ibex;
3306+create database ibex;
3307+use ibex;
3308+
3309+create table t1 (
3310+ i int,
3311+ unique key (i)
3312+) engine=innodb;
3313+
3314+insert into ibex.t1 values (1),(2),(3);
3315+
3316+create table t2 (
3317+ i int,
3318+ unique key (i)
3319+) engine=myisam;
3320+
3321+insert into ibex.t2 values (4),(5),(6);
3322
3323=== added file 'innobackupex/test-coverage.txt'
3324--- innobackupex/test-coverage.txt 1970-01-01 00:00:00 +0000
3325+++ innobackupex/test-coverage.txt 2010-01-22 01:31:15 +0000
3326@@ -0,0 +1,2854 @@
3327+Reading database from /home/daniel/dev/percona-xtrabackup/innobackupex/t/cover_db
3328+
3329+
3330+---------------------------- ------ ------ ------ ------ ------ ------ ------
3331+File stmt bran cond sub pod time total
3332+---------------------------- ------ ------ ------ ------ ------ ------ ------
3333+...innobackupex/innobackupex 51.7 32.6 33.7 76.4 n/a 100.0 46.0
3334+Total 51.7 32.6 33.7 76.4 n/a 100.0 46.0
3335+---------------------------- ------ ------ ------ ------ ------ ------ ------
3336+
3337+
3338+Run: 100_backup.t
3339+Perl version: 118.53.46.49.48.46.48
3340+OS: linux
3341+Start: Thu Jan 21 23:50:53 2010
3342+Finish: Thu Jan 21 23:50:55 2010
3343+
3344+Run: 100_backup.t
3345+Perl version: 118.53.46.49.48.46.48
3346+OS: linux
3347+Start: Thu Jan 21 23:50:53 2010
3348+Finish: Thu Jan 21 23:51:12 2010
3349+
3350+Run: 001_get_mysql_options.t
3351+Perl version: 118.53.46.49.48.46.48
3352+OS: linux
3353+Start: Thu Jan 21 23:55:00 2010
3354+Finish: Thu Jan 21 23:55:00 2010
3355+
3356+/home/daniel/dev/percona-xtrabackup/innobackupex/innobackupex
3357+
3358+line err stmt bran cond sub pod time code
3359+1 #!/usr/bin/env perl
3360+2
3361+3 # A script for making backups of InnoDB and MyISAM tables, indexes and .frm
3362+4 # files.
3363+5 #
3364+6 # Orginal code copyright 2003, 2009 Innobase Oy.
3365+7 # http://www.innodb.com/download/
3366+8 #
3367+9 # Modified code copyright 2010 Percona Inc.
3368+10 # https://launchpad.net/percona-xtrabackup/trunk
3369+11
3370+12 # ############################################################################
3371+13 # This is a a runnable module. Check at the end of this package for the call
3372+14 # to main() which actually runs the program.
3373+15 # ############################################################################
3374+16 package innobackupex;
3375+17
3376+18 3 3 23 use strict;
3377+ 3 6
3378+ 3 19
3379+19 3 3 20 use warnings FATAL => 'all';
3380+ 3 6
3381+ 3 29
3382+20 3 3 18 use English qw(-no_match_vars);
3383+ 3 7
3384+ 3 17
3385+21 3 3 32 use Getopt::Long qw(:config default);
3386+ 3 12
3387+ 3 23
3388+22 3 3 24 use File::Spec;
3389+ 3 6
3390+ 3 35
3391+23 3 3 47 use POSIX "strftime";
3392+ 3 10
3393+ 3 24
3394+24 3 3 55 use POSIX ":sys_wait_h";
3395+ 3 8
3396+ 3 24
3397+25 3 3 22 use POSIX "tmpnam";
3398+ 3 18
3399+ 3 21
3400+26 3 3 21 use FileHandle;
3401+ 3 9
3402+ 3 32
3403+27 3 3 24 use File::Basename;
3404+ 3 9
3405+ 3 41
3406+28
3407+29 *** 3 50 3 22 use constant IBEXDEBUG => $ENV{IBEXDEBUG} || 0;
3408+ 3 9
3409+ 3 57
3410+30
3411+31 # version of this script
3412+32 my $innobackup_version = '1.5.1-xtrabackup';
3413+33 my $innobackup_script = basename($0);
3414+34
3415+35 # copyright notice
3416+36 my $copyright_notice = "InnoDB Backup Utility v${innobackup_version}; Copyright 2003, 2009 Innobase Oy.
3417+37 All Rights Reserved.
3418+38
3419+39 This software is published under
3420+40 the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
3421+41
3422+42 ";
3423+43
3424+44 # required Perl version (5.005)
3425+45 my @required_perl_version = ( 5, 0, 5 );
3426+46 my $required_perl_version_old_style = 5.005;
3427+47
3428+48 # force flush after every write and print
3429+49 $OUTPUT_AUTOFLUSH = 1;
3430+50
3431+51 ######################################################################
3432+52 # modifiable parameters
3433+53 ######################################################################
3434+54
3435+55 # maximum number of files in a database directory which are
3436+56 # separately printed when a backup is made
3437+57 my $backup_file_print_limit = 9;
3438+58
3439+59 # timeout in seconds for a reply from mysql
3440+60 my $mysql_response_timeout = 900;
3441+61
3442+62 # default compression level (this is an argument to ibbackup)
3443+63 my $default_compression_level = 1;
3444+64
3445+65 # time in seconds after which a dummy query is sent to mysql server
3446+66 # in order to keep the database connection alive
3447+67 my $mysql_keep_alive_timeout = 1800;
3448+68
3449+69 ######################################################################
3450+70 # end of modifiable parameters
3451+71 ######################################################################
3452+72
3453+73 # command line options
3454+74 my $o = {
3455+75 'compress' => 999,
3456+76 'databases' => '',
3457+77 'ibbackup' => 'xtrabackup',
3458+78 'scp-opt' => '-Cp -c arcfour',
3459+79 'stream' => '',
3460+80 };
3461+81
3462+82 # root of the backup directory
3463+83 my $backup_root = '';
3464+84
3465+85 # backup directory pathname
3466+86 my $backup_dir = '';
3467+87
3468+88 # name of the ibbackup suspend-at-end file
3469+89 my $suspend_file = '';
3470+90
3471+91 # name of the temporary transaction log file during the backup
3472+92 my $tmp_logfile = '';
3473+93
3474+94 # home directory of innoDB log files
3475+95 my $innodb_log_group_home_dir = '';
3476+96
3477+97 # backup my.cnf file
3478+98 my $backup_config_file = '';
3479+99
3480+100 # options from the options file
3481+101 my %config;
3482+102
3483+103 # options from the backup options file
3484+104 #my %backup_config;
3485+105
3486+106 # list of databases to be included in a backup
3487+107 my %databases_list;
3488+108
3489+109 # prefix for output lines
3490+110 my $prefix = "$innobackup_script:";
3491+111
3492+112 # process id of mysql client program (runs as a child process of this script)
3493+113 my $mysql_pid = '';
3494+114
3495+115 # mysql server version string
3496+116 my $mysql_server_version = '';
3497+117
3498+118 # name of the file where stderr of mysql process is directed
3499+119 my $mysql_stderr;
3500+120
3501+121 # name of the file where stdout of mysql process is directed
3502+122 my $mysql_stdout;
3503+123
3504+124 # name of the file where binlog position info is written
3505+125 my $binlog_info;
3506+126
3507+127 # name of the file where slave info is written
3508+128 my $slave_info;
3509+129
3510+130 # mysql binlog position as given by "SHOW MASTER STATUS" command
3511+131 my $mysql_binlog_position = '';
3512+132
3513+133 # mysql master's binlog position as given by "SHOW SLAVE STATUS" command
3514+134 # run on a slave server
3515+135 my $mysql_slave_position = '';
3516+136
3517+137 # time of the most recent mysql_check call. (value returned by time() function)
3518+138 my $mysql_last_access_time = 0;
3519+139
3520+140 # process id of ibbackup program (runs as a child process of this script)
3521+141 my $ibbackup_pid = '';
3522+142
3523+143 # a counter for numbering mysql connection checks
3524+144 my $hello_id = 0;
3525+145
3526+146 # the request which has been sent to mysqld, but to which
3527+147 # mysqld has not yet replied. Empty string denotes that no
3528+148 # request has been sent to mysqld or that mysqld has replied
3529+149 # to all requests.
3530+150 my $current_mysql_request = '';
3531+151
3532+152 # escape sequences for options files
3533+153 my %option_value_escapes = (
3534+154 'b' => "\b",
3535+155 't' => "\t",
3536+156 'n' => "\n",
3537+157 'r' => "\r",
3538+158 "\\" => "\\",
3539+159 's' => ' '
3540+160 );
3541+161
3542+162 # signal that is sent to child processes when they are killed
3543+163 my $kill_signal = 15;
3544+164
3545+165 # current local time
3546+166 my $now;
3547+167
3548+168 # incremental backup base directory
3549+169 my $incremental_basedir = '';
3550+170
3551+171 my $src_name;
3552+172 my $dst_name;
3553+173 my $win = ( $^O eq 'MSWin32' ? 1 : 0 );
3554+174 my $CP_CMD = ( $win eq 1 ? "copy /Y" : "cp -p" );
3555+175
3556+176 ######################################################################
3557+177 # program execution begins here
3558+178 ######################################################################
3559+179 sub main {
3560+180 2 2 162 @ARGV = @_; # set global ARGV for this package
3561+181
3562+182 # check command-line args
3563+183 2 34 check_args();
3564+184
3565+185 # print program version and copyright
3566+186 2 14 print_version();
3567+187
3568+188 # initialize global variables and perform some checks
3569+189 2 32 init();
3570+190
3571+191 *** 2 50 26 if ( $o->{'copy-back'} ) {
3572+ *** 50
3573+192 # copy files from backup directory back to their original locations
3574+193 *** 0 0 copy_back();
3575+194 }
3576+195 elsif ( $o->{'apply-log'} ) {
3577+196 # expand data files in backup directory by applying log files to them
3578+197 *** 0 0 apply_log();
3579+198 }
3580+199 else {
3581+200 # make a backup of InnoDB and MyISAM tables, indexes and .frm files.
3582+201 2 40 backup();
3583+202 }
3584+203
3585+204 # program has completed successfully
3586+205 1 7 $now = current_time();
3587+206 1 24 print STDERR "$now $prefix completed OK!\n";
3588+207
3589+208 *** 1 50 14 if ( $o->{'stream'} eq 'tar' ) {
3590+209 *** 0 0 print STDERR "$prefix You must use -i (--ignore-zeros) option for extraction of the tar stream.\n";
3591+210 }
3592+211
3593+212 1 23 return 0;
3594+213 }
3595+214
3596+215 ##############################################################################
3597+216 # Subroutines
3598+217 ##############################################################################
3599+218
3600+219 #
3601+220 # print_version subroutine prints program version and copyright.
3602+221 #
3603+222 sub print_version {
3604+223 2 2 140 printf( STDERR $copyright_notice );
3605+224 }
3606+225
3607+226 #
3608+227 # usage subroutine prints instructions of how to use this program to stdout.
3609+228 #
3610+229 sub usage {
3611+230 *** 0 0 0 print STDOUT <<EOF;
3612+231
3613+232 Usage:
3614+233
3615+234 $innobackup_script BACKUP-ROOT-DIR [OPTIONS]
3616+235
3617+236 $innobackup_script --apply-log BACKUP-DIR [OPTIONS]
3618+237
3619+238 $innobackup_script --copy-back BACKUP-DIR [OPTIONS]
3620+239
3621+240 The first command line above makes a hot backup of a MySQL database.
3622+241 By default it creates a backup directory (named by the current date
3623+242 and time) in the given BACKUP-ROOT-DIR directory. With the --no-timestamp
3624+243 option it does not create a time-stamped backup directory, but it puts
3625+244 the backup in the given directory (which must not exist). This
3626+245 command makes a complete backup of all MyISAM and InnoDB tables and
3627+246 indexes in all databases or in all of the databases specified with the
3628+247 --databases option. The created backup contains .frm, .MRG, .MYD,
3629+248 .MYI, .TRG, .TRN, .ARM, .ARZ, .opt, .par, and InnoDB data and log files.
3630+249 The MY.CNF options file defines the location of the database. This command
3631+250 connects to the MySQL server using mysql client program, and runs
3632+251 ibbackup (InnoDB Hot Backup program) as a child process.
3633+252
3634+253 The command with --apply-log option prepares a backup for starting a MySQL
3635+254 server on the backup. This command expands InnoDB data files as specified
3636+255 in BACKUP-DIR/backup-my.cnf using BACKUP-DIR/ibbackup_logfile,
3637+256 and creates new InnoDB log files as specified in BACKUP-DIR/backup-my.cnf.
3638+257 The BACKUP-DIR should be a path name of a backup directory created by
3639+258 innobackup. This command runs ibbackup as a child process, but it does not
3640+259 connect to the database server.
3641+260
3642+261 The command with --copy-back option copies data, index, and log files
3643+262 from backup directory back to their original locations.
3644+263 The MY.CNF options file defines the original location of the database.
3645+264 The BACKUP-DIR is a path name of a backup directory created by innobackup.
3646+265
3647+266 On success the exit code of innobackup process is 0. A non-zero exit code
3648+267 indicates an error.
3649+268
3650+269 Options:
3651+270
3652+271 --help Display this helpscreen and exit.
3653+272
3654+273 --version Print version information and exit.
3655+274
3656+275 --defaults-file=[MY.CNF]
3657+276 This option is passed to the xtrabackup child process
3658+277 and the mysql child process. It specifies which file
3659+278 to read default options from.
3660+279
3661+280 --apply-log
3662+281 Prepare a backup for starting mysql server on the backup.
3663+282 Expand InnoDB data files as specified in
3664+283 backup-dir/backup-my.cnf, using backup-dir/ibbackup_logfile,
3665+284 and create new log files as specified in
3666+285 backup-dir/backup-my.cnf.
3667+286
3668+287 --copy-back
3669+288 Copy data and index files from backup directory back to
3670+289 their original locations.
3671+290
3672+291 --remote-host=[USER@]HOSTNAME
3673+292 If this option is specified, backup files will be created
3674+293 at the remote host by using ssh and scp. BACKUP-ROOT-DIR
3675+294 is relative to this host.
3676+295
3677+296 --stream=[tar|cpio(not implemented)]
3678+297 If this option is specified, backup to STDOUT as specified
3679+298 format.
3680+299
3681+300 --tmpdir=DIRECTORY
3682+301 When --remote-host or --stream specified, transaction log
3683+302 should be stored temporary. The default value is same to
3684+303 mysqld setting.
3685+304
3686+305 --use-memory=MB
3687+306 This option is passed to the ibbackup child process.
3688+307 It tells ibbackup that it can use MB megabytes of
3689+308 memory in restoration.
3690+309 Try 'ibbackup --help' for more details on this option.
3691+310
3692+311 --throttle=IOS
3693+312 This option is passed to the xtrabackup child process.
3694+313 It limits count of IO operations (pairs of read&write) per
3695+314 second to IOS values (for '--backup')
3696+315
3697+316 --sleep=MS
3698+317 This option is passed to the ibbackup child process.
3699+318 It instructs the ibbackup program to sleep
3700+319 MS milliseconds after each 1 MB of copied data.
3701+320 You can use this parameter to tune the additional
3702+321 disk i/o load the ibbackup program causes on the computer.
3703+322 Try 'ibbackup --help' for more details on this option.
3704+323 ** it is not supported by xtrabackup **
3705+324
3706+325 --compress[=LEVEL]
3707+326 This option is passed to the ibbackup child process.
3708+327 It instructs ibbackup to compress the backup copies of
3709+328 InnoDB data files. Compression level can be
3710+329 specified as an optional argument. Compression level is
3711+330 an integer between 0 and 9: 1 gives fastest compression,
3712+331 9 gives best compression, and 0 means no compression.
3713+332 If compression level is not given, the default level 1 is used.
3714+333 Try 'ibbackup --help' for more details on this option.
3715+334 ** it is not supported by xtrabackup yet **
3716+335
3717+336 --include=REGEXP
3718+337 This option is passed to the ibbackup child process.
3719+338 It tells ibbackup to backup only those per-table data
3720+339 files which match the given regular expression. For
3721+340 each table with a per-table data file a string of the
3722+341 form db_name.table_name is checked against the regular
3723+342 expression. If the regular expression matches the
3724+343 complete string db_name.table_name, the table is
3725+344 included in the backup. The regular expression should
3726+345 be of the POSIX 1003.2 "extended" form.
3727+346 Try 'ibbackup --help' for more details on this option.
3728+347
3729+348 --databases=LIST
3730+349 This option is used to specify the list of databases that
3731+350 innobackup should backup. The list is of the form
3732+351 "db_name[.table_name] db_name1[.table_name1] ...".
3733+352 If this option is not specified all databases containing
3734+353 MyISAM and InnoDB tables will be backed up.
3735+354 Please make sure that --databases contains all of the
3736+355 innodb databases & tables so that all of the innodb .frm
3737+356 files are also backed up. In case the list is very long,
3738+357 this can be specified in a file and the full path of the
3739+358 file can be specified instead of the list.
3740+359
3741+360 --uncompress
3742+361 This option is passed to the ibbackup child process.
3743+362 It tells ibbackup to uncompress compressed InnoDB data files.
3744+363 Try 'ibbackup --help' for more details on this option.
3745+364 ** it is not supported by xtrabackup yet **
3746+365
3747+366 --export
3748+367 This option is passed to the xtrabackup child process.
3749+368 It is effective with '--prepare' option.
3750+369 It tells xtrabackup to output "clean" .ibd files and .exp
3751+370 files for 'ALTER TABLE ... IMPORT TABLESPACE' command
3752+371 at innodb_expand_import option enabled XtraDB.
3753+372
3754+373 --user=NAME
3755+374 This option is passed to the mysql child process.
3756+375 It defines the user for database login if not current user.
3757+376 Try 'mysql --help' for more details on this option.
3758+377
3759+378 --password=WORD
3760+379 This option is passed to the mysql child process.
3761+380 It defines the password to use when connecting to database.
3762+381 Try 'mysql --help' for more details on this option.
3763+382
3764+383 --host=HOST
3765+384 This option is passed to the mysql child process.
3766+385 It defines the hostname or IP address to use when connecting
3767+386 to the database server.
3768+387 Try 'mysql --help' for more details on this option.
3769+388
3770+389 --port=PORT
3771+390 This option is passed to the mysql child process.
3772+391 It defines the port to use when connecting to local database
3773+392 server with TCP/IP.
3774+393 Try 'mysql --help' for more details on this option.
3775+394
3776+395 --slave-info
3777+396 This option is useful when backing up a replication
3778+397 slave server. It prints the binary log position and
3779+398 name of the binary log file of the master server.
3780+399 It also writes this information to the 'ibbackup_slave_info'
3781+400 file as a 'CHANGE MASTER' command. A new slave for this
3782+401 master can be set up by starting a slave server on this
3783+402 backup and issuing a 'CHANGE MASTER' command with the binary
3784+403 log position saved in the 'ibbackup_slave_info' file.
3785+404
3786+405 --socket=SOCKET
3787+406 This option is passed to the mysql child process.
3788+407 It defines the socket to use when connecting to local database
3789+408 server with UNIX domain socket.
3790+409 Try 'mysql --help' for more details on this option.
3791+410
3792+411 --no-timestamp
3793+412 This option prevents the creation of a new backup
3794+413 directory (named by the current date and time) under
3795+414 the backup root directory. Instead, the backup is put
3796+415 in the directory given on the command-line (in the
3797+416 place of BACKUP-ROOT-DIR argument). The directory must not
3798+417 exist, because innobackup creates it while making a backup.
3799+418
3800+419 --ibbackup=IBBACKUP-BINARY
3801+420 Use this option to specify which ibbackup (InnoDB Hot
3802+421 Backup) binary should be used. IBBACKUP-BINARY
3803+422 should be the command used to run ibbackup. This can
3804+423 be useful if ibbackup is not in your search path or
3805+424 working directory. If this option is not given,
3806+425 ibbackup is run with command "ibbackup".
3807+426 --no-lock
3808+427 Use this option to disable table lock with
3809+428 FLUSH TABLES WITH READ LOCK. use it only while ALL your
3810+429 tables are InnoDB and you DO NOT CARE about binary log
3811+430 position of backup.
3812+431
3813+432 --scpopt=SCP-OPTIONS
3814+433 Use this option to specify the command line options
3815+434 to pass to scp. The default options are '-Cp -c arcfour'.
3816+435
3817+436 EOF
3818+437
3819+438 *** 0 0 print STDOUT "Option values after processing arguments:\n\n";
3820+439 *** 0 0 0 print STDOUT
3821+440 *** 0 0 join( "\n", sort map { " --$_=" . $o->{$_} || '' } keys %$o ),
3822+441 "\n\n";
3823+442
3824+443 *** 0 0 return;
3825+444 }
3826+445
3827+446 #
3828+447 # return current local time as string in form "070816 12:23:15"
3829+448 #
3830+449 sub current_time {
3831+450 18 18 1294 return strftime( "%y%m%d %H:%M:%S", localtime() );
3832+451 }
3833+452
3834+453 #
3835+454 # Die subroutine kills all child processes and exits this process.
3836+455 # This subroutine takes the same argument as the built-in die function.
3837+456 # Parameters:
3838+457 # message string which is printed to stdout
3839+458 #
3840+459 sub Die {
3841+460 *** 0 0 0 my $message = shift;
3842+461 *** 0 0 my $extra_info = '';
3843+462
3844+463 # kill all child processes of this process
3845+464 *** 0 0 kill_child_processes();
3846+465
3847+466 *** 0 0 0 if ($current_mysql_request) {
3848+467 *** 0 0 $extra_info = " while waiting for reply to MySQL request:"
3849+468 . " '$current_mysql_request'";
3850+469 }
3851+470
3852+471 *** 0 0 IBEXDEBUG && _d('Die', $prefix, $message, $extra_info);
3853+472 *** 0 0 die "$prefix Error: $message$extra_info";
3854+473 }
3855+474
3856+475 #
3857+476 # backup subroutine makes a backup of InnoDB and MyISAM tables, indexes and
3858+477 # .frm files. It connects to the database server and runs ibbackup as a child
3859+478 # process.
3860+479 #
3861+480 sub backup {
3862+481 2 2 24 my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' );
3863+482
3864+483 # check that we can connect to the database. This done by
3865+484 # connecting, issuing a query, and closing the connection.
3866+485 2 36 mysql_open();
3867+486 2 30 mysql_check();
3868+487 2 452 mysql_close();
3869+488
3870+489 # start ibbackup as a child process
3871+490 2 20 start_ibbackup();
3872+491
3873+492 # wait for ibbackup to suspend itself
3874+493 *** 1 50 33 39 if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
3875+494 1 44 wait_for_ibbackup_suspend();
3876+495 }
3877+496
3878+497 *** 1 50 18 if ( !$o->{'incremental'} ) {
3879+498 1 15 mysql_open();
3880+499 1 660 mysql_check();
3881+500 *** 1 50 51 mysql_lockall() unless $o->{'no-lock'};
3882+501 1 11 backup_files();
3883+502 }
3884+503
3885+504 # resume ibbackup and wait till it has finished
3886+505 1 23 resume_ibbackup();
3887+506
3888+507 *** 1 50 44 if ( !$o->{'incremental'} ) {
3889+508 *** 1 50 18 mysql_unlockall() unless $o->{'no-lock'};
3890+509 1 20 mysql_close();
3891+510 }
3892+511
3893+512 *** 1 50 32 if ( $o->{'remote-host'} ) {
3894+ *** 50
3895+513 *** 0 0 system_call("scp $o->{'scp-opt'} '$tmp_logfile' "
3896+514 . "'$o->{'remote-host'}:$backup_dir/xtrabackup_logfile'");
3897+515
3898+516 *** 0 0 system_call("rm -f '$tmp_logfile'");
3899+517
3900+518 *** 0 0 system_call("scp $o->{'scp-opt'} '$orig_datadir/xtrabackup_checkpoints' "
3901+519 . "'$o->{'remote-host'}:$backup_dir/xtrabackup_checkpoints'");
3902+520
3903+521 *** 0 0 system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'");
3904+522 }
3905+523 elsif ( $o->{'stream'} eq 'tar' ) {
3906+524 *** 0 0 system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_logfile");
3907+525
3908+526 *** 0 0 system_call("rm -f '$tmp_logfile'");
3909+527
3910+528 *** 0 0 system_call("cd $orig_datadir; tar chf - xtrabackup_checkpoints");
3911+529
3912+530 *** 0 0 system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'");
3913+531 }
3914+532
3915+533 *** 1 50 35 print STDERR "\n$prefix Backup created in directory "
3916+534 . ($o->{'remote-host'} ? "$o->{'remote-host'}:" : '')
3917+535 . "'$backup_dir'\n";
3918+536
3919+537 *** 1 50 9 if ($mysql_binlog_position) {
3920+538 1 40 print STDERR "$prefix MySQL binlog position: $mysql_binlog_position\n";
3921+539 }
3922+540 *** 1 50 33 13 if ( $mysql_slave_position && $o->{'slave-info'} ) {
3923+541 *** 0 0 print STDERR "$prefix MySQL slave binlog position: "
3924+542 . "$mysql_slave_position\n";
3925+543 }
3926+544
3927+545 1 16 return;
3928+546 }
3929+547
3930+548 #
3931+549 # are_equal_innodb_data_file_paths subroutine checks if the given
3932+550 # InnoDB data file option values are equal.
3933+551 # Parameters:
3934+552 # str1 InnoDB data file path option value
3935+553 # str2 InnoDB data file path option value
3936+554 # Return value:
3937+555 # 1 if values are equal
3938+556 # 0 otherwise
3939+557 #
3940+558 sub are_equal_innodb_data_file_paths {
3941+559 *** 0 0 0 my $str1 = shift;
3942+560 *** 0 0 my $str2 = shift;
3943+561 *** 0 0 my @array1 = split( /;/, $str1 );
3944+562 *** 0 0 my @array2 = split( /;/, $str2 );
3945+563
3946+564 *** 0 0 0 if ( $#array1 != $#array2 ) { return 0; }
3947+ *** 0 0
3948+565
3949+566 for ( my $i = 0; $i <= $#array1; $i++ ) {
3950+567 *** 0 0 my @def1 = split( /:/, $array1[$i] );
3951+568 *** 0 0 my @def2 = split( /:/, $array2[$i] );
3952+569
3953+570 *** 0 0 0 if ( $#def1 != $#def2 ) { return 0; }
3954+ *** 0 0
3955+571
3956+572 for ( my $j = 0; $j <= $#def1; $j++ ) {
3957+573 *** 0 0 0 if ( $def1[$j] ne $def2[$j] ) { return 0; }
3958+ *** 0 0
3959+574 *** 0 0 }
3960+575 *** 0 0 }
3961+576 *** 0 0 return 1;
3962+577 }
3963+578
3964+579 #
3965+580 # is_in_array subroutine checks if the given string is in the array.
3966+581 # Parameters:
3967+582 # str a string
3968+583 # array_ref a reference to an array of strings
3969+584 # Return value:
3970+585 # 1 if string is in the array
3971+586 # 0 otherwise
3972+587 #
3973+588 sub is_in_array {
3974+589 *** 0 0 0 my $str = shift;
3975+590 *** 0 0 my $array_ref = shift;
3976+591
3977+592 *** 0 0 0 if ( grep { $str eq $_ } @{$array_ref} ) {
3978+ *** 0 0
3979+ *** 0 0
3980+593 *** 0 0 return 1;
3981+594 }
3982+595 *** 0 0 return 0;
3983+596 }
3984+597
3985+598 #
3986+599 # copy_back subroutine copies data and index files from backup directory
3987+600 # back to their original locations.
3988+601 #
3989+602 sub copy_back {
3990+603 *** 0 0 0 my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' );
3991+604 *** 0 0 my $orig_ibdata_dir =
3992+605 get_option( \%config, 'mysqld', 'innodb_data_home_dir' );
3993+606 *** 0 0 my $orig_innodb_data_file_path =
3994+607 get_option( \%config, 'mysqld', 'innodb_data_file_path' );
3995+608 *** 0 0 my $orig_iblog_dir =
3996+609 get_option( \%config, 'mysqld', 'innodb_log_group_home_dir' );
3997+610 *** 0 0 my $excluded_files =
3998+611 '^(\.\.?|backup-my\.cnf|xtrabackup_logfile|mysql-std(err|out)|.*\.ibz)$';
3999+612 *** 0 0 my @ibdata_files;
4000+613 *** 0 0 my $iblog_files = '^ib_logfile.*$';
4001+614 *** 0 0 my $compressed_data_file = '.*\.ibz$';
4002+615 *** 0 0 my $file;
4003+616 *** 0 0 my $backup_innodb_data_file_path;
4004+617
4005+618 # check that original data directory exists
4006+619 *** 0 0 0 if ( !-d $orig_datadir ) {
4007+620 *** 0 0 Die "Original data directory '$orig_datadir' does not exist!";
4008+621 }
4009+622
4010+623 # check that original InnoDB data directory exists
4011+624 *** 0 0 0 if ( !-d $orig_ibdata_dir ) {
4012+625 *** 0 0 Die "Original InnoDB data directory '$orig_ibdata_dir' does not exist!";
4013+626 }
4014+627
4015+628 # check that original InnoDB log directory exists
4016+629 *** 0 0 0 if ( !-d $orig_iblog_dir ) {
4017+630 *** 0 0 Die "Original InnoDB log directory '$orig_iblog_dir' does not exist!";
4018+631 }
4019+632
4020+633 # check that the original options file and the backup options file have
4021+634 # the same value for "innodb_data_file_path" option
4022+635 #$backup_innodb_data_file_path =
4023+636 # get_option(\%backup_config, 'mysqld', 'innodb_data_file_path');
4024+637 #if (!are_equal_innodb_data_file_paths($orig_innodb_data_file_path,
4025+638 # $backup_innodb_data_file_path)
4026+639 #) {
4027+640 # Die "The value of 'innodb_data_file_path' option in the original "
4028+641 # . "my.cnf file '$config_file' is different from the value "
4029+642 # . "in the backup my.cnf file '$backup_config_file'.\n(original: "
4030+643 # . "'$orig_innodb_data_file_path')\n"
4031+644 # . "(backup: '$backup_innodb_data_file_path')";
4032+645 #}
4033+646
4034+647 # make a list of all ibdata files in the backup directory and all
4035+648 # directories in the backup directory under which there are ibdata files
4036+649 *** 0 0 foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) {
4037+650 *** 0 0 my $path = ( split( /:/, $a ) )[0];
4038+651 *** 0 0 my $filename = ( split( /\/+/, $path ) )[-1];
4039+652
4040+653 # check that the backup data file exists
4041+654 *** 0 0 0 if ( !-e "$backup_dir/$filename" ) {
4042+655 *** 0 0 0 if ( -e "$backup_dir/${filename}.ibz" ) {
4043+656 *** 0 0 Die
4044+657 "Backup data file '$backup_dir/$filename' does not exist, but "
4045+658 . "its compressed copy '${path}.ibz' exists. Check that you "
4046+659 . "have run '$innobackup_script --apply-log --uncompress "
4047+660 . "...' before attempting '$innobackup_script --copy-back ...' !";
4048+661 }
4049+662 else {
4050+663 *** 0 0 Die "Backup data file '$backup_dir/$filename' does not exist.";
4051+664 }
4052+665 }
4053+666
4054+667 *** 0 0 0 if ( !is_in_array( $filename, \@ibdata_files ) ) {
4055+668 *** 0 0 push( @ibdata_files, $filename );
4056+669 }
4057+670 }
4058+671
4059+672 # copy files from backup dir to their original locations
4060+673
4061+674 # copy files to original data directory
4062+675 *** 0 0 0 opendir( DIR, $backup_dir )
4063+676 || Die "Can't open directory '$backup_dir': $OS_ERROR\n";
4064+677 *** 0 0 print STDERR "$prefix Starting to copy MyISAM tables, indexes,\n";
4065+678 *** 0 0 print STDERR
4066+679 "$prefix .MRG, .TRG, .TRN, .ARM, .ARZ, .opt, and .frm files\n";
4067+680 *** 0 0 print STDERR "$prefix in '$backup_dir'\n";
4068+681 *** 0 0 print STDERR "$prefix back to original data directory '$orig_datadir'\n";
4069+682 *** 0 0 while ( defined( $file = readdir(DIR) ) ) {
4070+683 *** 0 0 0 if ( $file =~ /$excluded_files/ ) { next; }
4071+ *** 0 0
4072+684 *** 0 0 0 if ( is_in_array( $file, \@ibdata_files ) ) { next; }
4073+ *** 0 0
4074+685 *** 0 0 0 if ( $file =~ /$iblog_files/ ) { next; }
4075+ *** 0 0
4076+686 *** 0 0 0 if ( -d "$backup_dir/$file" ) {
4077+687 *** 0 0 my $subdir = "$backup_dir/$file";
4078+688 *** 0 0 my $file2;
4079+689
4080+690 *** 0 0 print STDERR "$prefix Copying directory '$subdir'\n";
4081+691 *** 0 0 0 if ( !-x "\"" . escape_path("$orig_datadir/$file") . "\"" ) {
4082+692 *** 0 0 system_call("mkdir \"" . escape_path("$orig_datadir/$file") . "\"");
4083+693 }
4084+694 *** 0 0 0 opendir( SUBDIR, "$subdir" )
4085+695 || Die "Can't open directory '$subdir': $OS_ERROR\n";
4086+696 *** 0 0 while ( defined( $file2 = readdir(SUBDIR) ) ) {
4087+697 *** 0 0 0 if ( -d "$subdir/$file2" ) { next; }
4088+ *** 0 0
4089+698 *** 0 0 0 if ( $file2 =~ /$compressed_data_file/ ) { next; }
4090+ *** 0 0
4091+699 *** 0 0 $src_name = escape_path("$subdir/$file2");
4092+700 *** 0 0 $dst_name = escape_path("$orig_datadir/$file");
4093+701 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
4094+702 }
4095+703 *** 0 0 closedir(SUBDIR);
4096+704 }
4097+705 else {
4098+706 *** 0 0 print STDERR "$prefix Copying file " . "'$backup_dir/$file'\n";
4099+707 *** 0 0 $src_name = escape_path("$backup_dir/$file");
4100+708 *** 0 0 $dst_name = escape_path("$orig_datadir");
4101+709 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
4102+710 }
4103+711 }
4104+712 *** 0 0 closedir(DIR);
4105+713
4106+714 # copy InnoDB data files to original InnoDB data directory
4107+715 *** 0 0 print STDERR "\n$prefix Starting to copy InnoDB tables and indexes\n";
4108+716 *** 0 0 print STDERR "$prefix in '$backup_dir'\n";
4109+717 *** 0 0 print STDERR
4110+718 "$prefix back to original InnoDB data directory '$orig_ibdata_dir'\n";
4111+719 *** 0 0 foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) {
4112+720
4113+721 # get the relative pathname of a data file
4114+722 *** 0 0 my $path = ( split( /:/, $a ) )[0];
4115+723 *** 0 0 my $filename = ( split( /\/+/, $path ) )[-1];
4116+724 *** 0 0 print STDERR "$prefix Copying file '$backup_dir/$filename'\n";
4117+725 *** 0 0 $src_name = escape_path("$backup_dir/$filename");
4118+726 *** 0 0 $dst_name = escape_path("$orig_ibdata_dir/$path");
4119+727 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
4120+728 }
4121+729
4122+730 # copy InnoDB log files to original InnoDB log directory
4123+731 *** 0 0 0 opendir( DIR, $backup_dir )
4124+732 || Die "Can't open directory '$backup_dir': $OS_ERROR\n";
4125+733 *** 0 0 print STDERR "\n$prefix Starting to copy InnoDB log files\n";
4126+734 *** 0 0 print STDERR "$prefix in '$backup_dir'\n";
4127+735 *** 0 0 print STDERR
4128+736 "$prefix back to original InnoDB log directory '$orig_iblog_dir'\n";
4129+737 *** 0 0 while ( defined( $file = readdir(DIR) ) ) {
4130+738 *** 0 0 0 0 if ( $file =~ /$iblog_files/ && -f "$backup_dir/$file" ) {
4131+739 *** 0 0 print STDERR "$prefix Copying file '$backup_dir/$file'\n";
4132+740 *** 0 0 $src_name = escape_path("$backup_dir/$file");
4133+741 *** 0 0 $dst_name = escape_path("$orig_iblog_dir");
4134+742 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\"");
4135+743 }
4136+744 }
4137+745 *** 0 0 closedir(DIR);
4138+746
4139+747 *** 0 0 print STDERR "$prefix Finished copying back files.\n\n";
4140+748
4141+749 *** 0 0 return;
4142+750 }
4143+751
4144+752 #
4145+753 # apply_log subroutine prepares a backup for starting database server
4146+754 # on the backup. It applies InnoDB log files to the InnoDB data files.
4147+755 #
4148+756 sub apply_log {
4149+757 *** 0 0 0 my $rcode;
4150+758 *** 0 0 my $cmdline = '';
4151+759 *** 0 0 my $options = '';
4152+760
4153+761 *** 0 0 0 if ( $o->{'defaults-file'} ) {
4154+762 *** 0 0 $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" ";
4155+763 }
4156+764
4157+765 *** 0 0 $options = $options . "--prepare --target-dir=$backup_dir";
4158+766
4159+767 *** 0 0 0 if ( $o->{'uncompress'} ) {
4160+768 *** 0 0 $options = $options . ' --uncompress';
4161+769 }
4162+770 *** 0 0 0 if ( $o->{'export'} ) {
4163+771 *** 0 0 $options = $options . ' --export';
4164+772 }
4165+773 *** 0 0 0 if ( $o->{'use-memory'} ) {
4166+774 *** 0 0 $options = $options . " --use-memory=$o->{'use-memory'}";
4167+775 }
4168+776
4169+777 # run ibbackup as a child process
4170+778 *** 0 0 $cmdline = "$o->{'ibbackup'} $options";
4171+779 *** 0 0 $now = current_time();
4172+780 *** 0 0 print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n\n";
4173+781 *** 0 0 $rcode = system_call($cmdline, 1);
4174+782 *** 0 0 0 Die "\n$prefix ibbackup failed" if $rcode;
4175+783
4176+784 *** 0 0 $now = current_time();
4177+785 *** 0 0 print STDERR
4178+786 "\n$now $prefix Restarting xtrabackup with command: $cmdline\nfor creating ib_logfile*\n\n";
4179+787 *** 0 0 $rcode = system_call($cmdline, 1);
4180+788 *** 0 0 0 Die "\n$prefix xtrabackup (2nd execution) failed" if $rcode;
4181+789
4182+790 *** 0 0 return;
4183+791 }
4184+792
4185+793 #
4186+794 # wait_for_ibbackup_suspend subroutine waits until ibbackup has suspended
4187+795 # itself.
4188+796 #
4189+797 sub wait_for_ibbackup_suspend {
4190+798 1 1 165 print STDERR
4191+799 "$prefix Waiting for ibbackup (pid=$ibbackup_pid) to suspend\n";
4192+800 1 94 print STDERR "$prefix Suspend file '$suspend_file'\n\n";
4193+801 1 17 for ( ;; ) {
4194+802 1 2000223 sleep 2;
4195+803 *** 1 50 59 last if -e $suspend_file;
4196+804
4197+805 # check that ibbackup child process is still alive
4198+806 *** 0 0 0 if ( $ibbackup_pid == waitpid( $ibbackup_pid, &WNOHANG ) ) {
4199+807 *** 0 0 $ibbackup_pid = '';
4200+808 *** 0 0 Die "ibbackup child process has died";
4201+809 }
4202+810 }
4203+811 1 16 $now = current_time();
4204+812 1 200 print STDERR "\n$now $prefix Continuing after ibbackup has suspended\n";
4205+813 }
4206+814
4207+815 #
4208+816 # resume_ibbackup subroutine signals ibbackup to complete its execution
4209+817 # by deleting the 'ibbackup_suspended' file.
4210+818 #
4211+819 sub resume_ibbackup {
4212+820 1 1 222 print STDERR "$prefix Resuming ibbackup\n\n";
4213+821 *** 1 33 69 unlink $suspend_file || Die "Failed to delete '$suspend_file': $OS_ERROR";
4214+822
4215+823 # wait for ibbackup to finish
4216+824 1 302772 waitpid( $ibbackup_pid, 0 );
4217+825 1 22 $ibbackup_pid = '';
4218+826 }
4219+827
4220+828 #
4221+829 # start_ibbackup subroutine spawns a child process running ibbackup
4222+830 # program for backing up InnoDB tables and indexes.
4223+831 #
4224+832 sub start_ibbackup {
4225+833 2 2 12 my $options = '';
4226+834 2 14 my $cmdline = '';
4227+835 2 36 my $pid = undef;
4228+836
4229+837 *** 2 50 30 if ( $o->{'defaults-file'} ) {
4230+838 2 22 $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" ";
4231+839 }
4232+840
4233+841 2 12 $options = $options . "--backup --suspend-at-end";
4234+842
4235+843 *** 2 50 33 62 if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
4236+844 2 26 $options = $options . " --target-dir=$backup_dir";
4237+845 }
4238+846 else {
4239+847
4240+848 #(datadir) for 'xtrabackup_suspended' and 'xtrabackup_checkpoints'
4241+849 *** 0 0 $options = $options . " --log-stream --target-dir=./";
4242+850 }
4243+851
4244+852 # prepare command line for running ibbackup
4245+853 *** 2 50 20 if ( $o->{'throttle'} ) {
4246+854 *** 0 0 $options = $options . " --throttle=$o->{'throttle'}";
4247+855 }
4248+856 *** 2 50 18 if ( $o->{'sleep'} ) {
4249+857 *** 0 0 $options = $options . " --sleep=$o->{'sleep'}";
4250+858 }
4251+859 *** 2 50 16 if ( $o->{'compress'} ) {
4252+860 *** 0 0 $options = $options . " --compress=$o->{'compress'}";
4253+861 }
4254+862 *** 2 50 26 if ( $o->{'use-memory'} ) {
4255+863 *** 0 0 $options = $options . " --use-memory=$o->{'use-memory'}";
4256+864 }
4257+865 *** 2 50 16 if ( $o->{'include'} ) {
4258+866 *** 0 0 $options = $options . " --tables='$o->{'include'}'";
4259+867 }
4260+868 *** 2 50 16 if ( $o->{'incremental'} ) {
4261+869 *** 0 0 $options = $options . " --incremental-basedir='$incremental_basedir'";
4262+870 }
4263+871 2 24 $cmdline = "$o->{'ibbackup'} $options";
4264+872
4265+873 # run ibbackup as a child process
4266+874 2 10 $now = current_time();
4267+875 2 64 print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n";
4268+876 *** 2 50 23639 if ( defined($pid = fork) ) {
4269+877 2 100 64 if ($pid) {
4270+878 # parent process
4271+879 1 34 $ibbackup_pid = $pid;
4272+880
4273+881 # direct copy to remote
4274+882 *** 1 50 33 96 if ( $o->{'remote-host'} || $o->{'stream'} ) {
4275+883 *** 0 0 my $orig_datadir
4276+884 = get_option( \%config, 'mysqld', 'datadir' );
4277+885 *** 0 0 my $orig_ibdata_dir
4278+886 = get_option( \%config, 'mysqld', 'innodb_data_home_dir' );
4279+887 *** 0 0 my $orig_innodb_data_file_path
4280+888 = get_option( \%config, 'mysqld', 'innodb_data_file_path' );
4281+889 *** 0 0 my $subdir;
4282+890 *** 0 0 my @list;
4283+891
4284+892 *** 0 0 0 if ( $o->{'remote-host'} ) {
4285+893 *** 0 0 0 if ( system_call("ssh $o->{'remote-host'} test -e "
4286+894 . "$backup_dir/ib_logfile0", 1) == 0 ) {
4287+895 *** 0 0 print STDERR "$prefix Remove "
4288+896 . "$o->{'remote-host'}:$backup_dir/ib_logfile*\n";
4289+897 *** 0 0 system_call( "ssh $o->{'remote-host'} "
4290+898 . "rm $backup_dir/ib_logfile\*"
4291+899 )
4292+900 }
4293+901 }
4294+902
4295+903 *** 0 0 wait_for_ibbackup_suspend();
4296+904
4297+905 #InnoDB data files from original InnoDB data directory
4298+906 *** 0 0 print STDERR
4299+907 "\n$prefix Starting to backup InnoDB tables and indexes\n";
4300+908 *** 0 0 0 if ( $o->{'remote-host'} ) {
4301+909 *** 0 0 print STDERR "$prefix to '$backup_dir'\n";
4302+910 }
4303+911 *** 0 0 print STDERR
4304+912 "$prefix from original InnoDB data directory '$orig_ibdata_dir'\n";
4305+913 *** 0 0 foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) {
4306+914 *** 0 0 my $path = ( split( /:/, $a ) )[0];
4307+915 *** 0 0 $path =~ s/([\$\\\" ])/\\$1/g;
4308+916 *** 0 0 0 if ( $o->{'remote-host'} ) {
4309+ *** 0
4310+917 *** 0 0 print STDERR
4311+918 "$prefix Backing up file '$orig_ibdata_dir/$path'\n";
4312+919 *** 0 0 system_call("scp $o->{'scp-opt'} '$orig_ibdata_dir/$path' "
4313+920 . "'$o->{'remote-host'}:$backup_dir/$path'");
4314+921 }
4315+922 elsif ( $o->{'stream'} eq 'tar' ) {
4316+923 *** 0 0 my $ret = 0;
4317+924 *** 0 0 print STDERR "$prefix Backing up as tar stream '$path'\n";
4318+925 *** 0 0 0 if ( !$o->{'tar4ibd'} ) {
4319+926 *** 0 0 $ret = system_call(
4320+927 "cd $orig_ibdata_dir; tar chf - -b 32 $path", 1);
4321+928 }
4322+929 else {
4323+930 *** 0 0 $ret = system_call(
4324+931 "cd $orig_ibdata_dir; tar4ibd -c $path", 1);
4325+932 }
4326+933 *** 0 0 0 if ( $ret == 1 ) {
4327+ *** 0
4328+934 *** 0 0 print STDERR "$prefix If you use GNU tar, this warning "
4329+935 . "can be ignored.\n";
4330+936 }
4331+937 elsif ( $ret != 0 ) {
4332+938 *** 0 0 print STDERR "$prefix tar returned with exit code $ret.\n";
4333+939 *** 0 0 Die "Failed to stream '$path': $OS_ERROR";
4334+940 }
4335+941 }
4336+942 }
4337+943
4338+944 #copy *.ibd files
4339+945 *** 0 0 0 opendir( DIR, $orig_datadir )
4340+946 || Die "Can't open directory '$orig_datadir': $OS_ERROR\n";
4341+947 *** 0 0 while ( defined( $subdir = readdir(DIR) ) ) {
4342+948 *** 0 0 my $print_each_file = 0;
4343+949 *** 0 0 my $file_c;
4344+950 *** 0 0 my $file;
4345+951 *** 0 0 0 0 if ( $subdir eq '.' || $subdir eq '..' ) { next; }
4346+ *** 0 0
4347+952 *** 0 0 0 next unless -d "$orig_datadir/$subdir";
4348+953 *** 0 0 0 next unless check_if_required($subdir);
4349+954
4350+955 *** 0 0 @list = glob( "$orig_datadir/$subdir/" . '*.ibd' );
4351+956
4352+957 *** 0 0 $file_c = @list;
4353+958 *** 0 0 0 if ( $file_c <= $backup_file_print_limit ) {
4354+959 *** 0 0 $print_each_file = 1;
4355+960 }
4356+961 else {
4357+962 *** 0 0 print STDERR "$prefix Backing up files "
4358+963 . "'$orig_datadir/$subdir/*.ibd' ($file_c files)\n";
4359+964 }
4360+965
4361+966 *** 0 0 foreach $file ( @list ) {
4362+967 *** 0 0 0 if ( $o->{'include'} ) {
4363+968 *** 0 0 my $table_name;
4364+969 *** 0 0 $table_name = substr($file, rindex( $file, '/' ));
4365+970 *** 0 0 $table_name = substr( $table_name, 1,
4366+971 rindex( $table_name, '.' ) - 1);
4367+972 *** 0 0 $table_name = $subdir . "." . $table_name;
4368+973 *** 0 0 0 if ( $table_name !~ m/$o->{'include'}/o ) {
4369+974 *** 0 0 print STDERR "'$file' is skipped.\n";
4370+975 *** 0 0 next;
4371+976 }
4372+977 }
4373+978
4374+979 *** 0 0 0 if ($print_each_file) {
4375+980 *** 0 0 print STDERR "$prefix Backing up file '$file'\n";
4376+981 }
4377+982
4378+983 *** 0 0 0 if ( $o->{'remote-host'} ) {
4379+ *** 0
4380+984 *** 0 0 system_call(
4381+985 "ssh $o->{'remote-host'} mkdir '$backup_dir/$subdir'",
4382+986 1 # don't die on failure
4383+987 );
4384+988 *** 0 0 system_call("scp $o->{'scp-opt'} '$file' "
4385+989 . "'$o->{'remote-host'}:$backup_dir/$subdir/'");
4386+990 }
4387+991 elsif ( $o->{'stream'} eq 'tar' ) {
4388+992 *** 0 0 my $ret = 0;
4389+993 *** 0 0 my $file_name = substr($file, rindex( $file, '/' ) + 1);
4390+994 *** 0 0 $file_name =~ s/([\$\\\" ])/\\$1/g;
4391+995
4392+996 *** 0 0 0 if ( !$o->{'tar4ibd'} ) {
4393+997 *** 0 0 $ret = system_call("cd $orig_datadir; tar chf - -b 32 "
4394+998 . "$subdir/$file_name", 1);
4395+999 }
4396+1000 else {
4397+1001 *** 0 0 $ret = system_call("cd $orig_datadir; tar4ibd -c "
4398+1002 . "$subdir/$file_name", 1);
4399+1003 }
4400+1004
4401+1005 *** 0 0 0 if ( $ret == 1 ) {
4402+ *** 0
4403+1006 *** 0 0 print STDERR "$prefix If you use GNU tar, this warning "
4404+1007 . "can be ignored.\n";
4405+1008 }
4406+1009 elsif ( $ret != 0 ) {
4407+1010 *** 0 0 print STDERR "$prefix tar returned with exit code "
4408+1011 . "$ret.\n";
4409+1012 *** 0 0 Die "Failed to stream '$subdir/$file_name': $OS_ERROR";
4410+1013 }
4411+1014 }
4412+1015 } # each file
4413+1016 } # each subdir
4414+1017 *** 0 0 closedir(DIR);
4415+1018 }
4416+1019 } # parent process
4417+1020 else { # child process
4418+1021 *** 1 50 33 90 if ( $o->{'remote-host'} || $o->{'stream'} ) {
4419+1022 *** 0 0 0 open STDOUT, "> $tmp_logfile"
4420+1023 or Die "Failed to open file '$tmp_logfile': $OS_ERROR";
4421+1024 }
4422+1025 1 4 IBEXDEBUG && _d($cmdline);
4423+1026 *** 1 0 46 exec($cmdline) or Die "Failed to exec ibbackup: $OS_ERROR";
4424+1027 }
4425+1028 }
4426+1029 else {
4427+1030 *** 0 0 Die "failed to fork ibbackup child process: $OS_ERROR";
4428+1031 }
4429+1032 }
4430+1033
4431+1034 #
4432+1035 # get_mysql_options subroutine returns the options to mysql client program
4433+1036 # as a string. The options are determined from the options given by the
4434+1037 # user to innobackup.
4435+1038 #
4436+1039 sub get_mysql_options {
4437+1040 10 10 56 my @options;
4438+1041
4439+1042 # --defaults-file needs to be first.
4440+1043 10 100 82 if ( $o->{'defaults-file'} ) {
4441+1044 8 106 push @options, "--defaults-file=$o->{'defaults-file'}";
4442+1045 }
4443+1046
4444+1047 10 100 72 if ( $o->{'host'} ) {
4445+1048 1 5 push @options, "--host=$o->{'host'}";
4446+1049 }
4447+1050 10 100 65 if ( $o->{'port'} ) {
4448+1051 2 11 push @options, "--port=$o->{'port'}";
4449+1052 }
4450+1053 10 100 62 if ( $o->{'user'} ) {
4451+1054 2 10 push @options, "--user=$o->{'user'}";
4452+1055 }
4453+1056 10 100 61 if ( $o->{'password'} ) {
4454+1057 4 19 push @options, "--password=$o->{'password'}";
4455+1058 }
4456+1059 *** 10 50 65 if ( $o->{'socket'} ) {
4457+1060 *** 0 0 push @options, "--socket=$o->{'socket'}";
4458+1061 }
4459+1062
4460+1063 10 46 push @options, '--unbuffered --';
4461+1064
4462+1065 10 73 my $options = join( ' ', @options );
4463+1066 10 31 IBEXDEBUG && _d('mysql options:', $options);
4464+1067 10 83 return $options;
4465+1068 }
4466+1069
4467+1070 #
4468+1071 # mysql_open subroutine starts mysql as a child process with
4469+1072 # a pipe connection.
4470+1073 #
4471+1074 sub mysql_open {
4472+1075 3 3 31 my $options = get_mysql_options();
4473+1076
4474+1077 # run mysql as a child process with a pipe connection
4475+1078 3 30 $now = current_time();
4476+1079 3 398 print STDERR "$now $prefix Starting mysql with options: $options\n";
4477+1080 *** 3 50 14260 $mysql_pid = open(*MYSQL_WRITER,
4478+1081 "| mysql $options >$mysql_stdout 2>$mysql_stderr ")
4479+1082 or Die "Failed to spawn mysql child process: $OS_ERROR";
4480+1083 3 254 MYSQL_WRITER->autoflush(1);
4481+1084 3 113 $now = current_time();
4482+1085 3 5000 print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n";
4483+1086
4484+1087 3 70 return;
4485+1088 }
4486+1089
4487+1090 #
4488+1091 # mysql_check subroutine checks that the connection to mysql child process
4489+1092 # is ok.
4490+1093 #
4491+1094 sub mysql_check {
4492+1095 9 9 85 my $mysql_pid_copy = $mysql_pid;
4493+1096
4494+1097 # send a dummy query to mysql child process
4495+1098 9 69 $hello_id++;
4496+1099 9 73 my $hello_message = "innobackup hello $hello_id";
4497+1100 9 36 IBEXDEBUG && _d('mysql check:', $hello_message);
4498+1101 *** 9 50 5816 print MYSQL_WRITER "select '$hello_message';\n"
4499+1102 or Die "Connection to mysql child process failed: $OS_ERROR";
4500+1103
4501+1104 # wait for reply
4502+1105 9 82 eval {
4503+1106 *** 9 0 432 local $SIG{ALRM} = sub { die "alarm clock restart" };
4504+ *** 0 0
4505+1107 9 73 my $stdout = '';
4506+1108 9 55 my $stderr = '';
4507+1109 9 125 alarm $mysql_response_timeout;
4508+1110 9 93 while ( index( $stdout, $hello_message ) < 0 ) {
4509+1111 9 18001882 sleep 2;
4510+1112 *** 9 50 33 718 if ( $mysql_pid && $mysql_pid == waitpid( $mysql_pid, &WNOHANG ) ) {
4511+1113 *** 0 0 my $reason = `cat $mysql_stderr`;
4512+1114 *** 0 0 $mysql_pid = '';
4513+1115 *** 0 0 die "mysql child process has died: $reason";
4514+1116 }
4515+1117 9 53378 $stdout = `cat $mysql_stdout`;
4516+1118 9 47330 $stderr = `cat $mysql_stderr`;
4517+1119 *** 9 50 360 if ($stderr) {
4518+1120 # mysql has reported an error, do exit
4519+1121 *** 0 0 die "mysql error: $stderr";
4520+1122 }
4521+1123 }
4522+1124 9 667 alarm 0;
4523+1125 };
4524+1126 *** 9 50 176 if ( $EVAL_ERROR =~ /alarm clock restart/ ) {
4525+ *** 50
4526+1127 *** 0 0 Die "Connection to mysql child process (pid=$mysql_pid_copy) timedout."
4527+1128 . " (Time limit of $mysql_response_timeout seconds exceeded."
4528+1129 . " You may adjust time limit by editing the value of parameter"
4529+1130 . " \"\$mysql_response_timeout\" in this script.)";
4530+1131 }
4531+1132 elsif ( $EVAL_ERROR ) {
4532+1133 *** 0 0 Die $EVAL_ERROR;
4533+1134 }
4534+1135
4535+1136 9 81 $mysql_last_access_time = time();
4536+1137
4537+1138 9 171 return;
4538+1139 }
4539+1140
4540+1141 #
4541+1142 # mysql_keep_alive subroutine tries to keep connection to the mysqld database
4542+1143 # server alive by sending a dummy query when the connection has been idle
4543+1144 # for the specified time.
4544+1145 #
4545+1146 sub mysql_keep_alive {
4546+1147 *** 5 50 5 63 if ( ( time() - $mysql_last_access_time ) > $mysql_keep_alive_timeout ) {
4547+1148 # too long idle, send a dummy query
4548+1149 *** 0 0 mysql_check();
4549+1150 }
4550+1151 5 23 return;
4551+1152 }
4552+1153
4553+1154 #
4554+1155 # mysql_send subroutine send a request string to mysql child process.
4555+1156 # This subroutine appends a newline character to the request and checks
4556+1157 # that mysqld receives the query.
4557+1158 # Parameters:
4558+1159 # request request string
4559+1160 #
4560+1161 sub mysql_send {
4561+1162 6 6 59 my $request = shift;
4562+1163 6 41 $current_mysql_request = $request;
4563+1164 6 652 print MYSQL_WRITER "$request\n";
4564+1165 6 53 mysql_check();
4565+1166 6 99 $current_mysql_request = '';
4566+1167 6 57 return;
4567+1168 }
4568+1169
4569+1170 #
4570+1171 # mysql_close subroutine terminates mysql child process gracefully.
4571+1172 #
4572+1173 sub mysql_close {
4573+1174 3 3 95 print MYSQL_WRITER "quit\n";
4574+1175 3 50 $now = current_time();
4575+1176 3 94 print STDERR "$now $prefix Connection to database server closed\n";
4576+1177 3 34 $mysql_pid = '';
4577+1178 3 20 return;
4578+1179 }
4579+1180
4580+1181 #
4581+1182 # write_binlog_info subroutine retrieves MySQL binlog position and
4582+1183 # saves it in a file. It also prints it to stdout.
4583+1184 #
4584+1185 sub write_binlog_info {
4585+1186 1 1 9 my @lines;
4586+1187 1 22 my @info_lines = ();
4587+1188 1 12 my $position = '';
4588+1189 1 7 my $filename = '';
4589+1190
4590+1191 # get binlog position
4591+1192 1 13 mysql_send "SHOW MASTER STATUS;";
4592+1193
4593+1194 # get "show master status" output lines (2) from mysql output
4594+1195 1 69 file_to_array( $mysql_stdout, \@lines );
4595+1196 1 10 foreach my $line (@lines) {
4596+1197 14 100 112 if ( $line =~ m/innobackup hello/ ) {
4597+1198
4598+1199 # this is a hello message, ignore it
4599+1200 }
4600+1201 else {
4601+1202
4602+1203 # this is output line from "show master status"
4603+1204 2 13 push( @info_lines, $line );
4604+1205 }
4605+1206 }
4606+1207
4607+1208 # write binlog info file
4608+1209 *** 1 50 11 if ( !defined $info_lines[1] ) {
4609+1210 *** 0 0 $info_lines[1] = "";
4610+1211 }
4611+1212 *** 1 50 10 if ( !$o->{'remote-host'} ) {
4612+1213 *** 1 50 92 open( FILE, ">$binlog_info" )
4613+1214 || Die "Failed to open file '$binlog_info': $OS_ERROR";
4614+1215 }
4615+1216 else {
4616+1217 *** 0 0 0 open( FILE, "| ssh $o->{'remote-host'} 'cat > $binlog_info'" )
4617+1218 || Die
4618+1219 "Failed to open file '$o->{'remote-host'}:$binlog_info': $OS_ERROR";
4619+1220 }
4620+1221 1 21 print FILE "$info_lines[1]\n";
4621+1222 1 43 close(FILE);
4622+1223
4623+1224 *** 1 50 14 if ( $o->{'stream'} eq 'tar' ) {
4624+1225 *** 0 0 system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_binlog_info");
4625+1226 *** 0 0 0 unlink $binlog_info or Die "Failed to delete '$binlog_info': $OS_ERROR";
4626+1227 }
4627+1228
4628+1229 # get the name of the last binlog file and position in it
4629+1230 1 37 ( $filename, $position ) = $info_lines[1] =~ /^\s*([^\s]+)\s+(.*)$/;
4630+1231
4631+1232 *** 1 50 33 33 if ( defined $filename && defined $position ) {
4632+1233 1 11 $mysql_binlog_position = "filename '$filename', position $position";
4633+1234 }
4634+1235 else {
4635+1236 *** 0 0 $mysql_binlog_position = "filename '', position ";
4636+1237 }
4637+1238
4638+1239 1 8 return;
4639+1240 }
4640+1241
4641+1242 #
4642+1243 # write_slave_info subroutine retrieves MySQL binlog position of the
4643+1244 # master server in a replication setup and saves it in a file. It
4644+1245 # also saves it in $msql_slave_position variable.
4645+1246 #
4646+1247 sub write_slave_info {
4647+1248 *** 0 0 0 my @lines;
4648+1249 *** 0 0 my @info_lines;
4649+1250 *** 0 0 my $position = '';
4650+1251 *** 0 0 my $filename = '';
4651+1252 *** 0 0 my $master = '';
4652+1253
4653+1254 # get slave status. Use single quotes here, otherwise
4654+1255 # \G is evaluated as a control character.
4655+1256 *** 0 0 mysql_send 'SHOW SLAVE STATUS\G;';
4656+1257
4657+1258 # get output of the "show slave status" command from mysql output
4658+1259 # and extract binlog position of the master server
4659+1260 *** 0 0 file_to_array( $mysql_stdout, \@lines );
4660+1261 *** 0 0 for (@lines) {
4661+1262 *** 0 0 0 $master = $1 if /Master_Host:\s*(\S*)\s*$/;
4662+1263 *** 0 0 0 $filename = $1 if /Master_Log_File:\s*(\S*)\s*$/;
4663+1264 *** 0 0 0 $position = $1 if /Master_Log_Pos:\s*(\S*)\s*$/;
4664+1265 }
4665+1266
4666+1267 # print slave status to a file
4667+1268 *** 0 0 0 if ( !$o->{'remote-host'} ) {
4668+1269 *** 0 0 0 open( FILE, ">$slave_info" )
4669+1270 || Die "Failed to open file '$slave_info': $OS_ERROR";
4670+1271 }
4671+1272 else {
4672+1273 *** 0 0 0 open( FILE, "| ssh $o->{'remote-host'} 'cat > $slave_info'" )
4673+1274 || Die
4674+1275 "Failed to open file '$o->{'remote-host'}:$slave_info': $OS_ERROR";
4675+1276 }
4676+1277 *** 0 0 print FILE
4677+1278 "CHANGE MASTER TO MASTER_LOG_FILE='$filename', MASTER_LOG_POS=$position\n";
4678+1279 *** 0 0 close(FILE);
4679+1280
4680+1281 *** 0 0 0 if ( $o->{'stream'} eq 'tar' ) {
4681+1282 *** 0 0 system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_slave_info");
4682+1283 *** 0 0 0 unlink $slave_info or Die "Failed to delete '$slave_info': $OS_ERROR";
4683+1284 }
4684+1285
4685+1286 $mysql_slave_position =
4686+1287 *** 0 0 "master host '$master', filename '$filename', position $position";
4687+1288
4688+1289 *** 0 0 return;
4689+1290 }
4690+1291
4691+1292 #
4692+1293 # mysql_lockall subroutine puts a read lock on all tables in all databases.
4693+1294 #
4694+1295 sub mysql_lockall {
4695+1296 1 1 16 $now = current_time();
4696+1297 1 46 print STDERR "$now $prefix Starting to lock all tables...\n";
4697+1298
4698+1299 1 21 mysql_send "USE mysql;";
4699+1300
4700+1301 # mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;";
4701+1302 # if (compare_versions($mysql_server_version, '4.1.0') == -1) {
4702+1303 # # MySQL server version is 4.0 or older, ENGINE keyword not supported
4703+1304 # mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) TYPE=INNODB;";
4704+1305 # } else {
4705+1306 # # MySQL server version is 4.1 or newer, use ENGINE keyword
4706+1307 # mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) ENGINE=INNODB;";
4707+1308 # }
4708+1309 1 19 mysql_send "SET AUTOCOMMIT=0;";
4709+1310
4710+1311 # mysql_send "INSERT INTO ibbackup_binlog_marker VALUES (1);";
4711+1312 *** 1 50 33 42 if ( compare_versions( $mysql_server_version, '4.0.22' ) == 0
4712+1313 || compare_versions( $mysql_server_version, '4.1.7' ) == 0 )
4713+1314 {
4714+1315
4715+1316 # MySQL server version is 4.0.22 or 4.1.7
4716+1317 *** 0 0 mysql_send "COMMIT;";
4717+1318 *** 0 0 mysql_send "FLUSH TABLES WITH READ LOCK;";
4718+1319 }
4719+1320 else {
4720+1321
4721+1322 # MySQL server version is other than 4.0.22 or 4.1.7
4722+1323 1 13 mysql_send "FLUSH TABLES WITH READ LOCK;";
4723+1324 1 34 mysql_send "COMMIT;";
4724+1325 }
4725+1326 1 47 write_binlog_info;
4726+1327 *** 1 50 10 write_slave_info if $o->{'slave-info'};
4727+1328
4728+1329 1 19 $now = current_time();
4729+1330 1 39 print STDERR "$now $prefix All tables locked and flushed to disk\n";
4730+1331
4731+1332 1 5 return;
4732+1333 }
4733+1334
4734+1335 #
4735+1336 # mysql_unlockall subroutine releases read locks on all tables in all
4736+1337 # databases.
4737+1338 #
4738+1339 sub mysql_unlockall {
4739+1340 1 1 17 mysql_send "UNLOCK TABLES;";
4740+1341
4741+1342 # mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;";
4742+1343
4743+1344 1 35 $now = current_time();
4744+1345 1 58 print STDERR "$now $prefix All tables unlocked\n";
4745+1346 }
4746+1347
4747+1348 #
4748+1349 # catch_sigpipe subroutine is a signal handler for SIGPIPE.
4749+1350 #
4750+1351 sub catch_sigpipe {
4751+1352 *** 0 0 0 my $rcode;
4752+1353
4753+1354 *** 0 0 0 0 if (
4754+ *** 0
4755+1355 $mysql_pid
4756+1356 && ( -1 == ( $rcode = waitpid( $mysql_pid, &WNOHANG ) )
4757+1357 || $rcode == $mysql_pid )
4758+1358 )
4759+1359 {
4760+1360 *** 0 0 my $reason = `cat $mysql_stderr`;
4761+1361 *** 0 0 print STDERR "Pipe to mysql child process broken: $reason at";
4762+1362 *** 0 0 system("date +'%H:%M:%S'");
4763+1363 *** 0 0 exit(1);
4764+1364 }
4765+1365 else {
4766+1366 *** 0 0 Die "Broken pipe";
4767+1367 }
4768+1368 }
4769+1369
4770+1370 #
4771+1371 # kill_child_processes subroutine kills all child processes of this process.
4772+1372 #
4773+1373 sub kill_child_processes {
4774+1374 *** 0 0 0 0 if ($ibbackup_pid) {
4775+1375 *** 0 0 kill( $kill_signal, $ibbackup_pid );
4776+1376 *** 0 0 $ibbackup_pid = '';
4777+1377 }
4778+1378
4779+1379 *** 0 0 0 if ($mysql_pid) {
4780+1380 *** 0 0 kill( $kill_signal, $mysql_pid );
4781+1381 *** 0 0 $mysql_pid = '';
4782+1382 }
4783+1383 }
4784+1384
4785+1385 #
4786+1386 # require_external subroutine checks that an external program is runnable
4787+1387 # via the shell. This is tested by calling the program with the
4788+1388 # given arguments. It is checked that the program returns 0 and does
4789+1389 # not print anything to stderr. If this check fails, this subroutine exits.
4790+1390 # Parameters:
4791+1391 # program pathname of the program file
4792+1392 # args arguments to the program
4793+1393 # pattern a string containing a regular expression for finding
4794+1394 # the program version.
4795+1395 # this pattern should contain a subpattern enclosed
4796+1396 # in parentheses which is matched with the version.
4797+1397 # version_ref a reference to a variable where the program version
4798+1398 # string is returned. Example "2.0-beta2".
4799+1399 #
4800+1400 sub require_external {
4801+1401 2 2 12 my $program = shift;
4802+1402 2 10 my $args = shift;
4803+1403 2 10 my $pattern = shift;
4804+1404 2 10 my $version_ref = shift;
4805+1405 2 8 my @lines;
4806+1406 2 144 my $tmp_stdout = tmpnam();
4807+1407 2 46 my $tmp_stderr = tmpnam();
4808+1408 2 8 my $rcode;
4809+1409 2 6 my $error;
4810+1410
4811+1411 2 50 $rcode = system_call("$program $args >$tmp_stdout 2>$tmp_stderr", 1);
4812+1412 *** 2 50 32 if ($rcode) {
4813+1413 *** 0 0 $error = $OS_ERROR;
4814+1414 }
4815+1415 2 14814 my $stderr = `cat $tmp_stderr`;
4816+1416 2 216 unlink $tmp_stderr;
4817+1417 *** 2 50 68 if ( $stderr ne '' ) {
4818+ *** 50
4819+1418
4820+1419 # failure
4821+1420 *** 0 0 unlink $tmp_stdout;
4822+1421 *** 0 0 Die "Couldn't run $program: $stderr";
4823+1422 }
4824+1423 elsif ($rcode) {
4825+1424
4826+1425 # failure
4827+1426 *** 0 0 unlink $tmp_stdout;
4828+1427 *** 0 0 Die "Couldn't run $program: $error";
4829+1428 }
4830+1429
4831+1430 # success
4832+1431 2 11754 my $stdout = `cat $tmp_stdout`;
4833+1432 2 182 unlink $tmp_stdout;
4834+1433 2 108 @lines = split( /\n|;/, $stdout );
4835+1434 2 414 print STDERR "$prefix Using $lines[0]\n";
4836+1435
4837+1436 # get version string from the first output line of the program
4838+1437 2 46 ${$version_ref} = '';
4839+ 2 26
4840+1438 *** 2 50 154 if ( $lines[0] =~ /$pattern/ ) {
4841+1439 2 10 ${$version_ref} = $1;
4842+ 2 66
4843+1440 }
4844+1441 }
4845+1442
4846+1443 # compare_versions subroutine compares two GNU-style version strings.
4847+1444 # A GNU-style version string consists of three decimal numbers delimitted
4848+1445 # by dots, and optionally followed by extra attributes.
4849+1446 # Examples: "4.0.1", "4.1.1-alpha-debug".
4850+1447 # Parameters:
4851+1448 # str1 a GNU-style version string
4852+1449 # str2 a GNU-style version string
4853+1450 # Return value:
4854+1451 # -1 if str1 < str2
4855+1452 # 0 if str1 == str2
4856+1453 # 1 is str1 > str2
4857+1454 sub compare_versions {
4858+1455 2 2 22 my $str1 = shift;
4859+1456 2 16 my $str2 = shift;
4860+1457 2 11 my $extra1 = '';
4861+1458 2 10 my $extra2 = '';
4862+1459 2 11 my @array1 = ();
4863+1460 2 8 my @array2 = ();
4864+1461 2 8 my $i;
4865+1462
4866+1463 # remove possible extra attributes
4867+1464 2 42 ( $str1, $extra1 ) = $str1 =~ /^([0-9.]*)(.*)/;
4868+1465 2 24 ( $str2, $extra2 ) = $str2 =~ /^([0-9.]*)(.*)/;
4869+1466
4870+1467 # split "dotted" decimal number string into an array
4871+1468 2 21 @array1 = split( '\.', $str1 );
4872+1469 2 17 @array2 = split( '\.', $str2 );
4873+1470
4874+1471 # compare in lexicographic order
4875+1472 for ( $i = 0; $i <= $#array1 && $i <= $#array2; $i++ ) {
4876+1473 *** 0 0 0 if ( $array1[$i] < $array2[$i] ) {
4877+ *** 0
4878+1474 *** 0 0 return -1;
4879+1475 }
4880+1476 elsif ( $array1[$i] > $array2[$i] ) {
4881+1477 *** 0 0 return 1;
4882+1478 }
4883+1479 *** 2 33 18 }
4884+1480 *** 2 50 31 if ( $#array1 < $#array2 ) {
4885+ *** 0
4886+1481 2 47 return -1;
4887+1482 }
4888+1483 elsif ( $#array1 > $#array2 ) {
4889+1484 *** 0 0 return 1;
4890+1485 }
4891+1486 else {
4892+1487 *** 0 0 return 0;
4893+1488 }
4894+1489 }
4895+1490
4896+1491 #
4897+1492 # init subroutine initializes global variables and performs some checks on the
4898+1493 # system we are running on.
4899+1494 #
4900+1495 sub init {
4901+1496 2 2 12 my $mysql_version = '';
4902+1497 2 8 my $ibbackup_version = '';
4903+1498 2 10 my $run = '';
4904+1499
4905+1500 # print some instructions to the user
4906+1501 *** 2 50 33 92 if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) {
4907+ *** 0
4908+1502 2 12 $run = 'backup';
4909+1503 }
4910+1504 elsif ( $o->{'copy-back'} ) {
4911+1505 *** 0 0 $run = 'copy-back';
4912+1506 }
4913+1507 else {
4914+1508 *** 0 0 $run = 'apply-log';
4915+1509 }
4916+1510 2 36 print STDERR
4917+1511 "IMPORTANT: Please check that the $run run completes successfully.\n";
4918+1512 2 32 print STDERR
4919+1513 " At the end of a successful $run run $innobackup_script\n";
4920+1514 2 20 print STDERR " prints \"completed OK!\".\n\n";
4921+1515
4922+1516 # check that MySQL client program and InnoDB Hot Backup program
4923+1517 # are runnable via shell
4924+1518 *** 2 50 20 if ( !$o->{'copy-back'} ) {
4925+1519
4926+1520 # we are making a backup or applying log to backup
4927+1521 *** 2 50 14 if ( !$o->{'apply-log'} ) {
4928+1522
4929+1523 # we are making a backup, we need mysql server
4930+1524 2 10 my $output = '';
4931+1525 2 12 my @lines = ();
4932+1526
4933+1527 # check that we have mysql client program
4934+1528 2 34 require_external( 'mysql', '--version', 'Ver ([^,]+)',
4935+1529 \$mysql_version );
4936+1530
4937+1531 # get mysql server version
4938+1532 2 42 my $options = get_mysql_options();
4939+1533 2 27220 @lines = split( '\n', `mysql $options -e "select \@\@version"` );
4940+1534 2 108 $mysql_server_version = $lines[1];
4941+1535 2 146 print STDERR
4942+1536 "$prefix Using mysql server version $mysql_server_version\n";
4943+1537 }
4944+1538
4945+1539 #require_external($o->{'ibbackup-binary'}, '--license',
4946+1540 # 'version (\S+)', \$ibbackup_version);
4947+1541 2 32 print STDERR "\n";
4948+1542
4949+1543 *** 2 50 33 58 if ( $o->{'include'}
4950+ *** 33
4951+1544 && $ibbackup_version
4952+1545 && $ibbackup_version le "2.0" )
4953+1546 {
4954+1547
4955+1548 # --include option was given, but ibbackup is too
4956+1549 # old to support it
4957+1550 *** 0 0 Die "--include option was given, but ibbackup is too old"
4958+1551 . " to support it. You must upgrade to InnoDB Hot Backup"
4959+1552 . " v2.0 in order to use --include option.\n";
4960+1553 }
4961+1554 }
4962+1555
4963+1556 # set signal handlers
4964+1557 2 142 $SIG{PIPE} = \&catch_sigpipe;
4965+1558
4966+1559 # read MySQL options file
4967+1560 #read_config_file($config_file, \%config);
4968+1561 2 80 read_config_file( \%config );
4969+1562
4970+1563 *** 2 50 26 if ( !$o->{'tmpdir'} ) {
4971+1564 2 34 $o->{'tmpdir'} = get_option( \%config, 'mysqld', 'tmpdir' );
4972+1565 }
4973+1566
4974+1567 # get innodb log home directory from options file
4975+1568 #$innodb_log_group_home_dir =
4976+1569 # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir');
4977+1570
4978+1571 *** 2 50 33 54 if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) {
4979+ *** 0
4980+1572
4981+1573 # we are making a backup, create a new backup directory
4982+1574 *** 2 50 14 if ( !$o->{'remote-host'} ) {
4983+1575 2 30 $backup_dir = File::Spec->rel2abs( make_backup_dir() );
4984+1576 }
4985+1577 else {
4986+1578 *** 0 0 $backup_dir = make_backup_dir();
4987+1579 }
4988+1580 2 544 print STDERR "$prefix Created backup directory $backup_dir\n";
4989+1581 *** 2 50 33 66 if ( !$o->{'remote-host'} && !$o->{'stream'} ) {
4990+1582 2 18 $backup_config_file = $backup_dir . '/backup-my.cnf';
4991+1583 2 16 $suspend_file = $backup_dir . '/xtrabackup_suspended';
4992+1584 2 30 $mysql_stdout = $backup_dir . '/mysql-stdout';
4993+1585 2 16 $mysql_stderr = $backup_dir . '/mysql-stderr';
4994+1586 2 8 $binlog_info = $backup_dir . '/xtrabackup_binlog_info';
4995+1587 2 12 $slave_info = $backup_dir . '/xtrabackup_slave_info';
4996+1588 }
4997+1589 else {
4998+1590 *** 0 0 $suspend_file = get_option( \%config, 'mysqld', 'datadir' )
4999+1591 . '/xtrabackup_suspended';
5000+1592 *** 0 0 $tmp_logfile = $o->{'tmpdir'} . '/xtrabackup_logfile';
The diff has been truncated for viewing.

Subscribers

People subscribed via source and target branches