Merge lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite into lp:percona-xtrabackup/2.0
- bugfix510960-et-al-test-suite
- Merge into 2.0
Status: | Rejected |
---|---|
Rejected by: | Vadim Tkachenko |
Proposed branch: | lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite |
Merge into: | lp:percona-xtrabackup/2.0 |
Diff against target: |
6180 lines (+6137/-0) 8 files modified
innobackupex/common/DSNParser.pm (+417/-0) innobackupex/common/IbexTest.pm (+133/-0) innobackupex/common/Sandbox.pm (+168/-0) innobackupex/innobackupex (+2400/-0) innobackupex/t/001_get_mysql_options.t (+61/-0) innobackupex/t/100_backup.t (+87/-0) innobackupex/t/samples/basic-tables.sql (+17/-0) innobackupex/test-coverage.txt (+2854/-0) |
To merge this branch: | bzr merge lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Vadim Tkachenko | Needs Fixing | ||
Review via email: mp+17861@code.launchpad.net |
Commit message
Description of the change
Daniel Nichter (daniel-nichter) wrote : | # |
Vadim Tkachenko (vadim-tk) wrote : | # |
Daniel,
Could you re-allocate bugfix510960 and test suite into two different branches ?
bugfix should be included in main trunk, while I agree script with test suite could be in different dir.
Daniel Nichter (daniel-nichter) wrote : | # |
Vadim,
Vadim Tkachenko wrote:
> Review: Needs Fixing
> Daniel,
>
> Could you re-allocate bugfix510960 and test suite into two different branches ?
>
> bugfix should be included in main trunk, while I agree script with test suite could be in different dir.
So,
branch1 = apply fix(es) to original code in trunk
branch2 = separate innobackupex dir with my modified/fixed code and test suite
or,
branch1 = separate innodbackupex dir w/ my modified/fixed code
branch2 = add test suite in separate innobackupex dir
?
Vadim Tkachenko (vadim-tk) wrote : | # |
Daniel,
branch1 = apply fix(es) to original code in trunk
branch2 = separate innobackupex dir with my modified/fixed code and test suite
On Thu, Jan 21, 2010 at 9:00 PM, Daniel Nichter <email address hidden> wrote:
> Vadim,
>
> Vadim Tkachenko wrote:
>> Review: Needs Fixing
>> Daniel,
>>
>> Could you re-allocate bugfix510960 and test suite into two different branches ?
>>
>> bugfix should be included in main trunk, while I agree script with test suite could be in different dir.
>
> So,
>
> branch1 = apply fix(es) to original code in trunk
> branch2 = separate innobackupex dir with my modified/fixed code and test suite
>
> or,
>
> branch1 = separate innodbackupex dir w/ my modified/fixed code
> branch2 = add test suite in separate innobackupex dir
>
> ?
> --
> https:/
> You are reviewing the proposed merge of lp:~percona-dev/percona-xtrabackup/bugfix510960-et-al-test-suite into lp:percona-xtrabackup.
>
--
Vadim Tkachenko, CTO, Percona Inc.
Phone +1-888-401-3403, Skype: vadimtk153
Schedule meeting: http://
Unmerged revisions
- 115. By Daniel Nichter
-
Fix bugs 510960 510964 510965 510966 510969 510970. -- Add innobackupex/ with new, enhanced copy of innobackupex-1.5.1. First incarnation of a test suite.
Preview Diff
1 | === added directory 'innobackupex' |
2 | === added directory 'innobackupex/common' |
3 | === added file 'innobackupex/common/DSNParser.pm' |
4 | --- innobackupex/common/DSNParser.pm 1970-01-01 00:00:00 +0000 |
5 | +++ innobackupex/common/DSNParser.pm 2010-01-22 01:31:15 +0000 |
6 | @@ -0,0 +1,417 @@ |
7 | +# This program is copyright 2007-2009 Baron Schwartz. |
8 | +# Feedback and improvements are welcome. |
9 | +# |
10 | +# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED |
11 | +# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF |
12 | +# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. |
13 | +# |
14 | +# This program is free software; you can redistribute it and/or modify it under |
15 | +# the terms of the GNU General Public License as published by the Free Software |
16 | +# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar |
17 | +# systems, you can issue `man perlgpl' or `man perlartistic' to read these |
18 | +# licenses. |
19 | +# |
20 | +# You should have received a copy of the GNU General Public License along with |
21 | +# this program; if not, write to the Free Software Foundation, Inc., 59 Temple |
22 | +# Place, Suite 330, Boston, MA 02111-1307 USA. |
23 | +# ########################################################################### |
24 | +# DSNParser package $Revision: 5266 $ |
25 | +# ########################################################################### |
26 | +package DSNParser; |
27 | + |
28 | +use strict; |
29 | +use warnings FATAL => 'all'; |
30 | +use English qw(-no_match_vars); |
31 | +use Data::Dumper; |
32 | +$Data::Dumper::Indent = 0; |
33 | +$Data::Dumper::Quotekeys = 0; |
34 | + |
35 | +eval { |
36 | + require DBI; |
37 | +}; |
38 | +my $have_dbi = $EVAL_ERROR ? 0 : 1; |
39 | + |
40 | +use constant MKDEBUG => $ENV{MKDEBUG} || 0; |
41 | + |
42 | +# Defaults are built-in, but you can add/replace items by passing them as |
43 | +# hashrefs of {key, desc, copy, dsn}. The desc and dsn items are optional. |
44 | +# You can set properties with the prop() sub. Don't set the 'opts' property. |
45 | +sub new { |
46 | + my ( $class, @opts ) = @_; |
47 | + my $self = { |
48 | + opts => { |
49 | + A => { |
50 | + desc => 'Default character set', |
51 | + dsn => 'charset', |
52 | + copy => 1, |
53 | + }, |
54 | + D => { |
55 | + desc => 'Database to use', |
56 | + dsn => 'database', |
57 | + copy => 1, |
58 | + }, |
59 | + F => { |
60 | + desc => 'Only read default options from the given file', |
61 | + dsn => 'mysql_read_default_file', |
62 | + copy => 1, |
63 | + }, |
64 | + h => { |
65 | + desc => 'Connect to host', |
66 | + dsn => 'host', |
67 | + copy => 1, |
68 | + }, |
69 | + p => { |
70 | + desc => 'Password to use when connecting', |
71 | + dsn => 'password', |
72 | + copy => 1, |
73 | + }, |
74 | + P => { |
75 | + desc => 'Port number to use for connection', |
76 | + dsn => 'port', |
77 | + copy => 1, |
78 | + }, |
79 | + S => { |
80 | + desc => 'Socket file to use for connection', |
81 | + dsn => 'mysql_socket', |
82 | + copy => 1, |
83 | + }, |
84 | + u => { |
85 | + desc => 'User for login if not current user', |
86 | + dsn => 'user', |
87 | + copy => 1, |
88 | + }, |
89 | + }, |
90 | + }; |
91 | + foreach my $opt ( @opts ) { |
92 | + MKDEBUG && _d('Adding extra property', $opt->{key}); |
93 | + $self->{opts}->{$opt->{key}} = { desc => $opt->{desc}, copy => $opt->{copy} }; |
94 | + } |
95 | + return bless $self, $class; |
96 | +} |
97 | + |
98 | +# Recognized properties: |
99 | +# * dbidriver: which DBI driver to use; assumes mysql, supports Pg. |
100 | +# * required: which parts are required (hashref). |
101 | +# * set-vars: a list of variables to set after connecting |
102 | +sub prop { |
103 | + my ( $self, $prop, $value ) = @_; |
104 | + if ( @_ > 2 ) { |
105 | + MKDEBUG && _d('Setting', $prop, 'property'); |
106 | + $self->{$prop} = $value; |
107 | + } |
108 | + return $self->{$prop}; |
109 | +} |
110 | + |
111 | +# Parse DSN string, like "h=host,P=3306", and return hashref with |
112 | +# all DSN values, like: |
113 | +# { |
114 | +# D => undef, |
115 | +# F => undef, |
116 | +# h => 'host', |
117 | +# p => undef, |
118 | +# P => 3306, |
119 | +# S => undef, |
120 | +# t => undef, |
121 | +# u => undef, |
122 | +# A => undef, |
123 | +# } |
124 | +sub parse { |
125 | + my ( $self, $dsn, $prev, $defaults ) = @_; |
126 | + if ( !$dsn ) { |
127 | + MKDEBUG && _d('No DSN to parse'); |
128 | + return; |
129 | + } |
130 | + MKDEBUG && _d('Parsing', $dsn); |
131 | + $prev ||= {}; |
132 | + $defaults ||= {}; |
133 | + my %given_props; |
134 | + my %final_props; |
135 | + my %opts = %{$self->{opts}}; |
136 | + |
137 | + # Parse given props |
138 | + foreach my $dsn_part ( split(/,/, $dsn) ) { |
139 | + if ( my ($prop_key, $prop_val) = $dsn_part =~ m/^(.)=(.*)$/ ) { |
140 | + # Handle the typical DSN parts like h=host, P=3306, etc. |
141 | + $given_props{$prop_key} = $prop_val; |
142 | + } |
143 | + else { |
144 | + # Handle barewords |
145 | + MKDEBUG && _d('Interpreting', $dsn_part, 'as h=', $dsn_part); |
146 | + $given_props{h} = $dsn_part; |
147 | + } |
148 | + } |
149 | + |
150 | + # Fill in final props from given, previous, and/or default props |
151 | + foreach my $key ( keys %opts ) { |
152 | + MKDEBUG && _d('Finding value for', $key); |
153 | + $final_props{$key} = $given_props{$key}; |
154 | + if ( !defined $final_props{$key} |
155 | + && defined $prev->{$key} && $opts{$key}->{copy} ) |
156 | + { |
157 | + $final_props{$key} = $prev->{$key}; |
158 | + MKDEBUG && _d('Copying value for', $key, 'from previous DSN'); |
159 | + } |
160 | + if ( !defined $final_props{$key} ) { |
161 | + $final_props{$key} = $defaults->{$key}; |
162 | + MKDEBUG && _d('Copying value for', $key, 'from defaults'); |
163 | + } |
164 | + } |
165 | + |
166 | + # Sanity check props |
167 | + foreach my $key ( keys %given_props ) { |
168 | + die "Unrecognized DSN part '$key' in '$dsn'\n" |
169 | + unless exists $opts{$key}; |
170 | + } |
171 | + if ( (my $required = $self->prop('required')) ) { |
172 | + foreach my $key ( keys %$required ) { |
173 | + die "Missing DSN part '$key' in '$dsn'\n" unless $final_props{$key}; |
174 | + } |
175 | + } |
176 | + |
177 | + return \%final_props; |
178 | +} |
179 | + |
180 | +# Like parse() above but takes an OptionParser object instead of |
181 | +# a DSN string. |
182 | +sub parse_options { |
183 | + my ( $self, $o ) = @_; |
184 | + die 'I need an OptionParser object' unless ref $o eq 'OptionParser'; |
185 | + my $dsn_string |
186 | + = join(',', |
187 | + map { "$_=".$o->get($_); } |
188 | + grep { $o->has($_) && $o->get($_) } |
189 | + keys %{$self->{opts}} |
190 | + ); |
191 | + MKDEBUG && _d('DSN string made from options:', $dsn_string); |
192 | + return $self->parse($dsn_string); |
193 | +} |
194 | + |
195 | +sub as_string { |
196 | + my ( $self, $dsn ) = @_; |
197 | + return $dsn unless ref $dsn; |
198 | + return join(',', |
199 | + map { "$_=" . ($_ eq 'p' ? '...' : $dsn->{$_}) } |
200 | + grep { defined $dsn->{$_} && $self->{opts}->{$_} } |
201 | + sort keys %$dsn ); |
202 | +} |
203 | + |
204 | +sub usage { |
205 | + my ( $self ) = @_; |
206 | + my $usage |
207 | + = "DSN syntax is key=value[,key=value...] Allowable DSN keys:\n\n" |
208 | + . " KEY COPY MEANING\n" |
209 | + . " === ==== =============================================\n"; |
210 | + my %opts = %{$self->{opts}}; |
211 | + foreach my $key ( sort keys %opts ) { |
212 | + $usage .= " $key " |
213 | + . ($opts{$key}->{copy} ? 'yes ' : 'no ') |
214 | + . ($opts{$key}->{desc} || '[No description]') |
215 | + . "\n"; |
216 | + } |
217 | + $usage .= "\n If the DSN is a bareword, the word is treated as the 'h' key.\n"; |
218 | + return $usage; |
219 | +} |
220 | + |
221 | +# Supports PostgreSQL via the dbidriver element of $info, but assumes MySQL by |
222 | +# default. |
223 | +sub get_cxn_params { |
224 | + my ( $self, $info ) = @_; |
225 | + my $dsn; |
226 | + my %opts = %{$self->{opts}}; |
227 | + my $driver = $self->prop('dbidriver') || ''; |
228 | + if ( $driver eq 'Pg' ) { |
229 | + $dsn = 'DBI:Pg:dbname=' . ( $info->{D} || '' ) . ';' |
230 | + . join(';', map { "$opts{$_}->{dsn}=$info->{$_}" } |
231 | + grep { defined $info->{$_} } |
232 | + qw(h P)); |
233 | + } |
234 | + else { |
235 | + $dsn = 'DBI:mysql:' . ( $info->{D} || '' ) . ';' |
236 | + . join(';', map { "$opts{$_}->{dsn}=$info->{$_}" } |
237 | + grep { defined $info->{$_} } |
238 | + qw(F h P S A)) |
239 | + . ';mysql_read_default_group=client'; |
240 | + } |
241 | + MKDEBUG && _d($dsn); |
242 | + return ($dsn, $info->{u}, $info->{p}); |
243 | +} |
244 | + |
245 | +# Fills in missing info from a DSN after successfully connecting to the server. |
246 | +sub fill_in_dsn { |
247 | + my ( $self, $dbh, $dsn ) = @_; |
248 | + my $vars = $dbh->selectall_hashref('SHOW VARIABLES', 'Variable_name'); |
249 | + my ($user, $db) = $dbh->selectrow_array('SELECT USER(), DATABASE()'); |
250 | + $user =~ s/@.*//; |
251 | + $dsn->{h} ||= $vars->{hostname}->{Value}; |
252 | + $dsn->{S} ||= $vars->{'socket'}->{Value}; |
253 | + $dsn->{P} ||= $vars->{port}->{Value}; |
254 | + $dsn->{u} ||= $user; |
255 | + $dsn->{D} ||= $db; |
256 | +} |
257 | + |
258 | +# Actually opens a connection, then sets some things on the connection so it is |
259 | +# the way the Maatkit tools will expect. Tools should NEVER open their own |
260 | +# connection or use $dbh->reconnect, or these things will not take place! |
261 | +sub get_dbh { |
262 | + my ( $self, $cxn_string, $user, $pass, $opts ) = @_; |
263 | + $opts ||= {}; |
264 | + my $defaults = { |
265 | + AutoCommit => 0, |
266 | + RaiseError => 1, |
267 | + PrintError => 0, |
268 | + ShowErrorStatement => 1, |
269 | + mysql_enable_utf8 => ($cxn_string =~ m/charset=utf8/ ? 1 : 0), |
270 | + }; |
271 | + @{$defaults}{ keys %$opts } = values %$opts; |
272 | + |
273 | + if ( !$have_dbi ) { |
274 | + die "Cannot connect to MySQL because the Perl DBI module is not " |
275 | + . "installed or not found. Run 'perl -MDBI' to see the directories " |
276 | + . "that Perl searches for DBI. If DBI is not installed, try:\n" |
277 | + . " Debian/Ubuntu apt-get install libdbi-perl\n" |
278 | + . " RHEL/CentOS yum install perl-DBI\n" |
279 | + . " OpenSolaris pgk install pkg:/SUNWpmdbi\n"; |
280 | + |
281 | + } |
282 | + |
283 | + # Try twice to open the $dbh and set it up as desired. |
284 | + my $dbh; |
285 | + my $tries = 2; |
286 | + while ( !$dbh && $tries-- ) { |
287 | + MKDEBUG && _d($cxn_string, ' ', $user, ' ', $pass, ' {', |
288 | + join(', ', map { "$_=>$defaults->{$_}" } keys %$defaults ), '}'); |
289 | + |
290 | + eval { |
291 | + $dbh = DBI->connect($cxn_string, $user, $pass, $defaults); |
292 | + |
293 | + # If it's a MySQL connection, set some options. |
294 | + if ( $cxn_string =~ m/mysql/i ) { |
295 | + my $sql; |
296 | + |
297 | + # Set SQL_MODE and options for SHOW CREATE TABLE. |
298 | + $sql = q{SET @@SQL_QUOTE_SHOW_CREATE = 1} |
299 | + . q{/*!40101, @@SQL_MODE='NO_AUTO_VALUE_ON_ZERO'*/}; |
300 | + MKDEBUG && _d($dbh, ':', $sql); |
301 | + $dbh->do($sql); |
302 | + |
303 | + # Set character set and binmode on STDOUT. |
304 | + if ( my ($charset) = $cxn_string =~ m/charset=(\w+)/ ) { |
305 | + $sql = "/*!40101 SET NAMES $charset*/"; |
306 | + MKDEBUG && _d($dbh, ':', $sql); |
307 | + $dbh->do($sql); |
308 | + MKDEBUG && _d('Enabling charset for STDOUT'); |
309 | + if ( $charset eq 'utf8' ) { |
310 | + binmode(STDOUT, ':utf8') |
311 | + or die "Can't binmode(STDOUT, ':utf8'): $OS_ERROR"; |
312 | + } |
313 | + else { |
314 | + binmode(STDOUT) or die "Can't binmode(STDOUT): $OS_ERROR"; |
315 | + } |
316 | + } |
317 | + |
318 | + if ( $self->prop('set-vars') ) { |
319 | + $sql = "SET " . $self->prop('set-vars'); |
320 | + MKDEBUG && _d($dbh, ':', $sql); |
321 | + $dbh->do($sql); |
322 | + } |
323 | + } |
324 | + }; |
325 | + if ( !$dbh && $EVAL_ERROR ) { |
326 | + MKDEBUG && _d($EVAL_ERROR); |
327 | + if ( $EVAL_ERROR =~ m/not a compiled character set|character set utf8/ ) { |
328 | + MKDEBUG && _d('Going to try again without utf8 support'); |
329 | + delete $defaults->{mysql_enable_utf8}; |
330 | + } |
331 | + elsif ( $EVAL_ERROR =~ m/locate DBD\/mysql/i ) { |
332 | + die "Cannot connect to MySQL because the Perl DBD::mysql module is " |
333 | + . "not installed or not found. Run 'perl -MDBD::mysql' to see " |
334 | + . "the directories that Perl searches for DBD::mysql. If " |
335 | + . "DBD::mysql is not installed, try:\n" |
336 | + . " Debian/Ubuntu apt-get install libdbd-mysql-perl\n" |
337 | + . " RHEL/CentOS yum install perl-DBD-MySQL\n" |
338 | + . " OpenSolaris pgk install pkg:/SUNWapu13dbd-mysql\n"; |
339 | + } |
340 | + if ( !$tries ) { |
341 | + die $EVAL_ERROR; |
342 | + } |
343 | + } |
344 | + } |
345 | + |
346 | + MKDEBUG && _d('DBH info: ', |
347 | + $dbh, |
348 | + Dumper($dbh->selectrow_hashref( |
349 | + 'SELECT DATABASE(), CONNECTION_ID(), VERSION()/*!50038 , @@hostname*/')), |
350 | + 'Connection info:', $dbh->{mysql_hostinfo}, |
351 | + 'Character set info:', Dumper($dbh->selectall_arrayref( |
352 | + 'SHOW VARIABLES LIKE "character_set%"', { Slice => {}})), |
353 | + '$DBD::mysql::VERSION:', $DBD::mysql::VERSION, |
354 | + '$DBI::VERSION:', $DBI::VERSION, |
355 | + ); |
356 | + |
357 | + return $dbh; |
358 | +} |
359 | + |
360 | +# Tries to figure out a hostname for the connection. |
361 | +sub get_hostname { |
362 | + my ( $self, $dbh ) = @_; |
363 | + if ( my ($host) = ($dbh->{mysql_hostinfo} || '') =~ m/^(\w+) via/ ) { |
364 | + return $host; |
365 | + } |
366 | + my ( $hostname, $one ) = $dbh->selectrow_array( |
367 | + 'SELECT /*!50038 @@hostname, */ 1'); |
368 | + return $hostname; |
369 | +} |
370 | + |
371 | +# Disconnects a database handle, but complains verbosely if there are any active |
372 | +# children. These are usually $sth handles that haven't been finish()ed. |
373 | +sub disconnect { |
374 | + my ( $self, $dbh ) = @_; |
375 | + MKDEBUG && $self->print_active_handles($dbh); |
376 | + $dbh->disconnect; |
377 | +} |
378 | + |
379 | +sub print_active_handles { |
380 | + my ( $self, $thing, $level ) = @_; |
381 | + $level ||= 0; |
382 | + printf("# Active %sh: %s %s %s\n", ($thing->{Type} || 'undef'), "\t" x $level, |
383 | + $thing, (($thing->{Type} || '') eq 'st' ? $thing->{Statement} || '' : '')) |
384 | + or die "Cannot print: $OS_ERROR"; |
385 | + foreach my $handle ( grep {defined} @{ $thing->{ChildHandles} } ) { |
386 | + $self->print_active_handles( $handle, $level + 1 ); |
387 | + } |
388 | +} |
389 | + |
390 | +# Copy all set vals in dsn_1 to dsn_2. Existing val in dsn_2 are not |
391 | +# overwritten unless overwrite=>1 is given, but undef never overwrites a |
392 | +# val. |
393 | +sub copy { |
394 | + my ( $self, $dsn_1, $dsn_2, %args ) = @_; |
395 | + die 'I need a dsn_1 argument' unless $dsn_1; |
396 | + die 'I need a dsn_2 argument' unless $dsn_2; |
397 | + my %new_dsn = map { |
398 | + my $key = $_; |
399 | + my $val; |
400 | + if ( $args{overwrite} ) { |
401 | + $val = defined $dsn_1->{$key} ? $dsn_1->{$key} : $dsn_2->{$key}; |
402 | + } |
403 | + else { |
404 | + $val = defined $dsn_2->{$key} ? $dsn_2->{$key} : $dsn_1->{$key}; |
405 | + } |
406 | + $key => $val; |
407 | + } keys %{$self->{opts}}; |
408 | + return \%new_dsn; |
409 | +} |
410 | + |
411 | +sub _d { |
412 | + my ($package, undef, $line) = caller 0; |
413 | + @_ = map { (my $temp = $_) =~ s/\n/\n# /g; $temp; } |
414 | + map { defined $_ ? $_ : 'undef' } |
415 | + @_; |
416 | + print STDERR "# $package:$line $PID ", join(' ', @_), "\n"; |
417 | +} |
418 | + |
419 | +1; |
420 | + |
421 | +# ########################################################################### |
422 | +# End DSNParser package |
423 | +# ########################################################################### |
424 | |
425 | === added file 'innobackupex/common/IbexTest.pm' |
426 | --- innobackupex/common/IbexTest.pm 1970-01-01 00:00:00 +0000 |
427 | +++ innobackupex/common/IbexTest.pm 2010-01-22 01:31:15 +0000 |
428 | @@ -0,0 +1,133 @@ |
429 | +# This program is copyright 2010 Percona Inc. |
430 | +# Feedback and improvements are welcome. |
431 | +# |
432 | +# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED |
433 | +# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF |
434 | +# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. |
435 | +# |
436 | +# This program is free software; you can redistribute it and/or modify it under |
437 | +# the terms of the GNU General Public License as published by the Free Software |
438 | +# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar |
439 | +# systems, you can issue `man perlgpl' or `man perlartistic' to read these |
440 | +# licenses. |
441 | +# |
442 | +# You should have received a copy of the GNU General Public License along with |
443 | +# this program; if not, write to the Free Software Foundation, Inc., 59 Temple |
444 | +# Place, Suite 330, Boston, MA 02111-1307 USA. |
445 | +package IbexTest; |
446 | + |
447 | +BEGIN { |
448 | + die "The XTRABACKUP_TRUNK environment variable is not set." |
449 | + unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK}; |
450 | + unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common"; |
451 | +}; |
452 | +our $trunk = $ENV{XTRABACKUP_TRUNK}; |
453 | + |
454 | +use strict; |
455 | +use warnings FATAL => 'all'; |
456 | +use English qw(-no_match_vars); |
457 | +use Test::More; |
458 | +use Data::Dumper; |
459 | +$Data::Dumper::Indent = 1; |
460 | +$Data::Dumper::Sortkeys = 1; |
461 | +$Data::Dumper::Quotekeys = 0; |
462 | + |
463 | +require Exporter; |
464 | +our @ISA = qw(Exporter); |
465 | +our %EXPORT_TAGS = (); |
466 | +our @EXPORT = qw( |
467 | + output |
468 | + load_file |
469 | + wait_until |
470 | + wait_for |
471 | + no_diff |
472 | + $trunk |
473 | +); |
474 | +our @EXPORT_OK = qw(); |
475 | + |
476 | +sub output { |
477 | + my ( $code ) = @_; |
478 | + die "I need a code argument" unless $code; |
479 | + my $output = ''; |
480 | + open my $output_fh, '>', \$output |
481 | + or die "Cannot capture output to variable: $OS_ERROR"; |
482 | + select $output_fh; |
483 | + eval { $code->() }; |
484 | + close $output_fh; |
485 | + select STDOUT; |
486 | + die $EVAL_ERROR if $EVAL_ERROR; |
487 | + return $output; |
488 | +} |
489 | + |
490 | +# Slurp file and return its entire contents. |
491 | +sub load_file { |
492 | + my ( $file, %args ) = @_; |
493 | + $file = "$trunk/$file"; |
494 | + open my $fh, "<", $file or die "Cannot open $file: $OS_ERROR"; |
495 | + my $contents = do { local $/ = undef; <$fh> }; |
496 | + close $fh; |
497 | + chomp $contents if $args{chomp_contents}; |
498 | + return $contents; |
499 | +} |
500 | + |
501 | +# Wait until code returns true. |
502 | +sub wait_until { |
503 | + my ( $code, $t, $max_t ) = @_; |
504 | + my $slept = 0; |
505 | + my $sleep_int = $t || .5; |
506 | + $t ||= .5; |
507 | + $max_t ||= 5; |
508 | + $t *= 1_000_000; |
509 | + while ( $slept <= $max_t ) { |
510 | + return if $code->(); |
511 | + usleep($t); |
512 | + $slept += $sleep_int; |
513 | + } |
514 | + return; |
515 | +} |
516 | + |
517 | +# Wait t seconds for code to return. |
518 | +sub wait_for { |
519 | + my ( $code, $t ) = @_; |
520 | + $t ||= 0; |
521 | + my $mask = POSIX::SigSet->new(&POSIX::SIGALRM); |
522 | + my $action = POSIX::SigAction->new( |
523 | + sub { die }, |
524 | + $mask, |
525 | + ); |
526 | + my $oldaction = POSIX::SigAction->new(); |
527 | + sigaction(&POSIX::SIGALRM, $action, $oldaction); |
528 | + eval { |
529 | + alarm $t; |
530 | + $code->(); |
531 | + alarm 0; |
532 | + }; |
533 | + if ( $EVAL_ERROR ) { |
534 | + # alarm was raised |
535 | + return 1; |
536 | + } |
537 | + return 0; |
538 | +} |
539 | + |
540 | +# Returns true (1) if there's no difference between the |
541 | +# cmd's output and the expected output. If $update_sample |
542 | +# or the env var UPDATE_SAMPLES is true then the $cmd |
543 | +# output is written to the $expected_output file. |
544 | +sub no_diff { |
545 | + my ( $cmd, $expected_output, $update_sample ) = @_; |
546 | + die "I need a cmd argument" unless $cmd; |
547 | + die "I need an expected_output argument" unless $expected_output; |
548 | + my $output_file = '/tmp/ibex-test-output.txt'; |
549 | + $expected_output = "$trunk/$expected_output"; |
550 | + `$cmd > $output_file`; |
551 | + if ( $ENV{UPDATE_SAMPLES} || $update_sample ) { |
552 | + `cat $output_file > $expected_output`; |
553 | + print STDERR "Updated $expected_output\n"; |
554 | + } |
555 | + my $retval = system("diff $expected_output $output_file"); |
556 | + `rm -rf $output_file`; |
557 | + $retval = $retval >> 8; |
558 | + return !$retval; |
559 | +} |
560 | + |
561 | +1; |
562 | |
563 | === added file 'innobackupex/common/Sandbox.pm' |
564 | --- innobackupex/common/Sandbox.pm 1970-01-01 00:00:00 +0000 |
565 | +++ innobackupex/common/Sandbox.pm 2010-01-22 01:31:15 +0000 |
566 | @@ -0,0 +1,168 @@ |
567 | +# This program is copyright 2010 Percona Inc. |
568 | +# Feedback and improvements are welcome. |
569 | +# |
570 | +# THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED |
571 | +# WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF |
572 | +# MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. |
573 | +# |
574 | +# This program is free software; you can redistribute it and/or modify it under |
575 | +# the terms of the GNU General Public License as published by the Free Software |
576 | +# Foundation, version 2; OR the Perl Artistic License. On UNIX and similar |
577 | +# systems, you can issue `man perlgpl' or `man perlartistic' to read these |
578 | +# licenses. |
579 | +# |
580 | +# You should have received a copy of the GNU General Public License along with |
581 | +# this program; if not, write to the Free Software Foundation, Inc., 59 Temple |
582 | +# Place, Suite 330, Boston, MA 02111-1307 USA. |
583 | +package Sandbox; |
584 | + |
585 | +BEGIN { |
586 | + die "The XTRABACKUP_TRUNK environment variable is not set." |
587 | + unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK}; |
588 | + unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common"; |
589 | +}; |
590 | + |
591 | +use strict; |
592 | +use warnings FATAL => 'all'; |
593 | +use English qw(-no_match_vars); |
594 | + |
595 | +my $trunk = $ENV{XTRABACKUP_TRUNK}; |
596 | + |
597 | +use constant IBEXDEBUG => $ENV{IBEXDEBUG} || 0; |
598 | + |
599 | +my %port_for = ( |
600 | + master => 12345, |
601 | + slave1 => 12346, |
602 | + slave2 => 12347, |
603 | + master1 => 12348, # master-master |
604 | + master2 => 12349, # master-master |
605 | +); |
606 | + |
607 | +sub new { |
608 | + my ( $class, %args ) = @_; |
609 | + foreach my $arg ( qw(basedir DSNParser) ) { |
610 | + die "I need a $arg argument" unless defined $args{$arg}; |
611 | + } |
612 | + |
613 | + if ( !-d $args{basedir} ) { |
614 | + die "$args{basedir} is not a directory"; |
615 | + } |
616 | + |
617 | + return bless { %args }, $class; |
618 | +} |
619 | + |
620 | +sub use { |
621 | + my ( $self, $server, $cmd ) = @_; |
622 | + _check_server($server); |
623 | + return if !defined $cmd || !$cmd; |
624 | + my $use = $self->_use_for($server) . " $cmd"; |
625 | + IBEXDEBUG && _d('"Executing', $use, 'on', $server); |
626 | + eval { |
627 | + `$use`; |
628 | + }; |
629 | + if ( $EVAL_ERROR ) { |
630 | + die "Failed to execute $cmd on $server: $EVAL_ERROR"; |
631 | + } |
632 | + return; |
633 | +} |
634 | + |
635 | +sub create_dbs { |
636 | + my ( $self, $dbh, $dbs, %args ) = @_; |
637 | + die 'I need a dbh' if !$dbh; |
638 | + return if ( !ref $dbs || scalar @$dbs == 0 ); |
639 | + my %default_args = ( |
640 | + repl => 1, |
641 | + drop => 1, |
642 | + ); |
643 | + %args = ( %default_args, %args ); |
644 | + |
645 | + $dbh->do('SET SQL_LOG_BIN=0') unless $args{repl}; |
646 | + |
647 | + foreach my $db ( @$dbs ) { |
648 | + $dbh->do("DROP DATABASE IF EXISTS `$db`") if $args{drop}; |
649 | + |
650 | + my $sql = "CREATE DATABASE `$db`"; |
651 | + eval { |
652 | + $dbh->do($sql); |
653 | + }; |
654 | + die $EVAL_ERROR if $EVAL_ERROR; |
655 | + } |
656 | + |
657 | + $dbh->do('SET SQL_LOG_BIN=1'); |
658 | + |
659 | + return; |
660 | +} |
661 | + |
662 | +sub get_dbh_for { |
663 | + my ( $self, $server, $cxn_ops ) = @_; |
664 | + _check_server($server); |
665 | + $cxn_ops ||= { AutoCommit => 1 }; |
666 | + IBEXDEBUG && _d('dbh for', $server, 'on port', $port_for{$server}); |
667 | + my $dp = $self->{DSNParser}; |
668 | + my $dsn = $dp->parse('h=127.0.0.1,u=msandbox,p=msandbox,P=' . $port_for{$server}); |
669 | + my $dbh; |
670 | + eval { $dbh = $dp->get_dbh($dp->get_cxn_params($dsn), $cxn_ops) }; |
671 | + if ( $EVAL_ERROR ) { |
672 | + IBEXDEBUG && _d('Failed to get dbh for', $server, ':', $EVAL_ERROR); |
673 | + return 0; |
674 | + } |
675 | + return $dbh; |
676 | +} |
677 | + |
678 | +sub load_file { |
679 | + my ( $self, $server, $file, $use_db ) = @_; |
680 | + _check_server($server); |
681 | + $file = "$trunk/$file"; |
682 | + if ( !-f $file ) { |
683 | + die "$file is not a file"; |
684 | + } |
685 | + |
686 | + my $d = $use_db ? "-D $use_db" : ''; |
687 | + |
688 | + my $use = $self->_use_for($server) . " $d < $file"; |
689 | + IBEXDEBUG && _d('Loading', $file, 'on', $server, ':', $use); |
690 | + eval { `$use` }; |
691 | + if ( $EVAL_ERROR ) { |
692 | + die "Failed to execute $file on $server: $EVAL_ERROR"; |
693 | + } |
694 | + |
695 | + return; |
696 | +} |
697 | + |
698 | +sub _use_for { |
699 | + my ( $self, $server ) = @_; |
700 | + return "$self->{basedir}/$port_for{$server}/use"; |
701 | +} |
702 | + |
703 | +sub _check_server { |
704 | + my ( $server ) = @_; |
705 | + if ( !exists $port_for{$server} ) { |
706 | + die "Unknown server $server"; |
707 | + } |
708 | + return; |
709 | +} |
710 | + |
711 | +sub wipe_clean { |
712 | + my ( $self, $dbh ) = @_; |
713 | + foreach my $db ( @{$dbh->selectcol_arrayref('SHOW DATABASES')} ) { |
714 | + next if $db eq 'mysql'; |
715 | + next if $db eq 'information_schema'; |
716 | + next if $db eq 'sakila'; |
717 | + $dbh->do("DROP DATABASE IF EXISTS `$db`"); |
718 | + } |
719 | + return; |
720 | +} |
721 | + |
722 | +sub _d { |
723 | + my ($package, undef, $line) = caller 0; |
724 | + @_ = map { (my $temp = $_) =~ s/\n/\n# /g; $temp; } |
725 | + map { defined $_ ? $_ : 'undef' } |
726 | + @_; |
727 | + print STDERR "# $package:$line $PID ", join(' ', @_), "\n"; |
728 | +} |
729 | + |
730 | +1; |
731 | + |
732 | +# ########################################################################### |
733 | +# End Sandbox package |
734 | +# ########################################################################### |
735 | |
736 | === added file 'innobackupex/innobackupex' |
737 | --- innobackupex/innobackupex 1970-01-01 00:00:00 +0000 |
738 | +++ innobackupex/innobackupex 2010-01-22 01:31:15 +0000 |
739 | @@ -0,0 +1,2400 @@ |
740 | +#!/usr/bin/env perl |
741 | + |
742 | +# A script for making backups of InnoDB and MyISAM tables, indexes and .frm |
743 | +# files. |
744 | +# |
745 | +# Orginal code copyright 2003, 2009 Innobase Oy. |
746 | +# http://www.innodb.com/download/ |
747 | +# |
748 | +# Modified code copyright 2010 Percona Inc. |
749 | +# https://launchpad.net/percona-xtrabackup/trunk |
750 | + |
751 | +# ############################################################################ |
752 | +# This is a a runnable module. Check at the end of this package for the call |
753 | +# to main() which actually runs the program. |
754 | +# ############################################################################ |
755 | +package innobackupex; |
756 | + |
757 | +use strict; |
758 | +use warnings FATAL => 'all'; |
759 | +use English qw(-no_match_vars); |
760 | +use Getopt::Long qw(:config default); |
761 | +use File::Spec; |
762 | +use POSIX "strftime"; |
763 | +use POSIX ":sys_wait_h"; |
764 | +use POSIX "tmpnam"; |
765 | +use FileHandle; |
766 | +use File::Basename; |
767 | + |
768 | +use constant IBEXDEBUG => $ENV{IBEXDEBUG} || 0; |
769 | + |
770 | +# version of this script |
771 | +my $innobackup_version = '1.5.1-xtrabackup'; |
772 | +my $innobackup_script = basename($0); |
773 | + |
774 | +# copyright notice |
775 | +my $copyright_notice = "InnoDB Backup Utility v${innobackup_version}; Copyright 2003, 2009 Innobase Oy. |
776 | +All Rights Reserved. |
777 | + |
778 | +This software is published under |
779 | +the GNU GENERAL PUBLIC LICENSE Version 2, June 1991. |
780 | + |
781 | +"; |
782 | + |
783 | +# required Perl version (5.005) |
784 | +my @required_perl_version = ( 5, 0, 5 ); |
785 | +my $required_perl_version_old_style = 5.005; |
786 | + |
787 | +# force flush after every write and print |
788 | +$OUTPUT_AUTOFLUSH = 1; |
789 | + |
790 | +###################################################################### |
791 | +# modifiable parameters |
792 | +###################################################################### |
793 | + |
794 | +# maximum number of files in a database directory which are |
795 | +# separately printed when a backup is made |
796 | +my $backup_file_print_limit = 9; |
797 | + |
798 | +# timeout in seconds for a reply from mysql |
799 | +my $mysql_response_timeout = 900; |
800 | + |
801 | +# default compression level (this is an argument to ibbackup) |
802 | +my $default_compression_level = 1; |
803 | + |
804 | +# time in seconds after which a dummy query is sent to mysql server |
805 | +# in order to keep the database connection alive |
806 | +my $mysql_keep_alive_timeout = 1800; |
807 | + |
808 | +###################################################################### |
809 | +# end of modifiable parameters |
810 | +###################################################################### |
811 | + |
812 | +# command line options |
813 | +my $o = { |
814 | + 'compress' => 999, |
815 | + 'databases' => '', |
816 | + 'ibbackup' => 'xtrabackup', |
817 | + 'scp-opt' => '-Cp -c arcfour', |
818 | + 'stream' => '', |
819 | +}; |
820 | + |
821 | +# root of the backup directory |
822 | +my $backup_root = ''; |
823 | + |
824 | +# backup directory pathname |
825 | +my $backup_dir = ''; |
826 | + |
827 | +# name of the ibbackup suspend-at-end file |
828 | +my $suspend_file = ''; |
829 | + |
830 | +# name of the temporary transaction log file during the backup |
831 | +my $tmp_logfile = ''; |
832 | + |
833 | +# home directory of innoDB log files |
834 | +my $innodb_log_group_home_dir = ''; |
835 | + |
836 | +# backup my.cnf file |
837 | +my $backup_config_file = ''; |
838 | + |
839 | +# options from the options file |
840 | +my %config; |
841 | + |
842 | +# options from the backup options file |
843 | +#my %backup_config; |
844 | + |
845 | +# list of databases to be included in a backup |
846 | +my %databases_list; |
847 | + |
848 | +# prefix for output lines |
849 | +my $prefix = "$innobackup_script:"; |
850 | + |
851 | +# process id of mysql client program (runs as a child process of this script) |
852 | +my $mysql_pid = ''; |
853 | + |
854 | +# mysql server version string |
855 | +my $mysql_server_version = ''; |
856 | + |
857 | +# name of the file where stderr of mysql process is directed |
858 | +my $mysql_stderr; |
859 | + |
860 | +# name of the file where stdout of mysql process is directed |
861 | +my $mysql_stdout; |
862 | + |
863 | +# name of the file where binlog position info is written |
864 | +my $binlog_info; |
865 | + |
866 | +# name of the file where slave info is written |
867 | +my $slave_info; |
868 | + |
869 | +# mysql binlog position as given by "SHOW MASTER STATUS" command |
870 | +my $mysql_binlog_position = ''; |
871 | + |
872 | +# mysql master's binlog position as given by "SHOW SLAVE STATUS" command |
873 | +# run on a slave server |
874 | +my $mysql_slave_position = ''; |
875 | + |
876 | +# time of the most recent mysql_check call. (value returned by time() function) |
877 | +my $mysql_last_access_time = 0; |
878 | + |
879 | +# process id of ibbackup program (runs as a child process of this script) |
880 | +my $ibbackup_pid = ''; |
881 | + |
882 | +# a counter for numbering mysql connection checks |
883 | +my $hello_id = 0; |
884 | + |
885 | +# the request which has been sent to mysqld, but to which |
886 | +# mysqld has not yet replied. Empty string denotes that no |
887 | +# request has been sent to mysqld or that mysqld has replied |
888 | +# to all requests. |
889 | +my $current_mysql_request = ''; |
890 | + |
891 | +# escape sequences for options files |
892 | +my %option_value_escapes = ( |
893 | + 'b' => "\b", |
894 | + 't' => "\t", |
895 | + 'n' => "\n", |
896 | + 'r' => "\r", |
897 | + "\\" => "\\", |
898 | + 's' => ' ' |
899 | +); |
900 | + |
901 | +# signal that is sent to child processes when they are killed |
902 | +my $kill_signal = 15; |
903 | + |
904 | +# current local time |
905 | +my $now; |
906 | + |
907 | +# incremental backup base directory |
908 | +my $incremental_basedir = ''; |
909 | + |
910 | +my $src_name; |
911 | +my $dst_name; |
912 | +my $win = ( $^O eq 'MSWin32' ? 1 : 0 ); |
913 | +my $CP_CMD = ( $win eq 1 ? "copy /Y" : "cp -p" ); |
914 | + |
915 | +###################################################################### |
916 | +# program execution begins here |
917 | +###################################################################### |
918 | +sub main { |
919 | + @ARGV = @_; # set global ARGV for this package |
920 | + |
921 | + # check command-line args |
922 | + check_args(); |
923 | + |
924 | + # print program version and copyright |
925 | + print_version(); |
926 | + |
927 | + # initialize global variables and perform some checks |
928 | + init(); |
929 | + |
930 | + if ( $o->{'copy-back'} ) { |
931 | + # copy files from backup directory back to their original locations |
932 | + copy_back(); |
933 | + } |
934 | + elsif ( $o->{'apply-log'} ) { |
935 | + # expand data files in backup directory by applying log files to them |
936 | + apply_log(); |
937 | + } |
938 | + else { |
939 | + # make a backup of InnoDB and MyISAM tables, indexes and .frm files. |
940 | + backup(); |
941 | + } |
942 | + |
943 | + # program has completed successfully |
944 | + $now = current_time(); |
945 | + print STDERR "$now $prefix completed OK!\n"; |
946 | + |
947 | + if ( $o->{'stream'} eq 'tar' ) { |
948 | + print STDERR "$prefix You must use -i (--ignore-zeros) option for extraction of the tar stream.\n"; |
949 | + } |
950 | + |
951 | + return 0; |
952 | +} |
953 | + |
954 | +############################################################################## |
955 | +# Subroutines |
956 | +############################################################################## |
957 | + |
958 | +# |
959 | +# print_version subroutine prints program version and copyright. |
960 | +# |
961 | +sub print_version { |
962 | + printf( STDERR $copyright_notice ); |
963 | +} |
964 | + |
965 | +# |
966 | +# usage subroutine prints instructions of how to use this program to stdout. |
967 | +# |
968 | +sub usage { |
969 | + print STDOUT <<EOF; |
970 | + |
971 | +Usage: |
972 | + |
973 | + $innobackup_script BACKUP-ROOT-DIR [OPTIONS] |
974 | + |
975 | + $innobackup_script --apply-log BACKUP-DIR [OPTIONS] |
976 | + |
977 | + $innobackup_script --copy-back BACKUP-DIR [OPTIONS] |
978 | + |
979 | +The first command line above makes a hot backup of a MySQL database. |
980 | +By default it creates a backup directory (named by the current date |
981 | +and time) in the given BACKUP-ROOT-DIR directory. With the --no-timestamp |
982 | +option it does not create a time-stamped backup directory, but it puts |
983 | +the backup in the given directory (which must not exist). This |
984 | +command makes a complete backup of all MyISAM and InnoDB tables and |
985 | +indexes in all databases or in all of the databases specified with the |
986 | +--databases option. The created backup contains .frm, .MRG, .MYD, |
987 | +.MYI, .TRG, .TRN, .ARM, .ARZ, .opt, .par, and InnoDB data and log files. |
988 | +The MY.CNF options file defines the location of the database. This command |
989 | +connects to the MySQL server using mysql client program, and runs |
990 | +ibbackup (InnoDB Hot Backup program) as a child process. |
991 | + |
992 | +The command with --apply-log option prepares a backup for starting a MySQL |
993 | +server on the backup. This command expands InnoDB data files as specified |
994 | +in BACKUP-DIR/backup-my.cnf using BACKUP-DIR/ibbackup_logfile, |
995 | +and creates new InnoDB log files as specified in BACKUP-DIR/backup-my.cnf. |
996 | +The BACKUP-DIR should be a path name of a backup directory created by |
997 | +innobackup. This command runs ibbackup as a child process, but it does not |
998 | +connect to the database server. |
999 | + |
1000 | +The command with --copy-back option copies data, index, and log files |
1001 | +from backup directory back to their original locations. |
1002 | +The MY.CNF options file defines the original location of the database. |
1003 | +The BACKUP-DIR is a path name of a backup directory created by innobackup. |
1004 | + |
1005 | +On success the exit code of innobackup process is 0. A non-zero exit code |
1006 | +indicates an error. |
1007 | + |
1008 | +Options: |
1009 | + |
1010 | + --help Display this helpscreen and exit. |
1011 | + |
1012 | + --version Print version information and exit. |
1013 | + |
1014 | + --defaults-file=[MY.CNF] |
1015 | + This option is passed to the xtrabackup child process |
1016 | + and the mysql child process. It specifies which file |
1017 | + to read default options from. |
1018 | + |
1019 | + --apply-log |
1020 | + Prepare a backup for starting mysql server on the backup. |
1021 | + Expand InnoDB data files as specified in |
1022 | + backup-dir/backup-my.cnf, using backup-dir/ibbackup_logfile, |
1023 | + and create new log files as specified in |
1024 | + backup-dir/backup-my.cnf. |
1025 | + |
1026 | + --copy-back |
1027 | + Copy data and index files from backup directory back to |
1028 | + their original locations. |
1029 | + |
1030 | + --remote-host=[USER@]HOSTNAME |
1031 | + If this option is specified, backup files will be created |
1032 | + at the remote host by using ssh and scp. BACKUP-ROOT-DIR |
1033 | + is relative to this host. |
1034 | + |
1035 | + --stream=[tar|cpio(not implemented)] |
1036 | + If this option is specified, backup to STDOUT as specified |
1037 | + format. |
1038 | + |
1039 | + --tmpdir=DIRECTORY |
1040 | + When --remote-host or --stream specified, transaction log |
1041 | + should be stored temporary. The default value is same to |
1042 | + mysqld setting. |
1043 | + |
1044 | + --use-memory=MB |
1045 | + This option is passed to the ibbackup child process. |
1046 | + It tells ibbackup that it can use MB megabytes of |
1047 | + memory in restoration. |
1048 | + Try 'ibbackup --help' for more details on this option. |
1049 | + |
1050 | + --throttle=IOS |
1051 | + This option is passed to the xtrabackup child process. |
1052 | + It limits count of IO operations (pairs of read&write) per |
1053 | + second to IOS values (for '--backup') |
1054 | + |
1055 | + --sleep=MS |
1056 | + This option is passed to the ibbackup child process. |
1057 | + It instructs the ibbackup program to sleep |
1058 | + MS milliseconds after each 1 MB of copied data. |
1059 | + You can use this parameter to tune the additional |
1060 | + disk i/o load the ibbackup program causes on the computer. |
1061 | + Try 'ibbackup --help' for more details on this option. |
1062 | + ** it is not supported by xtrabackup ** |
1063 | + |
1064 | + --compress[=LEVEL] |
1065 | + This option is passed to the ibbackup child process. |
1066 | + It instructs ibbackup to compress the backup copies of |
1067 | + InnoDB data files. Compression level can be |
1068 | + specified as an optional argument. Compression level is |
1069 | + an integer between 0 and 9: 1 gives fastest compression, |
1070 | + 9 gives best compression, and 0 means no compression. |
1071 | + If compression level is not given, the default level 1 is used. |
1072 | + Try 'ibbackup --help' for more details on this option. |
1073 | + ** it is not supported by xtrabackup yet ** |
1074 | + |
1075 | + --include=REGEXP |
1076 | + This option is passed to the ibbackup child process. |
1077 | + It tells ibbackup to backup only those per-table data |
1078 | + files which match the given regular expression. For |
1079 | + each table with a per-table data file a string of the |
1080 | + form db_name.table_name is checked against the regular |
1081 | + expression. If the regular expression matches the |
1082 | + complete string db_name.table_name, the table is |
1083 | + included in the backup. The regular expression should |
1084 | + be of the POSIX 1003.2 "extended" form. |
1085 | + Try 'ibbackup --help' for more details on this option. |
1086 | + |
1087 | + --databases=LIST |
1088 | + This option is used to specify the list of databases that |
1089 | + innobackup should backup. The list is of the form |
1090 | + "db_name[.table_name] db_name1[.table_name1] ...". |
1091 | + If this option is not specified all databases containing |
1092 | + MyISAM and InnoDB tables will be backed up. |
1093 | + Please make sure that --databases contains all of the |
1094 | + innodb databases & tables so that all of the innodb .frm |
1095 | + files are also backed up. In case the list is very long, |
1096 | + this can be specified in a file and the full path of the |
1097 | + file can be specified instead of the list. |
1098 | + |
1099 | + --uncompress |
1100 | + This option is passed to the ibbackup child process. |
1101 | + It tells ibbackup to uncompress compressed InnoDB data files. |
1102 | + Try 'ibbackup --help' for more details on this option. |
1103 | + ** it is not supported by xtrabackup yet ** |
1104 | + |
1105 | + --export |
1106 | + This option is passed to the xtrabackup child process. |
1107 | + It is effective with '--prepare' option. |
1108 | + It tells xtrabackup to output "clean" .ibd files and .exp |
1109 | + files for 'ALTER TABLE ... IMPORT TABLESPACE' command |
1110 | + at innodb_expand_import option enabled XtraDB. |
1111 | + |
1112 | + --user=NAME |
1113 | + This option is passed to the mysql child process. |
1114 | + It defines the user for database login if not current user. |
1115 | + Try 'mysql --help' for more details on this option. |
1116 | + |
1117 | + --password=WORD |
1118 | + This option is passed to the mysql child process. |
1119 | + It defines the password to use when connecting to database. |
1120 | + Try 'mysql --help' for more details on this option. |
1121 | + |
1122 | + --host=HOST |
1123 | + This option is passed to the mysql child process. |
1124 | + It defines the hostname or IP address to use when connecting |
1125 | + to the database server. |
1126 | + Try 'mysql --help' for more details on this option. |
1127 | + |
1128 | + --port=PORT |
1129 | + This option is passed to the mysql child process. |
1130 | + It defines the port to use when connecting to local database |
1131 | + server with TCP/IP. |
1132 | + Try 'mysql --help' for more details on this option. |
1133 | + |
1134 | + --slave-info |
1135 | + This option is useful when backing up a replication |
1136 | + slave server. It prints the binary log position and |
1137 | + name of the binary log file of the master server. |
1138 | + It also writes this information to the 'ibbackup_slave_info' |
1139 | + file as a 'CHANGE MASTER' command. A new slave for this |
1140 | + master can be set up by starting a slave server on this |
1141 | + backup and issuing a 'CHANGE MASTER' command with the binary |
1142 | + log position saved in the 'ibbackup_slave_info' file. |
1143 | + |
1144 | + --socket=SOCKET |
1145 | + This option is passed to the mysql child process. |
1146 | + It defines the socket to use when connecting to local database |
1147 | + server with UNIX domain socket. |
1148 | + Try 'mysql --help' for more details on this option. |
1149 | + |
1150 | + --no-timestamp |
1151 | + This option prevents the creation of a new backup |
1152 | + directory (named by the current date and time) under |
1153 | + the backup root directory. Instead, the backup is put |
1154 | + in the directory given on the command-line (in the |
1155 | + place of BACKUP-ROOT-DIR argument). The directory must not |
1156 | + exist, because innobackup creates it while making a backup. |
1157 | + |
1158 | + --ibbackup=IBBACKUP-BINARY |
1159 | + Use this option to specify which ibbackup (InnoDB Hot |
1160 | + Backup) binary should be used. IBBACKUP-BINARY |
1161 | + should be the command used to run ibbackup. This can |
1162 | + be useful if ibbackup is not in your search path or |
1163 | + working directory. If this option is not given, |
1164 | + ibbackup is run with command "ibbackup". |
1165 | + --no-lock |
1166 | + Use this option to disable table lock with |
1167 | + FLUSH TABLES WITH READ LOCK. use it only while ALL your |
1168 | + tables are InnoDB and you DO NOT CARE about binary log |
1169 | + position of backup. |
1170 | + |
1171 | + --scpopt=SCP-OPTIONS |
1172 | + Use this option to specify the command line options |
1173 | + to pass to scp. The default options are '-Cp -c arcfour'. |
1174 | + |
1175 | +EOF |
1176 | + |
1177 | + print STDOUT "Option values after processing arguments:\n\n"; |
1178 | + print STDOUT |
1179 | + join( "\n", sort map { " --$_=" . $o->{$_} || '' } keys %$o ), |
1180 | + "\n\n"; |
1181 | + |
1182 | + return; |
1183 | +} |
1184 | + |
1185 | +# |
1186 | +# return current local time as string in form "070816 12:23:15" |
1187 | +# |
1188 | +sub current_time { |
1189 | + return strftime( "%y%m%d %H:%M:%S", localtime() ); |
1190 | +} |
1191 | + |
1192 | +# |
1193 | +# Die subroutine kills all child processes and exits this process. |
1194 | +# This subroutine takes the same argument as the built-in die function. |
1195 | +# Parameters: |
1196 | +# message string which is printed to stdout |
1197 | +# |
1198 | +sub Die { |
1199 | + my $message = shift; |
1200 | + my $extra_info = ''; |
1201 | + |
1202 | + # kill all child processes of this process |
1203 | + kill_child_processes(); |
1204 | + |
1205 | + if ($current_mysql_request) { |
1206 | + $extra_info = " while waiting for reply to MySQL request:" |
1207 | + . " '$current_mysql_request'"; |
1208 | + } |
1209 | + |
1210 | + IBEXDEBUG && _d('Die', $prefix, $message, $extra_info); |
1211 | + die "$prefix Error: $message$extra_info"; |
1212 | +} |
1213 | + |
1214 | +# |
1215 | +# backup subroutine makes a backup of InnoDB and MyISAM tables, indexes and |
1216 | +# .frm files. It connects to the database server and runs ibbackup as a child |
1217 | +# process. |
1218 | +# |
1219 | +sub backup { |
1220 | + my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' ); |
1221 | + |
1222 | + # check that we can connect to the database. This done by |
1223 | + # connecting, issuing a query, and closing the connection. |
1224 | + mysql_open(); |
1225 | + mysql_check(); |
1226 | + mysql_close(); |
1227 | + |
1228 | + # start ibbackup as a child process |
1229 | + start_ibbackup(); |
1230 | + |
1231 | + # wait for ibbackup to suspend itself |
1232 | + if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
1233 | + wait_for_ibbackup_suspend(); |
1234 | + } |
1235 | + |
1236 | + if ( !$o->{'incremental'} ) { |
1237 | + mysql_open(); |
1238 | + mysql_check(); |
1239 | + mysql_lockall() unless $o->{'no-lock'}; |
1240 | + backup_files(); |
1241 | + } |
1242 | + |
1243 | + # resume ibbackup and wait till it has finished |
1244 | + resume_ibbackup(); |
1245 | + |
1246 | + if ( !$o->{'incremental'} ) { |
1247 | + mysql_unlockall() unless $o->{'no-lock'}; |
1248 | + mysql_close(); |
1249 | + } |
1250 | + |
1251 | + if ( $o->{'remote-host'} ) { |
1252 | + system_call("scp $o->{'scp-opt'} '$tmp_logfile' " |
1253 | + . "'$o->{'remote-host'}:$backup_dir/xtrabackup_logfile'"); |
1254 | + |
1255 | + system_call("rm -f '$tmp_logfile'"); |
1256 | + |
1257 | + system_call("scp $o->{'scp-opt'} '$orig_datadir/xtrabackup_checkpoints' " |
1258 | + . "'$o->{'remote-host'}:$backup_dir/xtrabackup_checkpoints'"); |
1259 | + |
1260 | + system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'"); |
1261 | + } |
1262 | + elsif ( $o->{'stream'} eq 'tar' ) { |
1263 | + system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_logfile"); |
1264 | + |
1265 | + system_call("rm -f '$tmp_logfile'"); |
1266 | + |
1267 | + system_call("cd $orig_datadir; tar chf - xtrabackup_checkpoints"); |
1268 | + |
1269 | + system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'"); |
1270 | + } |
1271 | + |
1272 | + print STDERR "\n$prefix Backup created in directory " |
1273 | + . ($o->{'remote-host'} ? "$o->{'remote-host'}:" : '') |
1274 | + . "'$backup_dir'\n"; |
1275 | + |
1276 | + if ($mysql_binlog_position) { |
1277 | + print STDERR "$prefix MySQL binlog position: $mysql_binlog_position\n"; |
1278 | + } |
1279 | + if ( $mysql_slave_position && $o->{'slave-info'} ) { |
1280 | + print STDERR "$prefix MySQL slave binlog position: " |
1281 | + . "$mysql_slave_position\n"; |
1282 | + } |
1283 | + |
1284 | + return; |
1285 | +} |
1286 | + |
1287 | +# |
1288 | +# are_equal_innodb_data_file_paths subroutine checks if the given |
1289 | +# InnoDB data file option values are equal. |
1290 | +# Parameters: |
1291 | +# str1 InnoDB data file path option value |
1292 | +# str2 InnoDB data file path option value |
1293 | +# Return value: |
1294 | +# 1 if values are equal |
1295 | +# 0 otherwise |
1296 | +# |
1297 | +sub are_equal_innodb_data_file_paths { |
1298 | + my $str1 = shift; |
1299 | + my $str2 = shift; |
1300 | + my @array1 = split( /;/, $str1 ); |
1301 | + my @array2 = split( /;/, $str2 ); |
1302 | + |
1303 | + if ( $#array1 != $#array2 ) { return 0; } |
1304 | + |
1305 | + for ( my $i = 0; $i <= $#array1; $i++ ) { |
1306 | + my @def1 = split( /:/, $array1[$i] ); |
1307 | + my @def2 = split( /:/, $array2[$i] ); |
1308 | + |
1309 | + if ( $#def1 != $#def2 ) { return 0; } |
1310 | + |
1311 | + for ( my $j = 0; $j <= $#def1; $j++ ) { |
1312 | + if ( $def1[$j] ne $def2[$j] ) { return 0; } |
1313 | + } |
1314 | + } |
1315 | + return 1; |
1316 | +} |
1317 | + |
1318 | +# |
1319 | +# is_in_array subroutine checks if the given string is in the array. |
1320 | +# Parameters: |
1321 | +# str a string |
1322 | +# array_ref a reference to an array of strings |
1323 | +# Return value: |
1324 | +# 1 if string is in the array |
1325 | +# 0 otherwise |
1326 | +# |
1327 | +sub is_in_array { |
1328 | + my $str = shift; |
1329 | + my $array_ref = shift; |
1330 | + |
1331 | + if ( grep { $str eq $_ } @{$array_ref} ) { |
1332 | + return 1; |
1333 | + } |
1334 | + return 0; |
1335 | +} |
1336 | + |
1337 | +# |
1338 | +# copy_back subroutine copies data and index files from backup directory |
1339 | +# back to their original locations. |
1340 | +# |
1341 | +sub copy_back { |
1342 | + my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' ); |
1343 | + my $orig_ibdata_dir = |
1344 | + get_option( \%config, 'mysqld', 'innodb_data_home_dir' ); |
1345 | + my $orig_innodb_data_file_path = |
1346 | + get_option( \%config, 'mysqld', 'innodb_data_file_path' ); |
1347 | + my $orig_iblog_dir = |
1348 | + get_option( \%config, 'mysqld', 'innodb_log_group_home_dir' ); |
1349 | + my $excluded_files = |
1350 | + '^(\.\.?|backup-my\.cnf|xtrabackup_logfile|mysql-std(err|out)|.*\.ibz)$'; |
1351 | + my @ibdata_files; |
1352 | + my $iblog_files = '^ib_logfile.*$'; |
1353 | + my $compressed_data_file = '.*\.ibz$'; |
1354 | + my $file; |
1355 | + my $backup_innodb_data_file_path; |
1356 | + |
1357 | + # check that original data directory exists |
1358 | + if ( !-d $orig_datadir ) { |
1359 | + Die "Original data directory '$orig_datadir' does not exist!"; |
1360 | + } |
1361 | + |
1362 | + # check that original InnoDB data directory exists |
1363 | + if ( !-d $orig_ibdata_dir ) { |
1364 | + Die "Original InnoDB data directory '$orig_ibdata_dir' does not exist!"; |
1365 | + } |
1366 | + |
1367 | + # check that original InnoDB log directory exists |
1368 | + if ( !-d $orig_iblog_dir ) { |
1369 | + Die "Original InnoDB log directory '$orig_iblog_dir' does not exist!"; |
1370 | + } |
1371 | + |
1372 | + # check that the original options file and the backup options file have |
1373 | + # the same value for "innodb_data_file_path" option |
1374 | + #$backup_innodb_data_file_path = |
1375 | + # get_option(\%backup_config, 'mysqld', 'innodb_data_file_path'); |
1376 | + #if (!are_equal_innodb_data_file_paths($orig_innodb_data_file_path, |
1377 | + # $backup_innodb_data_file_path) |
1378 | + #) { |
1379 | + # Die "The value of 'innodb_data_file_path' option in the original " |
1380 | + # . "my.cnf file '$config_file' is different from the value " |
1381 | + # . "in the backup my.cnf file '$backup_config_file'.\n(original: " |
1382 | + # . "'$orig_innodb_data_file_path')\n" |
1383 | + # . "(backup: '$backup_innodb_data_file_path')"; |
1384 | + #} |
1385 | + |
1386 | + # make a list of all ibdata files in the backup directory and all |
1387 | + # directories in the backup directory under which there are ibdata files |
1388 | + foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) { |
1389 | + my $path = ( split( /:/, $a ) )[0]; |
1390 | + my $filename = ( split( /\/+/, $path ) )[-1]; |
1391 | + |
1392 | + # check that the backup data file exists |
1393 | + if ( !-e "$backup_dir/$filename" ) { |
1394 | + if ( -e "$backup_dir/${filename}.ibz" ) { |
1395 | + Die |
1396 | + "Backup data file '$backup_dir/$filename' does not exist, but " |
1397 | + . "its compressed copy '${path}.ibz' exists. Check that you " |
1398 | + . "have run '$innobackup_script --apply-log --uncompress " |
1399 | + . "...' before attempting '$innobackup_script --copy-back ...' !"; |
1400 | + } |
1401 | + else { |
1402 | + Die "Backup data file '$backup_dir/$filename' does not exist."; |
1403 | + } |
1404 | + } |
1405 | + |
1406 | + if ( !is_in_array( $filename, \@ibdata_files ) ) { |
1407 | + push( @ibdata_files, $filename ); |
1408 | + } |
1409 | + } |
1410 | + |
1411 | + # copy files from backup dir to their original locations |
1412 | + |
1413 | + # copy files to original data directory |
1414 | + opendir( DIR, $backup_dir ) |
1415 | + || Die "Can't open directory '$backup_dir': $OS_ERROR\n"; |
1416 | + print STDERR "$prefix Starting to copy MyISAM tables, indexes,\n"; |
1417 | + print STDERR |
1418 | + "$prefix .MRG, .TRG, .TRN, .ARM, .ARZ, .opt, and .frm files\n"; |
1419 | + print STDERR "$prefix in '$backup_dir'\n"; |
1420 | + print STDERR "$prefix back to original data directory '$orig_datadir'\n"; |
1421 | + while ( defined( $file = readdir(DIR) ) ) { |
1422 | + if ( $file =~ /$excluded_files/ ) { next; } |
1423 | + if ( is_in_array( $file, \@ibdata_files ) ) { next; } |
1424 | + if ( $file =~ /$iblog_files/ ) { next; } |
1425 | + if ( -d "$backup_dir/$file" ) { |
1426 | + my $subdir = "$backup_dir/$file"; |
1427 | + my $file2; |
1428 | + |
1429 | + print STDERR "$prefix Copying directory '$subdir'\n"; |
1430 | + if ( !-x "\"" . escape_path("$orig_datadir/$file") . "\"" ) { |
1431 | + system_call("mkdir \"" . escape_path("$orig_datadir/$file") . "\""); |
1432 | + } |
1433 | + opendir( SUBDIR, "$subdir" ) |
1434 | + || Die "Can't open directory '$subdir': $OS_ERROR\n"; |
1435 | + while ( defined( $file2 = readdir(SUBDIR) ) ) { |
1436 | + if ( -d "$subdir/$file2" ) { next; } |
1437 | + if ( $file2 =~ /$compressed_data_file/ ) { next; } |
1438 | + $src_name = escape_path("$subdir/$file2"); |
1439 | + $dst_name = escape_path("$orig_datadir/$file"); |
1440 | + system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
1441 | + } |
1442 | + closedir(SUBDIR); |
1443 | + } |
1444 | + else { |
1445 | + print STDERR "$prefix Copying file " . "'$backup_dir/$file'\n"; |
1446 | + $src_name = escape_path("$backup_dir/$file"); |
1447 | + $dst_name = escape_path("$orig_datadir"); |
1448 | + system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
1449 | + } |
1450 | + } |
1451 | + closedir(DIR); |
1452 | + |
1453 | + # copy InnoDB data files to original InnoDB data directory |
1454 | + print STDERR "\n$prefix Starting to copy InnoDB tables and indexes\n"; |
1455 | + print STDERR "$prefix in '$backup_dir'\n"; |
1456 | + print STDERR |
1457 | + "$prefix back to original InnoDB data directory '$orig_ibdata_dir'\n"; |
1458 | + foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) { |
1459 | + |
1460 | + # get the relative pathname of a data file |
1461 | + my $path = ( split( /:/, $a ) )[0]; |
1462 | + my $filename = ( split( /\/+/, $path ) )[-1]; |
1463 | + print STDERR "$prefix Copying file '$backup_dir/$filename'\n"; |
1464 | + $src_name = escape_path("$backup_dir/$filename"); |
1465 | + $dst_name = escape_path("$orig_ibdata_dir/$path"); |
1466 | + system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
1467 | + } |
1468 | + |
1469 | + # copy InnoDB log files to original InnoDB log directory |
1470 | + opendir( DIR, $backup_dir ) |
1471 | + || Die "Can't open directory '$backup_dir': $OS_ERROR\n"; |
1472 | + print STDERR "\n$prefix Starting to copy InnoDB log files\n"; |
1473 | + print STDERR "$prefix in '$backup_dir'\n"; |
1474 | + print STDERR |
1475 | + "$prefix back to original InnoDB log directory '$orig_iblog_dir'\n"; |
1476 | + while ( defined( $file = readdir(DIR) ) ) { |
1477 | + if ( $file =~ /$iblog_files/ && -f "$backup_dir/$file" ) { |
1478 | + print STDERR "$prefix Copying file '$backup_dir/$file'\n"; |
1479 | + $src_name = escape_path("$backup_dir/$file"); |
1480 | + $dst_name = escape_path("$orig_iblog_dir"); |
1481 | + system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
1482 | + } |
1483 | + } |
1484 | + closedir(DIR); |
1485 | + |
1486 | + print STDERR "$prefix Finished copying back files.\n\n"; |
1487 | + |
1488 | + return; |
1489 | +} |
1490 | + |
1491 | +# |
1492 | +# apply_log subroutine prepares a backup for starting database server |
1493 | +# on the backup. It applies InnoDB log files to the InnoDB data files. |
1494 | +# |
1495 | +sub apply_log { |
1496 | + my $rcode; |
1497 | + my $cmdline = ''; |
1498 | + my $options = ''; |
1499 | + |
1500 | + if ( $o->{'defaults-file'} ) { |
1501 | + $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" "; |
1502 | + } |
1503 | + |
1504 | + $options = $options . "--prepare --target-dir=$backup_dir"; |
1505 | + |
1506 | + if ( $o->{'uncompress'} ) { |
1507 | + $options = $options . ' --uncompress'; |
1508 | + } |
1509 | + if ( $o->{'export'} ) { |
1510 | + $options = $options . ' --export'; |
1511 | + } |
1512 | + if ( $o->{'use-memory'} ) { |
1513 | + $options = $options . " --use-memory=$o->{'use-memory'}"; |
1514 | + } |
1515 | + |
1516 | + # run ibbackup as a child process |
1517 | + $cmdline = "$o->{'ibbackup'} $options"; |
1518 | + $now = current_time(); |
1519 | + print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n\n"; |
1520 | + $rcode = system_call($cmdline, 1); |
1521 | + Die "\n$prefix ibbackup failed" if $rcode; |
1522 | + |
1523 | + $now = current_time(); |
1524 | + print STDERR |
1525 | + "\n$now $prefix Restarting xtrabackup with command: $cmdline\nfor creating ib_logfile*\n\n"; |
1526 | + $rcode = system_call($cmdline, 1); |
1527 | + Die "\n$prefix xtrabackup (2nd execution) failed" if $rcode; |
1528 | + |
1529 | + return; |
1530 | +} |
1531 | + |
1532 | +# |
1533 | +# wait_for_ibbackup_suspend subroutine waits until ibbackup has suspended |
1534 | +# itself. |
1535 | +# |
1536 | +sub wait_for_ibbackup_suspend { |
1537 | + print STDERR |
1538 | + "$prefix Waiting for ibbackup (pid=$ibbackup_pid) to suspend\n"; |
1539 | + print STDERR "$prefix Suspend file '$suspend_file'\n\n"; |
1540 | + for ( ;; ) { |
1541 | + sleep 2; |
1542 | + last if -e $suspend_file; |
1543 | + |
1544 | + # check that ibbackup child process is still alive |
1545 | + if ( $ibbackup_pid == waitpid( $ibbackup_pid, &WNOHANG ) ) { |
1546 | + $ibbackup_pid = ''; |
1547 | + Die "ibbackup child process has died"; |
1548 | + } |
1549 | + } |
1550 | + $now = current_time(); |
1551 | + print STDERR "\n$now $prefix Continuing after ibbackup has suspended\n"; |
1552 | +} |
1553 | + |
1554 | +# |
1555 | +# resume_ibbackup subroutine signals ibbackup to complete its execution |
1556 | +# by deleting the 'ibbackup_suspended' file. |
1557 | +# |
1558 | +sub resume_ibbackup { |
1559 | + print STDERR "$prefix Resuming ibbackup\n\n"; |
1560 | + unlink $suspend_file || Die "Failed to delete '$suspend_file': $OS_ERROR"; |
1561 | + |
1562 | + # wait for ibbackup to finish |
1563 | + waitpid( $ibbackup_pid, 0 ); |
1564 | + $ibbackup_pid = ''; |
1565 | +} |
1566 | + |
1567 | +# |
1568 | +# start_ibbackup subroutine spawns a child process running ibbackup |
1569 | +# program for backing up InnoDB tables and indexes. |
1570 | +# |
1571 | +sub start_ibbackup { |
1572 | + my $options = ''; |
1573 | + my $cmdline = ''; |
1574 | + my $pid = undef; |
1575 | + |
1576 | + if ( $o->{'defaults-file'} ) { |
1577 | + $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" "; |
1578 | + } |
1579 | + |
1580 | + $options = $options . "--backup --suspend-at-end"; |
1581 | + |
1582 | + if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
1583 | + $options = $options . " --target-dir=$backup_dir"; |
1584 | + } |
1585 | + else { |
1586 | + |
1587 | + #(datadir) for 'xtrabackup_suspended' and 'xtrabackup_checkpoints' |
1588 | + $options = $options . " --log-stream --target-dir=./"; |
1589 | + } |
1590 | + |
1591 | + # prepare command line for running ibbackup |
1592 | + if ( $o->{'throttle'} ) { |
1593 | + $options = $options . " --throttle=$o->{'throttle'}"; |
1594 | + } |
1595 | + if ( $o->{'sleep'} ) { |
1596 | + $options = $options . " --sleep=$o->{'sleep'}"; |
1597 | + } |
1598 | + if ( $o->{'compress'} ) { |
1599 | + $options = $options . " --compress=$o->{'compress'}"; |
1600 | + } |
1601 | + if ( $o->{'use-memory'} ) { |
1602 | + $options = $options . " --use-memory=$o->{'use-memory'}"; |
1603 | + } |
1604 | + if ( $o->{'include'} ) { |
1605 | + $options = $options . " --tables='$o->{'include'}'"; |
1606 | + } |
1607 | + if ( $o->{'incremental'} ) { |
1608 | + $options = $options . " --incremental-basedir='$incremental_basedir'"; |
1609 | + } |
1610 | + $cmdline = "$o->{'ibbackup'} $options"; |
1611 | + |
1612 | + # run ibbackup as a child process |
1613 | + $now = current_time(); |
1614 | + print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n"; |
1615 | + if ( defined($pid = fork) ) { |
1616 | + if ($pid) { |
1617 | + # parent process |
1618 | + $ibbackup_pid = $pid; |
1619 | + |
1620 | + # direct copy to remote |
1621 | + if ( $o->{'remote-host'} || $o->{'stream'} ) { |
1622 | + my $orig_datadir |
1623 | + = get_option( \%config, 'mysqld', 'datadir' ); |
1624 | + my $orig_ibdata_dir |
1625 | + = get_option( \%config, 'mysqld', 'innodb_data_home_dir' ); |
1626 | + my $orig_innodb_data_file_path |
1627 | + = get_option( \%config, 'mysqld', 'innodb_data_file_path' ); |
1628 | + my $subdir; |
1629 | + my @list; |
1630 | + |
1631 | + if ( $o->{'remote-host'} ) { |
1632 | + if ( system_call("ssh $o->{'remote-host'} test -e " |
1633 | + . "$backup_dir/ib_logfile0", 1) == 0 ) { |
1634 | + print STDERR "$prefix Remove " |
1635 | + . "$o->{'remote-host'}:$backup_dir/ib_logfile*\n"; |
1636 | + system_call( "ssh $o->{'remote-host'} " |
1637 | + . "rm $backup_dir/ib_logfile\*" |
1638 | + ) |
1639 | + } |
1640 | + } |
1641 | + |
1642 | + wait_for_ibbackup_suspend(); |
1643 | + |
1644 | + #InnoDB data files from original InnoDB data directory |
1645 | + print STDERR |
1646 | + "\n$prefix Starting to backup InnoDB tables and indexes\n"; |
1647 | + if ( $o->{'remote-host'} ) { |
1648 | + print STDERR "$prefix to '$backup_dir'\n"; |
1649 | + } |
1650 | + print STDERR |
1651 | + "$prefix from original InnoDB data directory '$orig_ibdata_dir'\n"; |
1652 | + foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) { |
1653 | + my $path = ( split( /:/, $a ) )[0]; |
1654 | + $path =~ s/([\$\\\" ])/\\$1/g; |
1655 | + if ( $o->{'remote-host'} ) { |
1656 | + print STDERR |
1657 | + "$prefix Backing up file '$orig_ibdata_dir/$path'\n"; |
1658 | + system_call("scp $o->{'scp-opt'} '$orig_ibdata_dir/$path' " |
1659 | + . "'$o->{'remote-host'}:$backup_dir/$path'"); |
1660 | + } |
1661 | + elsif ( $o->{'stream'} eq 'tar' ) { |
1662 | + my $ret = 0; |
1663 | + print STDERR "$prefix Backing up as tar stream '$path'\n"; |
1664 | + if ( !$o->{'tar4ibd'} ) { |
1665 | + $ret = system_call( |
1666 | + "cd $orig_ibdata_dir; tar chf - -b 32 $path", 1); |
1667 | + } |
1668 | + else { |
1669 | + $ret = system_call( |
1670 | + "cd $orig_ibdata_dir; tar4ibd -c $path", 1); |
1671 | + } |
1672 | + if ( $ret == 1 ) { |
1673 | + print STDERR "$prefix If you use GNU tar, this warning " |
1674 | + . "can be ignored.\n"; |
1675 | + } |
1676 | + elsif ( $ret != 0 ) { |
1677 | + print STDERR "$prefix tar returned with exit code $ret.\n"; |
1678 | + Die "Failed to stream '$path': $OS_ERROR"; |
1679 | + } |
1680 | + } |
1681 | + } |
1682 | + |
1683 | + #copy *.ibd files |
1684 | + opendir( DIR, $orig_datadir ) |
1685 | + || Die "Can't open directory '$orig_datadir': $OS_ERROR\n"; |
1686 | + while ( defined( $subdir = readdir(DIR) ) ) { |
1687 | + my $print_each_file = 0; |
1688 | + my $file_c; |
1689 | + my $file; |
1690 | + if ( $subdir eq '.' || $subdir eq '..' ) { next; } |
1691 | + next unless -d "$orig_datadir/$subdir"; |
1692 | + next unless check_if_required($subdir); |
1693 | + |
1694 | + @list = glob( "$orig_datadir/$subdir/" . '*.ibd' ); |
1695 | + |
1696 | + $file_c = @list; |
1697 | + if ( $file_c <= $backup_file_print_limit ) { |
1698 | + $print_each_file = 1; |
1699 | + } |
1700 | + else { |
1701 | + print STDERR "$prefix Backing up files " |
1702 | + . "'$orig_datadir/$subdir/*.ibd' ($file_c files)\n"; |
1703 | + } |
1704 | + |
1705 | + foreach $file ( @list ) { |
1706 | + if ( $o->{'include'} ) { |
1707 | + my $table_name; |
1708 | + $table_name = substr($file, rindex( $file, '/' )); |
1709 | + $table_name = substr( $table_name, 1, |
1710 | + rindex( $table_name, '.' ) - 1); |
1711 | + $table_name = $subdir . "." . $table_name; |
1712 | + if ( $table_name !~ m/$o->{'include'}/o ) { |
1713 | + print STDERR "'$file' is skipped.\n"; |
1714 | + next; |
1715 | + } |
1716 | + } |
1717 | + |
1718 | + if ($print_each_file) { |
1719 | + print STDERR "$prefix Backing up file '$file'\n"; |
1720 | + } |
1721 | + |
1722 | + if ( $o->{'remote-host'} ) { |
1723 | + system_call( |
1724 | + "ssh $o->{'remote-host'} mkdir '$backup_dir/$subdir'", |
1725 | + 1 # don't die on failure |
1726 | + ); |
1727 | + system_call("scp $o->{'scp-opt'} '$file' " |
1728 | + . "'$o->{'remote-host'}:$backup_dir/$subdir/'"); |
1729 | + } |
1730 | + elsif ( $o->{'stream'} eq 'tar' ) { |
1731 | + my $ret = 0; |
1732 | + my $file_name = substr($file, rindex( $file, '/' ) + 1); |
1733 | + $file_name =~ s/([\$\\\" ])/\\$1/g; |
1734 | + |
1735 | + if ( !$o->{'tar4ibd'} ) { |
1736 | + $ret = system_call("cd $orig_datadir; tar chf - -b 32 " |
1737 | + . "$subdir/$file_name", 1); |
1738 | + } |
1739 | + else { |
1740 | + $ret = system_call("cd $orig_datadir; tar4ibd -c " |
1741 | + . "$subdir/$file_name", 1); |
1742 | + } |
1743 | + |
1744 | + if ( $ret == 1 ) { |
1745 | + print STDERR "$prefix If you use GNU tar, this warning " |
1746 | + . "can be ignored.\n"; |
1747 | + } |
1748 | + elsif ( $ret != 0 ) { |
1749 | + print STDERR "$prefix tar returned with exit code " |
1750 | + . "$ret.\n"; |
1751 | + Die "Failed to stream '$subdir/$file_name': $OS_ERROR"; |
1752 | + } |
1753 | + } |
1754 | + } # each file |
1755 | + } # each subdir |
1756 | + closedir(DIR); |
1757 | + } |
1758 | + } # parent process |
1759 | + else { # child process |
1760 | + if ( $o->{'remote-host'} || $o->{'stream'} ) { |
1761 | + open STDOUT, "> $tmp_logfile" |
1762 | + or Die "Failed to open file '$tmp_logfile': $OS_ERROR"; |
1763 | + } |
1764 | + IBEXDEBUG && _d($cmdline); |
1765 | + exec($cmdline) or Die "Failed to exec ibbackup: $OS_ERROR"; |
1766 | + } |
1767 | + } |
1768 | + else { |
1769 | + Die "failed to fork ibbackup child process: $OS_ERROR"; |
1770 | + } |
1771 | +} |
1772 | + |
1773 | +# |
1774 | +# get_mysql_options subroutine returns the options to mysql client program |
1775 | +# as a string. The options are determined from the options given by the |
1776 | +# user to innobackup. |
1777 | +# |
1778 | +sub get_mysql_options { |
1779 | + my @options; |
1780 | + |
1781 | + # --defaults-file needs to be first. |
1782 | + if ( $o->{'defaults-file'} ) { |
1783 | + push @options, "--defaults-file=$o->{'defaults-file'}"; |
1784 | + } |
1785 | + |
1786 | + if ( $o->{'host'} ) { |
1787 | + push @options, "--host=$o->{'host'}"; |
1788 | + } |
1789 | + if ( $o->{'port'} ) { |
1790 | + push @options, "--port=$o->{'port'}"; |
1791 | + } |
1792 | + if ( $o->{'user'} ) { |
1793 | + push @options, "--user=$o->{'user'}"; |
1794 | + } |
1795 | + if ( $o->{'password'} ) { |
1796 | + push @options, "--password=$o->{'password'}"; |
1797 | + } |
1798 | + if ( $o->{'socket'} ) { |
1799 | + push @options, "--socket=$o->{'socket'}"; |
1800 | + } |
1801 | + |
1802 | + push @options, '--unbuffered --'; |
1803 | + |
1804 | + my $options = join( ' ', @options ); |
1805 | + IBEXDEBUG && _d('mysql options:', $options); |
1806 | + return $options; |
1807 | +} |
1808 | + |
1809 | +# |
1810 | +# mysql_open subroutine starts mysql as a child process with |
1811 | +# a pipe connection. |
1812 | +# |
1813 | +sub mysql_open { |
1814 | + my $options = get_mysql_options(); |
1815 | + |
1816 | + # run mysql as a child process with a pipe connection |
1817 | + $now = current_time(); |
1818 | + print STDERR "$now $prefix Starting mysql with options: $options\n"; |
1819 | + $mysql_pid = open(*MYSQL_WRITER, |
1820 | + "| mysql $options >$mysql_stdout 2>$mysql_stderr ") |
1821 | + or Die "Failed to spawn mysql child process: $OS_ERROR"; |
1822 | + MYSQL_WRITER->autoflush(1); |
1823 | + $now = current_time(); |
1824 | + print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n"; |
1825 | + |
1826 | + return; |
1827 | +} |
1828 | + |
1829 | +# |
1830 | +# mysql_check subroutine checks that the connection to mysql child process |
1831 | +# is ok. |
1832 | +# |
1833 | +sub mysql_check { |
1834 | + my $mysql_pid_copy = $mysql_pid; |
1835 | + |
1836 | + # send a dummy query to mysql child process |
1837 | + $hello_id++; |
1838 | + my $hello_message = "innobackup hello $hello_id"; |
1839 | + IBEXDEBUG && _d('mysql check:', $hello_message); |
1840 | + print MYSQL_WRITER "select '$hello_message';\n" |
1841 | + or Die "Connection to mysql child process failed: $OS_ERROR"; |
1842 | + |
1843 | + # wait for reply |
1844 | + eval { |
1845 | + local $SIG{ALRM} = sub { die "alarm clock restart" }; |
1846 | + my $stdout = ''; |
1847 | + my $stderr = ''; |
1848 | + alarm $mysql_response_timeout; |
1849 | + while ( index( $stdout, $hello_message ) < 0 ) { |
1850 | + sleep 2; |
1851 | + if ( $mysql_pid && $mysql_pid == waitpid( $mysql_pid, &WNOHANG ) ) { |
1852 | + my $reason = `cat $mysql_stderr`; |
1853 | + $mysql_pid = ''; |
1854 | + die "mysql child process has died: $reason"; |
1855 | + } |
1856 | + $stdout = `cat $mysql_stdout`; |
1857 | + $stderr = `cat $mysql_stderr`; |
1858 | + if ($stderr) { |
1859 | + # mysql has reported an error, do exit |
1860 | + die "mysql error: $stderr"; |
1861 | + } |
1862 | + } |
1863 | + alarm 0; |
1864 | + }; |
1865 | + if ( $EVAL_ERROR =~ /alarm clock restart/ ) { |
1866 | + Die "Connection to mysql child process (pid=$mysql_pid_copy) timedout." |
1867 | + . " (Time limit of $mysql_response_timeout seconds exceeded." |
1868 | + . " You may adjust time limit by editing the value of parameter" |
1869 | + . " \"\$mysql_response_timeout\" in this script.)"; |
1870 | + } |
1871 | + elsif ( $EVAL_ERROR ) { |
1872 | + Die $EVAL_ERROR; |
1873 | + } |
1874 | + |
1875 | + $mysql_last_access_time = time(); |
1876 | + |
1877 | + return; |
1878 | +} |
1879 | + |
1880 | +# |
1881 | +# mysql_keep_alive subroutine tries to keep connection to the mysqld database |
1882 | +# server alive by sending a dummy query when the connection has been idle |
1883 | +# for the specified time. |
1884 | +# |
1885 | +sub mysql_keep_alive { |
1886 | + if ( ( time() - $mysql_last_access_time ) > $mysql_keep_alive_timeout ) { |
1887 | + # too long idle, send a dummy query |
1888 | + mysql_check(); |
1889 | + } |
1890 | + return; |
1891 | +} |
1892 | + |
1893 | +# |
1894 | +# mysql_send subroutine send a request string to mysql child process. |
1895 | +# This subroutine appends a newline character to the request and checks |
1896 | +# that mysqld receives the query. |
1897 | +# Parameters: |
1898 | +# request request string |
1899 | +# |
1900 | +sub mysql_send { |
1901 | + my $request = shift; |
1902 | + $current_mysql_request = $request; |
1903 | + print MYSQL_WRITER "$request\n"; |
1904 | + mysql_check(); |
1905 | + $current_mysql_request = ''; |
1906 | + return; |
1907 | +} |
1908 | + |
1909 | +# |
1910 | +# mysql_close subroutine terminates mysql child process gracefully. |
1911 | +# |
1912 | +sub mysql_close { |
1913 | + print MYSQL_WRITER "quit\n"; |
1914 | + $now = current_time(); |
1915 | + print STDERR "$now $prefix Connection to database server closed\n"; |
1916 | + $mysql_pid = ''; |
1917 | + return; |
1918 | +} |
1919 | + |
1920 | +# |
1921 | +# write_binlog_info subroutine retrieves MySQL binlog position and |
1922 | +# saves it in a file. It also prints it to stdout. |
1923 | +# |
1924 | +sub write_binlog_info { |
1925 | + my @lines; |
1926 | + my @info_lines = (); |
1927 | + my $position = ''; |
1928 | + my $filename = ''; |
1929 | + |
1930 | + # get binlog position |
1931 | + mysql_send "SHOW MASTER STATUS;"; |
1932 | + |
1933 | + # get "show master status" output lines (2) from mysql output |
1934 | + file_to_array( $mysql_stdout, \@lines ); |
1935 | + foreach my $line (@lines) { |
1936 | + if ( $line =~ m/innobackup hello/ ) { |
1937 | + |
1938 | + # this is a hello message, ignore it |
1939 | + } |
1940 | + else { |
1941 | + |
1942 | + # this is output line from "show master status" |
1943 | + push( @info_lines, $line ); |
1944 | + } |
1945 | + } |
1946 | + |
1947 | + # write binlog info file |
1948 | + if ( !defined $info_lines[1] ) { |
1949 | + $info_lines[1] = ""; |
1950 | + } |
1951 | + if ( !$o->{'remote-host'} ) { |
1952 | + open( FILE, ">$binlog_info" ) |
1953 | + || Die "Failed to open file '$binlog_info': $OS_ERROR"; |
1954 | + } |
1955 | + else { |
1956 | + open( FILE, "| ssh $o->{'remote-host'} 'cat > $binlog_info'" ) |
1957 | + || Die |
1958 | + "Failed to open file '$o->{'remote-host'}:$binlog_info': $OS_ERROR"; |
1959 | + } |
1960 | + print FILE "$info_lines[1]\n"; |
1961 | + close(FILE); |
1962 | + |
1963 | + if ( $o->{'stream'} eq 'tar' ) { |
1964 | + system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_binlog_info"); |
1965 | + unlink $binlog_info or Die "Failed to delete '$binlog_info': $OS_ERROR"; |
1966 | + } |
1967 | + |
1968 | + # get the name of the last binlog file and position in it |
1969 | + ( $filename, $position ) = $info_lines[1] =~ /^\s*([^\s]+)\s+(.*)$/; |
1970 | + |
1971 | + if ( defined $filename && defined $position ) { |
1972 | + $mysql_binlog_position = "filename '$filename', position $position"; |
1973 | + } |
1974 | + else { |
1975 | + $mysql_binlog_position = "filename '', position "; |
1976 | + } |
1977 | + |
1978 | + return; |
1979 | +} |
1980 | + |
1981 | +# |
1982 | +# write_slave_info subroutine retrieves MySQL binlog position of the |
1983 | +# master server in a replication setup and saves it in a file. It |
1984 | +# also saves it in $msql_slave_position variable. |
1985 | +# |
1986 | +sub write_slave_info { |
1987 | + my @lines; |
1988 | + my @info_lines; |
1989 | + my $position = ''; |
1990 | + my $filename = ''; |
1991 | + my $master = ''; |
1992 | + |
1993 | + # get slave status. Use single quotes here, otherwise |
1994 | + # \G is evaluated as a control character. |
1995 | + mysql_send 'SHOW SLAVE STATUS\G;'; |
1996 | + |
1997 | + # get output of the "show slave status" command from mysql output |
1998 | + # and extract binlog position of the master server |
1999 | + file_to_array( $mysql_stdout, \@lines ); |
2000 | + for (@lines) { |
2001 | + $master = $1 if /Master_Host:\s*(\S*)\s*$/; |
2002 | + $filename = $1 if /Master_Log_File:\s*(\S*)\s*$/; |
2003 | + $position = $1 if /Master_Log_Pos:\s*(\S*)\s*$/; |
2004 | + } |
2005 | + |
2006 | + # print slave status to a file |
2007 | + if ( !$o->{'remote-host'} ) { |
2008 | + open( FILE, ">$slave_info" ) |
2009 | + || Die "Failed to open file '$slave_info': $OS_ERROR"; |
2010 | + } |
2011 | + else { |
2012 | + open( FILE, "| ssh $o->{'remote-host'} 'cat > $slave_info'" ) |
2013 | + || Die |
2014 | + "Failed to open file '$o->{'remote-host'}:$slave_info': $OS_ERROR"; |
2015 | + } |
2016 | + print FILE |
2017 | + "CHANGE MASTER TO MASTER_LOG_FILE='$filename', MASTER_LOG_POS=$position\n"; |
2018 | + close(FILE); |
2019 | + |
2020 | + if ( $o->{'stream'} eq 'tar' ) { |
2021 | + system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_slave_info"); |
2022 | + unlink $slave_info or Die "Failed to delete '$slave_info': $OS_ERROR"; |
2023 | + } |
2024 | + |
2025 | + $mysql_slave_position = |
2026 | + "master host '$master', filename '$filename', position $position"; |
2027 | + |
2028 | + return; |
2029 | +} |
2030 | + |
2031 | +# |
2032 | +# mysql_lockall subroutine puts a read lock on all tables in all databases. |
2033 | +# |
2034 | +sub mysql_lockall { |
2035 | + $now = current_time(); |
2036 | + print STDERR "$now $prefix Starting to lock all tables...\n"; |
2037 | + |
2038 | + mysql_send "USE mysql;"; |
2039 | + |
2040 | +# mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;"; |
2041 | +# if (compare_versions($mysql_server_version, '4.1.0') == -1) { |
2042 | +# # MySQL server version is 4.0 or older, ENGINE keyword not supported |
2043 | +# mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) TYPE=INNODB;"; |
2044 | +# } else { |
2045 | +# # MySQL server version is 4.1 or newer, use ENGINE keyword |
2046 | +# mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) ENGINE=INNODB;"; |
2047 | +# } |
2048 | + mysql_send "SET AUTOCOMMIT=0;"; |
2049 | + |
2050 | + # mysql_send "INSERT INTO ibbackup_binlog_marker VALUES (1);"; |
2051 | + if ( compare_versions( $mysql_server_version, '4.0.22' ) == 0 |
2052 | + || compare_versions( $mysql_server_version, '4.1.7' ) == 0 ) |
2053 | + { |
2054 | + |
2055 | + # MySQL server version is 4.0.22 or 4.1.7 |
2056 | + mysql_send "COMMIT;"; |
2057 | + mysql_send "FLUSH TABLES WITH READ LOCK;"; |
2058 | + } |
2059 | + else { |
2060 | + |
2061 | + # MySQL server version is other than 4.0.22 or 4.1.7 |
2062 | + mysql_send "FLUSH TABLES WITH READ LOCK;"; |
2063 | + mysql_send "COMMIT;"; |
2064 | + } |
2065 | + write_binlog_info; |
2066 | + write_slave_info if $o->{'slave-info'}; |
2067 | + |
2068 | + $now = current_time(); |
2069 | + print STDERR "$now $prefix All tables locked and flushed to disk\n"; |
2070 | + |
2071 | + return; |
2072 | +} |
2073 | + |
2074 | +# |
2075 | +# mysql_unlockall subroutine releases read locks on all tables in all |
2076 | +# databases. |
2077 | +# |
2078 | +sub mysql_unlockall { |
2079 | + mysql_send "UNLOCK TABLES;"; |
2080 | + |
2081 | + # mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;"; |
2082 | + |
2083 | + $now = current_time(); |
2084 | + print STDERR "$now $prefix All tables unlocked\n"; |
2085 | +} |
2086 | + |
2087 | +# |
2088 | +# catch_sigpipe subroutine is a signal handler for SIGPIPE. |
2089 | +# |
2090 | +sub catch_sigpipe { |
2091 | + my $rcode; |
2092 | + |
2093 | + if ( |
2094 | + $mysql_pid |
2095 | + && ( -1 == ( $rcode = waitpid( $mysql_pid, &WNOHANG ) ) |
2096 | + || $rcode == $mysql_pid ) |
2097 | + ) |
2098 | + { |
2099 | + my $reason = `cat $mysql_stderr`; |
2100 | + print STDERR "Pipe to mysql child process broken: $reason at"; |
2101 | + system("date +'%H:%M:%S'"); |
2102 | + exit(1); |
2103 | + } |
2104 | + else { |
2105 | + Die "Broken pipe"; |
2106 | + } |
2107 | +} |
2108 | + |
2109 | +# |
2110 | +# kill_child_processes subroutine kills all child processes of this process. |
2111 | +# |
2112 | +sub kill_child_processes { |
2113 | + if ($ibbackup_pid) { |
2114 | + kill( $kill_signal, $ibbackup_pid ); |
2115 | + $ibbackup_pid = ''; |
2116 | + } |
2117 | + |
2118 | + if ($mysql_pid) { |
2119 | + kill( $kill_signal, $mysql_pid ); |
2120 | + $mysql_pid = ''; |
2121 | + } |
2122 | +} |
2123 | + |
2124 | +# |
2125 | +# require_external subroutine checks that an external program is runnable |
2126 | +# via the shell. This is tested by calling the program with the |
2127 | +# given arguments. It is checked that the program returns 0 and does |
2128 | +# not print anything to stderr. If this check fails, this subroutine exits. |
2129 | +# Parameters: |
2130 | +# program pathname of the program file |
2131 | +# args arguments to the program |
2132 | +# pattern a string containing a regular expression for finding |
2133 | +# the program version. |
2134 | +# this pattern should contain a subpattern enclosed |
2135 | +# in parentheses which is matched with the version. |
2136 | +# version_ref a reference to a variable where the program version |
2137 | +# string is returned. Example "2.0-beta2". |
2138 | +# |
2139 | +sub require_external { |
2140 | + my $program = shift; |
2141 | + my $args = shift; |
2142 | + my $pattern = shift; |
2143 | + my $version_ref = shift; |
2144 | + my @lines; |
2145 | + my $tmp_stdout = tmpnam(); |
2146 | + my $tmp_stderr = tmpnam(); |
2147 | + my $rcode; |
2148 | + my $error; |
2149 | + |
2150 | + $rcode = system_call("$program $args >$tmp_stdout 2>$tmp_stderr", 1); |
2151 | + if ($rcode) { |
2152 | + $error = $OS_ERROR; |
2153 | + } |
2154 | + my $stderr = `cat $tmp_stderr`; |
2155 | + unlink $tmp_stderr; |
2156 | + if ( $stderr ne '' ) { |
2157 | + |
2158 | + # failure |
2159 | + unlink $tmp_stdout; |
2160 | + Die "Couldn't run $program: $stderr"; |
2161 | + } |
2162 | + elsif ($rcode) { |
2163 | + |
2164 | + # failure |
2165 | + unlink $tmp_stdout; |
2166 | + Die "Couldn't run $program: $error"; |
2167 | + } |
2168 | + |
2169 | + # success |
2170 | + my $stdout = `cat $tmp_stdout`; |
2171 | + unlink $tmp_stdout; |
2172 | + @lines = split( /\n|;/, $stdout ); |
2173 | + print STDERR "$prefix Using $lines[0]\n"; |
2174 | + |
2175 | + # get version string from the first output line of the program |
2176 | + ${$version_ref} = ''; |
2177 | + if ( $lines[0] =~ /$pattern/ ) { |
2178 | + ${$version_ref} = $1; |
2179 | + } |
2180 | +} |
2181 | + |
2182 | +# compare_versions subroutine compares two GNU-style version strings. |
2183 | +# A GNU-style version string consists of three decimal numbers delimitted |
2184 | +# by dots, and optionally followed by extra attributes. |
2185 | +# Examples: "4.0.1", "4.1.1-alpha-debug". |
2186 | +# Parameters: |
2187 | +# str1 a GNU-style version string |
2188 | +# str2 a GNU-style version string |
2189 | +# Return value: |
2190 | +# -1 if str1 < str2 |
2191 | +# 0 if str1 == str2 |
2192 | +# 1 is str1 > str2 |
2193 | +sub compare_versions { |
2194 | + my $str1 = shift; |
2195 | + my $str2 = shift; |
2196 | + my $extra1 = ''; |
2197 | + my $extra2 = ''; |
2198 | + my @array1 = (); |
2199 | + my @array2 = (); |
2200 | + my $i; |
2201 | + |
2202 | + # remove possible extra attributes |
2203 | + ( $str1, $extra1 ) = $str1 =~ /^([0-9.]*)(.*)/; |
2204 | + ( $str2, $extra2 ) = $str2 =~ /^([0-9.]*)(.*)/; |
2205 | + |
2206 | + # split "dotted" decimal number string into an array |
2207 | + @array1 = split( '\.', $str1 ); |
2208 | + @array2 = split( '\.', $str2 ); |
2209 | + |
2210 | + # compare in lexicographic order |
2211 | + for ( $i = 0; $i <= $#array1 && $i <= $#array2; $i++ ) { |
2212 | + if ( $array1[$i] < $array2[$i] ) { |
2213 | + return -1; |
2214 | + } |
2215 | + elsif ( $array1[$i] > $array2[$i] ) { |
2216 | + return 1; |
2217 | + } |
2218 | + } |
2219 | + if ( $#array1 < $#array2 ) { |
2220 | + return -1; |
2221 | + } |
2222 | + elsif ( $#array1 > $#array2 ) { |
2223 | + return 1; |
2224 | + } |
2225 | + else { |
2226 | + return 0; |
2227 | + } |
2228 | +} |
2229 | + |
2230 | +# |
2231 | +# init subroutine initializes global variables and performs some checks on the |
2232 | +# system we are running on. |
2233 | +# |
2234 | +sub init { |
2235 | + my $mysql_version = ''; |
2236 | + my $ibbackup_version = ''; |
2237 | + my $run = ''; |
2238 | + |
2239 | + # print some instructions to the user |
2240 | + if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) { |
2241 | + $run = 'backup'; |
2242 | + } |
2243 | + elsif ( $o->{'copy-back'} ) { |
2244 | + $run = 'copy-back'; |
2245 | + } |
2246 | + else { |
2247 | + $run = 'apply-log'; |
2248 | + } |
2249 | + print STDERR |
2250 | + "IMPORTANT: Please check that the $run run completes successfully.\n"; |
2251 | + print STDERR |
2252 | + " At the end of a successful $run run $innobackup_script\n"; |
2253 | + print STDERR " prints \"completed OK!\".\n\n"; |
2254 | + |
2255 | + # check that MySQL client program and InnoDB Hot Backup program |
2256 | + # are runnable via shell |
2257 | + if ( !$o->{'copy-back'} ) { |
2258 | + |
2259 | + # we are making a backup or applying log to backup |
2260 | + if ( !$o->{'apply-log'} ) { |
2261 | + |
2262 | + # we are making a backup, we need mysql server |
2263 | + my $output = ''; |
2264 | + my @lines = (); |
2265 | + |
2266 | + # check that we have mysql client program |
2267 | + require_external( 'mysql', '--version', 'Ver ([^,]+)', |
2268 | + \$mysql_version ); |
2269 | + |
2270 | + # get mysql server version |
2271 | + my $options = get_mysql_options(); |
2272 | + @lines = split( '\n', `mysql $options -e "select \@\@version"` ); |
2273 | + $mysql_server_version = $lines[1]; |
2274 | + print STDERR |
2275 | + "$prefix Using mysql server version $mysql_server_version\n"; |
2276 | + } |
2277 | + |
2278 | + #require_external($o->{'ibbackup-binary'}, '--license', |
2279 | + # 'version (\S+)', \$ibbackup_version); |
2280 | + print STDERR "\n"; |
2281 | + |
2282 | + if ( $o->{'include'} |
2283 | + && $ibbackup_version |
2284 | + && $ibbackup_version le "2.0" ) |
2285 | + { |
2286 | + |
2287 | + # --include option was given, but ibbackup is too |
2288 | + # old to support it |
2289 | + Die "--include option was given, but ibbackup is too old" |
2290 | + . " to support it. You must upgrade to InnoDB Hot Backup" |
2291 | + . " v2.0 in order to use --include option.\n"; |
2292 | + } |
2293 | + } |
2294 | + |
2295 | + # set signal handlers |
2296 | + $SIG{PIPE} = \&catch_sigpipe; |
2297 | + |
2298 | + # read MySQL options file |
2299 | + #read_config_file($config_file, \%config); |
2300 | + read_config_file( \%config ); |
2301 | + |
2302 | + if ( !$o->{'tmpdir'} ) { |
2303 | + $o->{'tmpdir'} = get_option( \%config, 'mysqld', 'tmpdir' ); |
2304 | + } |
2305 | + |
2306 | + # get innodb log home directory from options file |
2307 | + #$innodb_log_group_home_dir = |
2308 | + # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir'); |
2309 | + |
2310 | + if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) { |
2311 | + |
2312 | + # we are making a backup, create a new backup directory |
2313 | + if ( !$o->{'remote-host'} ) { |
2314 | + $backup_dir = File::Spec->rel2abs( make_backup_dir() ); |
2315 | + } |
2316 | + else { |
2317 | + $backup_dir = make_backup_dir(); |
2318 | + } |
2319 | + print STDERR "$prefix Created backup directory $backup_dir\n"; |
2320 | + if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
2321 | + $backup_config_file = $backup_dir . '/backup-my.cnf'; |
2322 | + $suspend_file = $backup_dir . '/xtrabackup_suspended'; |
2323 | + $mysql_stdout = $backup_dir . '/mysql-stdout'; |
2324 | + $mysql_stderr = $backup_dir . '/mysql-stderr'; |
2325 | + $binlog_info = $backup_dir . '/xtrabackup_binlog_info'; |
2326 | + $slave_info = $backup_dir . '/xtrabackup_slave_info'; |
2327 | + } |
2328 | + else { |
2329 | + $suspend_file = get_option( \%config, 'mysqld', 'datadir' ) |
2330 | + . '/xtrabackup_suspended'; |
2331 | + $tmp_logfile = $o->{'tmpdir'} . '/xtrabackup_logfile'; |
2332 | + $mysql_stdout = $o->{'tmpdir'} . '/mysql-stdout'; |
2333 | + $mysql_stderr = $o->{'tmpdir'} . '/mysql-stderr'; |
2334 | + if ( $o->{'stream'} ) { |
2335 | + $backup_config_file = $o->{'tmpdir'} . '/backup-my.cnf'; |
2336 | + $binlog_info = $o->{'tmpdir'} . '/xtrabackup_binlog_info'; |
2337 | + $slave_info = $o->{'tmpdir'} . '/xtrabackup_slave_info'; |
2338 | + } |
2339 | + else { |
2340 | + $backup_config_file = $backup_dir . '/backup-my.cnf'; |
2341 | + $binlog_info = $backup_dir . '/xtrabackup_binlog_info'; |
2342 | + $slave_info = $backup_dir . '/xtrabackup_slave_info'; |
2343 | + } |
2344 | + } |
2345 | + write_backup_config_file($backup_config_file); |
2346 | + } |
2347 | + elsif ( $o->{'copy-back'} ) { |
2348 | + #$backup_config_file = $backup_dir . '/backup-my.cnf'; |
2349 | + #read_config_file($backup_config_file, \%backup_config); |
2350 | + } |
2351 | + |
2352 | + return; |
2353 | +} |
2354 | + |
2355 | +# |
2356 | +# write_backup_config_file subroutine creates a backup options file for |
2357 | +# ibbackup program. It writes to the file only those options that |
2358 | +# are required by ibbackup. |
2359 | +# Parameters: |
2360 | +# filename name for the created options file |
2361 | +# |
2362 | +sub write_backup_config_file { |
2363 | + my $filename = shift; |
2364 | + my $innodb_data_file_path = |
2365 | + get_option( \%config, 'mysqld', 'innodb_data_file_path' ); |
2366 | + my $innodb_log_files_in_group = |
2367 | + get_option( \%config, 'mysqld', 'innodb_log_files_in_group' ); |
2368 | + my $innodb_log_file_size = |
2369 | + get_option( \%config, 'mysqld', 'innodb_log_file_size' ); |
2370 | + my $root; |
2371 | + |
2372 | + my @array = split( /;/, $innodb_data_file_path ); |
2373 | + for ( my $i = 0; $i <= $#array; $i++ ) { |
2374 | + my @tmp = split( /\/+/, $array[$i] ); |
2375 | + $array[$i] = $tmp[-1]; |
2376 | + } |
2377 | + $innodb_data_file_path = join( ";", @array ); |
2378 | + |
2379 | + if ( !$o->{'remote-host'} ) { |
2380 | + $root = $backup_dir; |
2381 | + open( FILE, "> $filename" ) |
2382 | + || Die "Failed to open file '$filename': $OS_ERROR"; |
2383 | + } |
2384 | + else { |
2385 | + $root = `ssh $o->{'remote-host'} 'cd $backup_dir; pwd'`; |
2386 | + open( FILE, "| ssh $o->{'remote-host'} 'cat > $filename'" ) |
2387 | + || Die |
2388 | + "Failed to open file '$o->{'remote-host'}:$filename': $OS_ERROR"; |
2389 | + } |
2390 | + |
2391 | + print FILE |
2392 | + "# This MySQL options file was generated by $innobackup_script.\n\n" |
2393 | + . "# The MySQL server\n" |
2394 | + . "[mysqld]\n" |
2395 | + . "datadir=$root\n" |
2396 | + . "innodb_data_home_dir=$root\n" |
2397 | + . "innodb_data_file_path=$innodb_data_file_path\n" |
2398 | + . "innodb_log_group_home_dir=$root\n" |
2399 | + . "innodb_log_files_in_group=$innodb_log_files_in_group\n" |
2400 | + . "innodb_log_file_size=$innodb_log_file_size\n"; |
2401 | + close(FILE); |
2402 | + |
2403 | + if ( $o->{'stream'} ) { |
2404 | + my $filename_dir = dirname($filename); |
2405 | + my $filename_name = basename($filename); |
2406 | + if ( $o->{'stream'} eq 'tar' ) { |
2407 | + system_call("cd $filename_dir; tar chf - $filename_name"); |
2408 | + } |
2409 | + unlink $filename or Die "Failed to delete '$filename': $OS_ERROR"; |
2410 | + } |
2411 | +} |
2412 | + |
2413 | +# |
2414 | +# check_args subroutine checks command line arguments. If there is a problem, |
2415 | +# this subroutine prints error message and exits. |
2416 | +# |
2417 | +sub check_args { |
2418 | + my $i; |
2419 | + my $rcode; |
2420 | + my $buf; |
2421 | + my $perl_version; |
2422 | + |
2423 | + # check the version of the perl we are running |
2424 | + if ( !defined $PERL_VERSION ) { |
2425 | + |
2426 | + # this perl is prior to 5.6.0 and uses old style version string |
2427 | + my $required_version = $required_perl_version_old_style; |
2428 | + if ( $] lt $required_version ) { |
2429 | + print STDERR "$prefix Warning: " |
2430 | + . "Your perl is too old! Innobackup requires\n"; |
2431 | + print STDERR "$prefix Warning: perl $required_version or newer!\n"; |
2432 | + } |
2433 | + } |
2434 | + else { |
2435 | + $perl_version = 'v' . join( '.', @required_perl_version ); |
2436 | + if ( $PERL_VERSION lt $perl_version ) { |
2437 | + my $version = |
2438 | + chr( 48 + $required_perl_version[0] ) . "." |
2439 | + . chr( 48 + $required_perl_version[1] ) . "." |
2440 | + . chr( 48 + $required_perl_version[2] ); |
2441 | + print STDERR "$prefix Warning: " |
2442 | + . "Your perl is too old! Innobackup requires\n"; |
2443 | + print STDERR "$prefix Warning: perl $version or newer!\n"; |
2444 | + } |
2445 | + } |
2446 | + |
2447 | + if ( @ARGV == 0 ) { |
2448 | + |
2449 | + # no command line arguments |
2450 | + usage(); |
2451 | + exit(1); |
2452 | + } |
2453 | + |
2454 | + # read command line options |
2455 | + $rcode = GetOptions( |
2456 | + $o, |
2457 | + 'compress:i', |
2458 | + 'help', |
2459 | + 'version', |
2460 | + 'throttle=i', |
2461 | + 'sleep=i', |
2462 | + 'apply-log', |
2463 | + 'copy-back', |
2464 | + 'include=s', |
2465 | + 'databases=s', |
2466 | + 'use-memory=s', |
2467 | + 'uncompress', |
2468 | + 'export', |
2469 | + 'password=s', |
2470 | + 'host=s', |
2471 | + 'user=s', |
2472 | + 'port=s', |
2473 | + 'slave-info', |
2474 | + 'socket=s', |
2475 | + 'no-timestamp', |
2476 | + 'defaults-file=s', |
2477 | + 'incremental', |
2478 | + 'remote-host=s', |
2479 | + 'stream=s', |
2480 | + 'tmpdir=s', |
2481 | + 'no-lock', |
2482 | + 'ibbackup=s', |
2483 | + 'scpopt=s', |
2484 | + ); |
2485 | + |
2486 | + if ( !$rcode ) { |
2487 | + print STDERR "$prefix Bad command line arguments\n"; |
2488 | + exit 1; |
2489 | + } |
2490 | + |
2491 | + if ( $o->{'help'} ) { |
2492 | + # print help text and exit |
2493 | + usage(); |
2494 | + exit 1; |
2495 | + } |
2496 | + |
2497 | + if ( $o->{'version'} ) { |
2498 | + # print program version and copyright |
2499 | + print_version(); |
2500 | + exit 0; |
2501 | + } |
2502 | + |
2503 | + if ( $o->{'compress'} == 0 ) { |
2504 | + # compression level no specified, use default level |
2505 | + $o->{'compress'} = $default_compression_level; |
2506 | + } |
2507 | + |
2508 | + if ( $o->{'compress'} == 999 ) { |
2509 | + # compress option not given in the command line |
2510 | + $o->{'compress'} = 0; |
2511 | + } |
2512 | + |
2513 | + if ( @ARGV < 1 ) { |
2514 | + print STDERR "$prefix Missing command line argument\n"; |
2515 | + usage(); |
2516 | + exit(1); |
2517 | + } |
2518 | + elsif ( @ARGV > 1 ) { |
2519 | + print STDERR "$prefix Too many command line arguments\n"; |
2520 | + usage(); |
2521 | + exit(1); |
2522 | + } |
2523 | + |
2524 | + if ( $o->{'stream'} ) { |
2525 | + if ( $o->{'stream'} eq 'tar' ) { |
2526 | + } |
2527 | + elsif ( $o->{'stream'} eq 'tar4ibd' ) { |
2528 | + $o->{'stream'} = 'tar'; |
2529 | + $o->{'tar4ibd'} = 'tar4ibd'; |
2530 | + } |
2531 | + elsif ( $o->{'stream'} eq 'cpio' ) { |
2532 | + print STDERR "$prefix --stream=cpio is not supported yet\n"; |
2533 | + exit 1; |
2534 | + } |
2535 | + else { |
2536 | + print STDERR "$prefix Unknown option --stream=$o->{'stream'}\n"; |
2537 | + exit 1; |
2538 | + } |
2539 | + } |
2540 | + |
2541 | + # get options file name |
2542 | + #$config_file = $ARGV[0]; |
2543 | + |
2544 | + if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) { |
2545 | + # we are making a backup, get backup root directory |
2546 | + $backup_root = $ARGV[0]; |
2547 | + if ( $o->{'incremental'} ) { |
2548 | + if ( $o->{'remote-host'} ) { |
2549 | + print STDERR "--incremental does not work with --remote-host.\n"; |
2550 | + exit 1; |
2551 | + } |
2552 | + my @dirs = `ls -1 -t $backup_root`; |
2553 | + my $inc_dir = $dirs[0]; |
2554 | + chomp($inc_dir); |
2555 | + $incremental_basedir = File::Spec->catfile( $backup_root, $inc_dir ); |
2556 | + |
2557 | + #print STDERR "--incremental_basedir=$incremental_basedir\n"; |
2558 | + #print STDERR "incremental backup is not supported for now.\n"; |
2559 | + #exit(1); |
2560 | + } |
2561 | + if( $o->{'remote-host'} ) { |
2562 | + my ($host, $dir) = split( ':', $o->{'remote-host'} ); |
2563 | + if ( $dir ) { |
2564 | + print STDERR "--remote-host should not have a ':directory' part.\n" |
2565 | + . "Specify the remote host directory asBACK-ROOT-DIR."; |
2566 | + exit 1; |
2567 | + } |
2568 | + } |
2569 | + } |
2570 | + else { |
2571 | + # get backup directory |
2572 | + $backup_dir = File::Spec->rel2abs( $ARGV[0] ); |
2573 | + } |
2574 | + |
2575 | + parse_databases_option_value(); |
2576 | + |
2577 | + return; |
2578 | +} |
2579 | + |
2580 | +# |
2581 | +# make_backup_dir subroutine creates a new backup directory and returns |
2582 | +# its name. |
2583 | +# |
2584 | +sub make_backup_dir { |
2585 | + my $dir; |
2586 | + my $innodb_data_file_path = |
2587 | + get_option( \%config, 'mysqld', 'innodb_data_file_path' ); |
2588 | + |
2589 | + # create backup directory |
2590 | + $dir = $backup_root; |
2591 | + if ( $o->{'stream'} ) { |
2592 | + return $dir; |
2593 | + } |
2594 | + |
2595 | + $dir .= '/' . strftime( "%Y-%m-%d_%H-%M-%S", localtime() ) |
2596 | + unless $o->{'no-timestamp'}; |
2597 | + if ( !$o->{'remote-host'} ) { |
2598 | + mkdir( $dir, 0777 ) |
2599 | + || Die "Failed to create backup directory $dir: $OS_ERROR"; |
2600 | + } |
2601 | + else { |
2602 | + system_call("ssh $o->{'remote-host'} mkdir '$dir'"); |
2603 | + } |
2604 | + |
2605 | + # create subdirectories for ibdata files if needed |
2606 | + # foreach my $a (split(/;/, $innodb_data_file_path)) { |
2607 | + # my $path = (split(/:/,$a))[0]; |
2608 | + # my @relative_path = split(/\/+/, $path); |
2609 | + # pop @relative_path; |
2610 | + # if (@relative_path) { |
2611 | + # # there is a non-trivial path from the backup directory |
2612 | + # # to the directory of this backup ibdata file, check |
2613 | + # # that all the directories in the path exist. |
2614 | + # create_path_if_needed($dir, \@relative_path); |
2615 | + # } |
2616 | + # } |
2617 | + |
2618 | + return $dir; |
2619 | +} |
2620 | + |
2621 | +# |
2622 | +# create_path_if_needed subroutine checks that all components |
2623 | +# in the given relative path are directories. If the |
2624 | +# directories do not exist, they are created. |
2625 | +# Parameters: |
2626 | +# root a path to the root directory of the relative pathname |
2627 | +# relative_path a relative pathname (a reference to an array of |
2628 | +# pathname components) |
2629 | +# |
2630 | +sub create_path_if_needed { |
2631 | + my $root = shift; |
2632 | + my $relative_path = shift; |
2633 | + my $path; |
2634 | + |
2635 | + $path = $root; |
2636 | + foreach $a ( @{$relative_path} ) { |
2637 | + $path = $path . "/" . $a; |
2638 | + if ( !$o->{'remote-host'} ) { |
2639 | + if ( !-d $path ) { |
2640 | + |
2641 | + # this directory does not exist, create it ! |
2642 | + mkdir( $path, 0777 ) |
2643 | + || Die "Failed to create backup directory: $OS_ERROR"; |
2644 | + } |
2645 | + } |
2646 | + else { |
2647 | + if ( system_call("ssh $o->{'remote-host'} test -d '$path'", 1) != 0 ) { |
2648 | + system_call("ssh $o->{'remote-host'} mkdir $path"); |
2649 | + } |
2650 | + } |
2651 | + } |
2652 | +} |
2653 | + |
2654 | +# |
2655 | +# remove_from_array subroutine removes excluded element from the array. |
2656 | +# Parameters: |
2657 | +# array_ref a reference to an array of strings |
2658 | +# excluded a string to be excluded from the copy |
2659 | +# |
2660 | +sub remove_from_array { |
2661 | + my $array_ref = shift; |
2662 | + my $excluded = shift; |
2663 | + my @copy = (); |
2664 | + my $size = 0; |
2665 | + |
2666 | + foreach my $str ( @{$array_ref} ) { |
2667 | + if ( $str ne $excluded ) { |
2668 | + $copy[$size] = $str; |
2669 | + $size = $size + 1; |
2670 | + } |
2671 | + } |
2672 | + @{$array_ref} = @copy; |
2673 | +} |
2674 | + |
2675 | +# |
2676 | +# backup_files subroutine copies .frm, .MRG, .MYD and .MYI files to |
2677 | +# backup directory. |
2678 | +# |
2679 | +sub backup_files { |
2680 | + my $source_dir = get_option( \%config, 'mysqld', 'datadir' ); |
2681 | + my @list; |
2682 | + my $file; |
2683 | + my $database; |
2684 | + my $wildcard = '*.{frm,MYD,MYI,MRG,TRG,TRN,ARM,ARZ,opt,par}'; |
2685 | + |
2686 | + opendir( DIR, $source_dir ) |
2687 | + or Die "Cannot open directory '$source_dir': $OS_ERROR\n"; |
2688 | + $now = current_time(); |
2689 | + print STDERR |
2690 | + "\n$now $prefix Starting to backup .frm, .MRG, .MYD, .MYI,\n"; |
2691 | + print STDERR "$prefix .TRG, .TRN, .ARM, .ARZ and .opt files in\n"; |
2692 | + print STDERR "$prefix subdirectories of '$source_dir'\n"; |
2693 | + |
2694 | + # loop through all database directories |
2695 | + while ( defined( $database = readdir(DIR) ) ) { |
2696 | + my $print_each_file = 0; |
2697 | + my $file_c; |
2698 | + my @scp_files; |
2699 | + |
2700 | + # skip files that are not database directories |
2701 | + next if $database eq '.' || $database eq '..'; |
2702 | + next unless -d "$source_dir/$database"; |
2703 | + next unless check_if_required($database); |
2704 | + |
2705 | + IBEXDEBUG && _d('Working in database', "$source_dir/$database"); |
2706 | + |
2707 | + if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
2708 | + if ( !-e "$backup_dir/$database" ) { |
2709 | + |
2710 | + # create database directory for the backup |
2711 | + mkdir( "$backup_dir/$database", 0777 ) |
2712 | + or Die "Cannot create directory '$backup_dir/$database': " |
2713 | + . $OS_ERROR; |
2714 | + } |
2715 | + } |
2716 | + elsif ( $o->{'remote-host'} ) { |
2717 | + system_call("ssh $o->{'remote-host'} mkdir '$backup_dir/$database'"); |
2718 | + } |
2719 | + |
2720 | + # copy files of this database |
2721 | + opendir( DBDIR, "$source_dir/$database" ); |
2722 | + @list = grep( |
2723 | + /\.(frm)|(MYD)|(MYI)|(MRG)|(TRG)|(TRN)|(ARM)|(ARZ)|(opt)|(par)$/, |
2724 | + readdir(DBDIR) ); |
2725 | + closedir DBDIR; |
2726 | + $file_c = @list; |
2727 | + if ( $file_c <= $backup_file_print_limit ) { |
2728 | + $print_each_file = 1; |
2729 | + } |
2730 | + else { |
2731 | + print STDERR "$prefix Backing up files " |
2732 | + . "'$source_dir/$database/$wildcard' ($file_c files)\n"; |
2733 | + } |
2734 | + foreach $file (@list) { |
2735 | + |
2736 | + # copying may take a long time, so we have to prevent |
2737 | + # mysql connection from timing out |
2738 | + mysql_keep_alive(); |
2739 | + next unless check_if_required( $database, $file ); |
2740 | + |
2741 | + if ( $print_each_file ) { |
2742 | + print STDERR "$prefix Backing up file " |
2743 | + . "'$source_dir/$database/$file'\n"; |
2744 | + } |
2745 | + |
2746 | + if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
2747 | + $src_name = escape_path("$source_dir/$database/$file"); |
2748 | + $dst_name = escape_path("$backup_dir/$database"); |
2749 | + system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
2750 | + } |
2751 | + elsif ( $o->{'remote-host'} ) { |
2752 | + # Queue up files for one single scp per database. |
2753 | + push @scp_files, "'$file'"; |
2754 | + } |
2755 | + elsif ( $o->{'stream'} eq 'tar' ) { |
2756 | + my $ret = 0; |
2757 | + my $file_name = substr( $file, rindex( $file, '/' ) + 1 ); |
2758 | + $file_name =~ s/([\$\\\" ])/\\$1/g; |
2759 | + $ret = system_call( |
2760 | + "cd $source_dir; tar cf - $database/$file_name", 1); |
2761 | + if ( $ret == 1 ) { |
2762 | + print STDERR "$prefix If you use GNU tar, " |
2763 | + . "this warning can be ignored.\n"; |
2764 | + } |
2765 | + elsif ( $ret != 0 ) { |
2766 | + print STDERR "$prefix tar returned with exit code $ret.\n"; |
2767 | + Die "Failed to stream '$database/$file_name': $OS_ERROR"; |
2768 | + } |
2769 | + } |
2770 | + } |
2771 | + if ( $o->{'remote-host'} ) { |
2772 | + my $scp_file_list = |
2773 | + join( " ", map {"$source_dir/$database/$_"} @scp_files ); |
2774 | + system_call("scp $o->{'scp-opt'} $scp_file_list " |
2775 | + . "'$o->{'remote-host'}:$backup_dir/$database/'"); |
2776 | + } |
2777 | + } |
2778 | + closedir(DIR); |
2779 | + |
2780 | + $now = current_time(); |
2781 | + print STDERR |
2782 | + "$now $prefix Finished backing up .frm, .MRG, .MYD, .MYI, .TRG, .TRN, .ARM, .ARZ and .opt files\n\n"; |
2783 | + |
2784 | + return; |
2785 | +} |
2786 | + |
2787 | +# |
2788 | +# file_to_array subroutine reads the given text file into an array and |
2789 | +# stores each line as an element of the array. The end-of-line |
2790 | +# character(s) are removed from the lines stored in the array. |
2791 | +# Parameters: |
2792 | +# filename name of a text file |
2793 | +# lines_ref a reference to an array |
2794 | +# |
2795 | +sub file_to_array { |
2796 | + my $filename = shift; |
2797 | + my $lines_ref = shift; |
2798 | + |
2799 | + open( FILE, $filename ) || Die "can't open file '$filename': $OS_ERROR"; |
2800 | + @{$lines_ref} = <FILE>; |
2801 | + close(FILE) || Die "can't close file '$filename': $OS_ERROR"; |
2802 | + |
2803 | + foreach my $a ( @{$lines_ref} ) { |
2804 | + chomp($a); |
2805 | + } |
2806 | +} |
2807 | + |
2808 | +# |
2809 | +# unescape_string subroutine expands escape sequences found in the string and |
2810 | +# returns the expanded string. It also removes possible single or double quotes |
2811 | +# around the value. |
2812 | +# Parameters: |
2813 | +# value a string |
2814 | +# Return value: |
2815 | +# a string with expanded escape sequences |
2816 | +# |
2817 | +sub unescape_string { |
2818 | + my $value = shift; |
2819 | + my $result = ''; |
2820 | + my $offset = 0; |
2821 | + |
2822 | + # remove quotes around the value if they exist |
2823 | + if ( length($value) >= 2 ) { |
2824 | + if ( ( substr( $value, 0, 1 ) eq "'" && substr( $value, -1, 1 ) eq "'" ) |
2825 | + || ( substr( $value, 0, 1 ) eq '"' |
2826 | + && substr( $value, -1, 1 ) eq '"' ) ) |
2827 | + { |
2828 | + $value = substr( $value, 1, -1 ); |
2829 | + } |
2830 | + } |
2831 | + |
2832 | + # expand escape sequences |
2833 | + while ( $offset < length($value) ) { |
2834 | + my $pos = index( $value, "\\", $offset ); |
2835 | + if ( $pos < 0 ) { |
2836 | + $pos = length($value); |
2837 | + $result = $result . substr( $value, $offset, $pos - $offset ); |
2838 | + $offset = $pos; |
2839 | + } |
2840 | + else { |
2841 | + my $replacement = substr( $value, $pos, 2 ); |
2842 | + my $escape_code = substr( $value, $pos + 1, 1 ); |
2843 | + if ( exists $o->{'value-escapes'}{$escape_code} ) { |
2844 | + $replacement = $o->{'value-escapes'}{$escape_code}; |
2845 | + } |
2846 | + $result = |
2847 | + $result |
2848 | + . substr( $value, $offset, $pos - $offset ) |
2849 | + . $replacement; |
2850 | + $offset = $pos + 2; |
2851 | + } |
2852 | + } |
2853 | + |
2854 | + return $result; |
2855 | +} |
2856 | + |
2857 | +# |
2858 | +# read_config_file subroutine reads MySQL options file and |
2859 | +# returns the options in a hash containing one hash per group. |
2860 | +# Parameters: |
2861 | +# filename name of a MySQL options file |
2862 | +# groups_ref a reference to hash variable where the read |
2863 | +# options are returned |
2864 | +# |
2865 | +sub read_config_file { |
2866 | + |
2867 | + #my $filename = shift; |
2868 | + my $groups_ref = shift; |
2869 | + my @lines; |
2870 | + my $i; |
2871 | + my $group; |
2872 | + my $group_hash_ref; |
2873 | + |
2874 | + my $cmdline = ''; |
2875 | + my $options = ''; |
2876 | + |
2877 | + if ( $o->{'defaults-file'} ) { |
2878 | + $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" "; |
2879 | + } |
2880 | + |
2881 | + $options = $options . "--print-param"; |
2882 | + |
2883 | + # read file to an array, one line per element |
2884 | + #file_to_array($filename, \@lines); |
2885 | + $cmdline = "$o->{'ibbackup'} $options"; |
2886 | + @lines = `$cmdline`; |
2887 | + |
2888 | + # classify lines and save option values |
2889 | + $group = 'default'; |
2890 | + $group_hash_ref = {}; |
2891 | + ${$groups_ref}{$group} = $group_hash_ref; |
2892 | + |
2893 | + # this pattern described an option value which may be |
2894 | + # quoted with single or double quotes. This pattern |
2895 | + # does not work by its own. It assumes that the first |
2896 | + # opening parenthesis in this string is the second opening |
2897 | + # parenthesis in the full pattern. |
2898 | + my $value_pattern = q/((["'])([^\\\4]|(\\[^\4]))*\4)|([^\s]+)/; |
2899 | + for ( $i = 0; $i < @lines; $i++ ) { |
2900 | + SWITCH: for ( $lines[$i] ) { |
2901 | + |
2902 | + # comment |
2903 | + /^\s*(#|;)/ |
2904 | + && do { last; }; |
2905 | + |
2906 | + # group |
2907 | + /^\s*\[(.*)\]/ |
2908 | + && do { |
2909 | + $group = $1; |
2910 | + if ( !exists ${$groups_ref}{$group} ) { |
2911 | + $group_hash_ref = {}; |
2912 | + ${$groups_ref}{$group} = $group_hash_ref; |
2913 | + } |
2914 | + else { |
2915 | + $group_hash_ref = ${$groups_ref}{$group}; |
2916 | + } |
2917 | + last; |
2918 | + }; |
2919 | + |
2920 | + # option |
2921 | + /^\s*([^\s=]+)\s*(#.*)?$/ |
2922 | + && do { |
2923 | + ${$group_hash_ref}{$1} = ''; |
2924 | + last; |
2925 | + }; |
2926 | + |
2927 | + # set-variable = option = value |
2928 | + /^\s*set-variable\s*=\s*([^\s=]+)\s*=\s*($value_pattern)\s*(#.*)?$/ |
2929 | + && do { ${$group_hash_ref}{$1} = unescape_string($2); last; }; |
2930 | + |
2931 | + # option = value |
2932 | + /^\s*([^\s=]+)\s*=\s*($value_pattern)\s*(#.*)?$/ |
2933 | + && do { ${$group_hash_ref}{$1} = unescape_string($2); last; }; |
2934 | + |
2935 | + # empty line |
2936 | + /^\s*$/ |
2937 | + && do { last; }; |
2938 | + |
2939 | + # unknown |
2940 | + print( "$prefix: Warning: Ignored unrecognized line ", |
2941 | + $i + 1, " in options : '${lines[$i]}'\n" ); |
2942 | + } |
2943 | + } |
2944 | +} |
2945 | + |
2946 | +# |
2947 | +# get_option subroutine returns the value of given option in the config |
2948 | +# structure. If option is missing, this subroutine calls exit. |
2949 | +# Parameters: |
2950 | +# config_ref a reference to a config data |
2951 | +# group option group name |
2952 | +# option_name name of the option |
2953 | +# Return value: |
2954 | +# option value as a string |
2955 | +# |
2956 | +sub get_option { |
2957 | + my $config_ref = shift; |
2958 | + my $group = shift; |
2959 | + my $o->{'name'} = shift; |
2960 | + my $group_hash_ref; |
2961 | + |
2962 | + if ( !exists $config{$group} ) { |
2963 | + # no group |
2964 | + print STDERR "$prefix fatal error: no '$group' group in MySQL options\n"; |
2965 | + print STDERR "$prefix fatal error: OR no 'datadir' option in group " |
2966 | + . "'$group' in MySQL options\n"; |
2967 | + exit(1); |
2968 | + } |
2969 | + |
2970 | + $group_hash_ref = ${$config_ref}{$group}; |
2971 | + if ( !exists ${$group_hash_ref}{ $o->{'name'} } ) { |
2972 | + # no option |
2973 | + print STDERR "$prefix fatal error: no '$o->{'name'}' option in group " |
2974 | + . "'$group' in MySQL options\n"; |
2975 | + exit(1); |
2976 | + } |
2977 | + |
2978 | + return ${$group_hash_ref}{ $o->{'name'} }; |
2979 | +} |
2980 | + |
2981 | +# check_if_required subroutine returns 1 if the specified database and |
2982 | +# table needs to be backed up. |
2983 | +# Parameters: |
2984 | +# $_[0] name of database to be checked |
2985 | +# $_[1] full path of table file (This argument is optional) |
2986 | +# Return value: |
2987 | +# 1 if backup should be done and 0 if not |
2988 | +# |
2989 | +sub check_if_required { |
2990 | + my $db = $_[0]; |
2991 | + my $db_count = keys %databases_list; |
2992 | + my $table_path; |
2993 | + if ( defined $_[1] ) { |
2994 | + $table_path = $_[1]; |
2995 | + } |
2996 | + else { |
2997 | + undef $table_path; |
2998 | + } |
2999 | + |
3000 | + if ( $db_count == 0 ) { |
3001 | + # no databases defined with --databases option, include all databases |
3002 | + return 1; |
3003 | + } |
3004 | + if ( defined $databases_list{$db} ) { |
3005 | + if ( defined $table_path ) { |
3006 | + |
3007 | + # get the last component in the table pathname |
3008 | + my $filename = ( reverse( split( /\//, $table_path ) ) )[0]; |
3009 | + |
3010 | + # get name of the table by removing file suffix |
3011 | + my $table = ( split( /\./, $filename ) )[0]; |
3012 | + |
3013 | + my $db_hash = $databases_list{$db}; |
3014 | + $db_count = keys %$db_hash; |
3015 | + if ( $db_count > 0 && !defined $databases_list{$db}->{$table} ) { |
3016 | + |
3017 | + # --databases option specified, but table is not included |
3018 | + return 0; |
3019 | + } |
3020 | + } |
3021 | + # include this database and table |
3022 | + return 1; |
3023 | + } |
3024 | + else { |
3025 | + # --databases option given, but database is not included |
3026 | + return 0; |
3027 | + } |
3028 | +} |
3029 | + |
3030 | +# parse_databases_option_value subroutine parses the value of |
3031 | +# --databases option. If the option value begins with a slash |
3032 | +# it is considered a pathname and the option value is read |
3033 | +# from the file. |
3034 | +# |
3035 | +# This subroutine sets the global "databases_list" variable. |
3036 | +# |
3037 | +sub parse_databases_option_value { |
3038 | + my $item; |
3039 | + |
3040 | + if ( $o->{'databases'} =~ /^\// ) { |
3041 | + |
3042 | + # the value of the --databases option begins with a slash, |
3043 | + # the option value is pathname of the file containing |
3044 | + # list of databases |
3045 | + if ( !-f $o->{'databases'} ) { |
3046 | + Die "can't find file '$o->{'databases'}'"; |
3047 | + } |
3048 | + |
3049 | + # read from file the value of --databases option |
3050 | + my @lines; |
3051 | + file_to_array( $o->{'databases'}, \@lines ); |
3052 | + $o->{'databases'} = join( " ", @lines ); |
3053 | + } |
3054 | + |
3055 | + # mark each database or database.table definition in the |
3056 | + # global databases_list. |
3057 | + foreach $item ( split( /\s/, $o->{'databases'} ) ) { |
3058 | + my $db = ""; |
3059 | + my $table = ""; |
3060 | + my %hash; |
3061 | + |
3062 | + if ( $item eq "" ) { |
3063 | + |
3064 | + # ignore empty strings |
3065 | + next; |
3066 | + } |
3067 | + |
3068 | + # get database and table names |
3069 | + if ( $item =~ /(\S*)\.(\S*)/ ) { |
3070 | + |
3071 | + # item is of the form DATABASE.TABLE |
3072 | + $db = $1; |
3073 | + $table = $2; |
3074 | + } |
3075 | + else { |
3076 | + |
3077 | + # item is database name, table is undefined |
3078 | + $db = $item; |
3079 | + } |
3080 | + |
3081 | + if ( !defined $databases_list{$db} ) { |
3082 | + |
3083 | + # create empty hash for the database |
3084 | + $databases_list{$db} = \%hash; |
3085 | + } |
3086 | + if ( $table ne "" ) { |
3087 | + |
3088 | + # add mapping table --> 1 to the database hash |
3089 | + my $h = $databases_list{$db}; |
3090 | + $h->{$table} = 1; |
3091 | + } |
3092 | + } |
3093 | +} |
3094 | + |
3095 | +sub escape_path { |
3096 | + my $str = shift; |
3097 | + if ( $win eq 1 ) { |
3098 | + $str =~ s/\//\\/g; |
3099 | + $str =~ s/\\\\/\\/g; |
3100 | + } |
3101 | + else { |
3102 | + $str =~ s/\/\//\//g; |
3103 | + } |
3104 | + return $str; |
3105 | + |
3106 | +} |
3107 | + |
3108 | +sub system_call { |
3109 | + my ( $cmd, $dont_check_retval ) = @_; |
3110 | + IBEXDEBUG && _d($cmd); |
3111 | + Die "I need a cmd argument" unless $cmd; |
3112 | + |
3113 | + my $retval = system($cmd); |
3114 | + IBEXDEBUG && _d('Exit status:', $retval); |
3115 | + |
3116 | + return $retval if $dont_check_retval; |
3117 | + |
3118 | + # A non-zero retval (exit status) means that cmd caused an error. |
3119 | + if ( $retval ) { |
3120 | + Die "Failed system call: $cmd\n" |
3121 | + . "Exit status: $retval\n" |
3122 | + } |
3123 | + |
3124 | + return $retval; |
3125 | +} |
3126 | + |
3127 | +sub _d { |
3128 | + my ( $package, undef, $line ) = caller 0; |
3129 | + @_ = map { ( my $temp = $_ ) =~ s/\n/\n# /g; $temp; } |
3130 | + map { defined $_ ? $_ : 'undef' } @_; |
3131 | + print STDERR "# $package:$line $PID ", join( ' ', @_ ), "\n"; |
3132 | +} |
3133 | + |
3134 | +# ############################################################################ |
3135 | +# Run the program. |
3136 | +# ############################################################################ |
3137 | +if ( !caller ) { exit main(@ARGV); } |
3138 | + |
3139 | +1; # Because this is a module as well as a script. |
3140 | |
3141 | === added directory 'innobackupex/t' |
3142 | === added file 'innobackupex/t/001_get_mysql_options.t' |
3143 | --- innobackupex/t/001_get_mysql_options.t 1970-01-01 00:00:00 +0000 |
3144 | +++ innobackupex/t/001_get_mysql_options.t 2010-01-22 01:31:15 +0000 |
3145 | @@ -0,0 +1,61 @@ |
3146 | +#!/usr/bin/perl |
3147 | + |
3148 | +BEGIN { |
3149 | + die "The XTRABACKUP_TRUNK environment variable is not set." |
3150 | + unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK}; |
3151 | + unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common"; |
3152 | +}; |
3153 | + |
3154 | +use strict; |
3155 | +use warnings FATAL => 'all'; |
3156 | +use English qw(-no_match_vars); |
3157 | +use Test::More tests => 5; |
3158 | + |
3159 | +use IbexTest; |
3160 | +require "$trunk/innobackupex/innobackupex"; |
3161 | + |
3162 | +is( |
3163 | + innobackupex::get_mysql_options(), |
3164 | + '--unbuffered --', |
3165 | + 'No options' |
3166 | +); |
3167 | + |
3168 | +@ARGV = qw(--password foo /tmp); |
3169 | +innobackupex::check_args(); |
3170 | +is( |
3171 | + innobackupex::get_mysql_options(), |
3172 | + '--password=foo --unbuffered --', |
3173 | + '--password' |
3174 | +); |
3175 | + |
3176 | +# check_args() doesn't clear opts from previous runs so this |
3177 | +# test inherits --password=foo. |
3178 | +@ARGV = qw(--defaults-file=/etc/my.cnf /tmp); |
3179 | +innobackupex::check_args(); |
3180 | +is( |
3181 | + innobackupex::get_mysql_options(), |
3182 | + '--defaults-file=/etc/my.cnf --password=foo --unbuffered --', |
3183 | + '--defaults-file' |
3184 | +); |
3185 | + |
3186 | +@ARGV = qw(--port=12345 --user=bob --password=bar /tmp); |
3187 | +innobackupex::check_args(); |
3188 | +is( |
3189 | + innobackupex::get_mysql_options(), |
3190 | + '--defaults-file=/etc/my.cnf --port=12345 --user=bob --password=bar --unbuffered --', |
3191 | + '--port and --user' |
3192 | +); |
3193 | + |
3194 | +# The whole enchilada. |
3195 | +@ARGV = qw(--host=10.0.0.1 /tmp); |
3196 | +innobackupex::check_args(); |
3197 | +is( |
3198 | + innobackupex::get_mysql_options(), |
3199 | + '--defaults-file=/etc/my.cnf --host=10.0.0.1 --port=12345 --user=bob --password=bar --unbuffered --', |
3200 | + '--host' |
3201 | +); |
3202 | + |
3203 | +# ############################################################################# |
3204 | +# Done. |
3205 | +# ############################################################################# |
3206 | +exit; |
3207 | |
3208 | === added file 'innobackupex/t/100_backup.t' |
3209 | --- innobackupex/t/100_backup.t 1970-01-01 00:00:00 +0000 |
3210 | +++ innobackupex/t/100_backup.t 2010-01-22 01:31:15 +0000 |
3211 | @@ -0,0 +1,87 @@ |
3212 | +#!/usr/bin/perl |
3213 | + |
3214 | +BEGIN { |
3215 | + die "The XTRABACKUP_TRUNK environment variable is not set." |
3216 | + unless $ENV{XTRABACKUP_TRUNK} && -d $ENV{XTRABACKUP_TRUNK}; |
3217 | + unshift @INC, "$ENV{XTRABACKUP_TRUNK}/innobackupex/common"; |
3218 | +}; |
3219 | + |
3220 | +use strict; |
3221 | +use warnings FATAL => 'all'; |
3222 | +use English qw(-no_match_vars); |
3223 | +use Test::More; |
3224 | + |
3225 | +use IbexTest; |
3226 | +use DSNParser; |
3227 | +use Sandbox; |
3228 | +require "$trunk/innobackupex/innobackupex"; |
3229 | + |
3230 | +my $dp = new DSNParser(); |
3231 | +my $sb = new Sandbox(basedir => '/tmp', DSNParser => $dp); |
3232 | +my $dbh = $sb->get_dbh_for('master'); |
3233 | + |
3234 | +if ( !$dbh ) { |
3235 | + plan skip_all => 'Cannot connect to sandbox master'; |
3236 | +} |
3237 | +else { |
3238 | + plan tests => 9; |
3239 | +} |
3240 | + |
3241 | +my $cnf = '/tmp/12345/my.sandbox.cnf'; |
3242 | +my $backup_root_dir = '/tmp/ibex'; |
3243 | + |
3244 | +diag(`rm -rf $backup_root_dir >/dev/null`); |
3245 | + |
3246 | +$sb->wipe_clean($dbh); |
3247 | +$sb->load_file('master', "innobackupex/t/samples/basic-tables.sql"); |
3248 | + |
3249 | +output( |
3250 | + sub { innobackupex::main('--defaults-file', $cnf, $backup_root_dir, |
3251 | + qw(--databases ibex --no-timestamp)) } |
3252 | +); |
3253 | + |
3254 | +ok( |
3255 | + -d $backup_root_dir, |
3256 | + "Created backup root dir $backup_root_dir" |
3257 | +); |
3258 | + |
3259 | +ok( |
3260 | + -d "$backup_root_dir/ibex", |
3261 | + "Created $backup_root_dir/ibex" |
3262 | +); |
3263 | + |
3264 | +ok( |
3265 | + !-d "$backup_root_dir/mysql", |
3266 | + "Did not backup mysql database" |
3267 | +); |
3268 | + |
3269 | +my @files = </tmp/12345/data/ibex/*>; |
3270 | + |
3271 | +is_deeply( |
3272 | + \@files, |
3273 | + [qw( |
3274 | + /tmp/12345/data/ibex/db.opt |
3275 | + /tmp/12345/data/ibex/t1.frm |
3276 | + /tmp/12345/data/ibex/t2.frm |
3277 | + /tmp/12345/data/ibex/t2.MYD |
3278 | + /tmp/12345/data/ibex/t2.MYI |
3279 | + )], |
3280 | + 'Copied all files from ibex db' |
3281 | +); |
3282 | + |
3283 | +foreach my $file ( @files ) { |
3284 | + my ($file2) = $file =~ m/([\w\.]+)$/; |
3285 | + my $output = `diff $file $backup_root_dir/ibex/$file2`; |
3286 | + is( |
3287 | + $output, |
3288 | + '', |
3289 | + "$file2 does not differ" |
3290 | + ); |
3291 | +} |
3292 | + |
3293 | +# ############################################################################# |
3294 | +# Done. |
3295 | +# ############################################################################# |
3296 | +diag(`rm -rf $backup_root_dir >/dev/null`); |
3297 | +$sb->wipe_clean($dbh); |
3298 | +exit; |
3299 | |
3300 | === added directory 'innobackupex/t/samples' |
3301 | === added file 'innobackupex/t/samples/basic-tables.sql' |
3302 | --- innobackupex/t/samples/basic-tables.sql 1970-01-01 00:00:00 +0000 |
3303 | +++ innobackupex/t/samples/basic-tables.sql 2010-01-22 01:31:15 +0000 |
3304 | @@ -0,0 +1,17 @@ |
3305 | +drop database if exists ibex; |
3306 | +create database ibex; |
3307 | +use ibex; |
3308 | + |
3309 | +create table t1 ( |
3310 | + i int, |
3311 | + unique key (i) |
3312 | +) engine=innodb; |
3313 | + |
3314 | +insert into ibex.t1 values (1),(2),(3); |
3315 | + |
3316 | +create table t2 ( |
3317 | + i int, |
3318 | + unique key (i) |
3319 | +) engine=myisam; |
3320 | + |
3321 | +insert into ibex.t2 values (4),(5),(6); |
3322 | |
3323 | === added file 'innobackupex/test-coverage.txt' |
3324 | --- innobackupex/test-coverage.txt 1970-01-01 00:00:00 +0000 |
3325 | +++ innobackupex/test-coverage.txt 2010-01-22 01:31:15 +0000 |
3326 | @@ -0,0 +1,2854 @@ |
3327 | +Reading database from /home/daniel/dev/percona-xtrabackup/innobackupex/t/cover_db |
3328 | + |
3329 | + |
3330 | +---------------------------- ------ ------ ------ ------ ------ ------ ------ |
3331 | +File stmt bran cond sub pod time total |
3332 | +---------------------------- ------ ------ ------ ------ ------ ------ ------ |
3333 | +...innobackupex/innobackupex 51.7 32.6 33.7 76.4 n/a 100.0 46.0 |
3334 | +Total 51.7 32.6 33.7 76.4 n/a 100.0 46.0 |
3335 | +---------------------------- ------ ------ ------ ------ ------ ------ ------ |
3336 | + |
3337 | + |
3338 | +Run: 100_backup.t |
3339 | +Perl version: 118.53.46.49.48.46.48 |
3340 | +OS: linux |
3341 | +Start: Thu Jan 21 23:50:53 2010 |
3342 | +Finish: Thu Jan 21 23:50:55 2010 |
3343 | + |
3344 | +Run: 100_backup.t |
3345 | +Perl version: 118.53.46.49.48.46.48 |
3346 | +OS: linux |
3347 | +Start: Thu Jan 21 23:50:53 2010 |
3348 | +Finish: Thu Jan 21 23:51:12 2010 |
3349 | + |
3350 | +Run: 001_get_mysql_options.t |
3351 | +Perl version: 118.53.46.49.48.46.48 |
3352 | +OS: linux |
3353 | +Start: Thu Jan 21 23:55:00 2010 |
3354 | +Finish: Thu Jan 21 23:55:00 2010 |
3355 | + |
3356 | +/home/daniel/dev/percona-xtrabackup/innobackupex/innobackupex |
3357 | + |
3358 | +line err stmt bran cond sub pod time code |
3359 | +1 #!/usr/bin/env perl |
3360 | +2 |
3361 | +3 # A script for making backups of InnoDB and MyISAM tables, indexes and .frm |
3362 | +4 # files. |
3363 | +5 # |
3364 | +6 # Orginal code copyright 2003, 2009 Innobase Oy. |
3365 | +7 # http://www.innodb.com/download/ |
3366 | +8 # |
3367 | +9 # Modified code copyright 2010 Percona Inc. |
3368 | +10 # https://launchpad.net/percona-xtrabackup/trunk |
3369 | +11 |
3370 | +12 # ############################################################################ |
3371 | +13 # This is a a runnable module. Check at the end of this package for the call |
3372 | +14 # to main() which actually runs the program. |
3373 | +15 # ############################################################################ |
3374 | +16 package innobackupex; |
3375 | +17 |
3376 | +18 3 3 23 use strict; |
3377 | + 3 6 |
3378 | + 3 19 |
3379 | +19 3 3 20 use warnings FATAL => 'all'; |
3380 | + 3 6 |
3381 | + 3 29 |
3382 | +20 3 3 18 use English qw(-no_match_vars); |
3383 | + 3 7 |
3384 | + 3 17 |
3385 | +21 3 3 32 use Getopt::Long qw(:config default); |
3386 | + 3 12 |
3387 | + 3 23 |
3388 | +22 3 3 24 use File::Spec; |
3389 | + 3 6 |
3390 | + 3 35 |
3391 | +23 3 3 47 use POSIX "strftime"; |
3392 | + 3 10 |
3393 | + 3 24 |
3394 | +24 3 3 55 use POSIX ":sys_wait_h"; |
3395 | + 3 8 |
3396 | + 3 24 |
3397 | +25 3 3 22 use POSIX "tmpnam"; |
3398 | + 3 18 |
3399 | + 3 21 |
3400 | +26 3 3 21 use FileHandle; |
3401 | + 3 9 |
3402 | + 3 32 |
3403 | +27 3 3 24 use File::Basename; |
3404 | + 3 9 |
3405 | + 3 41 |
3406 | +28 |
3407 | +29 *** 3 50 3 22 use constant IBEXDEBUG => $ENV{IBEXDEBUG} || 0; |
3408 | + 3 9 |
3409 | + 3 57 |
3410 | +30 |
3411 | +31 # version of this script |
3412 | +32 my $innobackup_version = '1.5.1-xtrabackup'; |
3413 | +33 my $innobackup_script = basename($0); |
3414 | +34 |
3415 | +35 # copyright notice |
3416 | +36 my $copyright_notice = "InnoDB Backup Utility v${innobackup_version}; Copyright 2003, 2009 Innobase Oy. |
3417 | +37 All Rights Reserved. |
3418 | +38 |
3419 | +39 This software is published under |
3420 | +40 the GNU GENERAL PUBLIC LICENSE Version 2, June 1991. |
3421 | +41 |
3422 | +42 "; |
3423 | +43 |
3424 | +44 # required Perl version (5.005) |
3425 | +45 my @required_perl_version = ( 5, 0, 5 ); |
3426 | +46 my $required_perl_version_old_style = 5.005; |
3427 | +47 |
3428 | +48 # force flush after every write and print |
3429 | +49 $OUTPUT_AUTOFLUSH = 1; |
3430 | +50 |
3431 | +51 ###################################################################### |
3432 | +52 # modifiable parameters |
3433 | +53 ###################################################################### |
3434 | +54 |
3435 | +55 # maximum number of files in a database directory which are |
3436 | +56 # separately printed when a backup is made |
3437 | +57 my $backup_file_print_limit = 9; |
3438 | +58 |
3439 | +59 # timeout in seconds for a reply from mysql |
3440 | +60 my $mysql_response_timeout = 900; |
3441 | +61 |
3442 | +62 # default compression level (this is an argument to ibbackup) |
3443 | +63 my $default_compression_level = 1; |
3444 | +64 |
3445 | +65 # time in seconds after which a dummy query is sent to mysql server |
3446 | +66 # in order to keep the database connection alive |
3447 | +67 my $mysql_keep_alive_timeout = 1800; |
3448 | +68 |
3449 | +69 ###################################################################### |
3450 | +70 # end of modifiable parameters |
3451 | +71 ###################################################################### |
3452 | +72 |
3453 | +73 # command line options |
3454 | +74 my $o = { |
3455 | +75 'compress' => 999, |
3456 | +76 'databases' => '', |
3457 | +77 'ibbackup' => 'xtrabackup', |
3458 | +78 'scp-opt' => '-Cp -c arcfour', |
3459 | +79 'stream' => '', |
3460 | +80 }; |
3461 | +81 |
3462 | +82 # root of the backup directory |
3463 | +83 my $backup_root = ''; |
3464 | +84 |
3465 | +85 # backup directory pathname |
3466 | +86 my $backup_dir = ''; |
3467 | +87 |
3468 | +88 # name of the ibbackup suspend-at-end file |
3469 | +89 my $suspend_file = ''; |
3470 | +90 |
3471 | +91 # name of the temporary transaction log file during the backup |
3472 | +92 my $tmp_logfile = ''; |
3473 | +93 |
3474 | +94 # home directory of innoDB log files |
3475 | +95 my $innodb_log_group_home_dir = ''; |
3476 | +96 |
3477 | +97 # backup my.cnf file |
3478 | +98 my $backup_config_file = ''; |
3479 | +99 |
3480 | +100 # options from the options file |
3481 | +101 my %config; |
3482 | +102 |
3483 | +103 # options from the backup options file |
3484 | +104 #my %backup_config; |
3485 | +105 |
3486 | +106 # list of databases to be included in a backup |
3487 | +107 my %databases_list; |
3488 | +108 |
3489 | +109 # prefix for output lines |
3490 | +110 my $prefix = "$innobackup_script:"; |
3491 | +111 |
3492 | +112 # process id of mysql client program (runs as a child process of this script) |
3493 | +113 my $mysql_pid = ''; |
3494 | +114 |
3495 | +115 # mysql server version string |
3496 | +116 my $mysql_server_version = ''; |
3497 | +117 |
3498 | +118 # name of the file where stderr of mysql process is directed |
3499 | +119 my $mysql_stderr; |
3500 | +120 |
3501 | +121 # name of the file where stdout of mysql process is directed |
3502 | +122 my $mysql_stdout; |
3503 | +123 |
3504 | +124 # name of the file where binlog position info is written |
3505 | +125 my $binlog_info; |
3506 | +126 |
3507 | +127 # name of the file where slave info is written |
3508 | +128 my $slave_info; |
3509 | +129 |
3510 | +130 # mysql binlog position as given by "SHOW MASTER STATUS" command |
3511 | +131 my $mysql_binlog_position = ''; |
3512 | +132 |
3513 | +133 # mysql master's binlog position as given by "SHOW SLAVE STATUS" command |
3514 | +134 # run on a slave server |
3515 | +135 my $mysql_slave_position = ''; |
3516 | +136 |
3517 | +137 # time of the most recent mysql_check call. (value returned by time() function) |
3518 | +138 my $mysql_last_access_time = 0; |
3519 | +139 |
3520 | +140 # process id of ibbackup program (runs as a child process of this script) |
3521 | +141 my $ibbackup_pid = ''; |
3522 | +142 |
3523 | +143 # a counter for numbering mysql connection checks |
3524 | +144 my $hello_id = 0; |
3525 | +145 |
3526 | +146 # the request which has been sent to mysqld, but to which |
3527 | +147 # mysqld has not yet replied. Empty string denotes that no |
3528 | +148 # request has been sent to mysqld or that mysqld has replied |
3529 | +149 # to all requests. |
3530 | +150 my $current_mysql_request = ''; |
3531 | +151 |
3532 | +152 # escape sequences for options files |
3533 | +153 my %option_value_escapes = ( |
3534 | +154 'b' => "\b", |
3535 | +155 't' => "\t", |
3536 | +156 'n' => "\n", |
3537 | +157 'r' => "\r", |
3538 | +158 "\\" => "\\", |
3539 | +159 's' => ' ' |
3540 | +160 ); |
3541 | +161 |
3542 | +162 # signal that is sent to child processes when they are killed |
3543 | +163 my $kill_signal = 15; |
3544 | +164 |
3545 | +165 # current local time |
3546 | +166 my $now; |
3547 | +167 |
3548 | +168 # incremental backup base directory |
3549 | +169 my $incremental_basedir = ''; |
3550 | +170 |
3551 | +171 my $src_name; |
3552 | +172 my $dst_name; |
3553 | +173 my $win = ( $^O eq 'MSWin32' ? 1 : 0 ); |
3554 | +174 my $CP_CMD = ( $win eq 1 ? "copy /Y" : "cp -p" ); |
3555 | +175 |
3556 | +176 ###################################################################### |
3557 | +177 # program execution begins here |
3558 | +178 ###################################################################### |
3559 | +179 sub main { |
3560 | +180 2 2 162 @ARGV = @_; # set global ARGV for this package |
3561 | +181 |
3562 | +182 # check command-line args |
3563 | +183 2 34 check_args(); |
3564 | +184 |
3565 | +185 # print program version and copyright |
3566 | +186 2 14 print_version(); |
3567 | +187 |
3568 | +188 # initialize global variables and perform some checks |
3569 | +189 2 32 init(); |
3570 | +190 |
3571 | +191 *** 2 50 26 if ( $o->{'copy-back'} ) { |
3572 | + *** 50 |
3573 | +192 # copy files from backup directory back to their original locations |
3574 | +193 *** 0 0 copy_back(); |
3575 | +194 } |
3576 | +195 elsif ( $o->{'apply-log'} ) { |
3577 | +196 # expand data files in backup directory by applying log files to them |
3578 | +197 *** 0 0 apply_log(); |
3579 | +198 } |
3580 | +199 else { |
3581 | +200 # make a backup of InnoDB and MyISAM tables, indexes and .frm files. |
3582 | +201 2 40 backup(); |
3583 | +202 } |
3584 | +203 |
3585 | +204 # program has completed successfully |
3586 | +205 1 7 $now = current_time(); |
3587 | +206 1 24 print STDERR "$now $prefix completed OK!\n"; |
3588 | +207 |
3589 | +208 *** 1 50 14 if ( $o->{'stream'} eq 'tar' ) { |
3590 | +209 *** 0 0 print STDERR "$prefix You must use -i (--ignore-zeros) option for extraction of the tar stream.\n"; |
3591 | +210 } |
3592 | +211 |
3593 | +212 1 23 return 0; |
3594 | +213 } |
3595 | +214 |
3596 | +215 ############################################################################## |
3597 | +216 # Subroutines |
3598 | +217 ############################################################################## |
3599 | +218 |
3600 | +219 # |
3601 | +220 # print_version subroutine prints program version and copyright. |
3602 | +221 # |
3603 | +222 sub print_version { |
3604 | +223 2 2 140 printf( STDERR $copyright_notice ); |
3605 | +224 } |
3606 | +225 |
3607 | +226 # |
3608 | +227 # usage subroutine prints instructions of how to use this program to stdout. |
3609 | +228 # |
3610 | +229 sub usage { |
3611 | +230 *** 0 0 0 print STDOUT <<EOF; |
3612 | +231 |
3613 | +232 Usage: |
3614 | +233 |
3615 | +234 $innobackup_script BACKUP-ROOT-DIR [OPTIONS] |
3616 | +235 |
3617 | +236 $innobackup_script --apply-log BACKUP-DIR [OPTIONS] |
3618 | +237 |
3619 | +238 $innobackup_script --copy-back BACKUP-DIR [OPTIONS] |
3620 | +239 |
3621 | +240 The first command line above makes a hot backup of a MySQL database. |
3622 | +241 By default it creates a backup directory (named by the current date |
3623 | +242 and time) in the given BACKUP-ROOT-DIR directory. With the --no-timestamp |
3624 | +243 option it does not create a time-stamped backup directory, but it puts |
3625 | +244 the backup in the given directory (which must not exist). This |
3626 | +245 command makes a complete backup of all MyISAM and InnoDB tables and |
3627 | +246 indexes in all databases or in all of the databases specified with the |
3628 | +247 --databases option. The created backup contains .frm, .MRG, .MYD, |
3629 | +248 .MYI, .TRG, .TRN, .ARM, .ARZ, .opt, .par, and InnoDB data and log files. |
3630 | +249 The MY.CNF options file defines the location of the database. This command |
3631 | +250 connects to the MySQL server using mysql client program, and runs |
3632 | +251 ibbackup (InnoDB Hot Backup program) as a child process. |
3633 | +252 |
3634 | +253 The command with --apply-log option prepares a backup for starting a MySQL |
3635 | +254 server on the backup. This command expands InnoDB data files as specified |
3636 | +255 in BACKUP-DIR/backup-my.cnf using BACKUP-DIR/ibbackup_logfile, |
3637 | +256 and creates new InnoDB log files as specified in BACKUP-DIR/backup-my.cnf. |
3638 | +257 The BACKUP-DIR should be a path name of a backup directory created by |
3639 | +258 innobackup. This command runs ibbackup as a child process, but it does not |
3640 | +259 connect to the database server. |
3641 | +260 |
3642 | +261 The command with --copy-back option copies data, index, and log files |
3643 | +262 from backup directory back to their original locations. |
3644 | +263 The MY.CNF options file defines the original location of the database. |
3645 | +264 The BACKUP-DIR is a path name of a backup directory created by innobackup. |
3646 | +265 |
3647 | +266 On success the exit code of innobackup process is 0. A non-zero exit code |
3648 | +267 indicates an error. |
3649 | +268 |
3650 | +269 Options: |
3651 | +270 |
3652 | +271 --help Display this helpscreen and exit. |
3653 | +272 |
3654 | +273 --version Print version information and exit. |
3655 | +274 |
3656 | +275 --defaults-file=[MY.CNF] |
3657 | +276 This option is passed to the xtrabackup child process |
3658 | +277 and the mysql child process. It specifies which file |
3659 | +278 to read default options from. |
3660 | +279 |
3661 | +280 --apply-log |
3662 | +281 Prepare a backup for starting mysql server on the backup. |
3663 | +282 Expand InnoDB data files as specified in |
3664 | +283 backup-dir/backup-my.cnf, using backup-dir/ibbackup_logfile, |
3665 | +284 and create new log files as specified in |
3666 | +285 backup-dir/backup-my.cnf. |
3667 | +286 |
3668 | +287 --copy-back |
3669 | +288 Copy data and index files from backup directory back to |
3670 | +289 their original locations. |
3671 | +290 |
3672 | +291 --remote-host=[USER@]HOSTNAME |
3673 | +292 If this option is specified, backup files will be created |
3674 | +293 at the remote host by using ssh and scp. BACKUP-ROOT-DIR |
3675 | +294 is relative to this host. |
3676 | +295 |
3677 | +296 --stream=[tar|cpio(not implemented)] |
3678 | +297 If this option is specified, backup to STDOUT as specified |
3679 | +298 format. |
3680 | +299 |
3681 | +300 --tmpdir=DIRECTORY |
3682 | +301 When --remote-host or --stream specified, transaction log |
3683 | +302 should be stored temporary. The default value is same to |
3684 | +303 mysqld setting. |
3685 | +304 |
3686 | +305 --use-memory=MB |
3687 | +306 This option is passed to the ibbackup child process. |
3688 | +307 It tells ibbackup that it can use MB megabytes of |
3689 | +308 memory in restoration. |
3690 | +309 Try 'ibbackup --help' for more details on this option. |
3691 | +310 |
3692 | +311 --throttle=IOS |
3693 | +312 This option is passed to the xtrabackup child process. |
3694 | +313 It limits count of IO operations (pairs of read&write) per |
3695 | +314 second to IOS values (for '--backup') |
3696 | +315 |
3697 | +316 --sleep=MS |
3698 | +317 This option is passed to the ibbackup child process. |
3699 | +318 It instructs the ibbackup program to sleep |
3700 | +319 MS milliseconds after each 1 MB of copied data. |
3701 | +320 You can use this parameter to tune the additional |
3702 | +321 disk i/o load the ibbackup program causes on the computer. |
3703 | +322 Try 'ibbackup --help' for more details on this option. |
3704 | +323 ** it is not supported by xtrabackup ** |
3705 | +324 |
3706 | +325 --compress[=LEVEL] |
3707 | +326 This option is passed to the ibbackup child process. |
3708 | +327 It instructs ibbackup to compress the backup copies of |
3709 | +328 InnoDB data files. Compression level can be |
3710 | +329 specified as an optional argument. Compression level is |
3711 | +330 an integer between 0 and 9: 1 gives fastest compression, |
3712 | +331 9 gives best compression, and 0 means no compression. |
3713 | +332 If compression level is not given, the default level 1 is used. |
3714 | +333 Try 'ibbackup --help' for more details on this option. |
3715 | +334 ** it is not supported by xtrabackup yet ** |
3716 | +335 |
3717 | +336 --include=REGEXP |
3718 | +337 This option is passed to the ibbackup child process. |
3719 | +338 It tells ibbackup to backup only those per-table data |
3720 | +339 files which match the given regular expression. For |
3721 | +340 each table with a per-table data file a string of the |
3722 | +341 form db_name.table_name is checked against the regular |
3723 | +342 expression. If the regular expression matches the |
3724 | +343 complete string db_name.table_name, the table is |
3725 | +344 included in the backup. The regular expression should |
3726 | +345 be of the POSIX 1003.2 "extended" form. |
3727 | +346 Try 'ibbackup --help' for more details on this option. |
3728 | +347 |
3729 | +348 --databases=LIST |
3730 | +349 This option is used to specify the list of databases that |
3731 | +350 innobackup should backup. The list is of the form |
3732 | +351 "db_name[.table_name] db_name1[.table_name1] ...". |
3733 | +352 If this option is not specified all databases containing |
3734 | +353 MyISAM and InnoDB tables will be backed up. |
3735 | +354 Please make sure that --databases contains all of the |
3736 | +355 innodb databases & tables so that all of the innodb .frm |
3737 | +356 files are also backed up. In case the list is very long, |
3738 | +357 this can be specified in a file and the full path of the |
3739 | +358 file can be specified instead of the list. |
3740 | +359 |
3741 | +360 --uncompress |
3742 | +361 This option is passed to the ibbackup child process. |
3743 | +362 It tells ibbackup to uncompress compressed InnoDB data files. |
3744 | +363 Try 'ibbackup --help' for more details on this option. |
3745 | +364 ** it is not supported by xtrabackup yet ** |
3746 | +365 |
3747 | +366 --export |
3748 | +367 This option is passed to the xtrabackup child process. |
3749 | +368 It is effective with '--prepare' option. |
3750 | +369 It tells xtrabackup to output "clean" .ibd files and .exp |
3751 | +370 files for 'ALTER TABLE ... IMPORT TABLESPACE' command |
3752 | +371 at innodb_expand_import option enabled XtraDB. |
3753 | +372 |
3754 | +373 --user=NAME |
3755 | +374 This option is passed to the mysql child process. |
3756 | +375 It defines the user for database login if not current user. |
3757 | +376 Try 'mysql --help' for more details on this option. |
3758 | +377 |
3759 | +378 --password=WORD |
3760 | +379 This option is passed to the mysql child process. |
3761 | +380 It defines the password to use when connecting to database. |
3762 | +381 Try 'mysql --help' for more details on this option. |
3763 | +382 |
3764 | +383 --host=HOST |
3765 | +384 This option is passed to the mysql child process. |
3766 | +385 It defines the hostname or IP address to use when connecting |
3767 | +386 to the database server. |
3768 | +387 Try 'mysql --help' for more details on this option. |
3769 | +388 |
3770 | +389 --port=PORT |
3771 | +390 This option is passed to the mysql child process. |
3772 | +391 It defines the port to use when connecting to local database |
3773 | +392 server with TCP/IP. |
3774 | +393 Try 'mysql --help' for more details on this option. |
3775 | +394 |
3776 | +395 --slave-info |
3777 | +396 This option is useful when backing up a replication |
3778 | +397 slave server. It prints the binary log position and |
3779 | +398 name of the binary log file of the master server. |
3780 | +399 It also writes this information to the 'ibbackup_slave_info' |
3781 | +400 file as a 'CHANGE MASTER' command. A new slave for this |
3782 | +401 master can be set up by starting a slave server on this |
3783 | +402 backup and issuing a 'CHANGE MASTER' command with the binary |
3784 | +403 log position saved in the 'ibbackup_slave_info' file. |
3785 | +404 |
3786 | +405 --socket=SOCKET |
3787 | +406 This option is passed to the mysql child process. |
3788 | +407 It defines the socket to use when connecting to local database |
3789 | +408 server with UNIX domain socket. |
3790 | +409 Try 'mysql --help' for more details on this option. |
3791 | +410 |
3792 | +411 --no-timestamp |
3793 | +412 This option prevents the creation of a new backup |
3794 | +413 directory (named by the current date and time) under |
3795 | +414 the backup root directory. Instead, the backup is put |
3796 | +415 in the directory given on the command-line (in the |
3797 | +416 place of BACKUP-ROOT-DIR argument). The directory must not |
3798 | +417 exist, because innobackup creates it while making a backup. |
3799 | +418 |
3800 | +419 --ibbackup=IBBACKUP-BINARY |
3801 | +420 Use this option to specify which ibbackup (InnoDB Hot |
3802 | +421 Backup) binary should be used. IBBACKUP-BINARY |
3803 | +422 should be the command used to run ibbackup. This can |
3804 | +423 be useful if ibbackup is not in your search path or |
3805 | +424 working directory. If this option is not given, |
3806 | +425 ibbackup is run with command "ibbackup". |
3807 | +426 --no-lock |
3808 | +427 Use this option to disable table lock with |
3809 | +428 FLUSH TABLES WITH READ LOCK. use it only while ALL your |
3810 | +429 tables are InnoDB and you DO NOT CARE about binary log |
3811 | +430 position of backup. |
3812 | +431 |
3813 | +432 --scpopt=SCP-OPTIONS |
3814 | +433 Use this option to specify the command line options |
3815 | +434 to pass to scp. The default options are '-Cp -c arcfour'. |
3816 | +435 |
3817 | +436 EOF |
3818 | +437 |
3819 | +438 *** 0 0 print STDOUT "Option values after processing arguments:\n\n"; |
3820 | +439 *** 0 0 0 print STDOUT |
3821 | +440 *** 0 0 join( "\n", sort map { " --$_=" . $o->{$_} || '' } keys %$o ), |
3822 | +441 "\n\n"; |
3823 | +442 |
3824 | +443 *** 0 0 return; |
3825 | +444 } |
3826 | +445 |
3827 | +446 # |
3828 | +447 # return current local time as string in form "070816 12:23:15" |
3829 | +448 # |
3830 | +449 sub current_time { |
3831 | +450 18 18 1294 return strftime( "%y%m%d %H:%M:%S", localtime() ); |
3832 | +451 } |
3833 | +452 |
3834 | +453 # |
3835 | +454 # Die subroutine kills all child processes and exits this process. |
3836 | +455 # This subroutine takes the same argument as the built-in die function. |
3837 | +456 # Parameters: |
3838 | +457 # message string which is printed to stdout |
3839 | +458 # |
3840 | +459 sub Die { |
3841 | +460 *** 0 0 0 my $message = shift; |
3842 | +461 *** 0 0 my $extra_info = ''; |
3843 | +462 |
3844 | +463 # kill all child processes of this process |
3845 | +464 *** 0 0 kill_child_processes(); |
3846 | +465 |
3847 | +466 *** 0 0 0 if ($current_mysql_request) { |
3848 | +467 *** 0 0 $extra_info = " while waiting for reply to MySQL request:" |
3849 | +468 . " '$current_mysql_request'"; |
3850 | +469 } |
3851 | +470 |
3852 | +471 *** 0 0 IBEXDEBUG && _d('Die', $prefix, $message, $extra_info); |
3853 | +472 *** 0 0 die "$prefix Error: $message$extra_info"; |
3854 | +473 } |
3855 | +474 |
3856 | +475 # |
3857 | +476 # backup subroutine makes a backup of InnoDB and MyISAM tables, indexes and |
3858 | +477 # .frm files. It connects to the database server and runs ibbackup as a child |
3859 | +478 # process. |
3860 | +479 # |
3861 | +480 sub backup { |
3862 | +481 2 2 24 my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' ); |
3863 | +482 |
3864 | +483 # check that we can connect to the database. This done by |
3865 | +484 # connecting, issuing a query, and closing the connection. |
3866 | +485 2 36 mysql_open(); |
3867 | +486 2 30 mysql_check(); |
3868 | +487 2 452 mysql_close(); |
3869 | +488 |
3870 | +489 # start ibbackup as a child process |
3871 | +490 2 20 start_ibbackup(); |
3872 | +491 |
3873 | +492 # wait for ibbackup to suspend itself |
3874 | +493 *** 1 50 33 39 if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
3875 | +494 1 44 wait_for_ibbackup_suspend(); |
3876 | +495 } |
3877 | +496 |
3878 | +497 *** 1 50 18 if ( !$o->{'incremental'} ) { |
3879 | +498 1 15 mysql_open(); |
3880 | +499 1 660 mysql_check(); |
3881 | +500 *** 1 50 51 mysql_lockall() unless $o->{'no-lock'}; |
3882 | +501 1 11 backup_files(); |
3883 | +502 } |
3884 | +503 |
3885 | +504 # resume ibbackup and wait till it has finished |
3886 | +505 1 23 resume_ibbackup(); |
3887 | +506 |
3888 | +507 *** 1 50 44 if ( !$o->{'incremental'} ) { |
3889 | +508 *** 1 50 18 mysql_unlockall() unless $o->{'no-lock'}; |
3890 | +509 1 20 mysql_close(); |
3891 | +510 } |
3892 | +511 |
3893 | +512 *** 1 50 32 if ( $o->{'remote-host'} ) { |
3894 | + *** 50 |
3895 | +513 *** 0 0 system_call("scp $o->{'scp-opt'} '$tmp_logfile' " |
3896 | +514 . "'$o->{'remote-host'}:$backup_dir/xtrabackup_logfile'"); |
3897 | +515 |
3898 | +516 *** 0 0 system_call("rm -f '$tmp_logfile'"); |
3899 | +517 |
3900 | +518 *** 0 0 system_call("scp $o->{'scp-opt'} '$orig_datadir/xtrabackup_checkpoints' " |
3901 | +519 . "'$o->{'remote-host'}:$backup_dir/xtrabackup_checkpoints'"); |
3902 | +520 |
3903 | +521 *** 0 0 system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'"); |
3904 | +522 } |
3905 | +523 elsif ( $o->{'stream'} eq 'tar' ) { |
3906 | +524 *** 0 0 system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_logfile"); |
3907 | +525 |
3908 | +526 *** 0 0 system_call("rm -f '$tmp_logfile'"); |
3909 | +527 |
3910 | +528 *** 0 0 system_call("cd $orig_datadir; tar chf - xtrabackup_checkpoints"); |
3911 | +529 |
3912 | +530 *** 0 0 system_call("rm -f '$orig_datadir/xtrabackup_checkpoints'"); |
3913 | +531 } |
3914 | +532 |
3915 | +533 *** 1 50 35 print STDERR "\n$prefix Backup created in directory " |
3916 | +534 . ($o->{'remote-host'} ? "$o->{'remote-host'}:" : '') |
3917 | +535 . "'$backup_dir'\n"; |
3918 | +536 |
3919 | +537 *** 1 50 9 if ($mysql_binlog_position) { |
3920 | +538 1 40 print STDERR "$prefix MySQL binlog position: $mysql_binlog_position\n"; |
3921 | +539 } |
3922 | +540 *** 1 50 33 13 if ( $mysql_slave_position && $o->{'slave-info'} ) { |
3923 | +541 *** 0 0 print STDERR "$prefix MySQL slave binlog position: " |
3924 | +542 . "$mysql_slave_position\n"; |
3925 | +543 } |
3926 | +544 |
3927 | +545 1 16 return; |
3928 | +546 } |
3929 | +547 |
3930 | +548 # |
3931 | +549 # are_equal_innodb_data_file_paths subroutine checks if the given |
3932 | +550 # InnoDB data file option values are equal. |
3933 | +551 # Parameters: |
3934 | +552 # str1 InnoDB data file path option value |
3935 | +553 # str2 InnoDB data file path option value |
3936 | +554 # Return value: |
3937 | +555 # 1 if values are equal |
3938 | +556 # 0 otherwise |
3939 | +557 # |
3940 | +558 sub are_equal_innodb_data_file_paths { |
3941 | +559 *** 0 0 0 my $str1 = shift; |
3942 | +560 *** 0 0 my $str2 = shift; |
3943 | +561 *** 0 0 my @array1 = split( /;/, $str1 ); |
3944 | +562 *** 0 0 my @array2 = split( /;/, $str2 ); |
3945 | +563 |
3946 | +564 *** 0 0 0 if ( $#array1 != $#array2 ) { return 0; } |
3947 | + *** 0 0 |
3948 | +565 |
3949 | +566 for ( my $i = 0; $i <= $#array1; $i++ ) { |
3950 | +567 *** 0 0 my @def1 = split( /:/, $array1[$i] ); |
3951 | +568 *** 0 0 my @def2 = split( /:/, $array2[$i] ); |
3952 | +569 |
3953 | +570 *** 0 0 0 if ( $#def1 != $#def2 ) { return 0; } |
3954 | + *** 0 0 |
3955 | +571 |
3956 | +572 for ( my $j = 0; $j <= $#def1; $j++ ) { |
3957 | +573 *** 0 0 0 if ( $def1[$j] ne $def2[$j] ) { return 0; } |
3958 | + *** 0 0 |
3959 | +574 *** 0 0 } |
3960 | +575 *** 0 0 } |
3961 | +576 *** 0 0 return 1; |
3962 | +577 } |
3963 | +578 |
3964 | +579 # |
3965 | +580 # is_in_array subroutine checks if the given string is in the array. |
3966 | +581 # Parameters: |
3967 | +582 # str a string |
3968 | +583 # array_ref a reference to an array of strings |
3969 | +584 # Return value: |
3970 | +585 # 1 if string is in the array |
3971 | +586 # 0 otherwise |
3972 | +587 # |
3973 | +588 sub is_in_array { |
3974 | +589 *** 0 0 0 my $str = shift; |
3975 | +590 *** 0 0 my $array_ref = shift; |
3976 | +591 |
3977 | +592 *** 0 0 0 if ( grep { $str eq $_ } @{$array_ref} ) { |
3978 | + *** 0 0 |
3979 | + *** 0 0 |
3980 | +593 *** 0 0 return 1; |
3981 | +594 } |
3982 | +595 *** 0 0 return 0; |
3983 | +596 } |
3984 | +597 |
3985 | +598 # |
3986 | +599 # copy_back subroutine copies data and index files from backup directory |
3987 | +600 # back to their original locations. |
3988 | +601 # |
3989 | +602 sub copy_back { |
3990 | +603 *** 0 0 0 my $orig_datadir = get_option( \%config, 'mysqld', 'datadir' ); |
3991 | +604 *** 0 0 my $orig_ibdata_dir = |
3992 | +605 get_option( \%config, 'mysqld', 'innodb_data_home_dir' ); |
3993 | +606 *** 0 0 my $orig_innodb_data_file_path = |
3994 | +607 get_option( \%config, 'mysqld', 'innodb_data_file_path' ); |
3995 | +608 *** 0 0 my $orig_iblog_dir = |
3996 | +609 get_option( \%config, 'mysqld', 'innodb_log_group_home_dir' ); |
3997 | +610 *** 0 0 my $excluded_files = |
3998 | +611 '^(\.\.?|backup-my\.cnf|xtrabackup_logfile|mysql-std(err|out)|.*\.ibz)$'; |
3999 | +612 *** 0 0 my @ibdata_files; |
4000 | +613 *** 0 0 my $iblog_files = '^ib_logfile.*$'; |
4001 | +614 *** 0 0 my $compressed_data_file = '.*\.ibz$'; |
4002 | +615 *** 0 0 my $file; |
4003 | +616 *** 0 0 my $backup_innodb_data_file_path; |
4004 | +617 |
4005 | +618 # check that original data directory exists |
4006 | +619 *** 0 0 0 if ( !-d $orig_datadir ) { |
4007 | +620 *** 0 0 Die "Original data directory '$orig_datadir' does not exist!"; |
4008 | +621 } |
4009 | +622 |
4010 | +623 # check that original InnoDB data directory exists |
4011 | +624 *** 0 0 0 if ( !-d $orig_ibdata_dir ) { |
4012 | +625 *** 0 0 Die "Original InnoDB data directory '$orig_ibdata_dir' does not exist!"; |
4013 | +626 } |
4014 | +627 |
4015 | +628 # check that original InnoDB log directory exists |
4016 | +629 *** 0 0 0 if ( !-d $orig_iblog_dir ) { |
4017 | +630 *** 0 0 Die "Original InnoDB log directory '$orig_iblog_dir' does not exist!"; |
4018 | +631 } |
4019 | +632 |
4020 | +633 # check that the original options file and the backup options file have |
4021 | +634 # the same value for "innodb_data_file_path" option |
4022 | +635 #$backup_innodb_data_file_path = |
4023 | +636 # get_option(\%backup_config, 'mysqld', 'innodb_data_file_path'); |
4024 | +637 #if (!are_equal_innodb_data_file_paths($orig_innodb_data_file_path, |
4025 | +638 # $backup_innodb_data_file_path) |
4026 | +639 #) { |
4027 | +640 # Die "The value of 'innodb_data_file_path' option in the original " |
4028 | +641 # . "my.cnf file '$config_file' is different from the value " |
4029 | +642 # . "in the backup my.cnf file '$backup_config_file'.\n(original: " |
4030 | +643 # . "'$orig_innodb_data_file_path')\n" |
4031 | +644 # . "(backup: '$backup_innodb_data_file_path')"; |
4032 | +645 #} |
4033 | +646 |
4034 | +647 # make a list of all ibdata files in the backup directory and all |
4035 | +648 # directories in the backup directory under which there are ibdata files |
4036 | +649 *** 0 0 foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) { |
4037 | +650 *** 0 0 my $path = ( split( /:/, $a ) )[0]; |
4038 | +651 *** 0 0 my $filename = ( split( /\/+/, $path ) )[-1]; |
4039 | +652 |
4040 | +653 # check that the backup data file exists |
4041 | +654 *** 0 0 0 if ( !-e "$backup_dir/$filename" ) { |
4042 | +655 *** 0 0 0 if ( -e "$backup_dir/${filename}.ibz" ) { |
4043 | +656 *** 0 0 Die |
4044 | +657 "Backup data file '$backup_dir/$filename' does not exist, but " |
4045 | +658 . "its compressed copy '${path}.ibz' exists. Check that you " |
4046 | +659 . "have run '$innobackup_script --apply-log --uncompress " |
4047 | +660 . "...' before attempting '$innobackup_script --copy-back ...' !"; |
4048 | +661 } |
4049 | +662 else { |
4050 | +663 *** 0 0 Die "Backup data file '$backup_dir/$filename' does not exist."; |
4051 | +664 } |
4052 | +665 } |
4053 | +666 |
4054 | +667 *** 0 0 0 if ( !is_in_array( $filename, \@ibdata_files ) ) { |
4055 | +668 *** 0 0 push( @ibdata_files, $filename ); |
4056 | +669 } |
4057 | +670 } |
4058 | +671 |
4059 | +672 # copy files from backup dir to their original locations |
4060 | +673 |
4061 | +674 # copy files to original data directory |
4062 | +675 *** 0 0 0 opendir( DIR, $backup_dir ) |
4063 | +676 || Die "Can't open directory '$backup_dir': $OS_ERROR\n"; |
4064 | +677 *** 0 0 print STDERR "$prefix Starting to copy MyISAM tables, indexes,\n"; |
4065 | +678 *** 0 0 print STDERR |
4066 | +679 "$prefix .MRG, .TRG, .TRN, .ARM, .ARZ, .opt, and .frm files\n"; |
4067 | +680 *** 0 0 print STDERR "$prefix in '$backup_dir'\n"; |
4068 | +681 *** 0 0 print STDERR "$prefix back to original data directory '$orig_datadir'\n"; |
4069 | +682 *** 0 0 while ( defined( $file = readdir(DIR) ) ) { |
4070 | +683 *** 0 0 0 if ( $file =~ /$excluded_files/ ) { next; } |
4071 | + *** 0 0 |
4072 | +684 *** 0 0 0 if ( is_in_array( $file, \@ibdata_files ) ) { next; } |
4073 | + *** 0 0 |
4074 | +685 *** 0 0 0 if ( $file =~ /$iblog_files/ ) { next; } |
4075 | + *** 0 0 |
4076 | +686 *** 0 0 0 if ( -d "$backup_dir/$file" ) { |
4077 | +687 *** 0 0 my $subdir = "$backup_dir/$file"; |
4078 | +688 *** 0 0 my $file2; |
4079 | +689 |
4080 | +690 *** 0 0 print STDERR "$prefix Copying directory '$subdir'\n"; |
4081 | +691 *** 0 0 0 if ( !-x "\"" . escape_path("$orig_datadir/$file") . "\"" ) { |
4082 | +692 *** 0 0 system_call("mkdir \"" . escape_path("$orig_datadir/$file") . "\""); |
4083 | +693 } |
4084 | +694 *** 0 0 0 opendir( SUBDIR, "$subdir" ) |
4085 | +695 || Die "Can't open directory '$subdir': $OS_ERROR\n"; |
4086 | +696 *** 0 0 while ( defined( $file2 = readdir(SUBDIR) ) ) { |
4087 | +697 *** 0 0 0 if ( -d "$subdir/$file2" ) { next; } |
4088 | + *** 0 0 |
4089 | +698 *** 0 0 0 if ( $file2 =~ /$compressed_data_file/ ) { next; } |
4090 | + *** 0 0 |
4091 | +699 *** 0 0 $src_name = escape_path("$subdir/$file2"); |
4092 | +700 *** 0 0 $dst_name = escape_path("$orig_datadir/$file"); |
4093 | +701 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
4094 | +702 } |
4095 | +703 *** 0 0 closedir(SUBDIR); |
4096 | +704 } |
4097 | +705 else { |
4098 | +706 *** 0 0 print STDERR "$prefix Copying file " . "'$backup_dir/$file'\n"; |
4099 | +707 *** 0 0 $src_name = escape_path("$backup_dir/$file"); |
4100 | +708 *** 0 0 $dst_name = escape_path("$orig_datadir"); |
4101 | +709 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
4102 | +710 } |
4103 | +711 } |
4104 | +712 *** 0 0 closedir(DIR); |
4105 | +713 |
4106 | +714 # copy InnoDB data files to original InnoDB data directory |
4107 | +715 *** 0 0 print STDERR "\n$prefix Starting to copy InnoDB tables and indexes\n"; |
4108 | +716 *** 0 0 print STDERR "$prefix in '$backup_dir'\n"; |
4109 | +717 *** 0 0 print STDERR |
4110 | +718 "$prefix back to original InnoDB data directory '$orig_ibdata_dir'\n"; |
4111 | +719 *** 0 0 foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) { |
4112 | +720 |
4113 | +721 # get the relative pathname of a data file |
4114 | +722 *** 0 0 my $path = ( split( /:/, $a ) )[0]; |
4115 | +723 *** 0 0 my $filename = ( split( /\/+/, $path ) )[-1]; |
4116 | +724 *** 0 0 print STDERR "$prefix Copying file '$backup_dir/$filename'\n"; |
4117 | +725 *** 0 0 $src_name = escape_path("$backup_dir/$filename"); |
4118 | +726 *** 0 0 $dst_name = escape_path("$orig_ibdata_dir/$path"); |
4119 | +727 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
4120 | +728 } |
4121 | +729 |
4122 | +730 # copy InnoDB log files to original InnoDB log directory |
4123 | +731 *** 0 0 0 opendir( DIR, $backup_dir ) |
4124 | +732 || Die "Can't open directory '$backup_dir': $OS_ERROR\n"; |
4125 | +733 *** 0 0 print STDERR "\n$prefix Starting to copy InnoDB log files\n"; |
4126 | +734 *** 0 0 print STDERR "$prefix in '$backup_dir'\n"; |
4127 | +735 *** 0 0 print STDERR |
4128 | +736 "$prefix back to original InnoDB log directory '$orig_iblog_dir'\n"; |
4129 | +737 *** 0 0 while ( defined( $file = readdir(DIR) ) ) { |
4130 | +738 *** 0 0 0 0 if ( $file =~ /$iblog_files/ && -f "$backup_dir/$file" ) { |
4131 | +739 *** 0 0 print STDERR "$prefix Copying file '$backup_dir/$file'\n"; |
4132 | +740 *** 0 0 $src_name = escape_path("$backup_dir/$file"); |
4133 | +741 *** 0 0 $dst_name = escape_path("$orig_iblog_dir"); |
4134 | +742 *** 0 0 system_call("$CP_CMD \"$src_name\" \"$dst_name\""); |
4135 | +743 } |
4136 | +744 } |
4137 | +745 *** 0 0 closedir(DIR); |
4138 | +746 |
4139 | +747 *** 0 0 print STDERR "$prefix Finished copying back files.\n\n"; |
4140 | +748 |
4141 | +749 *** 0 0 return; |
4142 | +750 } |
4143 | +751 |
4144 | +752 # |
4145 | +753 # apply_log subroutine prepares a backup for starting database server |
4146 | +754 # on the backup. It applies InnoDB log files to the InnoDB data files. |
4147 | +755 # |
4148 | +756 sub apply_log { |
4149 | +757 *** 0 0 0 my $rcode; |
4150 | +758 *** 0 0 my $cmdline = ''; |
4151 | +759 *** 0 0 my $options = ''; |
4152 | +760 |
4153 | +761 *** 0 0 0 if ( $o->{'defaults-file'} ) { |
4154 | +762 *** 0 0 $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" "; |
4155 | +763 } |
4156 | +764 |
4157 | +765 *** 0 0 $options = $options . "--prepare --target-dir=$backup_dir"; |
4158 | +766 |
4159 | +767 *** 0 0 0 if ( $o->{'uncompress'} ) { |
4160 | +768 *** 0 0 $options = $options . ' --uncompress'; |
4161 | +769 } |
4162 | +770 *** 0 0 0 if ( $o->{'export'} ) { |
4163 | +771 *** 0 0 $options = $options . ' --export'; |
4164 | +772 } |
4165 | +773 *** 0 0 0 if ( $o->{'use-memory'} ) { |
4166 | +774 *** 0 0 $options = $options . " --use-memory=$o->{'use-memory'}"; |
4167 | +775 } |
4168 | +776 |
4169 | +777 # run ibbackup as a child process |
4170 | +778 *** 0 0 $cmdline = "$o->{'ibbackup'} $options"; |
4171 | +779 *** 0 0 $now = current_time(); |
4172 | +780 *** 0 0 print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n\n"; |
4173 | +781 *** 0 0 $rcode = system_call($cmdline, 1); |
4174 | +782 *** 0 0 0 Die "\n$prefix ibbackup failed" if $rcode; |
4175 | +783 |
4176 | +784 *** 0 0 $now = current_time(); |
4177 | +785 *** 0 0 print STDERR |
4178 | +786 "\n$now $prefix Restarting xtrabackup with command: $cmdline\nfor creating ib_logfile*\n\n"; |
4179 | +787 *** 0 0 $rcode = system_call($cmdline, 1); |
4180 | +788 *** 0 0 0 Die "\n$prefix xtrabackup (2nd execution) failed" if $rcode; |
4181 | +789 |
4182 | +790 *** 0 0 return; |
4183 | +791 } |
4184 | +792 |
4185 | +793 # |
4186 | +794 # wait_for_ibbackup_suspend subroutine waits until ibbackup has suspended |
4187 | +795 # itself. |
4188 | +796 # |
4189 | +797 sub wait_for_ibbackup_suspend { |
4190 | +798 1 1 165 print STDERR |
4191 | +799 "$prefix Waiting for ibbackup (pid=$ibbackup_pid) to suspend\n"; |
4192 | +800 1 94 print STDERR "$prefix Suspend file '$suspend_file'\n\n"; |
4193 | +801 1 17 for ( ;; ) { |
4194 | +802 1 2000223 sleep 2; |
4195 | +803 *** 1 50 59 last if -e $suspend_file; |
4196 | +804 |
4197 | +805 # check that ibbackup child process is still alive |
4198 | +806 *** 0 0 0 if ( $ibbackup_pid == waitpid( $ibbackup_pid, &WNOHANG ) ) { |
4199 | +807 *** 0 0 $ibbackup_pid = ''; |
4200 | +808 *** 0 0 Die "ibbackup child process has died"; |
4201 | +809 } |
4202 | +810 } |
4203 | +811 1 16 $now = current_time(); |
4204 | +812 1 200 print STDERR "\n$now $prefix Continuing after ibbackup has suspended\n"; |
4205 | +813 } |
4206 | +814 |
4207 | +815 # |
4208 | +816 # resume_ibbackup subroutine signals ibbackup to complete its execution |
4209 | +817 # by deleting the 'ibbackup_suspended' file. |
4210 | +818 # |
4211 | +819 sub resume_ibbackup { |
4212 | +820 1 1 222 print STDERR "$prefix Resuming ibbackup\n\n"; |
4213 | +821 *** 1 33 69 unlink $suspend_file || Die "Failed to delete '$suspend_file': $OS_ERROR"; |
4214 | +822 |
4215 | +823 # wait for ibbackup to finish |
4216 | +824 1 302772 waitpid( $ibbackup_pid, 0 ); |
4217 | +825 1 22 $ibbackup_pid = ''; |
4218 | +826 } |
4219 | +827 |
4220 | +828 # |
4221 | +829 # start_ibbackup subroutine spawns a child process running ibbackup |
4222 | +830 # program for backing up InnoDB tables and indexes. |
4223 | +831 # |
4224 | +832 sub start_ibbackup { |
4225 | +833 2 2 12 my $options = ''; |
4226 | +834 2 14 my $cmdline = ''; |
4227 | +835 2 36 my $pid = undef; |
4228 | +836 |
4229 | +837 *** 2 50 30 if ( $o->{'defaults-file'} ) { |
4230 | +838 2 22 $options = $options . " --defaults-file=\"$o->{'defaults-file'}\" "; |
4231 | +839 } |
4232 | +840 |
4233 | +841 2 12 $options = $options . "--backup --suspend-at-end"; |
4234 | +842 |
4235 | +843 *** 2 50 33 62 if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
4236 | +844 2 26 $options = $options . " --target-dir=$backup_dir"; |
4237 | +845 } |
4238 | +846 else { |
4239 | +847 |
4240 | +848 #(datadir) for 'xtrabackup_suspended' and 'xtrabackup_checkpoints' |
4241 | +849 *** 0 0 $options = $options . " --log-stream --target-dir=./"; |
4242 | +850 } |
4243 | +851 |
4244 | +852 # prepare command line for running ibbackup |
4245 | +853 *** 2 50 20 if ( $o->{'throttle'} ) { |
4246 | +854 *** 0 0 $options = $options . " --throttle=$o->{'throttle'}"; |
4247 | +855 } |
4248 | +856 *** 2 50 18 if ( $o->{'sleep'} ) { |
4249 | +857 *** 0 0 $options = $options . " --sleep=$o->{'sleep'}"; |
4250 | +858 } |
4251 | +859 *** 2 50 16 if ( $o->{'compress'} ) { |
4252 | +860 *** 0 0 $options = $options . " --compress=$o->{'compress'}"; |
4253 | +861 } |
4254 | +862 *** 2 50 26 if ( $o->{'use-memory'} ) { |
4255 | +863 *** 0 0 $options = $options . " --use-memory=$o->{'use-memory'}"; |
4256 | +864 } |
4257 | +865 *** 2 50 16 if ( $o->{'include'} ) { |
4258 | +866 *** 0 0 $options = $options . " --tables='$o->{'include'}'"; |
4259 | +867 } |
4260 | +868 *** 2 50 16 if ( $o->{'incremental'} ) { |
4261 | +869 *** 0 0 $options = $options . " --incremental-basedir='$incremental_basedir'"; |
4262 | +870 } |
4263 | +871 2 24 $cmdline = "$o->{'ibbackup'} $options"; |
4264 | +872 |
4265 | +873 # run ibbackup as a child process |
4266 | +874 2 10 $now = current_time(); |
4267 | +875 2 64 print STDERR "\n$now $prefix Starting ibbackup with command: $cmdline\n"; |
4268 | +876 *** 2 50 23639 if ( defined($pid = fork) ) { |
4269 | +877 2 100 64 if ($pid) { |
4270 | +878 # parent process |
4271 | +879 1 34 $ibbackup_pid = $pid; |
4272 | +880 |
4273 | +881 # direct copy to remote |
4274 | +882 *** 1 50 33 96 if ( $o->{'remote-host'} || $o->{'stream'} ) { |
4275 | +883 *** 0 0 my $orig_datadir |
4276 | +884 = get_option( \%config, 'mysqld', 'datadir' ); |
4277 | +885 *** 0 0 my $orig_ibdata_dir |
4278 | +886 = get_option( \%config, 'mysqld', 'innodb_data_home_dir' ); |
4279 | +887 *** 0 0 my $orig_innodb_data_file_path |
4280 | +888 = get_option( \%config, 'mysqld', 'innodb_data_file_path' ); |
4281 | +889 *** 0 0 my $subdir; |
4282 | +890 *** 0 0 my @list; |
4283 | +891 |
4284 | +892 *** 0 0 0 if ( $o->{'remote-host'} ) { |
4285 | +893 *** 0 0 0 if ( system_call("ssh $o->{'remote-host'} test -e " |
4286 | +894 . "$backup_dir/ib_logfile0", 1) == 0 ) { |
4287 | +895 *** 0 0 print STDERR "$prefix Remove " |
4288 | +896 . "$o->{'remote-host'}:$backup_dir/ib_logfile*\n"; |
4289 | +897 *** 0 0 system_call( "ssh $o->{'remote-host'} " |
4290 | +898 . "rm $backup_dir/ib_logfile\*" |
4291 | +899 ) |
4292 | +900 } |
4293 | +901 } |
4294 | +902 |
4295 | +903 *** 0 0 wait_for_ibbackup_suspend(); |
4296 | +904 |
4297 | +905 #InnoDB data files from original InnoDB data directory |
4298 | +906 *** 0 0 print STDERR |
4299 | +907 "\n$prefix Starting to backup InnoDB tables and indexes\n"; |
4300 | +908 *** 0 0 0 if ( $o->{'remote-host'} ) { |
4301 | +909 *** 0 0 print STDERR "$prefix to '$backup_dir'\n"; |
4302 | +910 } |
4303 | +911 *** 0 0 print STDERR |
4304 | +912 "$prefix from original InnoDB data directory '$orig_ibdata_dir'\n"; |
4305 | +913 *** 0 0 foreach my $a ( split( /;/, $orig_innodb_data_file_path ) ) { |
4306 | +914 *** 0 0 my $path = ( split( /:/, $a ) )[0]; |
4307 | +915 *** 0 0 $path =~ s/([\$\\\" ])/\\$1/g; |
4308 | +916 *** 0 0 0 if ( $o->{'remote-host'} ) { |
4309 | + *** 0 |
4310 | +917 *** 0 0 print STDERR |
4311 | +918 "$prefix Backing up file '$orig_ibdata_dir/$path'\n"; |
4312 | +919 *** 0 0 system_call("scp $o->{'scp-opt'} '$orig_ibdata_dir/$path' " |
4313 | +920 . "'$o->{'remote-host'}:$backup_dir/$path'"); |
4314 | +921 } |
4315 | +922 elsif ( $o->{'stream'} eq 'tar' ) { |
4316 | +923 *** 0 0 my $ret = 0; |
4317 | +924 *** 0 0 print STDERR "$prefix Backing up as tar stream '$path'\n"; |
4318 | +925 *** 0 0 0 if ( !$o->{'tar4ibd'} ) { |
4319 | +926 *** 0 0 $ret = system_call( |
4320 | +927 "cd $orig_ibdata_dir; tar chf - -b 32 $path", 1); |
4321 | +928 } |
4322 | +929 else { |
4323 | +930 *** 0 0 $ret = system_call( |
4324 | +931 "cd $orig_ibdata_dir; tar4ibd -c $path", 1); |
4325 | +932 } |
4326 | +933 *** 0 0 0 if ( $ret == 1 ) { |
4327 | + *** 0 |
4328 | +934 *** 0 0 print STDERR "$prefix If you use GNU tar, this warning " |
4329 | +935 . "can be ignored.\n"; |
4330 | +936 } |
4331 | +937 elsif ( $ret != 0 ) { |
4332 | +938 *** 0 0 print STDERR "$prefix tar returned with exit code $ret.\n"; |
4333 | +939 *** 0 0 Die "Failed to stream '$path': $OS_ERROR"; |
4334 | +940 } |
4335 | +941 } |
4336 | +942 } |
4337 | +943 |
4338 | +944 #copy *.ibd files |
4339 | +945 *** 0 0 0 opendir( DIR, $orig_datadir ) |
4340 | +946 || Die "Can't open directory '$orig_datadir': $OS_ERROR\n"; |
4341 | +947 *** 0 0 while ( defined( $subdir = readdir(DIR) ) ) { |
4342 | +948 *** 0 0 my $print_each_file = 0; |
4343 | +949 *** 0 0 my $file_c; |
4344 | +950 *** 0 0 my $file; |
4345 | +951 *** 0 0 0 0 if ( $subdir eq '.' || $subdir eq '..' ) { next; } |
4346 | + *** 0 0 |
4347 | +952 *** 0 0 0 next unless -d "$orig_datadir/$subdir"; |
4348 | +953 *** 0 0 0 next unless check_if_required($subdir); |
4349 | +954 |
4350 | +955 *** 0 0 @list = glob( "$orig_datadir/$subdir/" . '*.ibd' ); |
4351 | +956 |
4352 | +957 *** 0 0 $file_c = @list; |
4353 | +958 *** 0 0 0 if ( $file_c <= $backup_file_print_limit ) { |
4354 | +959 *** 0 0 $print_each_file = 1; |
4355 | +960 } |
4356 | +961 else { |
4357 | +962 *** 0 0 print STDERR "$prefix Backing up files " |
4358 | +963 . "'$orig_datadir/$subdir/*.ibd' ($file_c files)\n"; |
4359 | +964 } |
4360 | +965 |
4361 | +966 *** 0 0 foreach $file ( @list ) { |
4362 | +967 *** 0 0 0 if ( $o->{'include'} ) { |
4363 | +968 *** 0 0 my $table_name; |
4364 | +969 *** 0 0 $table_name = substr($file, rindex( $file, '/' )); |
4365 | +970 *** 0 0 $table_name = substr( $table_name, 1, |
4366 | +971 rindex( $table_name, '.' ) - 1); |
4367 | +972 *** 0 0 $table_name = $subdir . "." . $table_name; |
4368 | +973 *** 0 0 0 if ( $table_name !~ m/$o->{'include'}/o ) { |
4369 | +974 *** 0 0 print STDERR "'$file' is skipped.\n"; |
4370 | +975 *** 0 0 next; |
4371 | +976 } |
4372 | +977 } |
4373 | +978 |
4374 | +979 *** 0 0 0 if ($print_each_file) { |
4375 | +980 *** 0 0 print STDERR "$prefix Backing up file '$file'\n"; |
4376 | +981 } |
4377 | +982 |
4378 | +983 *** 0 0 0 if ( $o->{'remote-host'} ) { |
4379 | + *** 0 |
4380 | +984 *** 0 0 system_call( |
4381 | +985 "ssh $o->{'remote-host'} mkdir '$backup_dir/$subdir'", |
4382 | +986 1 # don't die on failure |
4383 | +987 ); |
4384 | +988 *** 0 0 system_call("scp $o->{'scp-opt'} '$file' " |
4385 | +989 . "'$o->{'remote-host'}:$backup_dir/$subdir/'"); |
4386 | +990 } |
4387 | +991 elsif ( $o->{'stream'} eq 'tar' ) { |
4388 | +992 *** 0 0 my $ret = 0; |
4389 | +993 *** 0 0 my $file_name = substr($file, rindex( $file, '/' ) + 1); |
4390 | +994 *** 0 0 $file_name =~ s/([\$\\\" ])/\\$1/g; |
4391 | +995 |
4392 | +996 *** 0 0 0 if ( !$o->{'tar4ibd'} ) { |
4393 | +997 *** 0 0 $ret = system_call("cd $orig_datadir; tar chf - -b 32 " |
4394 | +998 . "$subdir/$file_name", 1); |
4395 | +999 } |
4396 | +1000 else { |
4397 | +1001 *** 0 0 $ret = system_call("cd $orig_datadir; tar4ibd -c " |
4398 | +1002 . "$subdir/$file_name", 1); |
4399 | +1003 } |
4400 | +1004 |
4401 | +1005 *** 0 0 0 if ( $ret == 1 ) { |
4402 | + *** 0 |
4403 | +1006 *** 0 0 print STDERR "$prefix If you use GNU tar, this warning " |
4404 | +1007 . "can be ignored.\n"; |
4405 | +1008 } |
4406 | +1009 elsif ( $ret != 0 ) { |
4407 | +1010 *** 0 0 print STDERR "$prefix tar returned with exit code " |
4408 | +1011 . "$ret.\n"; |
4409 | +1012 *** 0 0 Die "Failed to stream '$subdir/$file_name': $OS_ERROR"; |
4410 | +1013 } |
4411 | +1014 } |
4412 | +1015 } # each file |
4413 | +1016 } # each subdir |
4414 | +1017 *** 0 0 closedir(DIR); |
4415 | +1018 } |
4416 | +1019 } # parent process |
4417 | +1020 else { # child process |
4418 | +1021 *** 1 50 33 90 if ( $o->{'remote-host'} || $o->{'stream'} ) { |
4419 | +1022 *** 0 0 0 open STDOUT, "> $tmp_logfile" |
4420 | +1023 or Die "Failed to open file '$tmp_logfile': $OS_ERROR"; |
4421 | +1024 } |
4422 | +1025 1 4 IBEXDEBUG && _d($cmdline); |
4423 | +1026 *** 1 0 46 exec($cmdline) or Die "Failed to exec ibbackup: $OS_ERROR"; |
4424 | +1027 } |
4425 | +1028 } |
4426 | +1029 else { |
4427 | +1030 *** 0 0 Die "failed to fork ibbackup child process: $OS_ERROR"; |
4428 | +1031 } |
4429 | +1032 } |
4430 | +1033 |
4431 | +1034 # |
4432 | +1035 # get_mysql_options subroutine returns the options to mysql client program |
4433 | +1036 # as a string. The options are determined from the options given by the |
4434 | +1037 # user to innobackup. |
4435 | +1038 # |
4436 | +1039 sub get_mysql_options { |
4437 | +1040 10 10 56 my @options; |
4438 | +1041 |
4439 | +1042 # --defaults-file needs to be first. |
4440 | +1043 10 100 82 if ( $o->{'defaults-file'} ) { |
4441 | +1044 8 106 push @options, "--defaults-file=$o->{'defaults-file'}"; |
4442 | +1045 } |
4443 | +1046 |
4444 | +1047 10 100 72 if ( $o->{'host'} ) { |
4445 | +1048 1 5 push @options, "--host=$o->{'host'}"; |
4446 | +1049 } |
4447 | +1050 10 100 65 if ( $o->{'port'} ) { |
4448 | +1051 2 11 push @options, "--port=$o->{'port'}"; |
4449 | +1052 } |
4450 | +1053 10 100 62 if ( $o->{'user'} ) { |
4451 | +1054 2 10 push @options, "--user=$o->{'user'}"; |
4452 | +1055 } |
4453 | +1056 10 100 61 if ( $o->{'password'} ) { |
4454 | +1057 4 19 push @options, "--password=$o->{'password'}"; |
4455 | +1058 } |
4456 | +1059 *** 10 50 65 if ( $o->{'socket'} ) { |
4457 | +1060 *** 0 0 push @options, "--socket=$o->{'socket'}"; |
4458 | +1061 } |
4459 | +1062 |
4460 | +1063 10 46 push @options, '--unbuffered --'; |
4461 | +1064 |
4462 | +1065 10 73 my $options = join( ' ', @options ); |
4463 | +1066 10 31 IBEXDEBUG && _d('mysql options:', $options); |
4464 | +1067 10 83 return $options; |
4465 | +1068 } |
4466 | +1069 |
4467 | +1070 # |
4468 | +1071 # mysql_open subroutine starts mysql as a child process with |
4469 | +1072 # a pipe connection. |
4470 | +1073 # |
4471 | +1074 sub mysql_open { |
4472 | +1075 3 3 31 my $options = get_mysql_options(); |
4473 | +1076 |
4474 | +1077 # run mysql as a child process with a pipe connection |
4475 | +1078 3 30 $now = current_time(); |
4476 | +1079 3 398 print STDERR "$now $prefix Starting mysql with options: $options\n"; |
4477 | +1080 *** 3 50 14260 $mysql_pid = open(*MYSQL_WRITER, |
4478 | +1081 "| mysql $options >$mysql_stdout 2>$mysql_stderr ") |
4479 | +1082 or Die "Failed to spawn mysql child process: $OS_ERROR"; |
4480 | +1083 3 254 MYSQL_WRITER->autoflush(1); |
4481 | +1084 3 113 $now = current_time(); |
4482 | +1085 3 5000 print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n"; |
4483 | +1086 |
4484 | +1087 3 70 return; |
4485 | +1088 } |
4486 | +1089 |
4487 | +1090 # |
4488 | +1091 # mysql_check subroutine checks that the connection to mysql child process |
4489 | +1092 # is ok. |
4490 | +1093 # |
4491 | +1094 sub mysql_check { |
4492 | +1095 9 9 85 my $mysql_pid_copy = $mysql_pid; |
4493 | +1096 |
4494 | +1097 # send a dummy query to mysql child process |
4495 | +1098 9 69 $hello_id++; |
4496 | +1099 9 73 my $hello_message = "innobackup hello $hello_id"; |
4497 | +1100 9 36 IBEXDEBUG && _d('mysql check:', $hello_message); |
4498 | +1101 *** 9 50 5816 print MYSQL_WRITER "select '$hello_message';\n" |
4499 | +1102 or Die "Connection to mysql child process failed: $OS_ERROR"; |
4500 | +1103 |
4501 | +1104 # wait for reply |
4502 | +1105 9 82 eval { |
4503 | +1106 *** 9 0 432 local $SIG{ALRM} = sub { die "alarm clock restart" }; |
4504 | + *** 0 0 |
4505 | +1107 9 73 my $stdout = ''; |
4506 | +1108 9 55 my $stderr = ''; |
4507 | +1109 9 125 alarm $mysql_response_timeout; |
4508 | +1110 9 93 while ( index( $stdout, $hello_message ) < 0 ) { |
4509 | +1111 9 18001882 sleep 2; |
4510 | +1112 *** 9 50 33 718 if ( $mysql_pid && $mysql_pid == waitpid( $mysql_pid, &WNOHANG ) ) { |
4511 | +1113 *** 0 0 my $reason = `cat $mysql_stderr`; |
4512 | +1114 *** 0 0 $mysql_pid = ''; |
4513 | +1115 *** 0 0 die "mysql child process has died: $reason"; |
4514 | +1116 } |
4515 | +1117 9 53378 $stdout = `cat $mysql_stdout`; |
4516 | +1118 9 47330 $stderr = `cat $mysql_stderr`; |
4517 | +1119 *** 9 50 360 if ($stderr) { |
4518 | +1120 # mysql has reported an error, do exit |
4519 | +1121 *** 0 0 die "mysql error: $stderr"; |
4520 | +1122 } |
4521 | +1123 } |
4522 | +1124 9 667 alarm 0; |
4523 | +1125 }; |
4524 | +1126 *** 9 50 176 if ( $EVAL_ERROR =~ /alarm clock restart/ ) { |
4525 | + *** 50 |
4526 | +1127 *** 0 0 Die "Connection to mysql child process (pid=$mysql_pid_copy) timedout." |
4527 | +1128 . " (Time limit of $mysql_response_timeout seconds exceeded." |
4528 | +1129 . " You may adjust time limit by editing the value of parameter" |
4529 | +1130 . " \"\$mysql_response_timeout\" in this script.)"; |
4530 | +1131 } |
4531 | +1132 elsif ( $EVAL_ERROR ) { |
4532 | +1133 *** 0 0 Die $EVAL_ERROR; |
4533 | +1134 } |
4534 | +1135 |
4535 | +1136 9 81 $mysql_last_access_time = time(); |
4536 | +1137 |
4537 | +1138 9 171 return; |
4538 | +1139 } |
4539 | +1140 |
4540 | +1141 # |
4541 | +1142 # mysql_keep_alive subroutine tries to keep connection to the mysqld database |
4542 | +1143 # server alive by sending a dummy query when the connection has been idle |
4543 | +1144 # for the specified time. |
4544 | +1145 # |
4545 | +1146 sub mysql_keep_alive { |
4546 | +1147 *** 5 50 5 63 if ( ( time() - $mysql_last_access_time ) > $mysql_keep_alive_timeout ) { |
4547 | +1148 # too long idle, send a dummy query |
4548 | +1149 *** 0 0 mysql_check(); |
4549 | +1150 } |
4550 | +1151 5 23 return; |
4551 | +1152 } |
4552 | +1153 |
4553 | +1154 # |
4554 | +1155 # mysql_send subroutine send a request string to mysql child process. |
4555 | +1156 # This subroutine appends a newline character to the request and checks |
4556 | +1157 # that mysqld receives the query. |
4557 | +1158 # Parameters: |
4558 | +1159 # request request string |
4559 | +1160 # |
4560 | +1161 sub mysql_send { |
4561 | +1162 6 6 59 my $request = shift; |
4562 | +1163 6 41 $current_mysql_request = $request; |
4563 | +1164 6 652 print MYSQL_WRITER "$request\n"; |
4564 | +1165 6 53 mysql_check(); |
4565 | +1166 6 99 $current_mysql_request = ''; |
4566 | +1167 6 57 return; |
4567 | +1168 } |
4568 | +1169 |
4569 | +1170 # |
4570 | +1171 # mysql_close subroutine terminates mysql child process gracefully. |
4571 | +1172 # |
4572 | +1173 sub mysql_close { |
4573 | +1174 3 3 95 print MYSQL_WRITER "quit\n"; |
4574 | +1175 3 50 $now = current_time(); |
4575 | +1176 3 94 print STDERR "$now $prefix Connection to database server closed\n"; |
4576 | +1177 3 34 $mysql_pid = ''; |
4577 | +1178 3 20 return; |
4578 | +1179 } |
4579 | +1180 |
4580 | +1181 # |
4581 | +1182 # write_binlog_info subroutine retrieves MySQL binlog position and |
4582 | +1183 # saves it in a file. It also prints it to stdout. |
4583 | +1184 # |
4584 | +1185 sub write_binlog_info { |
4585 | +1186 1 1 9 my @lines; |
4586 | +1187 1 22 my @info_lines = (); |
4587 | +1188 1 12 my $position = ''; |
4588 | +1189 1 7 my $filename = ''; |
4589 | +1190 |
4590 | +1191 # get binlog position |
4591 | +1192 1 13 mysql_send "SHOW MASTER STATUS;"; |
4592 | +1193 |
4593 | +1194 # get "show master status" output lines (2) from mysql output |
4594 | +1195 1 69 file_to_array( $mysql_stdout, \@lines ); |
4595 | +1196 1 10 foreach my $line (@lines) { |
4596 | +1197 14 100 112 if ( $line =~ m/innobackup hello/ ) { |
4597 | +1198 |
4598 | +1199 # this is a hello message, ignore it |
4599 | +1200 } |
4600 | +1201 else { |
4601 | +1202 |
4602 | +1203 # this is output line from "show master status" |
4603 | +1204 2 13 push( @info_lines, $line ); |
4604 | +1205 } |
4605 | +1206 } |
4606 | +1207 |
4607 | +1208 # write binlog info file |
4608 | +1209 *** 1 50 11 if ( !defined $info_lines[1] ) { |
4609 | +1210 *** 0 0 $info_lines[1] = ""; |
4610 | +1211 } |
4611 | +1212 *** 1 50 10 if ( !$o->{'remote-host'} ) { |
4612 | +1213 *** 1 50 92 open( FILE, ">$binlog_info" ) |
4613 | +1214 || Die "Failed to open file '$binlog_info': $OS_ERROR"; |
4614 | +1215 } |
4615 | +1216 else { |
4616 | +1217 *** 0 0 0 open( FILE, "| ssh $o->{'remote-host'} 'cat > $binlog_info'" ) |
4617 | +1218 || Die |
4618 | +1219 "Failed to open file '$o->{'remote-host'}:$binlog_info': $OS_ERROR"; |
4619 | +1220 } |
4620 | +1221 1 21 print FILE "$info_lines[1]\n"; |
4621 | +1222 1 43 close(FILE); |
4622 | +1223 |
4623 | +1224 *** 1 50 14 if ( $o->{'stream'} eq 'tar' ) { |
4624 | +1225 *** 0 0 system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_binlog_info"); |
4625 | +1226 *** 0 0 0 unlink $binlog_info or Die "Failed to delete '$binlog_info': $OS_ERROR"; |
4626 | +1227 } |
4627 | +1228 |
4628 | +1229 # get the name of the last binlog file and position in it |
4629 | +1230 1 37 ( $filename, $position ) = $info_lines[1] =~ /^\s*([^\s]+)\s+(.*)$/; |
4630 | +1231 |
4631 | +1232 *** 1 50 33 33 if ( defined $filename && defined $position ) { |
4632 | +1233 1 11 $mysql_binlog_position = "filename '$filename', position $position"; |
4633 | +1234 } |
4634 | +1235 else { |
4635 | +1236 *** 0 0 $mysql_binlog_position = "filename '', position "; |
4636 | +1237 } |
4637 | +1238 |
4638 | +1239 1 8 return; |
4639 | +1240 } |
4640 | +1241 |
4641 | +1242 # |
4642 | +1243 # write_slave_info subroutine retrieves MySQL binlog position of the |
4643 | +1244 # master server in a replication setup and saves it in a file. It |
4644 | +1245 # also saves it in $msql_slave_position variable. |
4645 | +1246 # |
4646 | +1247 sub write_slave_info { |
4647 | +1248 *** 0 0 0 my @lines; |
4648 | +1249 *** 0 0 my @info_lines; |
4649 | +1250 *** 0 0 my $position = ''; |
4650 | +1251 *** 0 0 my $filename = ''; |
4651 | +1252 *** 0 0 my $master = ''; |
4652 | +1253 |
4653 | +1254 # get slave status. Use single quotes here, otherwise |
4654 | +1255 # \G is evaluated as a control character. |
4655 | +1256 *** 0 0 mysql_send 'SHOW SLAVE STATUS\G;'; |
4656 | +1257 |
4657 | +1258 # get output of the "show slave status" command from mysql output |
4658 | +1259 # and extract binlog position of the master server |
4659 | +1260 *** 0 0 file_to_array( $mysql_stdout, \@lines ); |
4660 | +1261 *** 0 0 for (@lines) { |
4661 | +1262 *** 0 0 0 $master = $1 if /Master_Host:\s*(\S*)\s*$/; |
4662 | +1263 *** 0 0 0 $filename = $1 if /Master_Log_File:\s*(\S*)\s*$/; |
4663 | +1264 *** 0 0 0 $position = $1 if /Master_Log_Pos:\s*(\S*)\s*$/; |
4664 | +1265 } |
4665 | +1266 |
4666 | +1267 # print slave status to a file |
4667 | +1268 *** 0 0 0 if ( !$o->{'remote-host'} ) { |
4668 | +1269 *** 0 0 0 open( FILE, ">$slave_info" ) |
4669 | +1270 || Die "Failed to open file '$slave_info': $OS_ERROR"; |
4670 | +1271 } |
4671 | +1272 else { |
4672 | +1273 *** 0 0 0 open( FILE, "| ssh $o->{'remote-host'} 'cat > $slave_info'" ) |
4673 | +1274 || Die |
4674 | +1275 "Failed to open file '$o->{'remote-host'}:$slave_info': $OS_ERROR"; |
4675 | +1276 } |
4676 | +1277 *** 0 0 print FILE |
4677 | +1278 "CHANGE MASTER TO MASTER_LOG_FILE='$filename', MASTER_LOG_POS=$position\n"; |
4678 | +1279 *** 0 0 close(FILE); |
4679 | +1280 |
4680 | +1281 *** 0 0 0 if ( $o->{'stream'} eq 'tar' ) { |
4681 | +1282 *** 0 0 system_call("cd $o->{'tmpdir'}; tar chf - xtrabackup_slave_info"); |
4682 | +1283 *** 0 0 0 unlink $slave_info or Die "Failed to delete '$slave_info': $OS_ERROR"; |
4683 | +1284 } |
4684 | +1285 |
4685 | +1286 $mysql_slave_position = |
4686 | +1287 *** 0 0 "master host '$master', filename '$filename', position $position"; |
4687 | +1288 |
4688 | +1289 *** 0 0 return; |
4689 | +1290 } |
4690 | +1291 |
4691 | +1292 # |
4692 | +1293 # mysql_lockall subroutine puts a read lock on all tables in all databases. |
4693 | +1294 # |
4694 | +1295 sub mysql_lockall { |
4695 | +1296 1 1 16 $now = current_time(); |
4696 | +1297 1 46 print STDERR "$now $prefix Starting to lock all tables...\n"; |
4697 | +1298 |
4698 | +1299 1 21 mysql_send "USE mysql;"; |
4699 | +1300 |
4700 | +1301 # mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;"; |
4701 | +1302 # if (compare_versions($mysql_server_version, '4.1.0') == -1) { |
4702 | +1303 # # MySQL server version is 4.0 or older, ENGINE keyword not supported |
4703 | +1304 # mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) TYPE=INNODB;"; |
4704 | +1305 # } else { |
4705 | +1306 # # MySQL server version is 4.1 or newer, use ENGINE keyword |
4706 | +1307 # mysql_send "CREATE TABLE ibbackup_binlog_marker(a INT) ENGINE=INNODB;"; |
4707 | +1308 # } |
4708 | +1309 1 19 mysql_send "SET AUTOCOMMIT=0;"; |
4709 | +1310 |
4710 | +1311 # mysql_send "INSERT INTO ibbackup_binlog_marker VALUES (1);"; |
4711 | +1312 *** 1 50 33 42 if ( compare_versions( $mysql_server_version, '4.0.22' ) == 0 |
4712 | +1313 || compare_versions( $mysql_server_version, '4.1.7' ) == 0 ) |
4713 | +1314 { |
4714 | +1315 |
4715 | +1316 # MySQL server version is 4.0.22 or 4.1.7 |
4716 | +1317 *** 0 0 mysql_send "COMMIT;"; |
4717 | +1318 *** 0 0 mysql_send "FLUSH TABLES WITH READ LOCK;"; |
4718 | +1319 } |
4719 | +1320 else { |
4720 | +1321 |
4721 | +1322 # MySQL server version is other than 4.0.22 or 4.1.7 |
4722 | +1323 1 13 mysql_send "FLUSH TABLES WITH READ LOCK;"; |
4723 | +1324 1 34 mysql_send "COMMIT;"; |
4724 | +1325 } |
4725 | +1326 1 47 write_binlog_info; |
4726 | +1327 *** 1 50 10 write_slave_info if $o->{'slave-info'}; |
4727 | +1328 |
4728 | +1329 1 19 $now = current_time(); |
4729 | +1330 1 39 print STDERR "$now $prefix All tables locked and flushed to disk\n"; |
4730 | +1331 |
4731 | +1332 1 5 return; |
4732 | +1333 } |
4733 | +1334 |
4734 | +1335 # |
4735 | +1336 # mysql_unlockall subroutine releases read locks on all tables in all |
4736 | +1337 # databases. |
4737 | +1338 # |
4738 | +1339 sub mysql_unlockall { |
4739 | +1340 1 1 17 mysql_send "UNLOCK TABLES;"; |
4740 | +1341 |
4741 | +1342 # mysql_send "DROP TABLE IF EXISTS ibbackup_binlog_marker;"; |
4742 | +1343 |
4743 | +1344 1 35 $now = current_time(); |
4744 | +1345 1 58 print STDERR "$now $prefix All tables unlocked\n"; |
4745 | +1346 } |
4746 | +1347 |
4747 | +1348 # |
4748 | +1349 # catch_sigpipe subroutine is a signal handler for SIGPIPE. |
4749 | +1350 # |
4750 | +1351 sub catch_sigpipe { |
4751 | +1352 *** 0 0 0 my $rcode; |
4752 | +1353 |
4753 | +1354 *** 0 0 0 0 if ( |
4754 | + *** 0 |
4755 | +1355 $mysql_pid |
4756 | +1356 && ( -1 == ( $rcode = waitpid( $mysql_pid, &WNOHANG ) ) |
4757 | +1357 || $rcode == $mysql_pid ) |
4758 | +1358 ) |
4759 | +1359 { |
4760 | +1360 *** 0 0 my $reason = `cat $mysql_stderr`; |
4761 | +1361 *** 0 0 print STDERR "Pipe to mysql child process broken: $reason at"; |
4762 | +1362 *** 0 0 system("date +'%H:%M:%S'"); |
4763 | +1363 *** 0 0 exit(1); |
4764 | +1364 } |
4765 | +1365 else { |
4766 | +1366 *** 0 0 Die "Broken pipe"; |
4767 | +1367 } |
4768 | +1368 } |
4769 | +1369 |
4770 | +1370 # |
4771 | +1371 # kill_child_processes subroutine kills all child processes of this process. |
4772 | +1372 # |
4773 | +1373 sub kill_child_processes { |
4774 | +1374 *** 0 0 0 0 if ($ibbackup_pid) { |
4775 | +1375 *** 0 0 kill( $kill_signal, $ibbackup_pid ); |
4776 | +1376 *** 0 0 $ibbackup_pid = ''; |
4777 | +1377 } |
4778 | +1378 |
4779 | +1379 *** 0 0 0 if ($mysql_pid) { |
4780 | +1380 *** 0 0 kill( $kill_signal, $mysql_pid ); |
4781 | +1381 *** 0 0 $mysql_pid = ''; |
4782 | +1382 } |
4783 | +1383 } |
4784 | +1384 |
4785 | +1385 # |
4786 | +1386 # require_external subroutine checks that an external program is runnable |
4787 | +1387 # via the shell. This is tested by calling the program with the |
4788 | +1388 # given arguments. It is checked that the program returns 0 and does |
4789 | +1389 # not print anything to stderr. If this check fails, this subroutine exits. |
4790 | +1390 # Parameters: |
4791 | +1391 # program pathname of the program file |
4792 | +1392 # args arguments to the program |
4793 | +1393 # pattern a string containing a regular expression for finding |
4794 | +1394 # the program version. |
4795 | +1395 # this pattern should contain a subpattern enclosed |
4796 | +1396 # in parentheses which is matched with the version. |
4797 | +1397 # version_ref a reference to a variable where the program version |
4798 | +1398 # string is returned. Example "2.0-beta2". |
4799 | +1399 # |
4800 | +1400 sub require_external { |
4801 | +1401 2 2 12 my $program = shift; |
4802 | +1402 2 10 my $args = shift; |
4803 | +1403 2 10 my $pattern = shift; |
4804 | +1404 2 10 my $version_ref = shift; |
4805 | +1405 2 8 my @lines; |
4806 | +1406 2 144 my $tmp_stdout = tmpnam(); |
4807 | +1407 2 46 my $tmp_stderr = tmpnam(); |
4808 | +1408 2 8 my $rcode; |
4809 | +1409 2 6 my $error; |
4810 | +1410 |
4811 | +1411 2 50 $rcode = system_call("$program $args >$tmp_stdout 2>$tmp_stderr", 1); |
4812 | +1412 *** 2 50 32 if ($rcode) { |
4813 | +1413 *** 0 0 $error = $OS_ERROR; |
4814 | +1414 } |
4815 | +1415 2 14814 my $stderr = `cat $tmp_stderr`; |
4816 | +1416 2 216 unlink $tmp_stderr; |
4817 | +1417 *** 2 50 68 if ( $stderr ne '' ) { |
4818 | + *** 50 |
4819 | +1418 |
4820 | +1419 # failure |
4821 | +1420 *** 0 0 unlink $tmp_stdout; |
4822 | +1421 *** 0 0 Die "Couldn't run $program: $stderr"; |
4823 | +1422 } |
4824 | +1423 elsif ($rcode) { |
4825 | +1424 |
4826 | +1425 # failure |
4827 | +1426 *** 0 0 unlink $tmp_stdout; |
4828 | +1427 *** 0 0 Die "Couldn't run $program: $error"; |
4829 | +1428 } |
4830 | +1429 |
4831 | +1430 # success |
4832 | +1431 2 11754 my $stdout = `cat $tmp_stdout`; |
4833 | +1432 2 182 unlink $tmp_stdout; |
4834 | +1433 2 108 @lines = split( /\n|;/, $stdout ); |
4835 | +1434 2 414 print STDERR "$prefix Using $lines[0]\n"; |
4836 | +1435 |
4837 | +1436 # get version string from the first output line of the program |
4838 | +1437 2 46 ${$version_ref} = ''; |
4839 | + 2 26 |
4840 | +1438 *** 2 50 154 if ( $lines[0] =~ /$pattern/ ) { |
4841 | +1439 2 10 ${$version_ref} = $1; |
4842 | + 2 66 |
4843 | +1440 } |
4844 | +1441 } |
4845 | +1442 |
4846 | +1443 # compare_versions subroutine compares two GNU-style version strings. |
4847 | +1444 # A GNU-style version string consists of three decimal numbers delimitted |
4848 | +1445 # by dots, and optionally followed by extra attributes. |
4849 | +1446 # Examples: "4.0.1", "4.1.1-alpha-debug". |
4850 | +1447 # Parameters: |
4851 | +1448 # str1 a GNU-style version string |
4852 | +1449 # str2 a GNU-style version string |
4853 | +1450 # Return value: |
4854 | +1451 # -1 if str1 < str2 |
4855 | +1452 # 0 if str1 == str2 |
4856 | +1453 # 1 is str1 > str2 |
4857 | +1454 sub compare_versions { |
4858 | +1455 2 2 22 my $str1 = shift; |
4859 | +1456 2 16 my $str2 = shift; |
4860 | +1457 2 11 my $extra1 = ''; |
4861 | +1458 2 10 my $extra2 = ''; |
4862 | +1459 2 11 my @array1 = (); |
4863 | +1460 2 8 my @array2 = (); |
4864 | +1461 2 8 my $i; |
4865 | +1462 |
4866 | +1463 # remove possible extra attributes |
4867 | +1464 2 42 ( $str1, $extra1 ) = $str1 =~ /^([0-9.]*)(.*)/; |
4868 | +1465 2 24 ( $str2, $extra2 ) = $str2 =~ /^([0-9.]*)(.*)/; |
4869 | +1466 |
4870 | +1467 # split "dotted" decimal number string into an array |
4871 | +1468 2 21 @array1 = split( '\.', $str1 ); |
4872 | +1469 2 17 @array2 = split( '\.', $str2 ); |
4873 | +1470 |
4874 | +1471 # compare in lexicographic order |
4875 | +1472 for ( $i = 0; $i <= $#array1 && $i <= $#array2; $i++ ) { |
4876 | +1473 *** 0 0 0 if ( $array1[$i] < $array2[$i] ) { |
4877 | + *** 0 |
4878 | +1474 *** 0 0 return -1; |
4879 | +1475 } |
4880 | +1476 elsif ( $array1[$i] > $array2[$i] ) { |
4881 | +1477 *** 0 0 return 1; |
4882 | +1478 } |
4883 | +1479 *** 2 33 18 } |
4884 | +1480 *** 2 50 31 if ( $#array1 < $#array2 ) { |
4885 | + *** 0 |
4886 | +1481 2 47 return -1; |
4887 | +1482 } |
4888 | +1483 elsif ( $#array1 > $#array2 ) { |
4889 | +1484 *** 0 0 return 1; |
4890 | +1485 } |
4891 | +1486 else { |
4892 | +1487 *** 0 0 return 0; |
4893 | +1488 } |
4894 | +1489 } |
4895 | +1490 |
4896 | +1491 # |
4897 | +1492 # init subroutine initializes global variables and performs some checks on the |
4898 | +1493 # system we are running on. |
4899 | +1494 # |
4900 | +1495 sub init { |
4901 | +1496 2 2 12 my $mysql_version = ''; |
4902 | +1497 2 8 my $ibbackup_version = ''; |
4903 | +1498 2 10 my $run = ''; |
4904 | +1499 |
4905 | +1500 # print some instructions to the user |
4906 | +1501 *** 2 50 33 92 if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) { |
4907 | + *** 0 |
4908 | +1502 2 12 $run = 'backup'; |
4909 | +1503 } |
4910 | +1504 elsif ( $o->{'copy-back'} ) { |
4911 | +1505 *** 0 0 $run = 'copy-back'; |
4912 | +1506 } |
4913 | +1507 else { |
4914 | +1508 *** 0 0 $run = 'apply-log'; |
4915 | +1509 } |
4916 | +1510 2 36 print STDERR |
4917 | +1511 "IMPORTANT: Please check that the $run run completes successfully.\n"; |
4918 | +1512 2 32 print STDERR |
4919 | +1513 " At the end of a successful $run run $innobackup_script\n"; |
4920 | +1514 2 20 print STDERR " prints \"completed OK!\".\n\n"; |
4921 | +1515 |
4922 | +1516 # check that MySQL client program and InnoDB Hot Backup program |
4923 | +1517 # are runnable via shell |
4924 | +1518 *** 2 50 20 if ( !$o->{'copy-back'} ) { |
4925 | +1519 |
4926 | +1520 # we are making a backup or applying log to backup |
4927 | +1521 *** 2 50 14 if ( !$o->{'apply-log'} ) { |
4928 | +1522 |
4929 | +1523 # we are making a backup, we need mysql server |
4930 | +1524 2 10 my $output = ''; |
4931 | +1525 2 12 my @lines = (); |
4932 | +1526 |
4933 | +1527 # check that we have mysql client program |
4934 | +1528 2 34 require_external( 'mysql', '--version', 'Ver ([^,]+)', |
4935 | +1529 \$mysql_version ); |
4936 | +1530 |
4937 | +1531 # get mysql server version |
4938 | +1532 2 42 my $options = get_mysql_options(); |
4939 | +1533 2 27220 @lines = split( '\n', `mysql $options -e "select \@\@version"` ); |
4940 | +1534 2 108 $mysql_server_version = $lines[1]; |
4941 | +1535 2 146 print STDERR |
4942 | +1536 "$prefix Using mysql server version $mysql_server_version\n"; |
4943 | +1537 } |
4944 | +1538 |
4945 | +1539 #require_external($o->{'ibbackup-binary'}, '--license', |
4946 | +1540 # 'version (\S+)', \$ibbackup_version); |
4947 | +1541 2 32 print STDERR "\n"; |
4948 | +1542 |
4949 | +1543 *** 2 50 33 58 if ( $o->{'include'} |
4950 | + *** 33 |
4951 | +1544 && $ibbackup_version |
4952 | +1545 && $ibbackup_version le "2.0" ) |
4953 | +1546 { |
4954 | +1547 |
4955 | +1548 # --include option was given, but ibbackup is too |
4956 | +1549 # old to support it |
4957 | +1550 *** 0 0 Die "--include option was given, but ibbackup is too old" |
4958 | +1551 . " to support it. You must upgrade to InnoDB Hot Backup" |
4959 | +1552 . " v2.0 in order to use --include option.\n"; |
4960 | +1553 } |
4961 | +1554 } |
4962 | +1555 |
4963 | +1556 # set signal handlers |
4964 | +1557 2 142 $SIG{PIPE} = \&catch_sigpipe; |
4965 | +1558 |
4966 | +1559 # read MySQL options file |
4967 | +1560 #read_config_file($config_file, \%config); |
4968 | +1561 2 80 read_config_file( \%config ); |
4969 | +1562 |
4970 | +1563 *** 2 50 26 if ( !$o->{'tmpdir'} ) { |
4971 | +1564 2 34 $o->{'tmpdir'} = get_option( \%config, 'mysqld', 'tmpdir' ); |
4972 | +1565 } |
4973 | +1566 |
4974 | +1567 # get innodb log home directory from options file |
4975 | +1568 #$innodb_log_group_home_dir = |
4976 | +1569 # get_option(\%config, 'mysqld', 'innodb_log_group_home_dir'); |
4977 | +1570 |
4978 | +1571 *** 2 50 33 54 if ( !$o->{'apply-log'} && !$o->{'copy-back'} ) { |
4979 | + *** 0 |
4980 | +1572 |
4981 | +1573 # we are making a backup, create a new backup directory |
4982 | +1574 *** 2 50 14 if ( !$o->{'remote-host'} ) { |
4983 | +1575 2 30 $backup_dir = File::Spec->rel2abs( make_backup_dir() ); |
4984 | +1576 } |
4985 | +1577 else { |
4986 | +1578 *** 0 0 $backup_dir = make_backup_dir(); |
4987 | +1579 } |
4988 | +1580 2 544 print STDERR "$prefix Created backup directory $backup_dir\n"; |
4989 | +1581 *** 2 50 33 66 if ( !$o->{'remote-host'} && !$o->{'stream'} ) { |
4990 | +1582 2 18 $backup_config_file = $backup_dir . '/backup-my.cnf'; |
4991 | +1583 2 16 $suspend_file = $backup_dir . '/xtrabackup_suspended'; |
4992 | +1584 2 30 $mysql_stdout = $backup_dir . '/mysql-stdout'; |
4993 | +1585 2 16 $mysql_stderr = $backup_dir . '/mysql-stderr'; |
4994 | +1586 2 8 $binlog_info = $backup_dir . '/xtrabackup_binlog_info'; |
4995 | +1587 2 12 $slave_info = $backup_dir . '/xtrabackup_slave_info'; |
4996 | +1588 } |
4997 | +1589 else { |
4998 | +1590 *** 0 0 $suspend_file = get_option( \%config, 'mysqld', 'datadir' ) |
4999 | +1591 . '/xtrabackup_suspended'; |
5000 | +1592 *** 0 0 $tmp_logfile = $o->{'tmpdir'} . '/xtrabackup_logfile'; |
I didn't want to make these kinds of extensive changes to innobackupex-1.5.1 so I made a new dir, innobackupex, in which innobackupex (the script) has been substantially altered. Changes include:
1) Debugging w/ IBEXDEBUG=1 (like MKDEBUG=1) to expose stuff that was hidden before.
2) All system() calls are now ran through system_call() so that every command can be seen when IBEXDEBUG=1.
3) Removed all $option_ vars, replaced with $o hashref.
4) Tweaked help/usage output a little, added to its end a list of the current option values after cmd line parsing.
5) Dogged it, which means I ran it through perltidy to format it according to the standards explained in the book Perl Best Practices.
6) innobackupex/t/ has the first incarnation of a test suite. It requires that the environment variable XTRABACKUP_TRUNK be set.
7) Made code a runable module so it can be tested and test coverage can be established. innobackupex/ test-coverage. txt shows what we have so far, about 50% coverage.
It doesn't seem like this script was designed from the start to be testable. Testing it will be difficult and require a good amount of work. xtrabackup is also very talkative which means that innobackupex's tests will be noisy; xtrabackup should have a --quiet option. Since innobackupex forks children and runs shell commands capturing STDOUT and STDERR is difficult to impossible.
All in all, my branch should be as reliable, if not more, than the original code.