Merge lp:~akopytov/percona-xtrabackup/bug408803-1.7 into lp:percona-xtrabackup/2.0
- bug408803-1.7
- Merge into 2.0
Proposed by
Alexey Kopytov
Status: | Merged | ||||
---|---|---|---|---|---|
Merged at revision: | 330 | ||||
Proposed branch: | lp:~akopytov/percona-xtrabackup/bug408803-1.7 | ||||
Merge into: | lp:percona-xtrabackup/2.0 | ||||
Diff against target: |
282 lines (+81/-60) 5 files modified
doc/source/innobackupex/how_innobackupex_works.rst (+1/-1) doc/source/innobackupex/privileges.rst (+1/-1) innobackupex (+61/-54) test/t/bug884737.sh (+14/-0) xtrabackup.c (+4/-4) |
||||
To merge this branch: | bzr merge lp:~akopytov/percona-xtrabackup/bug408803-1.7 | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Stewart Smith (community) | Approve | ||
Review via email: mp+82103@code.launchpad.net |
Commit message
Description of the change
To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) wrote : | # |
Revision history for this message
Stewart Smith (stewart) : | # |
review:
Approve
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'doc/source/innobackupex/how_innobackupex_works.rst' | |||
2 | --- doc/source/innobackupex/how_innobackupex_works.rst 2011-10-21 17:47:31 +0000 | |||
3 | +++ doc/source/innobackupex/how_innobackupex_works.rst 2011-11-14 07:14:52 +0000 | |||
4 | @@ -59,7 +59,7 @@ | |||
5 | 59 | 59 | ||
6 | 60 | Finally, the binary log position will be printed to ``STDERR`` and |innobackupex| will exit returning 0 if all went OK. | 60 | Finally, the binary log position will be printed to ``STDERR`` and |innobackupex| will exit returning 0 if all went OK. |
7 | 61 | 61 | ||
9 | 62 | Note that the ``STDERR`` of |innobackupex| is not written in any file. You will have to redirect it to a file, e.g., ``innobackupex OPTIONS > backupout.log``. | 62 | Note that the ``STDERR`` of |innobackupex| is not written in any file. You will have to redirect it to a file, e.g., ``innobackupex OPTIONS 2> backupout.log``. |
10 | 63 | 63 | ||
11 | 64 | .. _copy-back-ibk: | 64 | .. _copy-back-ibk: |
12 | 65 | 65 | ||
13 | 66 | 66 | ||
14 | === modified file 'doc/source/innobackupex/privileges.rst' | |||
15 | --- doc/source/innobackupex/privileges.rst 2011-07-28 05:29:04 +0000 | |||
16 | +++ doc/source/innobackupex/privileges.rst 2011-11-14 07:14:52 +0000 | |||
17 | @@ -8,7 +8,7 @@ | |||
18 | 8 | 8 | ||
19 | 9 | Permissions are those which permits a user to perform operations on the system, like reading, writing or executing on a certain directory or start/stop a system service. **They are set at a system level and only apply to system users**. | 9 | Permissions are those which permits a user to perform operations on the system, like reading, writing or executing on a certain directory or start/stop a system service. **They are set at a system level and only apply to system users**. |
20 | 10 | 10 | ||
22 | 11 | Whether |xtrabackup| or |innobackupex| is used, there are two actors involved: the user invoking the program - *a system user* - and the user performing action in the database server - *a database server*. Note that these are different users on different places, despite they may have the same username. | 11 | Whether |xtrabackup| or |innobackupex| is used, there are two actors involved: the user invoking the program - *a system user* - and the user performing action in the database server - *a database user*. Note that these are different users on different places, despite they may have the same username. |
23 | 12 | 12 | ||
24 | 13 | All the invocations of |innobackupex| and |xtrabackup| in this documentation assumes that the system user has the appropriate permissions and you are providing the relevant options for connecting the database server - besides the options for the action to be performed - and the database user has adequate privileges. | 13 | All the invocations of |innobackupex| and |xtrabackup| in this documentation assumes that the system user has the appropriate permissions and you are providing the relevant options for connecting the database server - besides the options for the action to be performed - and the database user has adequate privileges. |
25 | 14 | 14 | ||
26 | 15 | 15 | ||
27 | === modified file 'innobackupex' | |||
28 | --- innobackupex 2011-10-21 17:47:31 +0000 | |||
29 | +++ innobackupex 2011-11-14 07:14:52 +0000 | |||
30 | @@ -54,7 +54,7 @@ | |||
31 | 54 | 54 | ||
32 | 55 | # time in seconds after which a dummy query is sent to mysql server | 55 | # time in seconds after which a dummy query is sent to mysql server |
33 | 56 | # in order to keep the database connection alive | 56 | # in order to keep the database connection alive |
35 | 57 | my $mysql_keep_alive_timeout = 1800; | 57 | my $mysql_keep_alive = 5; |
36 | 58 | 58 | ||
37 | 59 | ###################################################################### | 59 | ###################################################################### |
38 | 60 | # end of modifiable parameters | 60 | # end of modifiable parameters |
39 | @@ -169,9 +169,6 @@ | |||
40 | 169 | # run on a slave server | 169 | # run on a slave server |
41 | 170 | my $mysql_slave_position = ''; | 170 | my $mysql_slave_position = ''; |
42 | 171 | 171 | ||
43 | 172 | # time of the most recent mysql_check call. (value returned by time() function) | ||
44 | 173 | my $mysql_last_access_time = 0; | ||
45 | 174 | |||
46 | 175 | # process id of ibbackup program (runs as a child process of this script) | 172 | # process id of ibbackup program (runs as a child process of this script) |
47 | 176 | my $ibbackup_pid = ''; | 173 | my $ibbackup_pid = ''; |
48 | 177 | 174 | ||
49 | @@ -366,7 +363,6 @@ | |||
50 | 366 | # check that we can connect to the database. This done by | 363 | # check that we can connect to the database. This done by |
51 | 367 | # connecting, issuing a query, and closing the connection. | 364 | # connecting, issuing a query, and closing the connection. |
52 | 368 | mysql_open(); | 365 | mysql_open(); |
53 | 369 | mysql_check(); | ||
54 | 370 | mysql_close(); | 366 | mysql_close(); |
55 | 371 | 367 | ||
56 | 372 | # start ibbackup as a child process | 368 | # start ibbackup as a child process |
57 | @@ -379,19 +375,15 @@ | |||
58 | 379 | 375 | ||
59 | 380 | # connect to database | 376 | # connect to database |
60 | 381 | mysql_open(); | 377 | mysql_open(); |
61 | 382 | mysql_check(); | ||
62 | 383 | 378 | ||
63 | 384 | if ($option_safe_slave_backup) { | 379 | if ($option_safe_slave_backup) { |
64 | 385 | mysql_check(); | ||
65 | 386 | wait_for_safe_slave(); | 380 | wait_for_safe_slave(); |
66 | 387 | } | 381 | } |
67 | 388 | 382 | ||
68 | 389 | # flush tables with read lock | 383 | # flush tables with read lock |
69 | 390 | mysql_check(); | ||
70 | 391 | mysql_lockall() if !$option_no_lock; | 384 | mysql_lockall() if !$option_no_lock; |
71 | 392 | 385 | ||
72 | 393 | if ($option_slave_info) { | 386 | if ($option_slave_info) { |
73 | 394 | mysql_check(); | ||
74 | 395 | write_slave_info(); | 387 | write_slave_info(); |
75 | 396 | } | 388 | } |
76 | 397 | 389 | ||
77 | @@ -406,7 +398,6 @@ | |||
78 | 406 | mysql_unlockall() if !$option_no_lock; | 398 | mysql_unlockall() if !$option_no_lock; |
79 | 407 | 399 | ||
80 | 408 | if ( $option_safe_slave_backup && $host_is_slave ) { | 400 | if ( $option_safe_slave_backup && $host_is_slave ) { |
81 | 409 | mysql_check(); | ||
82 | 410 | print STDERR "$prefix: Starting slave SQL thread\n"; | 401 | print STDERR "$prefix: Starting slave SQL thread\n"; |
83 | 411 | mysql_send('START SLAVE SQL_THREAD;'); | 402 | mysql_send('START SLAVE SQL_THREAD;'); |
84 | 412 | } | 403 | } |
85 | @@ -991,35 +982,15 @@ | |||
86 | 991 | 982 | ||
87 | 992 | 983 | ||
88 | 993 | # | 984 | # |
113 | 994 | # mysql_open subroutine starts mysql as a child process with | 985 | # Check that the server is responding to queries |
114 | 995 | # a pipe connection. | 986 | # |
115 | 996 | # | 987 | sub mysql_ping { |
92 | 997 | sub mysql_open { | ||
93 | 998 | my $options = get_mysql_options(); | ||
94 | 999 | # run mysql as a child process with a pipe connection | ||
95 | 1000 | $now = current_time(); | ||
96 | 1001 | (my $prt_options = $options) =~ s/--password=[^ ]+ /--password=xxxxxxxx /g; | ||
97 | 1002 | print STDERR "$now $prefix Starting mysql with options: $prt_options\n"; | ||
98 | 1003 | $mysql_pid = open(*MYSQL_WRITER, "| mysql $options >$mysql_stdout 2>$mysql_stderr ") or Die "Failed to spawn mysql child process: $!"; | ||
99 | 1004 | MYSQL_WRITER->autoflush(1); | ||
100 | 1005 | $now = current_time(); | ||
101 | 1006 | print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n"; | ||
102 | 1007 | print MYSQL_WRITER "SET SESSION wait_timeout = 2147000;\n" or die "Connection to mysql child process failed: $!"; | ||
103 | 1008 | |||
104 | 1009 | mysql_check(); | ||
105 | 1010 | } | ||
106 | 1011 | |||
107 | 1012 | |||
108 | 1013 | # | ||
109 | 1014 | # mysql_check subroutine checks that the connection to mysql child process | ||
110 | 1015 | # is ok. | ||
111 | 1016 | # | ||
112 | 1017 | sub mysql_check { | ||
116 | 1018 | my $mysql_pid_copy = $mysql_pid; | 988 | my $mysql_pid_copy = $mysql_pid; |
117 | 1019 | 989 | ||
118 | 1020 | # send a dummy query to mysql child process | 990 | # send a dummy query to mysql child process |
119 | 1021 | $hello_id++; | 991 | $hello_id++; |
121 | 1022 | my $hello_message = "innobackup hello $hello_id"; | 992 | |
122 | 993 | my $hello_message = "xtrabackup ping $hello_id"; | ||
123 | 1023 | print MYSQL_WRITER "select '$hello_message';\n" | 994 | print MYSQL_WRITER "select '$hello_message';\n" |
124 | 1024 | or Die "Connection to mysql child process failed: $!"; | 995 | or Die "Connection to mysql child process failed: $!"; |
125 | 1025 | 996 | ||
126 | @@ -1051,21 +1022,54 @@ | |||
127 | 1051 | . " You may adjust time limit by editing the value of parameter" | 1022 | . " You may adjust time limit by editing the value of parameter" |
128 | 1052 | . " \"\$mysql_response_timeout\" in this script.)"; | 1023 | . " \"\$mysql_response_timeout\" in this script.)"; |
129 | 1053 | } elsif ($@) { Die $@; } | 1024 | } elsif ($@) { Die $@; } |
145 | 1054 | 1025 | } | |
146 | 1055 | $mysql_last_access_time = time(); | 1026 | |
147 | 1056 | } | 1027 | |
148 | 1057 | 1028 | # | |
149 | 1058 | 1029 | # SIGALRM handler which sends a keepalive query to the server | |
150 | 1059 | # | 1030 | # |
151 | 1060 | # mysql_keep_alive subroutine tries to keep connection to the mysqld database | 1031 | sub catch_sigalrm { |
152 | 1061 | # server alive by sending a dummy query when the connection has been idle | 1032 | mysql_ping(); |
153 | 1062 | # for the specified time. | 1033 | # Reschedule SIGALRM |
154 | 1063 | # | 1034 | alarm $mysql_keep_alive; |
155 | 1064 | sub mysql_keep_alive { | 1035 | } |
156 | 1065 | if ((time() - $mysql_last_access_time) > $mysql_keep_alive_timeout) { | 1036 | |
157 | 1066 | # too long idle, send a dummy query | 1037 | |
158 | 1067 | mysql_check(); | 1038 | # |
159 | 1068 | } | 1039 | # Schedule periodic server pings |
160 | 1040 | # | ||
161 | 1041 | sub start_keepalives { | ||
162 | 1042 | $SIG{ALRM} = \&catch_sigalrm; | ||
163 | 1043 | alarm $mysql_keep_alive; | ||
164 | 1044 | } | ||
165 | 1045 | |||
166 | 1046 | # | ||
167 | 1047 | # Cancel periodic server pings | ||
168 | 1048 | # | ||
169 | 1049 | sub stop_keepalives { | ||
170 | 1050 | alarm 0; | ||
171 | 1051 | $SIG{ALRM} = "DEFAULT"; | ||
172 | 1052 | } | ||
173 | 1053 | |||
174 | 1054 | # | ||
175 | 1055 | # mysql_open subroutine starts mysql as a child process with | ||
176 | 1056 | # a pipe connection. | ||
177 | 1057 | # | ||
178 | 1058 | sub mysql_open { | ||
179 | 1059 | my $options = get_mysql_options(); | ||
180 | 1060 | # run mysql as a child process with a pipe connection | ||
181 | 1061 | $now = current_time(); | ||
182 | 1062 | (my $prt_options = $options) =~ s/--password=[^ ]+ /--password=xxxxxxxx /g; | ||
183 | 1063 | print STDERR "$now $prefix Starting mysql with options: $prt_options\n"; | ||
184 | 1064 | $mysql_pid = open(*MYSQL_WRITER, "| mysql $options >$mysql_stdout 2>$mysql_stderr ") or Die "Failed to spawn mysql child process: $!"; | ||
185 | 1065 | MYSQL_WRITER->autoflush(1); | ||
186 | 1066 | $now = current_time(); | ||
187 | 1067 | print STDERR "$now $prefix Connected to database with mysql child process (pid=$mysql_pid)\n"; | ||
188 | 1068 | print MYSQL_WRITER "SET SESSION wait_timeout = 2147000;\n" or die "Connection to mysql child process failed: $!"; | ||
189 | 1069 | |||
190 | 1070 | mysql_ping(); | ||
191 | 1071 | |||
192 | 1072 | start_keepalives(); | ||
193 | 1069 | } | 1073 | } |
194 | 1070 | 1074 | ||
195 | 1071 | 1075 | ||
196 | @@ -1079,10 +1083,14 @@ | |||
197 | 1079 | sub mysql_send { | 1083 | sub mysql_send { |
198 | 1080 | my $request = shift; | 1084 | my $request = shift; |
199 | 1081 | 1085 | ||
200 | 1086 | stop_keepalives(); | ||
201 | 1087 | |||
202 | 1082 | $current_mysql_request = $request; | 1088 | $current_mysql_request = $request; |
203 | 1083 | print MYSQL_WRITER "$request\n"; | 1089 | print MYSQL_WRITER "$request\n"; |
205 | 1084 | mysql_check(); | 1090 | mysql_ping(); |
206 | 1085 | $current_mysql_request = ''; | 1091 | $current_mysql_request = ''; |
207 | 1092 | |||
208 | 1093 | start_keepalives(); | ||
209 | 1086 | } | 1094 | } |
210 | 1087 | 1095 | ||
211 | 1088 | 1096 | ||
212 | @@ -1090,6 +1098,8 @@ | |||
213 | 1090 | # mysql_close subroutine terminates mysql child process gracefully. | 1098 | # mysql_close subroutine terminates mysql child process gracefully. |
214 | 1091 | # | 1099 | # |
215 | 1092 | sub mysql_close { | 1100 | sub mysql_close { |
216 | 1101 | stop_keepalives(); | ||
217 | 1102 | |||
218 | 1093 | print MYSQL_WRITER "quit\n"; | 1103 | print MYSQL_WRITER "quit\n"; |
219 | 1094 | $now = current_time(); | 1104 | $now = current_time(); |
220 | 1095 | print STDERR "$now $prefix Connection to database server closed\n"; | 1105 | print STDERR "$now $prefix Connection to database server closed\n"; |
221 | @@ -1852,9 +1862,6 @@ | |||
222 | 1852 | "'$source_dir/$database/$wildcard' ($file_c files)\n"; | 1862 | "'$source_dir/$database/$wildcard' ($file_c files)\n"; |
223 | 1853 | } | 1863 | } |
224 | 1854 | foreach $file (@list) { | 1864 | foreach $file (@list) { |
225 | 1855 | # copying may take a long time, so we have to prevent | ||
226 | 1856 | # mysql connection from timing out | ||
227 | 1857 | mysql_keep_alive(); | ||
228 | 1858 | next unless check_if_required($database, $file); | 1865 | next unless check_if_required($database, $file); |
229 | 1859 | 1866 | ||
230 | 1860 | if($option_include) { | 1867 | if($option_include) { |
231 | 1861 | 1868 | ||
232 | === added file 'test/t/bug884737.sh' | |||
233 | --- test/t/bug884737.sh 1970-01-01 00:00:00 +0000 | |||
234 | +++ test/t/bug884737.sh 2011-11-14 07:14:52 +0000 | |||
235 | @@ -0,0 +1,14 @@ | |||
236 | 1 | ######################################################################## | ||
237 | 2 | # Bug #884737: xtrabackup's --parallel option asserts / crashes with a | ||
238 | 3 | # value of -1 | ||
239 | 4 | ######################################################################## | ||
240 | 5 | |||
241 | 6 | . inc/common.sh | ||
242 | 7 | |||
243 | 8 | init | ||
244 | 9 | run_mysqld | ||
245 | 10 | |||
246 | 11 | # Check that --parallel=<negative value> doesn't blow up | ||
247 | 12 | vlog "Creating the backup directory: $topdir/backup" | ||
248 | 13 | mkdir -p $topdir/backup | ||
249 | 14 | innobackupex $topdir/backup --parallel=-1 | ||
250 | 0 | 15 | ||
251 | === modified file 'xtrabackup.c' | |||
252 | --- xtrabackup.c 2011-10-05 02:31:10 +0000 | |||
253 | +++ xtrabackup.c 2011-11-14 07:14:52 +0000 | |||
254 | @@ -708,7 +708,7 @@ | |||
255 | 708 | 708 | ||
256 | 709 | ibool xtrabackup_logfile_is_renamed = FALSE; | 709 | ibool xtrabackup_logfile_is_renamed = FALSE; |
257 | 710 | 710 | ||
259 | 711 | uint parallel; | 711 | int parallel; |
260 | 712 | 712 | ||
261 | 713 | /* === metadata of backup === */ | 713 | /* === metadata of backup === */ |
262 | 714 | #define XTRABACKUP_METADATA_FILENAME "xtrabackup_checkpoints" | 714 | #define XTRABACKUP_METADATA_FILENAME "xtrabackup_checkpoints" |
263 | @@ -1043,8 +1043,8 @@ | |||
264 | 1043 | {"parallel", OPT_XTRA_PARALLEL, | 1043 | {"parallel", OPT_XTRA_PARALLEL, |
265 | 1044 | "Number of threads to use for parallel datafiles transfer. Does not have " | 1044 | "Number of threads to use for parallel datafiles transfer. Does not have " |
266 | 1045 | "any effect in the stream mode. The default value is 1.", | 1045 | "any effect in the stream mode. The default value is 1.", |
269 | 1046 | (G_PTR*) ¶llel, (G_PTR*) ¶llel, 0, GET_UINT, REQUIRED_ARG, | 1046 | (G_PTR*) ¶llel, (G_PTR*) ¶llel, 0, GET_INT, REQUIRED_ARG, |
270 | 1047 | 1, 1, UINT_MAX, 0, 0, 0}, | 1047 | 1, 1, INT_MAX, 0, 0, 0}, |
271 | 1048 | 1048 | ||
272 | 1049 | {"innodb_adaptive_hash_index", OPT_INNODB_ADAPTIVE_HASH_INDEX, | 1049 | {"innodb_adaptive_hash_index", OPT_INNODB_ADAPTIVE_HASH_INDEX, |
273 | 1050 | "Enable InnoDB adaptive hash index (enabled by default). " | 1050 | "Enable InnoDB adaptive hash index (enabled by default). " |
274 | @@ -4107,7 +4107,7 @@ | |||
275 | 4107 | count = parallel; | 4107 | count = parallel; |
276 | 4108 | count_mutex = OS_MUTEX_CREATE(); | 4108 | count_mutex = OS_MUTEX_CREATE(); |
277 | 4109 | 4109 | ||
279 | 4110 | for (i = 0; i < parallel; i++) { | 4110 | for (i = 0; i < (uint) parallel; i++) { |
280 | 4111 | data_threads[i].it = it; | 4111 | data_threads[i].it = it; |
281 | 4112 | data_threads[i].num = i+1; | 4112 | data_threads[i].num = i+1; |
282 | 4113 | data_threads[i].count = &count; | 4113 | data_threads[i].count = &count; |
Jenkins build: http:// jenkins. percona. com/view/ Percona% 20Xtrabackup/ job/percona- xtrabackup- param/.
Lots of failures is a result slaves corruption as described in https:/ /code.launchpad .net/~akopytov/ percona- xtrabackup/ bug884737- 1.6/+merge/ 80963