Merge lp:~akopytov/percona-xtrabackup/bug1411157 into lp:percona-xtrabackup/2.3

Proposed by Alexey Kopytov
Status: Merged
Approved by: Alexey Kopytov
Approved revision: no longer in the source branch.
Merged at revision: 5063
Proposed branch: lp:~akopytov/percona-xtrabackup/bug1411157
Merge into: lp:percona-xtrabackup/2.3
Diff against target: 211 lines (+47/-47)
4 files modified
storage/innobase/xtrabackup/doc/source/innobackupex/improved_ftwrl.rst (+11/-11)
storage/innobase/xtrabackup/doc/source/innobackupex/innobackupex_option_reference.rst (+6/-6)
storage/innobase/xtrabackup/innobackupex.pl (+21/-21)
storage/innobase/xtrabackup/test/t/kill_long_selects.sh (+9/-9)
To merge this branch: bzr merge lp:~akopytov/percona-xtrabackup/bug1411157
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Approve
Review via email: mp+246760@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Alexey Kopytov (akopytov) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'storage/innobase/xtrabackup/doc/source/innobackupex/improved_ftwrl.rst'
--- storage/innobase/xtrabackup/doc/source/innobackupex/improved_ftwrl.rst 2014-12-18 12:48:27 +0000
+++ storage/innobase/xtrabackup/doc/source/innobackupex/improved_ftwrl.rst 2015-01-16 18:10:24 +0000
@@ -14,11 +14,11 @@
14Waiting for queries to finish14Waiting for queries to finish
15-----------------------------15-----------------------------
1616
17Good moment to issue a global lock is the moment when there are no long queries running. But waiting for a good moment to issue the global lock for extended period of time isn't always good approach, as it can extend the time needed for backup to take place. To prevent |innobackupex| from waiting to issue ``FLUSH TABLES WITH READ LOCK`` for too long, new option has been implemented: :option:`innobackupex --lock-wait-timeout` option can be used to limit the waiting time. If the good moment to issue the lock did not happen during this time, |innobackupex| will give up and exit with an error message and backup will not be taken. Zero value for this option turns off the feature (which is default).17Good moment to issue a global lock is the moment when there are no long queries running. But waiting for a good moment to issue the global lock for extended period of time isn't always good approach, as it can extend the time needed for backup to take place. To prevent |innobackupex| from waiting to issue ``FLUSH TABLES WITH READ LOCK`` for too long, new option has been implemented: :option:`innobackupex --ftwrl-wait-timeout` option can be used to limit the waiting time. If the good moment to issue the lock did not happen during this time, |innobackupex| will give up and exit with an error message and backup will not be taken. Zero value for this option turns off the feature (which is default).
1818
19Another possibility is to specify the type of query to wait on. In this case :option:`innobackupex --lock-wait-query-type`. Possible values are ``all`` and ``update``. When ``all`` is used |innobackupex| will wait for all long running queries (execution time longer than allowed by :option:`innobackupex --lock-wait-threshold`) to finish before running the ``FLUSH TABLES WITH READ LOCK``. When ``update`` is used |innobackupex| will wait on ``UPDATE/ALTER/REPLACE/INSERT`` queries to finish.19Another possibility is to specify the type of query to wait on. In this case :option:`innobackupex --ftwrl-wait-query-type`. Possible values are ``all`` and ``update``. When ``all`` is used |innobackupex| will wait for all long running queries (execution time longer than allowed by :option:`innobackupex --ftwrl-wait-threshold`) to finish before running the ``FLUSH TABLES WITH READ LOCK``. When ``update`` is used |innobackupex| will wait on ``UPDATE/ALTER/REPLACE/INSERT`` queries to finish.
2020
21Although time needed for specific query to complete is hard to predict, we can assume that queries that are running for a long time already will likely not be completed soon, and queries which are running for a short time will likely be completed shortly. |innobackupex| can use the value of :option:`innobackupex --lock-wait-threshold` option to specify which query is long running and will likely block global lock for a while. In order to use this option xtrabackup user should have ``PROCESS`` and ``SUPER`` privileges.21Although time needed for specific query to complete is hard to predict, we can assume that queries that are running for a long time already will likely not be completed soon, and queries which are running for a short time will likely be completed shortly. |innobackupex| can use the value of :option:`innobackupex --ftwrl-wait-threshold` option to specify which query is long running and will likely block global lock for a while. In order to use this option xtrabackup user should have ``PROCESS`` and ``SUPER`` privileges.
2222
23Killing the blocking queries23Killing the blocking queries
24----------------------------24----------------------------
@@ -30,11 +30,11 @@
30Options summary30Options summary
31---------------31---------------
3232
33* :option:`--lock-wait-timeout=N` (seconds) - how long to wait for a good moment. Default is 0, not to wait.33* :option:`--ftwrl-wait-timeout=N` (seconds) - how long to wait for a good moment. Default is 0, not to wait.
3434
35* :option:`--lock-wait-query-type={all|update}` - which long queries should be finished before ``FLUSH TABLES WITH READ LOCK`` is run. Default is all.35* :option:`--ftwrl-wait-query-type={all|update}` - which long queries should be finished before ``FLUSH TABLES WITH READ LOCK`` is run. Default is all.
3636
37* :option:`--lock-wait-threshold=N` (seconds) - how long query should be running before we consider it long running and potential blocker of global lock.37* :option:`--ftwrl-wait-threshold=N` (seconds) - how long query should be running before we consider it long running and potential blocker of global lock.
3838
39* :option:`--kill-long-queries-timeout=N` (seconds) - how many time we give for queries to complete after ``FLUSH TABLES WITH READ LOCK`` is issued before start to kill. Default if ``0``, not to kill.39* :option:`--kill-long-queries-timeout=N` (seconds) - how many time we give for queries to complete after ``FLUSH TABLES WITH READ LOCK`` is issued before start to kill. Default if ``0``, not to kill.
4040
@@ -45,7 +45,7 @@
4545
46Running the |innobackupex| with the following options: ::46Running the |innobackupex| with the following options: ::
4747
48 $ innobackupex --lock-wait-threshold=40 --lock-wait-query-type=all --lock-wait-timeout=180 --kill-long-queries-timeout=20 --kill-long-query-type=all /data/backups/48 $ innobackupex --ftwrl-wait-threshold=40 --ftwrl-wait-query-type=all --ftwrl-wait-timeout=180 --kill-long-queries-timeout=20 --kill-long-query-type=all /data/backups/
4949
50will cause |innobackupex| to spend no longer than 3 minutes waiting for all queries older than 40 seconds to complete. After ``FLUSH TABLES WITH READ LOCK`` is issued, |innobackupex| will wait 20 seconds for lock to be acquired. If lock is still not acquired after 20 seconds, it will kill all queries which are running longer that the ``FLUSH TABLES WITH READ LOCK``.50will cause |innobackupex| to spend no longer than 3 minutes waiting for all queries older than 40 seconds to complete. After ``FLUSH TABLES WITH READ LOCK`` is issued, |innobackupex| will wait 20 seconds for lock to be acquired. If lock is still not acquired after 20 seconds, it will kill all queries which are running longer that the ``FLUSH TABLES WITH READ LOCK``.
5151
5252
=== modified file 'storage/innobase/xtrabackup/doc/source/innobackupex/innobackupex_option_reference.rst'
--- storage/innobase/xtrabackup/doc/source/innobackupex/innobackupex_option_reference.rst 2014-12-18 10:39:15 +0000
+++ storage/innobase/xtrabackup/doc/source/innobackupex/innobackupex_option_reference.rst 2015-01-16 18:10:24 +0000
@@ -150,15 +150,15 @@
150150
151 This option specifies which types of queries should be killed to unblock the global lock. Default is "all".151 This option specifies which types of queries should be killed to unblock the global lock. Default is "all".
152152
153.. option:: --lock-wait-timeout=SECONDS153.. option:: --ftwrl-wait-timeout=SECONDS
154154
155 This option specifies time in seconds that innobackupex should wait for queries that would block ``FLUSH TABLES WITH READ LOCK`` before running it. If there are still such queries when the timeout expires, innobackupex terminates with an error. Default is 0, in which case innobackupex does not wait for queries to complete and starts ``FLUSH TABLES WITH READ LOCK`` immediately.155 This option specifies time in seconds that innobackupex should wait for queries that would block ``FLUSH TABLES WITH READ LOCK`` before running it. If there are still such queries when the timeout expires, innobackupex terminates with an error. Default is 0, in which case innobackupex does not wait for queries to complete and starts ``FLUSH TABLES WITH READ LOCK`` immediately.
156156
157.. option:: --lock-wait-threshold=SECONDS157.. option:: --ftwrl-wait-threshold=SECONDS
158158
159 This option specifies the query run time threshold which is used by innobackupex to detect long-running queries with a non-zero value of :option:`innobackupex --lock-wait-timeout`. FLUSH TABLES WITH READ LOCK`` is not started until such long-running queries exist. This option has no effect if --lock-wait-timeout is 0. Default value is 60 seconds.159 This option specifies the query run time threshold which is used by innobackupex to detect long-running queries with a non-zero value of :option:`innobackupex --ftwrl-wait-timeout`. FLUSH TABLES WITH READ LOCK`` is not started until such long-running queries exist. This option has no effect if --ftwrl-wait-timeout is 0. Default value is 60 seconds.
160160
161.. option:: --lock-wait-query-type=all|update161.. option:: --ftwrl-wait-query-type=all|update
162162
163 This option specifies which types of queries are allowed to complete before innobackupex will issue the global lock. Default is all.163 This option specifies which types of queries are allowed to complete before innobackupex will issue the global lock. Default is all.
164164
165165
=== modified file 'storage/innobase/xtrabackup/innobackupex.pl'
--- storage/innobase/xtrabackup/innobackupex.pl 2014-12-08 06:52:10 +0000
+++ storage/innobase/xtrabackup/innobackupex.pl 2015-01-16 18:10:24 +0000
@@ -229,14 +229,14 @@
229my $option_kill_long_queries_timeout = 0;229my $option_kill_long_queries_timeout = 0;
230230
231# waiting for an appropriate time to start FTWRL timeout231# waiting for an appropriate time to start FTWRL timeout
232my $option_lock_wait_timeout = 0;232my $option_ftwrl_wait_timeout = 0;
233233
234# how old should be query to be waited for234# how old should be query to be waited for
235my $option_lock_wait_threshold = 60;235my $option_ftwrl_wait_threshold = 60;
236236
237# which type of queries we are waiting for during the pre-FTWRL phase237# which type of queries we are waiting for during the pre-FTWRL phase
238# possible values are "update" and "all"238# possible values are "update" and "all"
239my $option_lock_wait_query_type = "all";239my $option_ftwrl_wait_query_type = "all";
240240
241# which type of queries wa are waiting when clearing the way for FTWRL241# which type of queries wa are waiting when clearing the way for FTWRL
242# by killing; possible values are "select" and "all"242# by killing; possible values are "select" and "all"
@@ -3342,7 +3342,7 @@
3342 while (my ($id, $process) = each %$processlist) {3342 while (my ($id, $process) = each %$processlist) {
3343 if (defined($process->{Info}) &&3343 if (defined($process->{Info}) &&
3344 $process->{Time} >= $threshold &&3344 $process->{Time} >= $threshold &&
3345 (($option_lock_wait_query_type eq "all" &&3345 (($option_ftwrl_wait_query_type eq "all" &&
3346 is_query($process->{Info})) ||3346 is_query($process->{Info})) ||
3347 is_update_query($process->{Info}))) {3347 is_update_query($process->{Info}))) {
3348 print STDERR "\n$now $prefix Waiting for query $id (duration " .3348 print STDERR "\n$now $prefix Waiting for query $id (duration " .
@@ -3456,9 +3456,9 @@
3456 return;3456 return;
3457 }3457 }
34583458
3459 if ($option_lock_wait_timeout) {3459 if ($option_ftwrl_wait_timeout) {
3460 wait_for_no_updates($con, $option_lock_wait_timeout,3460 wait_for_no_updates($con, $option_ftwrl_wait_timeout,
3461 $option_lock_wait_threshold);3461 $option_ftwrl_wait_threshold);
3462 }3462 }
34633463
3464 $now = current_time();3464 $now = current_time();
@@ -3831,10 +3831,10 @@
3831 \$option_kill_long_queries_timeout,3831 \$option_kill_long_queries_timeout,
3832 'kill-long-query-type=s' =>3832 'kill-long-query-type=s' =>
3833 \$option_kill_long_query_type,3833 \$option_kill_long_query_type,
3834 'lock-wait-timeout=i' => \$option_lock_wait_timeout,3834 'ftwrl-wait-timeout=i' => \$option_ftwrl_wait_timeout,
3835 'lock-wait-threshold=i' => \$option_lock_wait_threshold,3835 'ftwrl-wait-threshold=i' => \$option_ftwrl_wait_threshold,
3836 'lock-wait-query-type=s' =>3836 'ftwrl-wait-query-type=s' =>
3837 \$option_lock_wait_query_type,3837 \$option_ftwrl_wait_query_type,
3838 'version-check!' => \$option_version_check,3838 'version-check!' => \$option_version_check,
3839 'force-non-empty-directories' =>3839 'force-non-empty-directories' =>
3840 \$option_force_non_empty_dirs3840 \$option_force_non_empty_dirs
@@ -3902,10 +3902,10 @@
3902 $option_compress = 0;3902 $option_compress = 0;
3903 }3903 }
39043904
3905 # validate lock-wait-query-type and kill-long-query-type values3905 # validate ftwrl-wait-query-type and kill-long-query-type values
3906 if (!(grep {$_ eq $option_lock_wait_query_type} qw/all update/)) {3906 if (!(grep {$_ eq $option_ftwrl_wait_query_type} qw/all update/)) {
3907 die "Wrong value of lock-wait-query-type. ".3907 die "Wrong value of ftwrl-wait-query-type. ".
3908 "Possible values are all|update, but $option_lock_wait_query_type ".3908 "Possible values are all|update, but $option_ftwrl_wait_query_type ".
3909 "is specified.";3909 "is specified.";
3910 }3910 }
3911 if (!(grep {$_ eq $option_kill_long_query_type} qw/all select/)) {3911 if (!(grep {$_ eq $option_kill_long_query_type} qw/all select/)) {
@@ -5507,16 +5507,16 @@
55075507
5508This option specifies which types of queries should be killed to unblock the global lock. Default is "all".5508This option specifies which types of queries should be killed to unblock the global lock. Default is "all".
55095509
5510=item --lock-wait-timeout=SECONDS5510=item --ftwrl-wait-timeout=SECONDS
55115511
5512This option specifies time in seconds that innobackupex should wait for queries that would block FTWRL before running it. If there are still such queries when the timeout expires, innobackupex terminates with an error.5512This option specifies time in seconds that innobackupex should wait for queries that would block FTWRL before running it. If there are still such queries when the timeout expires, innobackupex terminates with an error.
5513Default is 0, in which case innobackupex does not wait for queries to complete and starts FTWRL immediately.5513Default is 0, in which case innobackupex does not wait for queries to complete and starts FTWRL immediately.
55145514
5515=item --lock-wait-threshold=SECONDS5515=item --ftwrl-wait-threshold=SECONDS
55165516
5517This option specifies the query run time threshold which is used by innobackupex to detect long-running queries with a non-zero value of --lock-wait-timeout. FTWRL is not started until such long-running queries exist. This option has no effect if --lock-wait-timeout is 0. Default value is 60 seconds.5517This option specifies the query run time threshold which is used by innobackupex to detect long-running queries with a non-zero value of --ftwrl-wait-timeout. FTWRL is not started until such long-running queries exist. This option has no effect if --ftwrl-wait-timeout is 0. Default value is 60 seconds.
55185518
5519=item --lock-wait-query-type=all|update5519=item --ftwrl-wait-query-type=all|update
55205520
5521This option specifies which types of queries are allowed to complete before innobackupex will issue the global lock. Default is all.5521This option specifies which types of queries are allowed to complete before innobackupex will issue the global lock. Default is all.
55225522
55235523
=== modified file 'storage/innobase/xtrabackup/test/t/kill_long_selects.sh'
--- storage/innobase/xtrabackup/test/t/kill_long_selects.sh 2014-09-04 16:53:33 +0000
+++ storage/innobase/xtrabackup/test/t/kill_long_selects.sh 2015-01-16 18:10:24 +0000
@@ -153,9 +153,9 @@
153wait_for_connection_count 2153wait_for_connection_count 2
154154
155run_cmd_expect_failure ${IB_BIN} ${IB_ARGS} $topdir/full \155run_cmd_expect_failure ${IB_BIN} ${IB_ARGS} $topdir/full \
156 --lock-wait-timeout=3 \156 --ftwrl-wait-timeout=3 \
157 --lock-wait-query-type=all \157 --ftwrl-wait-query-type=all \
158 --lock-wait-threshold=1 \158 --ftwrl-wait-threshold=1 \
159 --kill-long-queries-timeout=1 \159 --kill-long-queries-timeout=1 \
160 --kill-long-query-type=all160 --kill-long-query-type=all
161161
@@ -171,9 +171,9 @@
171wait_for_connection_count 2171wait_for_connection_count 2
172172
173run_cmd_expect_failure ${IB_BIN} ${IB_ARGS} $topdir/full \173run_cmd_expect_failure ${IB_BIN} ${IB_ARGS} $topdir/full \
174 --lock-wait-timeout=3 \174 --ftwrl-wait-timeout=3 \
175 --lock-wait-query-type=update \175 --ftwrl-wait-query-type=update \
176 --lock-wait-threshold=2 \176 --ftwrl-wait-threshold=2 \
177 --kill-long-queries-timeout=1 \177 --kill-long-queries-timeout=1 \
178 --kill-long-query-type=all178 --kill-long-query-type=all
179179
@@ -190,9 +190,9 @@
190wait_for_connection_count 2190wait_for_connection_count 2
191191
192innobackupex $topdir/full \192innobackupex $topdir/full \
193 --lock-wait-timeout=6 \193 --ftwrl-wait-timeout=6 \
194 --lock-wait-query-type=update \194 --ftwrl-wait-query-type=update \
195 --lock-wait-threshold=2 \195 --ftwrl-wait-threshold=2 \
196 --kill-long-queries-timeout=1 \196 --kill-long-queries-timeout=1 \
197 --kill-long-query-type=all197 --kill-long-query-type=all
198198

Subscribers

People subscribed via source and target branches

to all changes: