Merge lp:~laurynas-biveinis/percona-server/xtradb-cleaner-tuning-5.6 into lp:percona-server/5.6

Proposed by Laurynas Biveinis
Status: Merged
Approved by: Vadim Tkachenko
Approved revision: no longer in the source branch.
Merged at revision: 445
Proposed branch: lp:~laurynas-biveinis/percona-server/xtradb-cleaner-tuning-5.6
Merge into: lp:percona-server/5.6
Diff against target: 1222 lines (+795/-116)
16 files modified
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result (+31/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result (+31/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result (+35/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result (+31/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result (+25/-0)
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result (+25/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test (+35/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test (+33/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test (+35/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test (+33/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test (+31/-0)
Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test (+31/-0)
Percona-Server/storage/innobase/buf/buf0flu.cc (+323/-116)
Percona-Server/storage/innobase/handler/ha_innodb.cc (+42/-0)
Percona-Server/storage/innobase/include/srv0srv.h (+29/-0)
Percona-Server/storage/innobase/srv/srv0srv.cc (+25/-0)
To merge this branch: bzr merge lp:~laurynas-biveinis/percona-server/xtradb-cleaner-tuning-5.6
Reviewer Review Type Date Requested Status
Sergei Glushchenko (community) g2 Approve
Vadim Tkachenko algo Approve
Review via email: mp+188368@code.launchpad.net

Description of the change

Tune page cleaner thread, implementing
https://blueprints.launchpad.net/percona-server/+spec/cleaner-tuning-5.6.

Create new type, local tu buf0flu.cc, struct flush_counters_t, that
has separate flushed and evicted page counters. Adjust
buf_flush_LRU_list(), buf_do_LRU_batch(), buf_flush_batch(), and
buf_flush_LRU() to return it.

In buf_flush_LRU_list(), add new arg limited_scan that optionally
limits the total number of scanned pages to srv_LRU_scan_depth, limit
scans to not go deeper than srv_LRU_scan_depth in the LRU list.

In buf_flush_common(), only call buf_dblwr_flush_buffered_writes() if
there have been flushed pages, fixing
http://bugs.mysql.com/bug.php?id=69170 / bug 1231918 ("buf_flush_LRU
is lazy" | buf_flush_common() calls buf_dblwr_flush_buffered_writes()
even when no pages flushed).

Rewrite buf_flush_list() to divide flush requests into chunks that are
issued to buffer pool instances in parallel and limit total running
time to a new UNIV_PERF_DEBUG-only variable
innodb_cleaner_max_flush_time, default 1000 miliseconds. The chunk
size is specified by a new UNIV_PERF_DEBUG-only variable
innodb_cleaner_flush_chunk_size, default 100.

Rewrite buf_flush_LRU_tail() to add timeouts as in buf_flush_list,
governed by a new UNIV_PERF_DEBUG-only variable
innodb_cleaner_max_lru_time. Likewise issue chunk batches to buffer
pool instances in parallel, with the exception of buffer pool
instances with almost-empty free lists (as defined by
UNIV_PERF_DEBUG-only innodb_cleaner_free_list_lwm, default 10%), for
which batches are issued sequentially. Make the chunk size tunable by
a new UNIV_PERF_DEBUG-only variable innodb_cleaner_lru_chunk_size,
default 100.

The timeouts in buf_flush_list() and buf_flush_LRU_tail() fix
http://bugs.mysql.com/bug.php?id=70453 / bug 1232101 (Add hard
timeouts to page cleaner flushes).

Add new functions buf_get_total_free_list_length(),
buf_cleaner_adapt_lru_sleep_time(), and
page_cleaner_adapt_flush_sleep_time(). They implement adaptive sleep
time reduction that may reduce page cleaner iteration time below 1
second and thus implement furious flushing, fixing
http://bugs.mysql.com/bug.php?id=68481 / bug 1232406 (Implement
furious flushing for 5.6).

The adaptiveness is implemented as follows. Maintain a desired target
sleep time for LRU flushes. If the total length of all buffer pool
instances is <1%, no sleep; if <5%, sleep 50 ms less, if between 5%
and 20%, no change; if >20%, sleep 50ms longer. Then, if the
checkpoint age is in preflush sync zone, set the sleep time to zero.

Add a new UNIV_PERF_DEBUG-only variable
innodb_cleaner_eviction_factor, default FALSE, that, if enabled, makes
LRU tail flushing to use evicted instead of flushed page counts for
its heuristics.

Add sys_vars tests for these variables.

To post a comment you must log in.
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :
Revision history for this message
Vadim Tkachenko (vadim-tk) wrote :

Based on my talks with Alexey Stroganov
I approve this from algo point of view.

I leave tests/code style G2 review to Sergei_gl

review: Approve (algo)
Revision history for this message
Sergei Glushchenko (sergei.glushchenko) wrote :

Approve

review: Approve (g2)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result'
2--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result 1970-01-01 00:00:00 +0000
3+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result 2013-09-30 15:25:38 +0000
4@@ -0,0 +1,31 @@
5+SET @start_value = @@GLOBAL.innodb_cleaner_eviction_factor;
6+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
7+@@GLOBAL.innodb_cleaner_eviction_factor
8+0
9+SELECT @@SESSION.innodb_cleaner_eviction_factor;
10+ERROR HY000: Variable 'innodb_cleaner_eviction_factor' is a GLOBAL variable
11+SET GLOBAL innodb_cleaner_eviction_factor='OFF';
12+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
13+@@GLOBAL.innodb_cleaner_eviction_factor
14+0
15+SET GLOBAL innodb_cleaner_eviction_factor='ON';
16+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
17+@@GLOBAL.innodb_cleaner_eviction_factor
18+1
19+SET GLOBAL innodb_cleaner_eviction_factor=0;
20+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
21+@@GLOBAL.innodb_cleaner_eviction_factor
22+0
23+SET GLOBAL innodb_cleaner_eviction_factor=1;
24+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
25+@@GLOBAL.innodb_cleaner_eviction_factor
26+1
27+SET GLOBAL innodb_cleaner_eviction_factor=1.1;
28+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_eviction_factor'
29+SET GLOBAL innodb_cleaner_eviction_factor=1e1;
30+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_eviction_factor'
31+SET GLOBAL innodb_cleaner_eviction_factor=2;
32+ERROR 42000: Variable 'innodb_cleaner_eviction_factor' can't be set to the value of '2'
33+SET GLOBAL innodb_cleaner_eviction_factor='foo';
34+ERROR 42000: Variable 'innodb_cleaner_eviction_factor' can't be set to the value of 'foo'
35+SET GLOBAL innodb_cleaner_eviction_factor = @start_value;
36
37=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result'
38--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result 1970-01-01 00:00:00 +0000
39+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result 2013-09-30 15:25:38 +0000
40@@ -0,0 +1,31 @@
41+SET @start_value = @@GLOBAL.innodb_cleaner_flush_chunk_size;
42+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
43+@@GLOBAL.innodb_cleaner_flush_chunk_size
44+200
45+SELECT @@SESSION.innodb_cleaner_flush_chunk_size;
46+ERROR HY000: Variable 'innodb_cleaner_flush_chunk_size' is a GLOBAL variable
47+SET GLOBAL innodb_cleaner_flush_chunk_size=1;
48+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
49+@@GLOBAL.innodb_cleaner_flush_chunk_size
50+1
51+SET GLOBAL innodb_cleaner_flush_chunk_size=1000;
52+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
53+@@GLOBAL.innodb_cleaner_flush_chunk_size
54+1000
55+SET GLOBAL innodb_cleaner_flush_chunk_size=4294967295;
56+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
57+@@GLOBAL.innodb_cleaner_flush_chunk_size
58+4294967295
59+SET GLOBAL innodb_cleaner_flush_chunk_size=0;
60+Warnings:
61+Warning 1292 Truncated incorrect innodb_cleaner_flush_chunk_size value: '0'
62+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
63+@@GLOBAL.innodb_cleaner_flush_chunk_size
64+1
65+SET GLOBAL innodb_cleaner_flush_chunk_size=1.1;
66+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_flush_chunk_size'
67+SET GLOBAL innodb_cleaner_flush_chunk_size=1e1;
68+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_flush_chunk_size'
69+SET GLOBAL innodb_cleaner_flush_chunk_size='foo';
70+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_flush_chunk_size'
71+SET GLOBAL innodb_cleaner_flush_chunk_size = @start_value;
72
73=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result'
74--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result 1970-01-01 00:00:00 +0000
75+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result 2013-09-30 15:25:38 +0000
76@@ -0,0 +1,35 @@
77+SET @start_value = @@GLOBAL.innodb_cleaner_free_list_lwm;
78+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
79+@@GLOBAL.innodb_cleaner_free_list_lwm
80+10
81+SELECT @@SESSION.innodb_cleaner_free_list_lwm;
82+ERROR HY000: Variable 'innodb_cleaner_free_list_lwm' is a GLOBAL variable
83+SET GLOBAL innodb_cleaner_free_list_lwm=0;
84+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
85+@@GLOBAL.innodb_cleaner_free_list_lwm
86+0
87+SET GLOBAL innodb_cleaner_free_list_lwm=1;
88+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
89+@@GLOBAL.innodb_cleaner_free_list_lwm
90+1
91+SET GLOBAL innodb_cleaner_free_list_lwm=99;
92+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
93+@@GLOBAL.innodb_cleaner_free_list_lwm
94+99
95+SET GLOBAL innodb_cleaner_free_list_lwm=100;
96+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
97+@@GLOBAL.innodb_cleaner_free_list_lwm
98+100
99+SET GLOBAL innodb_cleaner_free_list_lwm=101;
100+Warnings:
101+Warning 1292 Truncated incorrect innodb_cleaner_free_list_lwm value: '101'
102+SELECT @@innodb_cleaner_free_list_lwm;
103+@@innodb_cleaner_free_list_lwm
104+100
105+SET GLOBAL innodb_cleaner_free_list_lwm=1.1;
106+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_free_list_lwm'
107+SET GLOBAL innodb_cleaner_free_list_lwm=1e1;
108+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_free_list_lwm'
109+SET GLOBAL innodb_cleaner_free_list_lwm='foo';
110+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_free_list_lwm'
111+SET GLOBAL innodb_cleaner_free_list_lwm = @start_value;
112
113=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result'
114--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result 1970-01-01 00:00:00 +0000
115+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result 2013-09-30 15:25:38 +0000
116@@ -0,0 +1,31 @@
117+SET @start_value = @@GLOBAL.innodb_cleaner_lru_chunk_size;
118+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
119+@@GLOBAL.innodb_cleaner_lru_chunk_size
120+100
121+SELECT @@SESSION.innodb_cleaner_lru_chunk_size;
122+ERROR HY000: Variable 'innodb_cleaner_lru_chunk_size' is a GLOBAL variable
123+SET GLOBAL innodb_cleaner_lru_chunk_size=1;
124+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
125+@@GLOBAL.innodb_cleaner_lru_chunk_size
126+1
127+SET GLOBAL innodb_cleaner_lru_chunk_size=1000;
128+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
129+@@GLOBAL.innodb_cleaner_lru_chunk_size
130+1000
131+SET GLOBAL innodb_cleaner_lru_chunk_size=4294967295;
132+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
133+@@GLOBAL.innodb_cleaner_lru_chunk_size
134+4294967295
135+SET GLOBAL innodb_cleaner_lru_chunk_size=0;
136+Warnings:
137+Warning 1292 Truncated incorrect innodb_cleaner_lru_chunk_size value: '0'
138+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
139+@@GLOBAL.innodb_cleaner_lru_chunk_size
140+1
141+SET GLOBAL innodb_cleaner_lru_chunk_size=1.1;
142+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_lru_chunk_size'
143+SET GLOBAL innodb_cleaner_lru_chunk_size=1e1;
144+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_lru_chunk_size'
145+SET GLOBAL innodb_cleaner_lru_chunk_size='foo';
146+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_lru_chunk_size'
147+SET GLOBAL innodb_cleaner_lru_chunk_size = @start_value;
148
149=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result'
150--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result 1970-01-01 00:00:00 +0000
151+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result 2013-09-30 15:25:38 +0000
152@@ -0,0 +1,25 @@
153+SET @start_value = @@GLOBAL.innodb_cleaner_max_flush_time;
154+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
155+@@GLOBAL.innodb_cleaner_max_flush_time
156+1000
157+SELECT @@SESSION.innodb_cleaner_max_flush_time;
158+ERROR HY000: Variable 'innodb_cleaner_max_flush_time' is a GLOBAL variable
159+SET GLOBAL innodb_cleaner_max_flush_time=0;
160+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
161+@@GLOBAL.innodb_cleaner_max_flush_time
162+0
163+SET GLOBAL innodb_cleaner_max_flush_time=1000;
164+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
165+@@GLOBAL.innodb_cleaner_max_flush_time
166+1000
167+SET GLOBAL innodb_cleaner_max_flush_time=4294967295;
168+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
169+@@GLOBAL.innodb_cleaner_max_flush_time
170+4294967295
171+SET GLOBAL innodb_cleaner_max_flush_time=1.1;
172+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_flush_time'
173+SET GLOBAL innodb_cleaner_max_flush_time=1e1;
174+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_flush_time'
175+SET GLOBAL innodb_cleaner_max_flush_time='foo';
176+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_flush_time'
177+SET GLOBAL innodb_cleaner_max_flush_time = @start_value;
178
179=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result'
180--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result 1970-01-01 00:00:00 +0000
181+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result 2013-09-30 15:25:38 +0000
182@@ -0,0 +1,25 @@
183+SET @start_value = @@GLOBAL.innodb_cleaner_max_lru_time;
184+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
185+@@GLOBAL.innodb_cleaner_max_lru_time
186+1000
187+SELECT @@SESSION.innodb_cleaner_max_lru_time;
188+ERROR HY000: Variable 'innodb_cleaner_max_lru_time' is a GLOBAL variable
189+SET GLOBAL innodb_cleaner_max_lru_time=0;
190+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
191+@@GLOBAL.innodb_cleaner_max_lru_time
192+0
193+SET GLOBAL innodb_cleaner_max_lru_time=1000;
194+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
195+@@GLOBAL.innodb_cleaner_max_lru_time
196+1000
197+SET GLOBAL innodb_cleaner_max_lru_time=4294967295;
198+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
199+@@GLOBAL.innodb_cleaner_max_lru_time
200+4294967295
201+SET GLOBAL innodb_cleaner_max_lru_time=1.1;
202+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_lru_time'
203+SET GLOBAL innodb_cleaner_max_lru_time=1e1;
204+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_lru_time'
205+SET GLOBAL innodb_cleaner_max_lru_time='foo';
206+ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_lru_time'
207+SET GLOBAL innodb_cleaner_max_lru_time = @start_value;
208
209=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test'
210--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test 1970-01-01 00:00:00 +0000
211+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test 2013-09-30 15:25:38 +0000
212@@ -0,0 +1,35 @@
213+--source include/have_debug.inc
214+--source include/have_innodb.inc
215+
216+# A dynamic, global variable
217+
218+SET @start_value = @@GLOBAL.innodb_cleaner_eviction_factor;
219+
220+# Default value
221+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
222+
223+# Global only
224+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
225+SELECT @@SESSION.innodb_cleaner_eviction_factor;
226+
227+# Correct values
228+SET GLOBAL innodb_cleaner_eviction_factor='OFF';
229+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
230+SET GLOBAL innodb_cleaner_eviction_factor='ON';
231+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
232+SET GLOBAL innodb_cleaner_eviction_factor=0;
233+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
234+SET GLOBAL innodb_cleaner_eviction_factor=1;
235+SELECT @@GLOBAL.innodb_cleaner_eviction_factor;
236+
237+# Incorrect values
238+--error ER_WRONG_TYPE_FOR_VAR
239+SET GLOBAL innodb_cleaner_eviction_factor=1.1;
240+--error ER_WRONG_TYPE_FOR_VAR
241+SET GLOBAL innodb_cleaner_eviction_factor=1e1;
242+--error ER_WRONG_VALUE_FOR_VAR
243+SET GLOBAL innodb_cleaner_eviction_factor=2;
244+--error ER_WRONG_VALUE_FOR_VAR
245+SET GLOBAL innodb_cleaner_eviction_factor='foo';
246+
247+SET GLOBAL innodb_cleaner_eviction_factor = @start_value;
248
249=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test'
250--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test 1970-01-01 00:00:00 +0000
251+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test 2013-09-30 15:25:38 +0000
252@@ -0,0 +1,33 @@
253+--source include/have_debug.inc
254+--source include/have_innodb.inc
255+
256+# A dynamic, global variable
257+
258+SET @start_value = @@GLOBAL.innodb_cleaner_flush_chunk_size;
259+
260+# Default value
261+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
262+
263+# Global only
264+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
265+SELECT @@SESSION.innodb_cleaner_flush_chunk_size;
266+
267+# Correct values
268+SET GLOBAL innodb_cleaner_flush_chunk_size=1;
269+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
270+SET GLOBAL innodb_cleaner_flush_chunk_size=1000;
271+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
272+SET GLOBAL innodb_cleaner_flush_chunk_size=4294967295;
273+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
274+
275+# Incorrect values
276+SET GLOBAL innodb_cleaner_flush_chunk_size=0;
277+SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size;
278+--error ER_WRONG_TYPE_FOR_VAR
279+SET GLOBAL innodb_cleaner_flush_chunk_size=1.1;
280+--error ER_WRONG_TYPE_FOR_VAR
281+SET GLOBAL innodb_cleaner_flush_chunk_size=1e1;
282+--error ER_WRONG_TYPE_FOR_VAR
283+SET GLOBAL innodb_cleaner_flush_chunk_size='foo';
284+
285+SET GLOBAL innodb_cleaner_flush_chunk_size = @start_value;
286
287=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test'
288--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test 1970-01-01 00:00:00 +0000
289+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test 2013-09-30 15:25:38 +0000
290@@ -0,0 +1,35 @@
291+--source include/have_debug.inc
292+--source include/have_innodb.inc
293+
294+# A dynamic, global variable
295+
296+SET @start_value = @@GLOBAL.innodb_cleaner_free_list_lwm;
297+
298+# Default value
299+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
300+
301+# Global only
302+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
303+SELECT @@SESSION.innodb_cleaner_free_list_lwm;
304+
305+# Correct values
306+SET GLOBAL innodb_cleaner_free_list_lwm=0;
307+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
308+SET GLOBAL innodb_cleaner_free_list_lwm=1;
309+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
310+SET GLOBAL innodb_cleaner_free_list_lwm=99;
311+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
312+SET GLOBAL innodb_cleaner_free_list_lwm=100;
313+SELECT @@GLOBAL.innodb_cleaner_free_list_lwm;
314+
315+# Incorrect values
316+SET GLOBAL innodb_cleaner_free_list_lwm=101;
317+SELECT @@innodb_cleaner_free_list_lwm;
318+--error ER_WRONG_TYPE_FOR_VAR
319+SET GLOBAL innodb_cleaner_free_list_lwm=1.1;
320+--error ER_WRONG_TYPE_FOR_VAR
321+SET GLOBAL innodb_cleaner_free_list_lwm=1e1;
322+--error ER_WRONG_TYPE_FOR_VAR
323+SET GLOBAL innodb_cleaner_free_list_lwm='foo';
324+
325+SET GLOBAL innodb_cleaner_free_list_lwm = @start_value;
326
327=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test'
328--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test 1970-01-01 00:00:00 +0000
329+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test 2013-09-30 15:25:38 +0000
330@@ -0,0 +1,33 @@
331+--source include/have_debug.inc
332+--source include/have_innodb.inc
333+
334+# A dynamic, global variable
335+
336+SET @start_value = @@GLOBAL.innodb_cleaner_lru_chunk_size;
337+
338+# Default value
339+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
340+
341+# Global only
342+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
343+SELECT @@SESSION.innodb_cleaner_lru_chunk_size;
344+
345+# Correct values
346+SET GLOBAL innodb_cleaner_lru_chunk_size=1;
347+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
348+SET GLOBAL innodb_cleaner_lru_chunk_size=1000;
349+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
350+SET GLOBAL innodb_cleaner_lru_chunk_size=4294967295;
351+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
352+
353+# Incorrect values
354+SET GLOBAL innodb_cleaner_lru_chunk_size=0;
355+SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size;
356+--error ER_WRONG_TYPE_FOR_VAR
357+SET GLOBAL innodb_cleaner_lru_chunk_size=1.1;
358+--error ER_WRONG_TYPE_FOR_VAR
359+SET GLOBAL innodb_cleaner_lru_chunk_size=1e1;
360+--error ER_WRONG_TYPE_FOR_VAR
361+SET GLOBAL innodb_cleaner_lru_chunk_size='foo';
362+
363+SET GLOBAL innodb_cleaner_lru_chunk_size = @start_value;
364
365=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test'
366--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test 1970-01-01 00:00:00 +0000
367+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test 2013-09-30 15:25:38 +0000
368@@ -0,0 +1,31 @@
369+--source include/have_debug.inc
370+--source include/have_innodb.inc
371+
372+# A dynamic, global variable
373+
374+SET @start_value = @@GLOBAL.innodb_cleaner_max_flush_time;
375+
376+# Default value
377+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
378+
379+# Global only
380+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
381+SELECT @@SESSION.innodb_cleaner_max_flush_time;
382+
383+# Correct values
384+SET GLOBAL innodb_cleaner_max_flush_time=0;
385+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
386+SET GLOBAL innodb_cleaner_max_flush_time=1000;
387+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
388+SET GLOBAL innodb_cleaner_max_flush_time=4294967295;
389+SELECT @@GLOBAL.innodb_cleaner_max_flush_time;
390+
391+# Incorrect values
392+--error ER_WRONG_TYPE_FOR_VAR
393+SET GLOBAL innodb_cleaner_max_flush_time=1.1;
394+--error ER_WRONG_TYPE_FOR_VAR
395+SET GLOBAL innodb_cleaner_max_flush_time=1e1;
396+--error ER_WRONG_TYPE_FOR_VAR
397+SET GLOBAL innodb_cleaner_max_flush_time='foo';
398+
399+SET GLOBAL innodb_cleaner_max_flush_time = @start_value;
400
401=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test'
402--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test 1970-01-01 00:00:00 +0000
403+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test 2013-09-30 15:25:38 +0000
404@@ -0,0 +1,31 @@
405+--source include/have_debug.inc
406+--source include/have_innodb.inc
407+
408+# A dynamic, global variable
409+
410+SET @start_value = @@GLOBAL.innodb_cleaner_max_lru_time;
411+
412+# Default value
413+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
414+
415+# Global only
416+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
417+SELECT @@SESSION.innodb_cleaner_max_lru_time;
418+
419+# Correct values
420+SET GLOBAL innodb_cleaner_max_lru_time=0;
421+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
422+SET GLOBAL innodb_cleaner_max_lru_time=1000;
423+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
424+SET GLOBAL innodb_cleaner_max_lru_time=4294967295;
425+SELECT @@GLOBAL.innodb_cleaner_max_lru_time;
426+
427+# Incorrect values
428+--error ER_WRONG_TYPE_FOR_VAR
429+SET GLOBAL innodb_cleaner_max_lru_time=1.1;
430+--error ER_WRONG_TYPE_FOR_VAR
431+SET GLOBAL innodb_cleaner_max_lru_time=1e1;
432+--error ER_WRONG_TYPE_FOR_VAR
433+SET GLOBAL innodb_cleaner_max_lru_time='foo';
434+
435+SET GLOBAL innodb_cleaner_max_lru_time = @start_value;
436
437=== modified file 'Percona-Server/storage/innobase/buf/buf0flu.cc'
438--- Percona-Server/storage/innobase/buf/buf0flu.cc 2013-09-27 14:09:03 +0000
439+++ Percona-Server/storage/innobase/buf/buf0flu.cc 2013-09-30 15:25:38 +0000
440@@ -59,10 +59,6 @@
441 doing the shutdown */
442 UNIV_INTERN ibool buf_page_cleaner_is_active = FALSE;
443
444-/** LRU flush batch is further divided into this chunk size to
445-reduce the wait time for the threads waiting for a clean block */
446-#define PAGE_CLEANER_LRU_BATCH_CHUNK_SIZE 100
447-
448 #ifdef UNIV_PFS_THREAD
449 UNIV_INTERN mysql_pfs_key_t buf_page_cleaner_thread_key;
450 #endif /* UNIV_PFS_THREAD */
451@@ -75,6 +71,12 @@
452
453 /* @} */
454
455+/** Handled page counters for a single flush */
456+struct flush_counters_t {
457+ ulint flushed; /*!< number of dirty pages flushed */
458+ ulint evicted; /*!< number of clean pages evicted */
459+};
460+
461 /******************************************************************//**
462 Increases flush_list size in bytes with zip_size for compressed page,
463 UNIV_PAGE_SIZE for uncompressed page in inline function */
464@@ -1508,32 +1510,50 @@
465 it is a best effort attempt and it is not guaranteed that after a call
466 to this function there will be 'max' blocks in the free list.
467 @return number of blocks for which the write request was queued. */
468+__attribute__((nonnull))
469 static
470 ulint
471 buf_flush_LRU_list_batch(
472 /*=====================*/
473 buf_pool_t* buf_pool, /*!< in: buffer pool instance */
474- ulint max) /*!< in: desired number of
475+ ulint max, /*!< in: desired number of
476 blocks in the free_list */
477+ bool limited_scan, /*!< in: if true, allow to scan only up
478+ to srv_LRU_scan_depth pages in total */
479+ flush_counters_t* n) /*!< out: flushed/evicted page
480+ counts */
481 {
482 buf_page_t* bpage;
483 ulint scanned = 0;
484+ ulint lru_position = 0;
485+ ulint max_lru_position;
486+ ulint max_scanned_pages;
487 ulint count = 0;
488 ulint free_len = UT_LIST_GET_LEN(buf_pool->free);
489 ulint lru_len = UT_LIST_GET_LEN(buf_pool->LRU);
490
491+ n->flushed = 0;
492+ n->evicted = 0;
493+
494 ut_ad(mutex_own(&buf_pool->LRU_list_mutex));
495
496+ max_scanned_pages = limited_scan ? srv_LRU_scan_depth : lru_len * max;
497+ max_lru_position = ut_min(srv_LRU_scan_depth, lru_len);
498+
499 bpage = UT_LIST_GET_LAST(buf_pool->LRU);
500- while (bpage != NULL && count < max
501+ while (bpage != NULL
502+ && (srv_cleaner_eviction_factor ? n->evicted : n->flushed) < max
503 && free_len < srv_LRU_scan_depth
504- && lru_len > BUF_LRU_MIN_LEN) {
505+ && lru_len > BUF_LRU_MIN_LEN
506+ && lru_position < max_lru_position
507+ && scanned < max_scanned_pages) {
508
509 ib_mutex_t* block_mutex = buf_page_get_mutex(bpage);
510 ibool evict;
511 ulint failed_acquire;
512
513 ++scanned;
514+ ++lru_position;
515
516 failed_acquire = mutex_enter_nowait(block_mutex);
517
518@@ -1560,6 +1580,8 @@
519 if (buf_LRU_free_page(bpage, true)) {
520
521 mutex_exit(block_mutex);
522+ n->evicted++;
523+ lru_position = 0;
524 mutex_enter(&buf_pool->LRU_list_mutex);
525 bpage = UT_LIST_GET_LAST(buf_pool->LRU);
526 } else {
527@@ -1573,6 +1595,9 @@
528 bpage,
529 BUF_FLUSH_LRU, max, &count)) {
530
531+ n->flushed += count;
532+ lru_position = 0;
533+
534 /* LRU list mutex was released.
535 Restart the scan. */
536 bpage = UT_LIST_GET_LAST(buf_pool->LRU);
537@@ -1610,13 +1635,18 @@
538 @return number of blocks for which either the write request was queued
539 or in case of unzip_LRU the number of blocks actually moved to the
540 free list */
541+__attribute__((nonnull))
542 static
543-ulint
544+void
545 buf_do_LRU_batch(
546 /*=============*/
547 buf_pool_t* buf_pool, /*!< in: buffer pool instance */
548- ulint max) /*!< in: desired number of
549+ ulint max, /*!< in: desired number of
550 blocks in the free_list */
551+ bool limited_scan, /*!< in: if true, allow to scan only up
552+ to srv_LRU_scan_depth pages in total */
553+ flush_counters_t* n) /*!< out: flushed/evicted page
554+ counts */
555 {
556 ulint count = 0;
557
558@@ -1625,10 +1655,14 @@
559 }
560
561 if (max > count) {
562- count += buf_flush_LRU_list_batch(buf_pool, max - count);
563+ buf_flush_LRU_list_batch(buf_pool, max - count, limited_scan,
564+ n);
565+ } else {
566+ n->evicted = 0;
567+ n->flushed = 0;
568 }
569
570- return(count);
571+ n->evicted += count;
572 }
573
574 /*******************************************************************//**
575@@ -1718,8 +1752,9 @@
576 end up waiting for these latches! NOTE 2: in the case of a flush list flush,
577 the calling thread is not allowed to own any latches on pages!
578 @return number of blocks for which the write request was queued */
579+__attribute__((nonnull))
580 static
581-ulint
582+void
583 buf_flush_batch(
584 /*============*/
585 buf_pool_t* buf_pool, /*!< in: buffer pool instance */
586@@ -1730,14 +1765,17 @@
587 ulint min_n, /*!< in: wished minimum mumber of blocks
588 flushed (it is not guaranteed that the
589 actual number is that big, though) */
590- lsn_t lsn_limit) /*!< in: in the case of BUF_FLUSH_LIST
591+ lsn_t lsn_limit, /*!< in: in the case of BUF_FLUSH_LIST
592 all blocks whose oldest_modification is
593 smaller than this should be flushed
594 (if their number does not exceed
595 min_n), otherwise ignored */
596+ bool limited_lru_scan,/*!< in: for LRU flushes, if true,
597+ allow to scan only up to
598+ srv_LRU_scan_depth pages in total */
599+ flush_counters_t* n) /*!< out: flushed/evicted page
600+ counts */
601 {
602- ulint count = 0;
603-
604 ut_ad(flush_type == BUF_FLUSH_LRU || flush_type == BUF_FLUSH_LIST);
605 #ifdef UNIV_SYNC_DEBUG
606 ut_ad((flush_type != BUF_FLUSH_LIST)
607@@ -1749,26 +1787,27 @@
608 switch (flush_type) {
609 case BUF_FLUSH_LRU:
610 mutex_enter(&buf_pool->LRU_list_mutex);
611- count = buf_do_LRU_batch(buf_pool, min_n);
612+ buf_do_LRU_batch(buf_pool, min_n, limited_lru_scan, n);
613 mutex_exit(&buf_pool->LRU_list_mutex);
614 break;
615 case BUF_FLUSH_LIST:
616- count = buf_do_flush_list_batch(buf_pool, min_n, lsn_limit);
617+ ut_ad(!limited_lru_scan);
618+ n->flushed = buf_do_flush_list_batch(buf_pool, min_n,
619+ lsn_limit);
620+ n->evicted = 0;
621 break;
622 default:
623 ut_error;
624 }
625
626 #ifdef UNIV_DEBUG
627- if (buf_debug_prints && count > 0) {
628+ if (buf_debug_prints && n->flushed > 0) {
629 fprintf(stderr, flush_type == BUF_FLUSH_LRU
630 ? "Flushed %lu pages in LRU flush\n"
631 : "Flushed %lu pages in flush list flush\n",
632- (ulong) count);
633+ (ulong) n->flushed);
634 }
635 #endif /* UNIV_DEBUG */
636-
637- return(count);
638 }
639
640 /******************************************************************//**
641@@ -1780,7 +1819,9 @@
642 buf_flush_t flush_type, /*!< in: type of flush */
643 ulint page_count) /*!< in: number of pages flushed */
644 {
645- buf_dblwr_flush_buffered_writes();
646+ if (page_count) {
647+ buf_dblwr_flush_buffered_writes();
648+ }
649
650 ut_a(flush_type == BUF_FLUSH_LRU || flush_type == BUF_FLUSH_LIST);
651
652@@ -1889,6 +1930,7 @@
653 NOTE: The calling thread is not allowed to own any latches on pages!
654 @return true if a batch was queued successfully. false if another batch
655 of same type was already running. */
656+__attribute__((nonnull))
657 static
658 bool
659 buf_flush_LRU(
660@@ -1897,29 +1939,23 @@
661 ulint min_n, /*!< in: wished minimum mumber of blocks
662 flushed (it is not guaranteed that the
663 actual number is that big, though) */
664- ulint* n_processed) /*!< out: the number of pages
665- which were processed is passed
666- back to caller. Ignored if NULL */
667+ bool limited_scan, /*!< in: if true, allow to scan
668+ only up to srv_LRU_scan_depth
669+ pages in total */
670+ flush_counters_t *n) /*!< out: flushed/evicted page
671+ counts */
672 {
673- ulint page_count;
674-
675- if (n_processed) {
676- *n_processed = 0;
677- }
678-
679 if (!buf_flush_start(buf_pool, BUF_FLUSH_LRU)) {
680+ n->flushed = 0;
681+ n->evicted = 0;
682 return(false);
683 }
684
685- page_count = buf_flush_batch(buf_pool, BUF_FLUSH_LRU, min_n, 0);
686+ buf_flush_batch(buf_pool, BUF_FLUSH_LRU, min_n, 0, limited_scan, n);
687
688 buf_flush_end(buf_pool, BUF_FLUSH_LRU);
689
690- buf_flush_common(BUF_FLUSH_LRU, page_count);
691-
692- if (n_processed) {
693- *n_processed = page_count;
694- }
695+ buf_flush_common(BUF_FLUSH_LRU, n->flushed);
696
697 return(true);
698 }
699@@ -1949,7 +1985,17 @@
700
701 {
702 ulint i;
703- bool success = true;
704+
705+ ulint requested_pages[MAX_BUFFER_POOLS];
706+ bool active_instance[MAX_BUFFER_POOLS];
707+ ulint remaining_instances = srv_buf_pool_instances;
708+ bool timeout = false;
709+ ulint flush_start_time = 0;
710+
711+ for (i = 0; i < srv_buf_pool_instances; i++) {
712+ requested_pages[i] = 0;
713+ active_instance[i] = true;
714+ }
715
716 if (n_processed) {
717 *n_processed = 0;
718@@ -1962,52 +2008,81 @@
719 so no limit here. */
720 min_n = (min_n + srv_buf_pool_instances - 1)
721 / srv_buf_pool_instances;
722+ flush_start_time = ut_time_ms();
723 }
724
725 /* Flush to lsn_limit in all buffer pool instances */
726+ while (remaining_instances && !timeout) {
727+
728+ for (i = 0; i < srv_buf_pool_instances; i++) {
729+
730+ if (flush_start_time
731+ && (ut_time_ms() - flush_start_time
732+ >= srv_cleaner_max_flush_time)) {
733+
734+ timeout = true;
735+ break;
736+ }
737+
738+ if (active_instance[i]) {
739+
740+ buf_pool_t* buf_pool;
741+ ulint chunk_size;
742+ flush_counters_t n;
743+
744+ chunk_size = ut_min(
745+ srv_cleaner_flush_chunk_size,
746+ min_n - requested_pages[i]);
747+
748+ buf_pool = buf_pool_from_array(i);
749+
750+ if (!buf_flush_start(buf_pool,
751+ BUF_FLUSH_LIST)) {
752+
753+ continue;
754+ }
755+
756+ buf_flush_batch(buf_pool, BUF_FLUSH_LIST,
757+ chunk_size, lsn_limit, false,
758+ &n);
759+
760+ buf_flush_end(buf_pool, BUF_FLUSH_LIST);
761+
762+ buf_flush_common(BUF_FLUSH_LIST, n.flushed);
763+
764+ if (n_processed) {
765+ *n_processed += n.flushed;
766+ }
767+
768+ requested_pages[i] += chunk_size;
769+
770+ if (requested_pages[i] >= min_n
771+ || !n.flushed) {
772+
773+ active_instance[i] = false;
774+ remaining_instances--;
775+ }
776+
777+ if (n.flushed) {
778+ MONITOR_INC_VALUE_CUMULATIVE(
779+ MONITOR_FLUSH_BATCH_TOTAL_PAGE,
780+ MONITOR_FLUSH_BATCH_COUNT,
781+ MONITOR_FLUSH_BATCH_PAGES,
782+ n.flushed);
783+ }
784+ }
785+ }
786+ }
787+
788+ /* If we haven't flushed all the instances due to timeout or a repeat
789+ failure to start a flush, return failure */
790 for (i = 0; i < srv_buf_pool_instances; i++) {
791- buf_pool_t* buf_pool;
792- ulint page_count = 0;
793-
794- buf_pool = buf_pool_from_array(i);
795-
796- if (!buf_flush_start(buf_pool, BUF_FLUSH_LIST)) {
797- /* We have two choices here. If lsn_limit was
798- specified then skipping an instance of buffer
799- pool means we cannot guarantee that all pages
800- up to lsn_limit has been flushed. We can
801- return right now with failure or we can try
802- to flush remaining buffer pools up to the
803- lsn_limit. We attempt to flush other buffer
804- pools based on the assumption that it will
805- help in the retry which will follow the
806- failure. */
807- success = false;
808-
809- continue;
810- }
811-
812- page_count = buf_flush_batch(
813- buf_pool, BUF_FLUSH_LIST, min_n, lsn_limit);
814-
815- buf_flush_end(buf_pool, BUF_FLUSH_LIST);
816-
817- buf_flush_common(BUF_FLUSH_LIST, page_count);
818-
819- if (n_processed) {
820- *n_processed += page_count;
821- }
822-
823- if (page_count) {
824- MONITOR_INC_VALUE_CUMULATIVE(
825- MONITOR_FLUSH_BATCH_TOTAL_PAGE,
826- MONITOR_FLUSH_BATCH_COUNT,
827- MONITOR_FLUSH_BATCH_PAGES,
828- page_count);
829+ if (active_instance[i]) {
830+ return(false);
831 }
832 }
833
834- return(success);
835+ return(true);
836 }
837
838 /******************************************************************//**
839@@ -2117,47 +2192,94 @@
840 /*====================*/
841 {
842 ulint total_flushed = 0;
843+ ulint start_time = ut_time_ms();
844+ ulint scan_depth[MAX_BUFFER_POOLS];
845+ ulint requested_pages[MAX_BUFFER_POOLS];
846+ bool active_instance[MAX_BUFFER_POOLS];
847+ bool limited_scan[MAX_BUFFER_POOLS];
848+ ulint previous_evicted[MAX_BUFFER_POOLS];
849+ ulint remaining_instances = srv_buf_pool_instances;
850+ ulint lru_chunk_size = srv_cleaner_lru_chunk_size;
851+ ulint free_list_lwm = srv_LRU_scan_depth / 100
852+ * srv_cleaner_free_list_lwm;
853+ bool timeout = false;
854
855 for (ulint i = 0; i < srv_buf_pool_instances; i++) {
856
857- buf_pool_t* buf_pool = buf_pool_from_array(i);
858- ulint scan_depth;
859-
860- /* srv_LRU_scan_depth can be arbitrarily large value.
861- We cap it with current LRU size. */
862- scan_depth = UT_LIST_GET_LEN(buf_pool->LRU);
863-
864- scan_depth = ut_min(srv_LRU_scan_depth, scan_depth);
865-
866- /* We divide LRU flush into smaller chunks because
867- there may be user threads waiting for the flush to
868- end in buf_LRU_get_free_block(). */
869- for (ulint j = 0;
870- j < scan_depth;
871- j += PAGE_CLEANER_LRU_BATCH_CHUNK_SIZE) {
872-
873- ulint n_flushed = 0;
874-
875- /* Currently page_cleaner is the only thread
876- that can trigger an LRU flush. It is possible
877- that a batch triggered during last iteration is
878- still running, */
879- if (buf_flush_LRU(buf_pool,
880- PAGE_CLEANER_LRU_BATCH_CHUNK_SIZE,
881- &n_flushed)) {
882-
883- /* Allowed only one batch per
884- buffer pool instance. */
885- buf_flush_wait_batch_end(
886- buf_pool, BUF_FLUSH_LRU);
887- }
888-
889- if (n_flushed) {
890- total_flushed += n_flushed;
891- } else {
892- /* Nothing to flush */
893- break;
894- }
895+ const buf_pool_t* buf_pool = buf_pool_from_array(i);
896+
897+ scan_depth[i] = ut_min(srv_LRU_scan_depth,
898+ UT_LIST_GET_LEN(buf_pool->LRU));
899+ requested_pages[i] = 0;
900+ active_instance[i] = true;
901+ limited_scan[i] = true;
902+ previous_evicted[i] = 0;
903+ }
904+
905+ while (remaining_instances && !timeout) {
906+
907+ for (ulint i = 0; i < srv_buf_pool_instances && !timeout;
908+ i++) {
909+
910+ if (!active_instance[i]) {
911+ continue;
912+ }
913+
914+ ulint free_len = free_list_lwm;
915+ buf_pool_t* buf_pool = buf_pool_from_array(i);
916+
917+ do {
918+ flush_counters_t n;
919+
920+ if (ut_time_ms() - start_time
921+ >= srv_cleaner_max_lru_time) {
922+
923+ timeout = true;
924+ break;
925+ }
926+
927+ ut_ad(requested_pages[i] <= scan_depth[i]);
928+
929+ /* Currently page_cleaner is the only thread
930+ that can trigger an LRU flush. It is possible
931+ that a batch triggered during last iteration is
932+ still running, */
933+ if (buf_flush_LRU(buf_pool, lru_chunk_size,
934+ limited_scan[i], &n)) {
935+
936+ /* Allowed only one batch per
937+ buffer pool instance. */
938+ buf_flush_wait_batch_end(
939+ buf_pool, BUF_FLUSH_LRU);
940+ } else {
941+
942+ total_flushed += n.flushed;
943+
944+ /* When we evict less pages than we did
945+ on a previous try we relax the LRU scan
946+ limit in order to attempt to evict
947+ more */
948+ limited_scan[i]
949+ = (previous_evicted[i]
950+ > n.evicted);
951+ previous_evicted[i] = n.evicted;
952+
953+ requested_pages[i] += lru_chunk_size;
954+ }
955+
956+ if (requested_pages[i] >= scan_depth[i]
957+ || (srv_cleaner_eviction_factor
958+ ? n.evicted : n.flushed)) {
959+
960+ active_instance[i] = false;
961+ remaining_instances--;
962+ } else {
963+
964+ free_len = UT_LIST_GET_LEN(
965+ buf_pool->free);
966+ }
967+ } while (active_instance[i]
968+ && free_len <= free_list_lwm);
969 }
970 }
971
972@@ -2421,6 +2543,80 @@
973 }
974 }
975
976+/*********************************************************************//**
977+Returns the aggregate free list length over all buffer pool instances.
978+@return total free list length. */
979+__attribute__((warn_unused_result))
980+static
981+ulint
982+buf_get_total_free_list_length(void)
983+/*================================*/
984+{
985+ ulint result = 0;
986+
987+ for (ulint i = 0; i < srv_buf_pool_instances; i++) {
988+
989+ result += UT_LIST_GET_LEN(buf_pool_from_array(i)->free);
990+ }
991+
992+ return result;
993+}
994+
995+/*********************************************************************//**
996+Adjust the desired page cleaner thread sleep time for LRU flushes. */
997+__attribute__((nonnull))
998+static
999+void
1000+page_cleaner_adapt_lru_sleep_time(
1001+/*==============================*/
1002+ ulint* lru_sleep_time) /*!< in/out: desired page cleaner thread sleep
1003+ time for LRU flushes */
1004+{
1005+ ulint free_len = buf_get_total_free_list_length();
1006+ ulint max_free_len = srv_LRU_scan_depth * srv_buf_pool_instances;
1007+
1008+ if (free_len < max_free_len / 100) {
1009+
1010+ /* Free lists filled less than 1%, no sleep */
1011+ *lru_sleep_time = 0;
1012+ } else if (free_len > max_free_len / 5) {
1013+
1014+ /* Free lists filled more than 20%, sleep a bit more */
1015+ *lru_sleep_time += 50;
1016+ if (*lru_sleep_time > srv_cleaner_max_lru_time)
1017+ *lru_sleep_time = srv_cleaner_max_lru_time;
1018+ } else if (free_len < max_free_len / 20 && *lru_sleep_time >= 50) {
1019+
1020+ /* Free lists filled less than 5%, sleep a bit less */
1021+ *lru_sleep_time -= 50;
1022+ } else {
1023+
1024+ /* Free lists filled between 5% and 20%, no change */
1025+ }
1026+}
1027+
1028+/*********************************************************************//**
1029+Get the desired page cleaner thread sleep time for flush list flushes.
1030+@return desired sleep time */
1031+__attribute__((warn_unused_result))
1032+static
1033+ulint
1034+page_cleaner_adapt_flush_sleep_time(void)
1035+/*=====================================*/
1036+{
1037+ lsn_t age = log_get_lsn() - log_sys->last_checkpoint_lsn;
1038+
1039+ if (age > log_sys->max_modified_age_sync) {
1040+
1041+ /* No sleep if in sync preflush zone */
1042+ return(0);
1043+ }
1044+
1045+ /* In all other cases flush list factors do not influence the page
1046+ cleaner sleep time */
1047+ return(srv_cleaner_max_flush_time);
1048+}
1049+
1050 /******************************************************************//**
1051 page_cleaner thread tasked with flushing dirty pages from the buffer
1052 pools. As of now we'll have only one instance of this thread.
1053@@ -2436,6 +2632,7 @@
1054 ulint next_loop_time = ut_time_ms() + 1000;
1055 ulint n_flushed = 0;
1056 ulint last_activity = srv_get_activity_count();
1057+ ulint lru_sleep_time = srv_cleaner_max_lru_time;
1058
1059 ut_ad(!srv_read_only_mode);
1060
1061@@ -2452,6 +2649,9 @@
1062
1063 while (srv_shutdown_state == SRV_SHUTDOWN_NONE) {
1064
1065+ ulint flush_sleep_time;
1066+ ulint page_cleaner_sleep_time;
1067+
1068 srv_current_thread_priority = srv_cleaner_thread_priority;
1069
1070 /* The page_cleaner skips sleep if the server is
1071@@ -2463,7 +2663,14 @@
1072 page_cleaner_sleep_if_needed(next_loop_time);
1073 }
1074
1075- next_loop_time = ut_time_ms() + 1000;
1076+ page_cleaner_adapt_lru_sleep_time(&lru_sleep_time);
1077+
1078+ flush_sleep_time = page_cleaner_adapt_flush_sleep_time();
1079+
1080+ page_cleaner_sleep_time = ut_min(lru_sleep_time,
1081+ flush_sleep_time);
1082+
1083+ next_loop_time = ut_time_ms() + page_cleaner_sleep_time;
1084
1085 if (srv_check_activity(last_activity)) {
1086 last_activity = srv_get_activity_count();
1087
1088=== modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc'
1089--- Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-09-27 14:09:03 +0000
1090+++ Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-09-30 15:25:38 +0000
1091@@ -16574,6 +16574,42 @@
1092
1093 #endif /* UNIV_LINUX */
1094
1095+static MYSQL_SYSVAR_ULONG(cleaner_max_lru_time, srv_cleaner_max_lru_time,
1096+ PLUGIN_VAR_RQCMDARG,
1097+ "The maximum time limit for a single LRU tail flush iteration by the page "
1098+ "cleaner thread in miliseconds",
1099+ NULL, NULL, 1000, 0, ~0UL, 0);
1100+
1101+static MYSQL_SYSVAR_ULONG(cleaner_max_flush_time, srv_cleaner_max_flush_time,
1102+ PLUGIN_VAR_RQCMDARG,
1103+ "The maximum time limit for a single flush list flush iteration by the page "
1104+ "cleaner thread in miliseconds",
1105+ NULL, NULL, 1000, 0, ~0UL, 0);
1106+
1107+static MYSQL_SYSVAR_ULONG(cleaner_flush_chunk_size,
1108+ srv_cleaner_flush_chunk_size,
1109+ PLUGIN_VAR_RQCMDARG,
1110+ "Divide page cleaner flush list flush batches into chunks of this size",
1111+ NULL, NULL, 200, 1, ~0UL, 0);
1112+
1113+static MYSQL_SYSVAR_ULONG(cleaner_lru_chunk_size,
1114+ srv_cleaner_lru_chunk_size,
1115+ PLUGIN_VAR_RQCMDARG,
1116+ "Divide page cleaner LRU list flush batches into chunks of this size",
1117+ NULL, NULL, 100, 1, ~0UL, 0);
1118+
1119+static MYSQL_SYSVAR_ULONG(cleaner_free_list_lwm, srv_cleaner_free_list_lwm,
1120+ PLUGIN_VAR_RQCMDARG,
1121+ "Page cleaner will keep on flushing the same buffer pool instance if its "
1122+ "free list length is below this percentage of innodb_lru_scan_depth",
1123+ NULL, NULL, 10, 0, 100, 0);
1124+
1125+static MYSQL_SYSVAR_BOOL(cleaner_eviction_factor, srv_cleaner_eviction_factor,
1126+ PLUGIN_VAR_OPCMDARG,
1127+ "Make page cleaner LRU flushes use evicted instead of flushed page counts "
1128+ "for its heuristics",
1129+ NULL, NULL, FALSE);
1130+
1131 #endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */
1132
1133 static MYSQL_SYSVAR_LONG(buffer_pool_instances, innobase_buffer_pool_instances,
1134@@ -17241,6 +17277,12 @@
1135 MYSQL_SYSVAR(priority_cleaner),
1136 MYSQL_SYSVAR(priority_master),
1137 #endif /* UNIV_LINUX */
1138+ MYSQL_SYSVAR(cleaner_max_lru_time),
1139+ MYSQL_SYSVAR(cleaner_max_flush_time),
1140+ MYSQL_SYSVAR(cleaner_flush_chunk_size),
1141+ MYSQL_SYSVAR(cleaner_lru_chunk_size),
1142+ MYSQL_SYSVAR(cleaner_free_list_lwm),
1143+ MYSQL_SYSVAR(cleaner_eviction_factor),
1144 #endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */
1145 MYSQL_SYSVAR(print_all_deadlocks),
1146 MYSQL_SYSVAR(cmp_per_index_enabled),
1147
1148=== modified file 'Percona-Server/storage/innobase/include/srv0srv.h'
1149--- Percona-Server/storage/innobase/include/srv0srv.h 2013-09-27 14:09:03 +0000
1150+++ Percona-Server/storage/innobase/include/srv0srv.h 2013-09-30 15:25:38 +0000
1151@@ -309,6 +309,35 @@
1152 extern ulint srv_mem_pool_size;
1153 extern ulint srv_lock_table_size;
1154
1155+extern ulint srv_cleaner_max_lru_time;/*!< the maximum time limit for a
1156+ single LRU tail flush iteration by the
1157+ page cleaner thread */
1158+
1159+extern ulint srv_cleaner_max_flush_time;/*!< the maximum time limit for a
1160+ single flush list flush iteration by
1161+ the page cleaner thread */
1162+
1163+extern ulint srv_cleaner_flush_chunk_size;
1164+ /*!< page cleaner flush list flush
1165+ batches are further divided into this
1166+ chunk size */
1167+
1168+extern ulint srv_cleaner_lru_chunk_size;
1169+ /*!< page cleaner LRU list flush
1170+ batches are further divided into this
1171+ chunk size */
1172+
1173+extern ulint srv_cleaner_free_list_lwm;/*!< if free list length is lower
1174+ than this percentage of
1175+ srv_LRU_scan_depth, page cleaner LRU
1176+ flushes will issue flush batches to the
1177+ same instance in a row */
1178+
1179+extern my_bool srv_cleaner_eviction_factor;
1180+ /*!< if TRUE, page cleaner heuristics
1181+ use evicted instead of flushed page
1182+ counts for its heuristics */
1183+
1184 extern ulint srv_n_file_io_threads;
1185 extern my_bool srv_random_read_ahead;
1186 extern ulong srv_read_ahead_threshold;
1187
1188=== modified file 'Percona-Server/storage/innobase/srv/srv0srv.cc'
1189--- Percona-Server/storage/innobase/srv/srv0srv.cc 2013-09-27 14:09:03 +0000
1190+++ Percona-Server/storage/innobase/srv/srv0srv.cc 2013-09-30 15:25:38 +0000
1191@@ -261,6 +261,31 @@
1192 UNIV_INTERN ulint srv_mem_pool_size = ULINT_MAX;
1193 UNIV_INTERN ulint srv_lock_table_size = ULINT_MAX;
1194
1195+/** The maximum time limit for a single LRU tail flush iteration by the page
1196+cleaner thread */
1197+UNIV_INTERN ulint srv_cleaner_max_lru_time = 1000;
1198+
1199+/** The maximum time limit for a single flush list flush iteration by the page
1200+cleaner thread */
1201+UNIV_INTERN ulint srv_cleaner_max_flush_time = 1000;
1202+
1203+/** Page cleaner flush list flush batches are further divided into this chunk
1204+size */
1205+UNIV_INTERN ulint srv_cleaner_flush_chunk_size = 100;
1206+
1207+/** Page cleaner LRU list flush batches are further divided into this chunk
1208+size */
1209+UNIV_INTERN ulint srv_cleaner_lru_chunk_size = 100;
1210+
1211+/** If free list length is lower than this percentage of srv_LRU_scan_depth,
1212+page cleaner LRU flushes will issue flush batches to the same instance in a
1213+row */
1214+UNIV_INTERN ulint srv_cleaner_free_list_lwm = 10;
1215+
1216+/** If TRUE, page cleaner heuristics use evicted instead of flushed page counts
1217+for its heuristics */
1218+UNIV_INTERN my_bool srv_cleaner_eviction_factor = FALSE;
1219+
1220 /* This parameter is deprecated. Use srv_n_io_[read|write]_threads
1221 instead. */
1222 UNIV_INTERN ulint srv_n_file_io_threads = ULINT_MAX;

Subscribers

People subscribed via source and target branches