Merge lp:~laurynas-biveinis/percona-server/xtradb-cleaner-tuning-5.6 into lp:percona-server/5.6
- xtradb-cleaner-tuning-5.6
- Merge into 5.6
Status: | Merged | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Approved by: | Vadim Tkachenko | ||||||||||||
Approved revision: | no longer in the source branch. | ||||||||||||
Merged at revision: | 445 | ||||||||||||
Proposed branch: | lp:~laurynas-biveinis/percona-server/xtradb-cleaner-tuning-5.6 | ||||||||||||
Merge into: | lp:percona-server/5.6 | ||||||||||||
Diff against target: |
1222 lines (+795/-116) 16 files modified
Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result (+31/-0) Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result (+31/-0) Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result (+35/-0) Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result (+31/-0) Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result (+25/-0) Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result (+25/-0) Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test (+35/-0) Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test (+33/-0) Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test (+35/-0) Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test (+33/-0) Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test (+31/-0) Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test (+31/-0) Percona-Server/storage/innobase/buf/buf0flu.cc (+323/-116) Percona-Server/storage/innobase/handler/ha_innodb.cc (+42/-0) Percona-Server/storage/innobase/include/srv0srv.h (+29/-0) Percona-Server/storage/innobase/srv/srv0srv.cc (+25/-0) |
||||||||||||
To merge this branch: | bzr merge lp:~laurynas-biveinis/percona-server/xtradb-cleaner-tuning-5.6 | ||||||||||||
Related bugs: |
|
||||||||||||
Related blueprints: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Sergei Glushchenko (community) | g2 | Approve | |
Vadim Tkachenko | algo | Approve | |
Review via email: mp+188368@code.launchpad.net |
Commit message
Description of the change
Tune page cleaner thread, implementing
https:/
Create new type, local tu buf0flu.cc, struct flush_counters_t, that
has separate flushed and evicted page counters. Adjust
buf_flush_
buf_flush_LRU() to return it.
In buf_flush_
limits the total number of scanned pages to srv_LRU_scan_depth, limit
scans to not go deeper than srv_LRU_scan_depth in the LRU list.
In buf_flush_common(), only call buf_dblwr_
there have been flushed pages, fixing
http://
is lazy" | buf_flush_common() calls buf_dblwr_
even when no pages flushed).
Rewrite buf_flush_list() to divide flush requests into chunks that are
issued to buffer pool instances in parallel and limit total running
time to a new UNIV_PERF_
innodb_
size is specified by a new UNIV_PERF_
innodb_
Rewrite buf_flush_
governed by a new UNIV_PERF_
innodb_
pool instances in parallel, with the exception of buffer pool
instances with almost-empty free lists (as defined by
UNIV_PERF_
which batches are issued sequentially. Make the chunk size tunable by
a new UNIV_PERF_
default 100.
The timeouts in buf_flush_list() and buf_flush_
http://
timeouts to page cleaner flushes).
Add new functions buf_get_
buf_cleaner_
page_cleaner_
time reduction that may reduce page cleaner iteration time below 1
second and thus implement furious flushing, fixing
http://
furious flushing for 5.6).
The adaptiveness is implemented as follows. Maintain a desired target
sleep time for LRU flushes. If the total length of all buffer pool
instances is <1%, no sleep; if <5%, sleep 50 ms less, if between 5%
and 20%, no change; if >20%, sleep 50ms longer. Then, if the
checkpoint age is in preflush sync zone, set the sleep time to zero.
Add a new UNIV_PERF_
innodb_
LRU tail flushing to use evicted instead of flushed page counts for
its heuristics.
Add sys_vars tests for these variables.
Laurynas Biveinis (laurynas-biveinis) wrote : | # |
Vadim Tkachenko (vadim-tk) wrote : | # |
Based on my talks with Alexey Stroganov
I approve this from algo point of view.
I leave tests/code style G2 review to Sergei_gl
Sergei Glushchenko (sergei.glushchenko) wrote : | # |
Approve
Preview Diff
1 | === added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result' |
2 | --- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result 1970-01-01 00:00:00 +0000 |
3 | +++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_eviction_factor_basic.result 2013-09-30 15:25:38 +0000 |
4 | @@ -0,0 +1,31 @@ |
5 | +SET @start_value = @@GLOBAL.innodb_cleaner_eviction_factor; |
6 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
7 | +@@GLOBAL.innodb_cleaner_eviction_factor |
8 | +0 |
9 | +SELECT @@SESSION.innodb_cleaner_eviction_factor; |
10 | +ERROR HY000: Variable 'innodb_cleaner_eviction_factor' is a GLOBAL variable |
11 | +SET GLOBAL innodb_cleaner_eviction_factor='OFF'; |
12 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
13 | +@@GLOBAL.innodb_cleaner_eviction_factor |
14 | +0 |
15 | +SET GLOBAL innodb_cleaner_eviction_factor='ON'; |
16 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
17 | +@@GLOBAL.innodb_cleaner_eviction_factor |
18 | +1 |
19 | +SET GLOBAL innodb_cleaner_eviction_factor=0; |
20 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
21 | +@@GLOBAL.innodb_cleaner_eviction_factor |
22 | +0 |
23 | +SET GLOBAL innodb_cleaner_eviction_factor=1; |
24 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
25 | +@@GLOBAL.innodb_cleaner_eviction_factor |
26 | +1 |
27 | +SET GLOBAL innodb_cleaner_eviction_factor=1.1; |
28 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_eviction_factor' |
29 | +SET GLOBAL innodb_cleaner_eviction_factor=1e1; |
30 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_eviction_factor' |
31 | +SET GLOBAL innodb_cleaner_eviction_factor=2; |
32 | +ERROR 42000: Variable 'innodb_cleaner_eviction_factor' can't be set to the value of '2' |
33 | +SET GLOBAL innodb_cleaner_eviction_factor='foo'; |
34 | +ERROR 42000: Variable 'innodb_cleaner_eviction_factor' can't be set to the value of 'foo' |
35 | +SET GLOBAL innodb_cleaner_eviction_factor = @start_value; |
36 | |
37 | === added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result' |
38 | --- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result 1970-01-01 00:00:00 +0000 |
39 | +++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_flush_chunk_size_basic.result 2013-09-30 15:25:38 +0000 |
40 | @@ -0,0 +1,31 @@ |
41 | +SET @start_value = @@GLOBAL.innodb_cleaner_flush_chunk_size; |
42 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
43 | +@@GLOBAL.innodb_cleaner_flush_chunk_size |
44 | +200 |
45 | +SELECT @@SESSION.innodb_cleaner_flush_chunk_size; |
46 | +ERROR HY000: Variable 'innodb_cleaner_flush_chunk_size' is a GLOBAL variable |
47 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1; |
48 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
49 | +@@GLOBAL.innodb_cleaner_flush_chunk_size |
50 | +1 |
51 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1000; |
52 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
53 | +@@GLOBAL.innodb_cleaner_flush_chunk_size |
54 | +1000 |
55 | +SET GLOBAL innodb_cleaner_flush_chunk_size=4294967295; |
56 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
57 | +@@GLOBAL.innodb_cleaner_flush_chunk_size |
58 | +4294967295 |
59 | +SET GLOBAL innodb_cleaner_flush_chunk_size=0; |
60 | +Warnings: |
61 | +Warning 1292 Truncated incorrect innodb_cleaner_flush_chunk_size value: '0' |
62 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
63 | +@@GLOBAL.innodb_cleaner_flush_chunk_size |
64 | +1 |
65 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1.1; |
66 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_flush_chunk_size' |
67 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1e1; |
68 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_flush_chunk_size' |
69 | +SET GLOBAL innodb_cleaner_flush_chunk_size='foo'; |
70 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_flush_chunk_size' |
71 | +SET GLOBAL innodb_cleaner_flush_chunk_size = @start_value; |
72 | |
73 | === added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result' |
74 | --- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result 1970-01-01 00:00:00 +0000 |
75 | +++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_free_list_lwm_basic.result 2013-09-30 15:25:38 +0000 |
76 | @@ -0,0 +1,35 @@ |
77 | +SET @start_value = @@GLOBAL.innodb_cleaner_free_list_lwm; |
78 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
79 | +@@GLOBAL.innodb_cleaner_free_list_lwm |
80 | +10 |
81 | +SELECT @@SESSION.innodb_cleaner_free_list_lwm; |
82 | +ERROR HY000: Variable 'innodb_cleaner_free_list_lwm' is a GLOBAL variable |
83 | +SET GLOBAL innodb_cleaner_free_list_lwm=0; |
84 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
85 | +@@GLOBAL.innodb_cleaner_free_list_lwm |
86 | +0 |
87 | +SET GLOBAL innodb_cleaner_free_list_lwm=1; |
88 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
89 | +@@GLOBAL.innodb_cleaner_free_list_lwm |
90 | +1 |
91 | +SET GLOBAL innodb_cleaner_free_list_lwm=99; |
92 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
93 | +@@GLOBAL.innodb_cleaner_free_list_lwm |
94 | +99 |
95 | +SET GLOBAL innodb_cleaner_free_list_lwm=100; |
96 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
97 | +@@GLOBAL.innodb_cleaner_free_list_lwm |
98 | +100 |
99 | +SET GLOBAL innodb_cleaner_free_list_lwm=101; |
100 | +Warnings: |
101 | +Warning 1292 Truncated incorrect innodb_cleaner_free_list_lwm value: '101' |
102 | +SELECT @@innodb_cleaner_free_list_lwm; |
103 | +@@innodb_cleaner_free_list_lwm |
104 | +100 |
105 | +SET GLOBAL innodb_cleaner_free_list_lwm=1.1; |
106 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_free_list_lwm' |
107 | +SET GLOBAL innodb_cleaner_free_list_lwm=1e1; |
108 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_free_list_lwm' |
109 | +SET GLOBAL innodb_cleaner_free_list_lwm='foo'; |
110 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_free_list_lwm' |
111 | +SET GLOBAL innodb_cleaner_free_list_lwm = @start_value; |
112 | |
113 | === added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result' |
114 | --- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result 1970-01-01 00:00:00 +0000 |
115 | +++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_lru_chunk_size_basic.result 2013-09-30 15:25:38 +0000 |
116 | @@ -0,0 +1,31 @@ |
117 | +SET @start_value = @@GLOBAL.innodb_cleaner_lru_chunk_size; |
118 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
119 | +@@GLOBAL.innodb_cleaner_lru_chunk_size |
120 | +100 |
121 | +SELECT @@SESSION.innodb_cleaner_lru_chunk_size; |
122 | +ERROR HY000: Variable 'innodb_cleaner_lru_chunk_size' is a GLOBAL variable |
123 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1; |
124 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
125 | +@@GLOBAL.innodb_cleaner_lru_chunk_size |
126 | +1 |
127 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1000; |
128 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
129 | +@@GLOBAL.innodb_cleaner_lru_chunk_size |
130 | +1000 |
131 | +SET GLOBAL innodb_cleaner_lru_chunk_size=4294967295; |
132 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
133 | +@@GLOBAL.innodb_cleaner_lru_chunk_size |
134 | +4294967295 |
135 | +SET GLOBAL innodb_cleaner_lru_chunk_size=0; |
136 | +Warnings: |
137 | +Warning 1292 Truncated incorrect innodb_cleaner_lru_chunk_size value: '0' |
138 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
139 | +@@GLOBAL.innodb_cleaner_lru_chunk_size |
140 | +1 |
141 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1.1; |
142 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_lru_chunk_size' |
143 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1e1; |
144 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_lru_chunk_size' |
145 | +SET GLOBAL innodb_cleaner_lru_chunk_size='foo'; |
146 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_lru_chunk_size' |
147 | +SET GLOBAL innodb_cleaner_lru_chunk_size = @start_value; |
148 | |
149 | === added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result' |
150 | --- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result 1970-01-01 00:00:00 +0000 |
151 | +++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_flush_time_basic.result 2013-09-30 15:25:38 +0000 |
152 | @@ -0,0 +1,25 @@ |
153 | +SET @start_value = @@GLOBAL.innodb_cleaner_max_flush_time; |
154 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
155 | +@@GLOBAL.innodb_cleaner_max_flush_time |
156 | +1000 |
157 | +SELECT @@SESSION.innodb_cleaner_max_flush_time; |
158 | +ERROR HY000: Variable 'innodb_cleaner_max_flush_time' is a GLOBAL variable |
159 | +SET GLOBAL innodb_cleaner_max_flush_time=0; |
160 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
161 | +@@GLOBAL.innodb_cleaner_max_flush_time |
162 | +0 |
163 | +SET GLOBAL innodb_cleaner_max_flush_time=1000; |
164 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
165 | +@@GLOBAL.innodb_cleaner_max_flush_time |
166 | +1000 |
167 | +SET GLOBAL innodb_cleaner_max_flush_time=4294967295; |
168 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
169 | +@@GLOBAL.innodb_cleaner_max_flush_time |
170 | +4294967295 |
171 | +SET GLOBAL innodb_cleaner_max_flush_time=1.1; |
172 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_flush_time' |
173 | +SET GLOBAL innodb_cleaner_max_flush_time=1e1; |
174 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_flush_time' |
175 | +SET GLOBAL innodb_cleaner_max_flush_time='foo'; |
176 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_flush_time' |
177 | +SET GLOBAL innodb_cleaner_max_flush_time = @start_value; |
178 | |
179 | === added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result' |
180 | --- Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result 1970-01-01 00:00:00 +0000 |
181 | +++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_cleaner_max_lru_time_basic.result 2013-09-30 15:25:38 +0000 |
182 | @@ -0,0 +1,25 @@ |
183 | +SET @start_value = @@GLOBAL.innodb_cleaner_max_lru_time; |
184 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
185 | +@@GLOBAL.innodb_cleaner_max_lru_time |
186 | +1000 |
187 | +SELECT @@SESSION.innodb_cleaner_max_lru_time; |
188 | +ERROR HY000: Variable 'innodb_cleaner_max_lru_time' is a GLOBAL variable |
189 | +SET GLOBAL innodb_cleaner_max_lru_time=0; |
190 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
191 | +@@GLOBAL.innodb_cleaner_max_lru_time |
192 | +0 |
193 | +SET GLOBAL innodb_cleaner_max_lru_time=1000; |
194 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
195 | +@@GLOBAL.innodb_cleaner_max_lru_time |
196 | +1000 |
197 | +SET GLOBAL innodb_cleaner_max_lru_time=4294967295; |
198 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
199 | +@@GLOBAL.innodb_cleaner_max_lru_time |
200 | +4294967295 |
201 | +SET GLOBAL innodb_cleaner_max_lru_time=1.1; |
202 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_lru_time' |
203 | +SET GLOBAL innodb_cleaner_max_lru_time=1e1; |
204 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_lru_time' |
205 | +SET GLOBAL innodb_cleaner_max_lru_time='foo'; |
206 | +ERROR 42000: Incorrect argument type to variable 'innodb_cleaner_max_lru_time' |
207 | +SET GLOBAL innodb_cleaner_max_lru_time = @start_value; |
208 | |
209 | === added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test' |
210 | --- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test 1970-01-01 00:00:00 +0000 |
211 | +++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_eviction_factor_basic.test 2013-09-30 15:25:38 +0000 |
212 | @@ -0,0 +1,35 @@ |
213 | +--source include/have_debug.inc |
214 | +--source include/have_innodb.inc |
215 | + |
216 | +# A dynamic, global variable |
217 | + |
218 | +SET @start_value = @@GLOBAL.innodb_cleaner_eviction_factor; |
219 | + |
220 | +# Default value |
221 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
222 | + |
223 | +# Global only |
224 | +--error ER_INCORRECT_GLOBAL_LOCAL_VAR |
225 | +SELECT @@SESSION.innodb_cleaner_eviction_factor; |
226 | + |
227 | +# Correct values |
228 | +SET GLOBAL innodb_cleaner_eviction_factor='OFF'; |
229 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
230 | +SET GLOBAL innodb_cleaner_eviction_factor='ON'; |
231 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
232 | +SET GLOBAL innodb_cleaner_eviction_factor=0; |
233 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
234 | +SET GLOBAL innodb_cleaner_eviction_factor=1; |
235 | +SELECT @@GLOBAL.innodb_cleaner_eviction_factor; |
236 | + |
237 | +# Incorrect values |
238 | +--error ER_WRONG_TYPE_FOR_VAR |
239 | +SET GLOBAL innodb_cleaner_eviction_factor=1.1; |
240 | +--error ER_WRONG_TYPE_FOR_VAR |
241 | +SET GLOBAL innodb_cleaner_eviction_factor=1e1; |
242 | +--error ER_WRONG_VALUE_FOR_VAR |
243 | +SET GLOBAL innodb_cleaner_eviction_factor=2; |
244 | +--error ER_WRONG_VALUE_FOR_VAR |
245 | +SET GLOBAL innodb_cleaner_eviction_factor='foo'; |
246 | + |
247 | +SET GLOBAL innodb_cleaner_eviction_factor = @start_value; |
248 | |
249 | === added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test' |
250 | --- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test 1970-01-01 00:00:00 +0000 |
251 | +++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_flush_chunk_size_basic.test 2013-09-30 15:25:38 +0000 |
252 | @@ -0,0 +1,33 @@ |
253 | +--source include/have_debug.inc |
254 | +--source include/have_innodb.inc |
255 | + |
256 | +# A dynamic, global variable |
257 | + |
258 | +SET @start_value = @@GLOBAL.innodb_cleaner_flush_chunk_size; |
259 | + |
260 | +# Default value |
261 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
262 | + |
263 | +# Global only |
264 | +--error ER_INCORRECT_GLOBAL_LOCAL_VAR |
265 | +SELECT @@SESSION.innodb_cleaner_flush_chunk_size; |
266 | + |
267 | +# Correct values |
268 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1; |
269 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
270 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1000; |
271 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
272 | +SET GLOBAL innodb_cleaner_flush_chunk_size=4294967295; |
273 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
274 | + |
275 | +# Incorrect values |
276 | +SET GLOBAL innodb_cleaner_flush_chunk_size=0; |
277 | +SELECT @@GLOBAL.innodb_cleaner_flush_chunk_size; |
278 | +--error ER_WRONG_TYPE_FOR_VAR |
279 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1.1; |
280 | +--error ER_WRONG_TYPE_FOR_VAR |
281 | +SET GLOBAL innodb_cleaner_flush_chunk_size=1e1; |
282 | +--error ER_WRONG_TYPE_FOR_VAR |
283 | +SET GLOBAL innodb_cleaner_flush_chunk_size='foo'; |
284 | + |
285 | +SET GLOBAL innodb_cleaner_flush_chunk_size = @start_value; |
286 | |
287 | === added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test' |
288 | --- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test 1970-01-01 00:00:00 +0000 |
289 | +++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_free_list_lwm_basic.test 2013-09-30 15:25:38 +0000 |
290 | @@ -0,0 +1,35 @@ |
291 | +--source include/have_debug.inc |
292 | +--source include/have_innodb.inc |
293 | + |
294 | +# A dynamic, global variable |
295 | + |
296 | +SET @start_value = @@GLOBAL.innodb_cleaner_free_list_lwm; |
297 | + |
298 | +# Default value |
299 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
300 | + |
301 | +# Global only |
302 | +--error ER_INCORRECT_GLOBAL_LOCAL_VAR |
303 | +SELECT @@SESSION.innodb_cleaner_free_list_lwm; |
304 | + |
305 | +# Correct values |
306 | +SET GLOBAL innodb_cleaner_free_list_lwm=0; |
307 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
308 | +SET GLOBAL innodb_cleaner_free_list_lwm=1; |
309 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
310 | +SET GLOBAL innodb_cleaner_free_list_lwm=99; |
311 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
312 | +SET GLOBAL innodb_cleaner_free_list_lwm=100; |
313 | +SELECT @@GLOBAL.innodb_cleaner_free_list_lwm; |
314 | + |
315 | +# Incorrect values |
316 | +SET GLOBAL innodb_cleaner_free_list_lwm=101; |
317 | +SELECT @@innodb_cleaner_free_list_lwm; |
318 | +--error ER_WRONG_TYPE_FOR_VAR |
319 | +SET GLOBAL innodb_cleaner_free_list_lwm=1.1; |
320 | +--error ER_WRONG_TYPE_FOR_VAR |
321 | +SET GLOBAL innodb_cleaner_free_list_lwm=1e1; |
322 | +--error ER_WRONG_TYPE_FOR_VAR |
323 | +SET GLOBAL innodb_cleaner_free_list_lwm='foo'; |
324 | + |
325 | +SET GLOBAL innodb_cleaner_free_list_lwm = @start_value; |
326 | |
327 | === added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test' |
328 | --- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test 1970-01-01 00:00:00 +0000 |
329 | +++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_lru_chunk_size_basic.test 2013-09-30 15:25:38 +0000 |
330 | @@ -0,0 +1,33 @@ |
331 | +--source include/have_debug.inc |
332 | +--source include/have_innodb.inc |
333 | + |
334 | +# A dynamic, global variable |
335 | + |
336 | +SET @start_value = @@GLOBAL.innodb_cleaner_lru_chunk_size; |
337 | + |
338 | +# Default value |
339 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
340 | + |
341 | +# Global only |
342 | +--error ER_INCORRECT_GLOBAL_LOCAL_VAR |
343 | +SELECT @@SESSION.innodb_cleaner_lru_chunk_size; |
344 | + |
345 | +# Correct values |
346 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1; |
347 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
348 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1000; |
349 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
350 | +SET GLOBAL innodb_cleaner_lru_chunk_size=4294967295; |
351 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
352 | + |
353 | +# Incorrect values |
354 | +SET GLOBAL innodb_cleaner_lru_chunk_size=0; |
355 | +SELECT @@GLOBAL.innodb_cleaner_lru_chunk_size; |
356 | +--error ER_WRONG_TYPE_FOR_VAR |
357 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1.1; |
358 | +--error ER_WRONG_TYPE_FOR_VAR |
359 | +SET GLOBAL innodb_cleaner_lru_chunk_size=1e1; |
360 | +--error ER_WRONG_TYPE_FOR_VAR |
361 | +SET GLOBAL innodb_cleaner_lru_chunk_size='foo'; |
362 | + |
363 | +SET GLOBAL innodb_cleaner_lru_chunk_size = @start_value; |
364 | |
365 | === added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test' |
366 | --- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test 1970-01-01 00:00:00 +0000 |
367 | +++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_flush_time_basic.test 2013-09-30 15:25:38 +0000 |
368 | @@ -0,0 +1,31 @@ |
369 | +--source include/have_debug.inc |
370 | +--source include/have_innodb.inc |
371 | + |
372 | +# A dynamic, global variable |
373 | + |
374 | +SET @start_value = @@GLOBAL.innodb_cleaner_max_flush_time; |
375 | + |
376 | +# Default value |
377 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
378 | + |
379 | +# Global only |
380 | +--error ER_INCORRECT_GLOBAL_LOCAL_VAR |
381 | +SELECT @@SESSION.innodb_cleaner_max_flush_time; |
382 | + |
383 | +# Correct values |
384 | +SET GLOBAL innodb_cleaner_max_flush_time=0; |
385 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
386 | +SET GLOBAL innodb_cleaner_max_flush_time=1000; |
387 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
388 | +SET GLOBAL innodb_cleaner_max_flush_time=4294967295; |
389 | +SELECT @@GLOBAL.innodb_cleaner_max_flush_time; |
390 | + |
391 | +# Incorrect values |
392 | +--error ER_WRONG_TYPE_FOR_VAR |
393 | +SET GLOBAL innodb_cleaner_max_flush_time=1.1; |
394 | +--error ER_WRONG_TYPE_FOR_VAR |
395 | +SET GLOBAL innodb_cleaner_max_flush_time=1e1; |
396 | +--error ER_WRONG_TYPE_FOR_VAR |
397 | +SET GLOBAL innodb_cleaner_max_flush_time='foo'; |
398 | + |
399 | +SET GLOBAL innodb_cleaner_max_flush_time = @start_value; |
400 | |
401 | === added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test' |
402 | --- Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test 1970-01-01 00:00:00 +0000 |
403 | +++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_cleaner_max_lru_time_basic.test 2013-09-30 15:25:38 +0000 |
404 | @@ -0,0 +1,31 @@ |
405 | +--source include/have_debug.inc |
406 | +--source include/have_innodb.inc |
407 | + |
408 | +# A dynamic, global variable |
409 | + |
410 | +SET @start_value = @@GLOBAL.innodb_cleaner_max_lru_time; |
411 | + |
412 | +# Default value |
413 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
414 | + |
415 | +# Global only |
416 | +--error ER_INCORRECT_GLOBAL_LOCAL_VAR |
417 | +SELECT @@SESSION.innodb_cleaner_max_lru_time; |
418 | + |
419 | +# Correct values |
420 | +SET GLOBAL innodb_cleaner_max_lru_time=0; |
421 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
422 | +SET GLOBAL innodb_cleaner_max_lru_time=1000; |
423 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
424 | +SET GLOBAL innodb_cleaner_max_lru_time=4294967295; |
425 | +SELECT @@GLOBAL.innodb_cleaner_max_lru_time; |
426 | + |
427 | +# Incorrect values |
428 | +--error ER_WRONG_TYPE_FOR_VAR |
429 | +SET GLOBAL innodb_cleaner_max_lru_time=1.1; |
430 | +--error ER_WRONG_TYPE_FOR_VAR |
431 | +SET GLOBAL innodb_cleaner_max_lru_time=1e1; |
432 | +--error ER_WRONG_TYPE_FOR_VAR |
433 | +SET GLOBAL innodb_cleaner_max_lru_time='foo'; |
434 | + |
435 | +SET GLOBAL innodb_cleaner_max_lru_time = @start_value; |
436 | |
437 | === modified file 'Percona-Server/storage/innobase/buf/buf0flu.cc' |
438 | --- Percona-Server/storage/innobase/buf/buf0flu.cc 2013-09-27 14:09:03 +0000 |
439 | +++ Percona-Server/storage/innobase/buf/buf0flu.cc 2013-09-30 15:25:38 +0000 |
440 | @@ -59,10 +59,6 @@ |
441 | doing the shutdown */ |
442 | UNIV_INTERN ibool buf_page_cleaner_is_active = FALSE; |
443 | |
444 | -/** LRU flush batch is further divided into this chunk size to |
445 | -reduce the wait time for the threads waiting for a clean block */ |
446 | -#define PAGE_CLEANER_LRU_BATCH_CHUNK_SIZE 100 |
447 | - |
448 | #ifdef UNIV_PFS_THREAD |
449 | UNIV_INTERN mysql_pfs_key_t buf_page_cleaner_thread_key; |
450 | #endif /* UNIV_PFS_THREAD */ |
451 | @@ -75,6 +71,12 @@ |
452 | |
453 | /* @} */ |
454 | |
455 | +/** Handled page counters for a single flush */ |
456 | +struct flush_counters_t { |
457 | + ulint flushed; /*!< number of dirty pages flushed */ |
458 | + ulint evicted; /*!< number of clean pages evicted */ |
459 | +}; |
460 | + |
461 | /******************************************************************//** |
462 | Increases flush_list size in bytes with zip_size for compressed page, |
463 | UNIV_PAGE_SIZE for uncompressed page in inline function */ |
464 | @@ -1508,32 +1510,50 @@ |
465 | it is a best effort attempt and it is not guaranteed that after a call |
466 | to this function there will be 'max' blocks in the free list. |
467 | @return number of blocks for which the write request was queued. */ |
468 | +__attribute__((nonnull)) |
469 | static |
470 | ulint |
471 | buf_flush_LRU_list_batch( |
472 | /*=====================*/ |
473 | buf_pool_t* buf_pool, /*!< in: buffer pool instance */ |
474 | - ulint max) /*!< in: desired number of |
475 | + ulint max, /*!< in: desired number of |
476 | blocks in the free_list */ |
477 | + bool limited_scan, /*!< in: if true, allow to scan only up |
478 | + to srv_LRU_scan_depth pages in total */ |
479 | + flush_counters_t* n) /*!< out: flushed/evicted page |
480 | + counts */ |
481 | { |
482 | buf_page_t* bpage; |
483 | ulint scanned = 0; |
484 | + ulint lru_position = 0; |
485 | + ulint max_lru_position; |
486 | + ulint max_scanned_pages; |
487 | ulint count = 0; |
488 | ulint free_len = UT_LIST_GET_LEN(buf_pool->free); |
489 | ulint lru_len = UT_LIST_GET_LEN(buf_pool->LRU); |
490 | |
491 | + n->flushed = 0; |
492 | + n->evicted = 0; |
493 | + |
494 | ut_ad(mutex_own(&buf_pool->LRU_list_mutex)); |
495 | |
496 | + max_scanned_pages = limited_scan ? srv_LRU_scan_depth : lru_len * max; |
497 | + max_lru_position = ut_min(srv_LRU_scan_depth, lru_len); |
498 | + |
499 | bpage = UT_LIST_GET_LAST(buf_pool->LRU); |
500 | - while (bpage != NULL && count < max |
501 | + while (bpage != NULL |
502 | + && (srv_cleaner_eviction_factor ? n->evicted : n->flushed) < max |
503 | && free_len < srv_LRU_scan_depth |
504 | - && lru_len > BUF_LRU_MIN_LEN) { |
505 | + && lru_len > BUF_LRU_MIN_LEN |
506 | + && lru_position < max_lru_position |
507 | + && scanned < max_scanned_pages) { |
508 | |
509 | ib_mutex_t* block_mutex = buf_page_get_mutex(bpage); |
510 | ibool evict; |
511 | ulint failed_acquire; |
512 | |
513 | ++scanned; |
514 | + ++lru_position; |
515 | |
516 | failed_acquire = mutex_enter_nowait(block_mutex); |
517 | |
518 | @@ -1560,6 +1580,8 @@ |
519 | if (buf_LRU_free_page(bpage, true)) { |
520 | |
521 | mutex_exit(block_mutex); |
522 | + n->evicted++; |
523 | + lru_position = 0; |
524 | mutex_enter(&buf_pool->LRU_list_mutex); |
525 | bpage = UT_LIST_GET_LAST(buf_pool->LRU); |
526 | } else { |
527 | @@ -1573,6 +1595,9 @@ |
528 | bpage, |
529 | BUF_FLUSH_LRU, max, &count)) { |
530 | |
531 | + n->flushed += count; |
532 | + lru_position = 0; |
533 | + |
534 | /* LRU list mutex was released. |
535 | Restart the scan. */ |
536 | bpage = UT_LIST_GET_LAST(buf_pool->LRU); |
537 | @@ -1610,13 +1635,18 @@ |
538 | @return number of blocks for which either the write request was queued |
539 | or in case of unzip_LRU the number of blocks actually moved to the |
540 | free list */ |
541 | +__attribute__((nonnull)) |
542 | static |
543 | -ulint |
544 | +void |
545 | buf_do_LRU_batch( |
546 | /*=============*/ |
547 | buf_pool_t* buf_pool, /*!< in: buffer pool instance */ |
548 | - ulint max) /*!< in: desired number of |
549 | + ulint max, /*!< in: desired number of |
550 | blocks in the free_list */ |
551 | + bool limited_scan, /*!< in: if true, allow to scan only up |
552 | + to srv_LRU_scan_depth pages in total */ |
553 | + flush_counters_t* n) /*!< out: flushed/evicted page |
554 | + counts */ |
555 | { |
556 | ulint count = 0; |
557 | |
558 | @@ -1625,10 +1655,14 @@ |
559 | } |
560 | |
561 | if (max > count) { |
562 | - count += buf_flush_LRU_list_batch(buf_pool, max - count); |
563 | + buf_flush_LRU_list_batch(buf_pool, max - count, limited_scan, |
564 | + n); |
565 | + } else { |
566 | + n->evicted = 0; |
567 | + n->flushed = 0; |
568 | } |
569 | |
570 | - return(count); |
571 | + n->evicted += count; |
572 | } |
573 | |
574 | /*******************************************************************//** |
575 | @@ -1718,8 +1752,9 @@ |
576 | end up waiting for these latches! NOTE 2: in the case of a flush list flush, |
577 | the calling thread is not allowed to own any latches on pages! |
578 | @return number of blocks for which the write request was queued */ |
579 | +__attribute__((nonnull)) |
580 | static |
581 | -ulint |
582 | +void |
583 | buf_flush_batch( |
584 | /*============*/ |
585 | buf_pool_t* buf_pool, /*!< in: buffer pool instance */ |
586 | @@ -1730,14 +1765,17 @@ |
587 | ulint min_n, /*!< in: wished minimum mumber of blocks |
588 | flushed (it is not guaranteed that the |
589 | actual number is that big, though) */ |
590 | - lsn_t lsn_limit) /*!< in: in the case of BUF_FLUSH_LIST |
591 | + lsn_t lsn_limit, /*!< in: in the case of BUF_FLUSH_LIST |
592 | all blocks whose oldest_modification is |
593 | smaller than this should be flushed |
594 | (if their number does not exceed |
595 | min_n), otherwise ignored */ |
596 | + bool limited_lru_scan,/*!< in: for LRU flushes, if true, |
597 | + allow to scan only up to |
598 | + srv_LRU_scan_depth pages in total */ |
599 | + flush_counters_t* n) /*!< out: flushed/evicted page |
600 | + counts */ |
601 | { |
602 | - ulint count = 0; |
603 | - |
604 | ut_ad(flush_type == BUF_FLUSH_LRU || flush_type == BUF_FLUSH_LIST); |
605 | #ifdef UNIV_SYNC_DEBUG |
606 | ut_ad((flush_type != BUF_FLUSH_LIST) |
607 | @@ -1749,26 +1787,27 @@ |
608 | switch (flush_type) { |
609 | case BUF_FLUSH_LRU: |
610 | mutex_enter(&buf_pool->LRU_list_mutex); |
611 | - count = buf_do_LRU_batch(buf_pool, min_n); |
612 | + buf_do_LRU_batch(buf_pool, min_n, limited_lru_scan, n); |
613 | mutex_exit(&buf_pool->LRU_list_mutex); |
614 | break; |
615 | case BUF_FLUSH_LIST: |
616 | - count = buf_do_flush_list_batch(buf_pool, min_n, lsn_limit); |
617 | + ut_ad(!limited_lru_scan); |
618 | + n->flushed = buf_do_flush_list_batch(buf_pool, min_n, |
619 | + lsn_limit); |
620 | + n->evicted = 0; |
621 | break; |
622 | default: |
623 | ut_error; |
624 | } |
625 | |
626 | #ifdef UNIV_DEBUG |
627 | - if (buf_debug_prints && count > 0) { |
628 | + if (buf_debug_prints && n->flushed > 0) { |
629 | fprintf(stderr, flush_type == BUF_FLUSH_LRU |
630 | ? "Flushed %lu pages in LRU flush\n" |
631 | : "Flushed %lu pages in flush list flush\n", |
632 | - (ulong) count); |
633 | + (ulong) n->flushed); |
634 | } |
635 | #endif /* UNIV_DEBUG */ |
636 | - |
637 | - return(count); |
638 | } |
639 | |
640 | /******************************************************************//** |
641 | @@ -1780,7 +1819,9 @@ |
642 | buf_flush_t flush_type, /*!< in: type of flush */ |
643 | ulint page_count) /*!< in: number of pages flushed */ |
644 | { |
645 | - buf_dblwr_flush_buffered_writes(); |
646 | + if (page_count) { |
647 | + buf_dblwr_flush_buffered_writes(); |
648 | + } |
649 | |
650 | ut_a(flush_type == BUF_FLUSH_LRU || flush_type == BUF_FLUSH_LIST); |
651 | |
652 | @@ -1889,6 +1930,7 @@ |
653 | NOTE: The calling thread is not allowed to own any latches on pages! |
654 | @return true if a batch was queued successfully. false if another batch |
655 | of same type was already running. */ |
656 | +__attribute__((nonnull)) |
657 | static |
658 | bool |
659 | buf_flush_LRU( |
660 | @@ -1897,29 +1939,23 @@ |
661 | ulint min_n, /*!< in: wished minimum mumber of blocks |
662 | flushed (it is not guaranteed that the |
663 | actual number is that big, though) */ |
664 | - ulint* n_processed) /*!< out: the number of pages |
665 | - which were processed is passed |
666 | - back to caller. Ignored if NULL */ |
667 | + bool limited_scan, /*!< in: if true, allow to scan |
668 | + only up to srv_LRU_scan_depth |
669 | + pages in total */ |
670 | + flush_counters_t *n) /*!< out: flushed/evicted page |
671 | + counts */ |
672 | { |
673 | - ulint page_count; |
674 | - |
675 | - if (n_processed) { |
676 | - *n_processed = 0; |
677 | - } |
678 | - |
679 | if (!buf_flush_start(buf_pool, BUF_FLUSH_LRU)) { |
680 | + n->flushed = 0; |
681 | + n->evicted = 0; |
682 | return(false); |
683 | } |
684 | |
685 | - page_count = buf_flush_batch(buf_pool, BUF_FLUSH_LRU, min_n, 0); |
686 | + buf_flush_batch(buf_pool, BUF_FLUSH_LRU, min_n, 0, limited_scan, n); |
687 | |
688 | buf_flush_end(buf_pool, BUF_FLUSH_LRU); |
689 | |
690 | - buf_flush_common(BUF_FLUSH_LRU, page_count); |
691 | - |
692 | - if (n_processed) { |
693 | - *n_processed = page_count; |
694 | - } |
695 | + buf_flush_common(BUF_FLUSH_LRU, n->flushed); |
696 | |
697 | return(true); |
698 | } |
699 | @@ -1949,7 +1985,17 @@ |
700 | |
701 | { |
702 | ulint i; |
703 | - bool success = true; |
704 | + |
705 | + ulint requested_pages[MAX_BUFFER_POOLS]; |
706 | + bool active_instance[MAX_BUFFER_POOLS]; |
707 | + ulint remaining_instances = srv_buf_pool_instances; |
708 | + bool timeout = false; |
709 | + ulint flush_start_time = 0; |
710 | + |
711 | + for (i = 0; i < srv_buf_pool_instances; i++) { |
712 | + requested_pages[i] = 0; |
713 | + active_instance[i] = true; |
714 | + } |
715 | |
716 | if (n_processed) { |
717 | *n_processed = 0; |
718 | @@ -1962,52 +2008,81 @@ |
719 | so no limit here. */ |
720 | min_n = (min_n + srv_buf_pool_instances - 1) |
721 | / srv_buf_pool_instances; |
722 | + flush_start_time = ut_time_ms(); |
723 | } |
724 | |
725 | /* Flush to lsn_limit in all buffer pool instances */ |
726 | + while (remaining_instances && !timeout) { |
727 | + |
728 | + for (i = 0; i < srv_buf_pool_instances; i++) { |
729 | + |
730 | + if (flush_start_time |
731 | + && (ut_time_ms() - flush_start_time |
732 | + >= srv_cleaner_max_flush_time)) { |
733 | + |
734 | + timeout = true; |
735 | + break; |
736 | + } |
737 | + |
738 | + if (active_instance[i]) { |
739 | + |
740 | + buf_pool_t* buf_pool; |
741 | + ulint chunk_size; |
742 | + flush_counters_t n; |
743 | + |
744 | + chunk_size = ut_min( |
745 | + srv_cleaner_flush_chunk_size, |
746 | + min_n - requested_pages[i]); |
747 | + |
748 | + buf_pool = buf_pool_from_array(i); |
749 | + |
750 | + if (!buf_flush_start(buf_pool, |
751 | + BUF_FLUSH_LIST)) { |
752 | + |
753 | + continue; |
754 | + } |
755 | + |
756 | + buf_flush_batch(buf_pool, BUF_FLUSH_LIST, |
757 | + chunk_size, lsn_limit, false, |
758 | + &n); |
759 | + |
760 | + buf_flush_end(buf_pool, BUF_FLUSH_LIST); |
761 | + |
762 | + buf_flush_common(BUF_FLUSH_LIST, n.flushed); |
763 | + |
764 | + if (n_processed) { |
765 | + *n_processed += n.flushed; |
766 | + } |
767 | + |
768 | + requested_pages[i] += chunk_size; |
769 | + |
770 | + if (requested_pages[i] >= min_n |
771 | + || !n.flushed) { |
772 | + |
773 | + active_instance[i] = false; |
774 | + remaining_instances--; |
775 | + } |
776 | + |
777 | + if (n.flushed) { |
778 | + MONITOR_INC_VALUE_CUMULATIVE( |
779 | + MONITOR_FLUSH_BATCH_TOTAL_PAGE, |
780 | + MONITOR_FLUSH_BATCH_COUNT, |
781 | + MONITOR_FLUSH_BATCH_PAGES, |
782 | + n.flushed); |
783 | + } |
784 | + } |
785 | + } |
786 | + } |
787 | + |
788 | + /* If we haven't flushed all the instances due to timeout or a repeat |
789 | + failure to start a flush, return failure */ |
790 | for (i = 0; i < srv_buf_pool_instances; i++) { |
791 | - buf_pool_t* buf_pool; |
792 | - ulint page_count = 0; |
793 | - |
794 | - buf_pool = buf_pool_from_array(i); |
795 | - |
796 | - if (!buf_flush_start(buf_pool, BUF_FLUSH_LIST)) { |
797 | - /* We have two choices here. If lsn_limit was |
798 | - specified then skipping an instance of buffer |
799 | - pool means we cannot guarantee that all pages |
800 | - up to lsn_limit has been flushed. We can |
801 | - return right now with failure or we can try |
802 | - to flush remaining buffer pools up to the |
803 | - lsn_limit. We attempt to flush other buffer |
804 | - pools based on the assumption that it will |
805 | - help in the retry which will follow the |
806 | - failure. */ |
807 | - success = false; |
808 | - |
809 | - continue; |
810 | - } |
811 | - |
812 | - page_count = buf_flush_batch( |
813 | - buf_pool, BUF_FLUSH_LIST, min_n, lsn_limit); |
814 | - |
815 | - buf_flush_end(buf_pool, BUF_FLUSH_LIST); |
816 | - |
817 | - buf_flush_common(BUF_FLUSH_LIST, page_count); |
818 | - |
819 | - if (n_processed) { |
820 | - *n_processed += page_count; |
821 | - } |
822 | - |
823 | - if (page_count) { |
824 | - MONITOR_INC_VALUE_CUMULATIVE( |
825 | - MONITOR_FLUSH_BATCH_TOTAL_PAGE, |
826 | - MONITOR_FLUSH_BATCH_COUNT, |
827 | - MONITOR_FLUSH_BATCH_PAGES, |
828 | - page_count); |
829 | + if (active_instance[i]) { |
830 | + return(false); |
831 | } |
832 | } |
833 | |
834 | - return(success); |
835 | + return(true); |
836 | } |
837 | |
838 | /******************************************************************//** |
839 | @@ -2117,47 +2192,94 @@ |
840 | /*====================*/ |
841 | { |
842 | ulint total_flushed = 0; |
843 | + ulint start_time = ut_time_ms(); |
844 | + ulint scan_depth[MAX_BUFFER_POOLS]; |
845 | + ulint requested_pages[MAX_BUFFER_POOLS]; |
846 | + bool active_instance[MAX_BUFFER_POOLS]; |
847 | + bool limited_scan[MAX_BUFFER_POOLS]; |
848 | + ulint previous_evicted[MAX_BUFFER_POOLS]; |
849 | + ulint remaining_instances = srv_buf_pool_instances; |
850 | + ulint lru_chunk_size = srv_cleaner_lru_chunk_size; |
851 | + ulint free_list_lwm = srv_LRU_scan_depth / 100 |
852 | + * srv_cleaner_free_list_lwm; |
853 | + bool timeout = false; |
854 | |
855 | for (ulint i = 0; i < srv_buf_pool_instances; i++) { |
856 | |
857 | - buf_pool_t* buf_pool = buf_pool_from_array(i); |
858 | - ulint scan_depth; |
859 | - |
860 | - /* srv_LRU_scan_depth can be arbitrarily large value. |
861 | - We cap it with current LRU size. */ |
862 | - scan_depth = UT_LIST_GET_LEN(buf_pool->LRU); |
863 | - |
864 | - scan_depth = ut_min(srv_LRU_scan_depth, scan_depth); |
865 | - |
866 | - /* We divide LRU flush into smaller chunks because |
867 | - there may be user threads waiting for the flush to |
868 | - end in buf_LRU_get_free_block(). */ |
869 | - for (ulint j = 0; |
870 | - j < scan_depth; |
871 | - j += PAGE_CLEANER_LRU_BATCH_CHUNK_SIZE) { |
872 | - |
873 | - ulint n_flushed = 0; |
874 | - |
875 | - /* Currently page_cleaner is the only thread |
876 | - that can trigger an LRU flush. It is possible |
877 | - that a batch triggered during last iteration is |
878 | - still running, */ |
879 | - if (buf_flush_LRU(buf_pool, |
880 | - PAGE_CLEANER_LRU_BATCH_CHUNK_SIZE, |
881 | - &n_flushed)) { |
882 | - |
883 | - /* Allowed only one batch per |
884 | - buffer pool instance. */ |
885 | - buf_flush_wait_batch_end( |
886 | - buf_pool, BUF_FLUSH_LRU); |
887 | - } |
888 | - |
889 | - if (n_flushed) { |
890 | - total_flushed += n_flushed; |
891 | - } else { |
892 | - /* Nothing to flush */ |
893 | - break; |
894 | - } |
895 | + const buf_pool_t* buf_pool = buf_pool_from_array(i); |
896 | + |
897 | + scan_depth[i] = ut_min(srv_LRU_scan_depth, |
898 | + UT_LIST_GET_LEN(buf_pool->LRU)); |
899 | + requested_pages[i] = 0; |
900 | + active_instance[i] = true; |
901 | + limited_scan[i] = true; |
902 | + previous_evicted[i] = 0; |
903 | + } |
904 | + |
905 | + while (remaining_instances && !timeout) { |
906 | + |
907 | + for (ulint i = 0; i < srv_buf_pool_instances && !timeout; |
908 | + i++) { |
909 | + |
910 | + if (!active_instance[i]) { |
911 | + continue; |
912 | + } |
913 | + |
914 | + ulint free_len = free_list_lwm; |
915 | + buf_pool_t* buf_pool = buf_pool_from_array(i); |
916 | + |
917 | + do { |
918 | + flush_counters_t n; |
919 | + |
920 | + if (ut_time_ms() - start_time |
921 | + >= srv_cleaner_max_lru_time) { |
922 | + |
923 | + timeout = true; |
924 | + break; |
925 | + } |
926 | + |
927 | + ut_ad(requested_pages[i] <= scan_depth[i]); |
928 | + |
929 | + /* Currently page_cleaner is the only thread |
930 | + that can trigger an LRU flush. It is possible |
931 | + that a batch triggered during last iteration is |
932 | + still running, */ |
933 | + if (buf_flush_LRU(buf_pool, lru_chunk_size, |
934 | + limited_scan[i], &n)) { |
935 | + |
936 | + /* Allowed only one batch per |
937 | + buffer pool instance. */ |
938 | + buf_flush_wait_batch_end( |
939 | + buf_pool, BUF_FLUSH_LRU); |
940 | + } else { |
941 | + |
942 | + total_flushed += n.flushed; |
943 | + |
944 | + /* When we evict less pages than we did |
945 | + on a previous try we relax the LRU scan |
946 | + limit in order to attempt to evict |
947 | + more */ |
948 | + limited_scan[i] |
949 | + = (previous_evicted[i] |
950 | + > n.evicted); |
951 | + previous_evicted[i] = n.evicted; |
952 | + |
953 | + requested_pages[i] += lru_chunk_size; |
954 | + } |
955 | + |
956 | + if (requested_pages[i] >= scan_depth[i] |
957 | + || (srv_cleaner_eviction_factor |
958 | + ? n.evicted : n.flushed)) { |
959 | + |
960 | + active_instance[i] = false; |
961 | + remaining_instances--; |
962 | + } else { |
963 | + |
964 | + free_len = UT_LIST_GET_LEN( |
965 | + buf_pool->free); |
966 | + } |
967 | + } while (active_instance[i] |
968 | + && free_len <= free_list_lwm); |
969 | } |
970 | } |
971 | |
972 | @@ -2421,6 +2543,80 @@ |
973 | } |
974 | } |
975 | |
976 | +/*********************************************************************//** |
977 | +Returns the aggregate free list length over all buffer pool instances. |
978 | +@return total free list length. */ |
979 | +__attribute__((warn_unused_result)) |
980 | +static |
981 | +ulint |
982 | +buf_get_total_free_list_length(void) |
983 | +/*================================*/ |
984 | +{ |
985 | + ulint result = 0; |
986 | + |
987 | + for (ulint i = 0; i < srv_buf_pool_instances; i++) { |
988 | + |
989 | + result += UT_LIST_GET_LEN(buf_pool_from_array(i)->free); |
990 | + } |
991 | + |
992 | + return result; |
993 | +} |
994 | + |
995 | +/*********************************************************************//** |
996 | +Adjust the desired page cleaner thread sleep time for LRU flushes. */ |
997 | +__attribute__((nonnull)) |
998 | +static |
999 | +void |
1000 | +page_cleaner_adapt_lru_sleep_time( |
1001 | +/*==============================*/ |
1002 | + ulint* lru_sleep_time) /*!< in/out: desired page cleaner thread sleep |
1003 | + time for LRU flushes */ |
1004 | +{ |
1005 | + ulint free_len = buf_get_total_free_list_length(); |
1006 | + ulint max_free_len = srv_LRU_scan_depth * srv_buf_pool_instances; |
1007 | + |
1008 | + if (free_len < max_free_len / 100) { |
1009 | + |
1010 | + /* Free lists filled less than 1%, no sleep */ |
1011 | + *lru_sleep_time = 0; |
1012 | + } else if (free_len > max_free_len / 5) { |
1013 | + |
1014 | + /* Free lists filled more than 20%, sleep a bit more */ |
1015 | + *lru_sleep_time += 50; |
1016 | + if (*lru_sleep_time > srv_cleaner_max_lru_time) |
1017 | + *lru_sleep_time = srv_cleaner_max_lru_time; |
1018 | + } else if (free_len < max_free_len / 20 && *lru_sleep_time >= 50) { |
1019 | + |
1020 | + /* Free lists filled less than 5%, sleep a bit less */ |
1021 | + *lru_sleep_time -= 50; |
1022 | + } else { |
1023 | + |
1024 | + /* Free lists filled between 5% and 20%, no change */ |
1025 | + } |
1026 | +} |
1027 | + |
1028 | +/*********************************************************************//** |
1029 | +Get the desired page cleaner thread sleep time for flush list flushes. |
1030 | +@return desired sleep time */ |
1031 | +__attribute__((warn_unused_result)) |
1032 | +static |
1033 | +ulint |
1034 | +page_cleaner_adapt_flush_sleep_time(void) |
1035 | +/*=====================================*/ |
1036 | +{ |
1037 | + lsn_t age = log_get_lsn() - log_sys->last_checkpoint_lsn; |
1038 | + |
1039 | + if (age > log_sys->max_modified_age_sync) { |
1040 | + |
1041 | + /* No sleep if in sync preflush zone */ |
1042 | + return(0); |
1043 | + } |
1044 | + |
1045 | + /* In all other cases flush list factors do not influence the page |
1046 | + cleaner sleep time */ |
1047 | + return(srv_cleaner_max_flush_time); |
1048 | +} |
1049 | + |
1050 | /******************************************************************//** |
1051 | page_cleaner thread tasked with flushing dirty pages from the buffer |
1052 | pools. As of now we'll have only one instance of this thread. |
1053 | @@ -2436,6 +2632,7 @@ |
1054 | ulint next_loop_time = ut_time_ms() + 1000; |
1055 | ulint n_flushed = 0; |
1056 | ulint last_activity = srv_get_activity_count(); |
1057 | + ulint lru_sleep_time = srv_cleaner_max_lru_time; |
1058 | |
1059 | ut_ad(!srv_read_only_mode); |
1060 | |
1061 | @@ -2452,6 +2649,9 @@ |
1062 | |
1063 | while (srv_shutdown_state == SRV_SHUTDOWN_NONE) { |
1064 | |
1065 | + ulint flush_sleep_time; |
1066 | + ulint page_cleaner_sleep_time; |
1067 | + |
1068 | srv_current_thread_priority = srv_cleaner_thread_priority; |
1069 | |
1070 | /* The page_cleaner skips sleep if the server is |
1071 | @@ -2463,7 +2663,14 @@ |
1072 | page_cleaner_sleep_if_needed(next_loop_time); |
1073 | } |
1074 | |
1075 | - next_loop_time = ut_time_ms() + 1000; |
1076 | + page_cleaner_adapt_lru_sleep_time(&lru_sleep_time); |
1077 | + |
1078 | + flush_sleep_time = page_cleaner_adapt_flush_sleep_time(); |
1079 | + |
1080 | + page_cleaner_sleep_time = ut_min(lru_sleep_time, |
1081 | + flush_sleep_time); |
1082 | + |
1083 | + next_loop_time = ut_time_ms() + page_cleaner_sleep_time; |
1084 | |
1085 | if (srv_check_activity(last_activity)) { |
1086 | last_activity = srv_get_activity_count(); |
1087 | |
1088 | === modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc' |
1089 | --- Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-09-27 14:09:03 +0000 |
1090 | +++ Percona-Server/storage/innobase/handler/ha_innodb.cc 2013-09-30 15:25:38 +0000 |
1091 | @@ -16574,6 +16574,42 @@ |
1092 | |
1093 | #endif /* UNIV_LINUX */ |
1094 | |
1095 | +static MYSQL_SYSVAR_ULONG(cleaner_max_lru_time, srv_cleaner_max_lru_time, |
1096 | + PLUGIN_VAR_RQCMDARG, |
1097 | + "The maximum time limit for a single LRU tail flush iteration by the page " |
1098 | + "cleaner thread in miliseconds", |
1099 | + NULL, NULL, 1000, 0, ~0UL, 0); |
1100 | + |
1101 | +static MYSQL_SYSVAR_ULONG(cleaner_max_flush_time, srv_cleaner_max_flush_time, |
1102 | + PLUGIN_VAR_RQCMDARG, |
1103 | + "The maximum time limit for a single flush list flush iteration by the page " |
1104 | + "cleaner thread in miliseconds", |
1105 | + NULL, NULL, 1000, 0, ~0UL, 0); |
1106 | + |
1107 | +static MYSQL_SYSVAR_ULONG(cleaner_flush_chunk_size, |
1108 | + srv_cleaner_flush_chunk_size, |
1109 | + PLUGIN_VAR_RQCMDARG, |
1110 | + "Divide page cleaner flush list flush batches into chunks of this size", |
1111 | + NULL, NULL, 200, 1, ~0UL, 0); |
1112 | + |
1113 | +static MYSQL_SYSVAR_ULONG(cleaner_lru_chunk_size, |
1114 | + srv_cleaner_lru_chunk_size, |
1115 | + PLUGIN_VAR_RQCMDARG, |
1116 | + "Divide page cleaner LRU list flush batches into chunks of this size", |
1117 | + NULL, NULL, 100, 1, ~0UL, 0); |
1118 | + |
1119 | +static MYSQL_SYSVAR_ULONG(cleaner_free_list_lwm, srv_cleaner_free_list_lwm, |
1120 | + PLUGIN_VAR_RQCMDARG, |
1121 | + "Page cleaner will keep on flushing the same buffer pool instance if its " |
1122 | + "free list length is below this percentage of innodb_lru_scan_depth", |
1123 | + NULL, NULL, 10, 0, 100, 0); |
1124 | + |
1125 | +static MYSQL_SYSVAR_BOOL(cleaner_eviction_factor, srv_cleaner_eviction_factor, |
1126 | + PLUGIN_VAR_OPCMDARG, |
1127 | + "Make page cleaner LRU flushes use evicted instead of flushed page counts " |
1128 | + "for its heuristics", |
1129 | + NULL, NULL, FALSE); |
1130 | + |
1131 | #endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */ |
1132 | |
1133 | static MYSQL_SYSVAR_LONG(buffer_pool_instances, innobase_buffer_pool_instances, |
1134 | @@ -17241,6 +17277,12 @@ |
1135 | MYSQL_SYSVAR(priority_cleaner), |
1136 | MYSQL_SYSVAR(priority_master), |
1137 | #endif /* UNIV_LINUX */ |
1138 | + MYSQL_SYSVAR(cleaner_max_lru_time), |
1139 | + MYSQL_SYSVAR(cleaner_max_flush_time), |
1140 | + MYSQL_SYSVAR(cleaner_flush_chunk_size), |
1141 | + MYSQL_SYSVAR(cleaner_lru_chunk_size), |
1142 | + MYSQL_SYSVAR(cleaner_free_list_lwm), |
1143 | + MYSQL_SYSVAR(cleaner_eviction_factor), |
1144 | #endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */ |
1145 | MYSQL_SYSVAR(print_all_deadlocks), |
1146 | MYSQL_SYSVAR(cmp_per_index_enabled), |
1147 | |
1148 | === modified file 'Percona-Server/storage/innobase/include/srv0srv.h' |
1149 | --- Percona-Server/storage/innobase/include/srv0srv.h 2013-09-27 14:09:03 +0000 |
1150 | +++ Percona-Server/storage/innobase/include/srv0srv.h 2013-09-30 15:25:38 +0000 |
1151 | @@ -309,6 +309,35 @@ |
1152 | extern ulint srv_mem_pool_size; |
1153 | extern ulint srv_lock_table_size; |
1154 | |
1155 | +extern ulint srv_cleaner_max_lru_time;/*!< the maximum time limit for a |
1156 | + single LRU tail flush iteration by the |
1157 | + page cleaner thread */ |
1158 | + |
1159 | +extern ulint srv_cleaner_max_flush_time;/*!< the maximum time limit for a |
1160 | + single flush list flush iteration by |
1161 | + the page cleaner thread */ |
1162 | + |
1163 | +extern ulint srv_cleaner_flush_chunk_size; |
1164 | + /*!< page cleaner flush list flush |
1165 | + batches are further divided into this |
1166 | + chunk size */ |
1167 | + |
1168 | +extern ulint srv_cleaner_lru_chunk_size; |
1169 | + /*!< page cleaner LRU list flush |
1170 | + batches are further divided into this |
1171 | + chunk size */ |
1172 | + |
1173 | +extern ulint srv_cleaner_free_list_lwm;/*!< if free list length is lower |
1174 | + than this percentage of |
1175 | + srv_LRU_scan_depth, page cleaner LRU |
1176 | + flushes will issue flush batches to the |
1177 | + same instance in a row */ |
1178 | + |
1179 | +extern my_bool srv_cleaner_eviction_factor; |
1180 | + /*!< if TRUE, page cleaner heuristics |
1181 | + use evicted instead of flushed page |
1182 | + counts for its heuristics */ |
1183 | + |
1184 | extern ulint srv_n_file_io_threads; |
1185 | extern my_bool srv_random_read_ahead; |
1186 | extern ulong srv_read_ahead_threshold; |
1187 | |
1188 | === modified file 'Percona-Server/storage/innobase/srv/srv0srv.cc' |
1189 | --- Percona-Server/storage/innobase/srv/srv0srv.cc 2013-09-27 14:09:03 +0000 |
1190 | +++ Percona-Server/storage/innobase/srv/srv0srv.cc 2013-09-30 15:25:38 +0000 |
1191 | @@ -261,6 +261,31 @@ |
1192 | UNIV_INTERN ulint srv_mem_pool_size = ULINT_MAX; |
1193 | UNIV_INTERN ulint srv_lock_table_size = ULINT_MAX; |
1194 | |
1195 | +/** The maximum time limit for a single LRU tail flush iteration by the page |
1196 | +cleaner thread */ |
1197 | +UNIV_INTERN ulint srv_cleaner_max_lru_time = 1000; |
1198 | + |
1199 | +/** The maximum time limit for a single flush list flush iteration by the page |
1200 | +cleaner thread */ |
1201 | +UNIV_INTERN ulint srv_cleaner_max_flush_time = 1000; |
1202 | + |
1203 | +/** Page cleaner flush list flush batches are further divided into this chunk |
1204 | +size */ |
1205 | +UNIV_INTERN ulint srv_cleaner_flush_chunk_size = 100; |
1206 | + |
1207 | +/** Page cleaner LRU list flush batches are further divided into this chunk |
1208 | +size */ |
1209 | +UNIV_INTERN ulint srv_cleaner_lru_chunk_size = 100; |
1210 | + |
1211 | +/** If free list length is lower than this percentage of srv_LRU_scan_depth, |
1212 | +page cleaner LRU flushes will issue flush batches to the same instance in a |
1213 | +row */ |
1214 | +UNIV_INTERN ulint srv_cleaner_free_list_lwm = 10; |
1215 | + |
1216 | +/** If TRUE, page cleaner heuristics use evicted instead of flushed page counts |
1217 | +for its heuristics */ |
1218 | +UNIV_INTERN my_bool srv_cleaner_eviction_factor = FALSE; |
1219 | + |
1220 | /* This parameter is deprecated. Use srv_n_io_[read|write]_threads |
1221 | instead. */ |
1222 | UNIV_INTERN ulint srv_n_file_io_threads = ULINT_MAX; |
http:// jenkins. percona. com/job/ percona- server- 5.6-param/ 330/