Merge lp:~zephyrleaves/percona-server/row_cache into lp:percona-server/5.5
- row_cache
- Merge into 5.5
Status: | Work in progress |
---|---|
Proposed branch: | lp:~zephyrleaves/percona-server/row_cache |
Merge into: | lp:percona-server/5.5 |
Diff against target: |
3321 lines (+3317/-0) 1 file modified
patches/innodb_row_cache.diff (+3317/-0) |
To merge this branch: | bzr merge lp:~zephyrleaves/percona-server/row_cache |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Oleg Tsarev | Pending | ||
Vadim Tkachenko | Pending | ||
Review via email: mp+88310@code.launchpad.net |
Commit message
Description of the change
Row Cache For Innodb is designed for increase memory utilization when query is Key-Value-like.
To solve the problem that it is mainly in popular data and popular data distribution is discrete cases Page in memory cache utilization rate is low.
Some detail can see : http://
Benchmarks can use this sysbench which i modified for row cache scene.
http://
- 206. By
-
fix bug for memory leak on shutdown
fix bug for crash on rollback - 207. By Kevin.Huang
-
fix bug for row cache is not evict when truncate
Vadim Tkachenko (vadim-tk) wrote : | # |
Kevin,
can you pls provide merge proposal against current lp:percona-server ?
We do not have patches anymore, this is full bzr branch now.
Vadim Tkachenko (vadim-tk) wrote : | # |
ok,
it was easier than I thought, I ported it there
lp:~vadim-tk/percona-server/5.5.22-rowcache
Vadim Tkachenko (vadim-tk) wrote : | # |
I have a problem with compiling current patch
"(byte*) key_val_buff" is not defined.
Kevin.Huang (zephyrleaves) wrote : | # |
> I have a problem with compiling current patch
> "(byte*) key_val_buff" is not defined.
I will fix it with 5.5.22.
Kevin.Huang (zephyrleaves) wrote : | # |
> I have a problem with compiling current patch
> "(byte*) key_val_buff" is not defined.
I fixed the compatibility problems for percona 5.5.22.
I ported it here
lp:~zephyrleaves/percona-server/5.5.22-rowcache
Vadim Tkachenko (vadim-tk) wrote : | # |
I can't get good performance numbers with row cache.
I am running sysbench oltp, ZipF distribution, 100GB database, 16 threads
mysqld --innodb-
with innodb_
and in any case it is worse then with no row cache at all.
I suspect it maybe a some mutex contention issue.
Kevin.Huang (zephyrleaves) wrote : | # |
> I can't get good performance numbers with row cache.
>
> I am running sysbench oltp, ZipF distribution, 100GB database, 16 threads
>
> mysqld --innodb-
> --innodb_
> --innodb_
>
> with innodb_
> with innodb_
>
> and in any case it is worse then with no row cache at all.
>
> I suspect it maybe a some mutex contention issue.
You need use this sysbench
http://
for test.
This sysbench which i modified for row cache's scene.
You can use command like this to run test:
sysbench --test=oltp --oltp-
Because of the common sysbench's oltp-dist-type is not random enough.
Vadim Tkachenko (vadim-tk) wrote : | # |
Kevin,
I am not sure what do you mean by "not random" enough.
I am using sysbench 0.5 with new zipf distribution.
In anycase I do not expect a performance degradation when I use bigger cache, but
this is what I see.
Kevin.Huang (zephyrleaves) wrote : | # |
> > I can't get good performance numbers with row cache.
> >
> > I am running sysbench oltp, ZipF distribution, 100GB database, 16 threads
> >
> > mysqld --innodb-
> > --innodb_
> > --innodb_
> >
> > with innodb_
> > with innodb_
> >
> > and in any case it is worse then with no row cache at all.
> >
> > I suspect it maybe a some mutex contention issue.
>
> You need use this sysbench
> http://
> innodb/
> for test.
> This sysbench which i modified for row cache's scene.
> You can use command like this to run test:
> sysbench --test=oltp --oltp-
> size=80000000 --oltp-range-size=1 --mysql-
> --mysql-password=xx --oltp-read-only=on --init-rng=on --num-threads=70 --oltp-
> dist-type=special --oltp-dist-pct=1 --oltp-dist-res=80 --max-requests=0 --max-
> time=1800 run
>
> Because of the common sysbench's oltp-dist-type is not random enough.
RowCache is good at for hot data and the hot data is distributed in random ,not continuous.
Vadim Tkachenko (vadim-tk) wrote : | # |
Kevin,
How does that explain that with
--innodb-
I am getting worse throughput then with
--innodb-
Again, whatever distribution is, I do not expect to see worse results with bigger cache.
Kevin.Huang (zephyrleaves) wrote : | # |
> Kevin,
>
> I am not sure what do you mean by "not random" enough.
>
> I am using sysbench 0.5 with new zipf distribution.
>
> In anycase I do not expect a performance degradation when I use bigger cache,
> but
> this is what I see.
Vadim,
I see.
I will test in sysbench 0.5 with zipf.
And improve the row cache.
thanks.
Kevin.Huang (zephyrleaves) wrote : | # |
> > Kevin,
> >
> > I am not sure what do you mean by "not random" enough.
> >
> > I am using sysbench 0.5 with new zipf distribution.
> >
> > In anycase I do not expect a performance degradation when I use bigger
> cache,
> > but
> > this is what I see.
>
> Vadim,
>
> I see.
> I will test in sysbench 0.5 with zipf.
> And improve the row cache.
>
> thanks.
Hi Vadim,
Can you provide the command which you use to test the row cache with sysbench?
tks.
Kevin.Huang (zephyrleaves) wrote : | # |
> Kevin,
>
> How does that explain that with
> --innodb-
>
> I am getting worse throughput then with
> --innodb-
>
> Again, whatever distribution is, I do not expect to see worse results with
> bigger cache.
Hi Vadim,
I think i found the reason why bigger cache cause worse performance.
If you set bigger innodb-
I use hashtable to store the records , so bigger innodb-
So set the bigger innodb_
Vadim Tkachenko (vadim-tk) wrote : | # |
Kevin,
command is
sysbench --test=
what value should I use for innodb_
Kevin.Huang (zephyrleaves) wrote : | # |
> Kevin,
>
> command is
> sysbench --test=
> size=$ROWS --rand-init=on --num-threads=
> --report-
> socket=
> user=root --percentile=99 run
>
> what value should I use for innodb_
Vadim,
The default value of innodb_
You can set --innodb-
and set --innodb-
for test.
Stewart Smith (stewart) wrote : | # |
On Mon, 14 May 2012 01:24:17 -0000, "Kevin.Huang" <email address hidden> wrote:
> The default value of innodb_
> You can set --innodb-
> and set --innodb-
> for test.
I suggest perhaps having a sensible default calculation for it, maybe
1000/GB of row cache? does that seems sensible?
--
Stewart Smith
Kevin.Huang (zephyrleaves) wrote : | # |
> On Mon, 14 May 2012 01:24:17 -0000, "Kevin.Huang" <email address hidden>
> wrote:
> > The default value of innodb_
> > You can set --innodb-
> --innodb_
> > and set --innodb-
> --innodb_
> > for test.
>
> I suggest perhaps having a sensible default calculation for it, maybe
> 1000/GB of row cache? does that seems sensible?
>
> --
> Stewart Smith
Hi Stewart,
Innodb_
The smaller record need more innodb_
Stewart Smith (stewart) wrote : | # |
On Wed, 16 May 2012 02:15:23 -0000, "Kevin.Huang" <email address hidden> wrote:
> Innodb_
> The smaller record need more innodb_
I completely understand, we do however see a lot of people who don't
know the intricacies of server configuration and will go with defaults
or something close to the default. Making it as easy for these people as
possible is a good feature, even if it may not be optimal for everyone
(people can change the variable by hand of course).
--
Stewart Smith
Kevin.Huang (zephyrleaves) wrote : | # |
> On Wed, 16 May 2012 02:15:23 -0000, "Kevin.Huang" <email address hidden>
> wrote:
> > Innodb_
> be cached by row cache.
> > The smaller record need more innodb_
> so i make it can be configured.
>
> I completely understand, we do however see a lot of people who don't
> know the intricacies of server configuration and will go with defaults
> or something close to the default. Making it as easy for these people as
> possible is a good feature, even if it may not be optimal for everyone
> (people can change the variable by hand of course).
>
> --
> Stewart Smith
Yeah, I will improve it ,the default value will be self-adaption ,and can be configured by people .
Unmerged revisions
- 207. By Kevin.Huang
-
fix bug for row cache is not evict when truncate
- 206. By
-
fix bug for memory leak on shutdown
fix bug for crash on rollback - 205. By
-
fix bug for memory leak on shutdown
fix bug for crash on rollback - 204. By Kevin.Huang
-
add row cache for innodb
Preview Diff
1 | === added file 'patches/innodb_row_cache.diff' | |||
2 | --- patches/innodb_row_cache.diff 1970-01-01 00:00:00 +0000 | |||
3 | +++ patches/innodb_row_cache.diff 2012-03-22 02:13:18 +0000 | |||
4 | @@ -0,0 +1,3317 @@ | |||
5 | 1 | Index: storage/innobase/mtr/mtr0mtr.c | ||
6 | 2 | =================================================================== | ||
7 | 3 | --- storage/innobase/mtr/mtr0mtr.c (revision 498) | ||
8 | 4 | +++ storage/innobase/mtr/mtr0mtr.c (revision 724) | ||
9 | 5 | @@ -279,6 +279,11 @@ | ||
10 | 6 | } | ||
11 | 7 | #endif /* UNIV_DEBUG_VALGRIND */ | ||
12 | 8 | ut_d(mtr->state = MTR_COMMITTED); | ||
13 | 9 | + | ||
14 | 10 | + if (mtr->row_cache_value_queue_base.count > 0) { | ||
15 | 11 | + //release row cache value | ||
16 | 12 | + release_row_cache_value_in_mtr(mtr); | ||
17 | 13 | + } | ||
18 | 14 | } | ||
19 | 15 | |||
20 | 16 | #ifndef UNIV_HOTBACKUP | ||
21 | 17 | Index: storage/innobase/cache/row0cache0lru.c | ||
22 | 18 | =================================================================== | ||
23 | 19 | --- storage/innobase/cache/row0cache0lru.c (revision 0) | ||
24 | 20 | +++ storage/innobase/cache/row0cache0lru.c (revision 724) | ||
25 | 21 | @@ -0,0 +1,155 @@ | ||
26 | 22 | +/******************************************************************** | ||
27 | 23 | +created: 2011/03/23 | ||
28 | 24 | +created: 23:3:2011 15:15 | ||
29 | 25 | +file base: row0cache0lru | ||
30 | 26 | +file ext: c | ||
31 | 27 | +author: wentong@taobao.com | ||
32 | 28 | + | ||
33 | 29 | +purpose: | ||
34 | 30 | +*********************************************************************/ | ||
35 | 31 | +#include "row0cache0lru.h" | ||
36 | 32 | +#include "row0cache0hash.h" | ||
37 | 33 | +#include "row0cache0mempool.h" | ||
38 | 34 | +#include "ut0rbt.h" | ||
39 | 35 | +#include "ut0lst.h" | ||
40 | 36 | + | ||
41 | 37 | +#define ROW_CACHE_FREE_DISANCE 100 | ||
42 | 38 | + | ||
43 | 39 | +static ROW_CACHE_VALUE_LIST_BASE *innodb_row_cache_lru; | ||
44 | 40 | + | ||
45 | 41 | +static row_cache_lru_stat_t _row_cache_lru_stat; | ||
46 | 42 | + | ||
47 | 43 | +UNIV_INTERN row_cache_lru_stat_t* row_cache_lru_stat = &_row_cache_lru_stat; | ||
48 | 44 | + | ||
49 | 45 | +UNIV_INTERN my_bool innodb_row_cache_clean_cache = FALSE; | ||
50 | 46 | + | ||
51 | 47 | +void init_innodb_row_cache_lru(){ | ||
52 | 48 | + if (innodb_row_cache_mutex_num > 0){ | ||
53 | 49 | + ulint i; | ||
54 | 50 | + innodb_row_cache_lru = (ROW_CACHE_VALUE_LIST_BASE*) ut_malloc(innodb_row_cache_mutex_num * sizeof(ROW_CACHE_VALUE_LIST_BASE)); | ||
55 | 51 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
56 | 52 | + { | ||
57 | 53 | + UT_LIST_INIT(*(innodb_row_cache_lru+i)); | ||
58 | 54 | + } | ||
59 | 55 | + } | ||
60 | 56 | + memset(row_cache_lru_stat , 0 , sizeof(row_cache_lru_stat_t)); | ||
61 | 57 | +} | ||
62 | 58 | + | ||
63 | 59 | +void deinit_innodb_row_cache_lru(){ | ||
64 | 60 | + if (innodb_row_cache_mutex_num > 0){ | ||
65 | 61 | + ulint i; | ||
66 | 62 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
67 | 63 | + { | ||
68 | 64 | + row_cache_value_t* value; | ||
69 | 65 | + ROW_CACHE_VALUE_LIST_BASE* free_lru = innodb_row_cache_lru + i; | ||
70 | 66 | + row_cache_enter_mutex_by_no(i); | ||
71 | 67 | + for (value = UT_LIST_GET_LAST(*free_lru); | ||
72 | 68 | + value!=NULL;){ | ||
73 | 69 | + row_cache_value_t* free_value; | ||
74 | 70 | + ulint fold; | ||
75 | 71 | + free_value = value; | ||
76 | 72 | + fold = free_value->fold; | ||
77 | 73 | + value=UT_LIST_GET_PREV(list,value); | ||
78 | 74 | + UT_LIST_REMOVE(list,*free_lru,free_value); | ||
79 | 75 | + delete_row_cache_value(free_value); | ||
80 | 76 | + ca_free(free_value->buf ,free_value->buf_size ,fold); | ||
81 | 77 | + ca_free_for_value(free_value , fold); | ||
82 | 78 | + } | ||
83 | 79 | + row_cache_exit_mutex_by_no(i); | ||
84 | 80 | + } | ||
85 | 81 | + ut_free(innodb_row_cache_lru); | ||
86 | 82 | + } | ||
87 | 83 | +} | ||
88 | 84 | + | ||
89 | 85 | +void clean_row_cache(){ | ||
90 | 86 | + if (innodb_row_cache_mutex_num > 0){ | ||
91 | 87 | + ulint i; | ||
92 | 88 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
93 | 89 | + { | ||
94 | 90 | + row_cache_value_t* value; | ||
95 | 91 | + ROW_CACHE_VALUE_LIST_BASE* free_lru = innodb_row_cache_lru + i; | ||
96 | 92 | + row_cache_enter_mutex_by_no(i); | ||
97 | 93 | + for (value = UT_LIST_GET_LAST(*free_lru); | ||
98 | 94 | + value!=NULL;){ | ||
99 | 95 | + row_cache_value_t* free_value; | ||
100 | 96 | + ulint fold; | ||
101 | 97 | + free_value = value; | ||
102 | 98 | + fold = free_value->fold; | ||
103 | 99 | + value=UT_LIST_GET_PREV(list,value); | ||
104 | 100 | + if(free_value->ref_num==0){ | ||
105 | 101 | + UT_LIST_REMOVE(list,*free_lru,free_value); | ||
106 | 102 | + delete_row_cache_value(free_value); | ||
107 | 103 | + ca_free(free_value->buf ,free_value->buf_size, fold); | ||
108 | 104 | + ca_free_for_value(free_value , fold); | ||
109 | 105 | + } | ||
110 | 106 | + } | ||
111 | 107 | + row_cache_exit_mutex_by_no(i); | ||
112 | 108 | + } | ||
113 | 109 | + } | ||
114 | 110 | +} | ||
115 | 111 | + | ||
116 | 112 | +static ROW_CACHE_VALUE_LIST_BASE* get_current_lru_base(const ulint fold){ | ||
117 | 113 | + return innodb_row_cache_lru + row_cache_get_mutex_no(fold); | ||
118 | 114 | +} | ||
119 | 115 | + | ||
120 | 116 | +void add_row_cache_value_to_lru( row_cache_value_t* value ) { | ||
121 | 117 | + UT_LIST_ADD_FIRST(list,*get_current_lru_base(value->fold),value); | ||
122 | 118 | + row_cache_lru_stat->n_add++; | ||
123 | 119 | +} | ||
124 | 120 | + | ||
125 | 121 | +void make_row_cache_value_first_from_lru( row_cache_value_t* value ) | ||
126 | 122 | +{ | ||
127 | 123 | + ROW_CACHE_VALUE_LIST_BASE* lru_base = get_current_lru_base(value->fold); | ||
128 | 124 | + UT_LIST_REMOVE(list,*lru_base,value); | ||
129 | 125 | + UT_LIST_ADD_FIRST(list,*lru_base,value); | ||
130 | 126 | + row_cache_lru_stat->n_make_first++; | ||
131 | 127 | +} | ||
132 | 128 | + | ||
133 | 129 | +ulint free_from_lru(const ulint size , const ulint used_fold) | ||
134 | 130 | +{ | ||
135 | 131 | + ulint iteration_size = 0; | ||
136 | 132 | + int free_distance = 0; | ||
137 | 133 | + ulint free_size = 0; | ||
138 | 134 | + row_cache_value_t* value; | ||
139 | 135 | + ROW_CACHE_VALUE_LIST_BASE* lru_base = get_current_lru_base(used_fold); | ||
140 | 136 | + for (value = UT_LIST_GET_LAST(*lru_base); | ||
141 | 137 | + free_distance < ROW_CACHE_FREE_DISANCE && value!=NULL;) | ||
142 | 138 | + { | ||
143 | 139 | + row_cache_value_t* free_value; | ||
144 | 140 | + free_value = value; | ||
145 | 141 | + value=UT_LIST_GET_PREV(list,value); | ||
146 | 142 | + iteration_size += free_value->buf_size; | ||
147 | 143 | + if(free_value->ref_num==0 && free_value->fold!=used_fold){ | ||
148 | 144 | + //the value can't be used or can't be locked by the upper function | ||
149 | 145 | + UT_LIST_REMOVE(list,*lru_base,free_value); | ||
150 | 146 | + row_cache_lru_stat->n_evict++; | ||
151 | 147 | + free_size+=free_value->buf_size; | ||
152 | 148 | + delete_row_cache_value(free_value); | ||
153 | 149 | + ca_free(free_value->buf ,free_value->buf_size , used_fold); | ||
154 | 150 | + ca_free_for_value(free_value , used_fold); | ||
155 | 151 | + } | ||
156 | 152 | + if(iteration_size > size){ | ||
157 | 153 | + //when may free enough mem and calc free distance | ||
158 | 154 | + free_distance++; | ||
159 | 155 | + } | ||
160 | 156 | + } | ||
161 | 157 | + return free_size; | ||
162 | 158 | +} | ||
163 | 159 | + | ||
164 | 160 | +void remove_row_cache_value_from_lru( row_cache_value_t* value ) | ||
165 | 161 | +{ | ||
166 | 162 | + UT_LIST_REMOVE(list,*get_current_lru_base(value->fold),value); | ||
167 | 163 | + row_cache_lru_stat->n_evict++; | ||
168 | 164 | +} | ||
169 | 165 | + | ||
170 | 166 | +ulint get_row_cache_lru_count() | ||
171 | 167 | +{ | ||
172 | 168 | + ulint i; | ||
173 | 169 | + ulint ret = 0; | ||
174 | 170 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
175 | 171 | + { | ||
176 | 172 | + ret += UT_LIST_GET_LEN(*(innodb_row_cache_lru+i)); | ||
177 | 173 | + } | ||
178 | 174 | + return ret; | ||
179 | 175 | +} | ||
180 | 176 | + | ||
181 | 177 | |||
182 | 178 | Property changes on: storage/innobase/cache/row0cache0lru.c | ||
183 | 179 | ___________________________________________________________________ | ||
184 | 180 | Added: svn:mime-type | ||
185 | 181 | + text/plain | ||
186 | 182 | |||
187 | 183 | Index: storage/innobase/cache/row0cache0hash.c | ||
188 | 184 | =================================================================== | ||
189 | 185 | --- storage/innobase/cache/row0cache0hash.c (revision 0) | ||
190 | 186 | +++ storage/innobase/cache/row0cache0hash.c (revision 724) | ||
191 | 187 | @@ -0,0 +1,196 @@ | ||
192 | 188 | +/******************************************************************** | ||
193 | 189 | +created: 2011/03/24 | ||
194 | 190 | +created: 24:3:2011 8:49 | ||
195 | 191 | +file base: row0cache0hash | ||
196 | 192 | +file ext: c | ||
197 | 193 | +author: wentong@taobao.com | ||
198 | 194 | + | ||
199 | 195 | +purpose: | ||
200 | 196 | +*********************************************************************/ | ||
201 | 197 | + | ||
202 | 198 | +#include "row0cache0hash.h" | ||
203 | 199 | +#include "ha0ha.h" | ||
204 | 200 | +#include "hash0hash.h" | ||
205 | 201 | +#include "rem0rec.h" | ||
206 | 202 | +#include "rem0cmp.h" | ||
207 | 203 | +#include "ut0byte.h" | ||
208 | 204 | + | ||
209 | 205 | + | ||
210 | 206 | + | ||
211 | 207 | +UNIV_INTERN unsigned long innodb_row_cache_cell_num = 10000L; | ||
212 | 208 | + | ||
213 | 209 | +UNIV_INTERN unsigned int innodb_row_cache_mutex_num_shift = 6; | ||
214 | 210 | + | ||
215 | 211 | +UNIV_INTERN ulint innodb_row_cache_mutex_num = 0; | ||
216 | 212 | + | ||
217 | 213 | +static row_cache_t _innodb_row_cache; | ||
218 | 214 | + | ||
219 | 215 | +UNIV_INTERN row_cache_t* innodb_row_cache = &_innodb_row_cache; | ||
220 | 216 | + | ||
221 | 217 | +int init_row_cache_hash(my_bool innodb_row_cache_on) | ||
222 | 218 | +{ | ||
223 | 219 | + memset(innodb_row_cache, 0 , sizeof(row_cache_t)); | ||
224 | 220 | + if (innodb_row_cache_on) | ||
225 | 221 | + { | ||
226 | 222 | + innodb_row_cache_mutex_num = (1<<innodb_row_cache_mutex_num_shift); | ||
227 | 223 | + innodb_row_cache->row_cache = ha_create(innodb_row_cache_cell_num,innodb_row_cache_mutex_num,0); | ||
228 | 224 | + | ||
229 | 225 | + }else{ | ||
230 | 226 | + innodb_row_cache->row_cache = NULL; | ||
231 | 227 | + } | ||
232 | 228 | + return 0; | ||
233 | 229 | +} | ||
234 | 230 | + | ||
235 | 231 | +static void free_hash_table_mutex(hash_table_t* table){ | ||
236 | 232 | + //mutex was freed by sync_close(); | ||
237 | 233 | + ulint i; | ||
238 | 234 | + for (i = 0; i < table->n_mutexes; i++) { | ||
239 | 235 | + mem_heap_free(table->heaps[i]); | ||
240 | 236 | + } | ||
241 | 237 | + mem_free(table->mutexes); | ||
242 | 238 | + table->mutexes = NULL; | ||
243 | 239 | + table->n_mutexes = 0; | ||
244 | 240 | + mem_free(table->heaps); | ||
245 | 241 | + table->heaps = NULL; | ||
246 | 242 | +} | ||
247 | 243 | + | ||
248 | 244 | +void deinit_row_cache_hash(my_bool innodb_row_cache_on){ | ||
249 | 245 | + if (innodb_row_cache_on) | ||
250 | 246 | + { | ||
251 | 247 | +// ha_clear(innodb_row_cache->row_cache); | ||
252 | 248 | + free_hash_table_mutex(innodb_row_cache->row_cache); | ||
253 | 249 | + hash_table_free(innodb_row_cache->row_cache); | ||
254 | 250 | + } | ||
255 | 251 | +} | ||
256 | 252 | + | ||
257 | 253 | +row_cache_value_t* search_row_cache_value(const dtuple_t* tuple, const dict_index_t* index, const ulint fold) | ||
258 | 254 | +{ | ||
259 | 255 | + ulint offsets_[REC_OFFS_NORMAL_SIZE]; | ||
260 | 256 | + ulint* offsets = offsets_; | ||
261 | 257 | + mem_heap_t* heap = NULL; | ||
262 | 258 | + row_cache_chain_t* chain = NULL; | ||
263 | 259 | + rec_offs_init(offsets_); | ||
264 | 260 | + | ||
265 | 261 | + HASH_SEARCH( | ||
266 | 262 | + /* hash_chain->"next" */ | ||
267 | 263 | + next, | ||
268 | 264 | + /* the hash table */ | ||
269 | 265 | + innodb_row_cache->row_cache, | ||
270 | 266 | + /* fold */ | ||
271 | 267 | + fold, | ||
272 | 268 | + /* the type of the next variable */ | ||
273 | 269 | + row_cache_chain_t*, | ||
274 | 270 | + /* auxiliary variable */ | ||
275 | 271 | + chain, | ||
276 | 272 | + /* assertion on every traversed item */ | ||
277 | 273 | + , | ||
278 | 274 | + /* this determines if we have found the lock */ | ||
279 | 275 | + (chain->value!=NULL && chain->value->tree_id == index->id && chain->value->table_id == index->table->id && | ||
280 | 276 | + cmp_dtuple_rec(tuple,chain->value->rec,rec_get_offsets(chain->value->rec, index, offsets, ULINT_UNDEFINED, &heap)) == 0)); | ||
281 | 277 | + if (UNIV_LIKELY_NULL(heap)) { | ||
282 | 278 | + mem_heap_free(heap); | ||
283 | 279 | + } | ||
284 | 280 | + if (chain == NULL) { | ||
285 | 281 | + return(NULL); | ||
286 | 282 | + } | ||
287 | 283 | + /* else */ | ||
288 | 284 | + return(chain->value); | ||
289 | 285 | +} | ||
290 | 286 | + | ||
291 | 287 | + | ||
292 | 288 | +row_cache_value_t* search_row_cache_value_with_rec( const rec_t* rec, const ulint* rec_offsets, dict_index_t* index, const ulint fold ) | ||
293 | 289 | +{ | ||
294 | 290 | + ulint offsets_[REC_OFFS_NORMAL_SIZE]; | ||
295 | 291 | + ulint* offsets = offsets_; | ||
296 | 292 | + mem_heap_t* heap = NULL; | ||
297 | 293 | + row_cache_chain_t* chain = NULL; | ||
298 | 294 | + rec_offs_init(offsets_); | ||
299 | 295 | + | ||
300 | 296 | + HASH_SEARCH( | ||
301 | 297 | + /* hash_chain->"next" */ | ||
302 | 298 | + next, | ||
303 | 299 | + /* the hash table */ | ||
304 | 300 | + innodb_row_cache->row_cache, | ||
305 | 301 | + /* fold */ | ||
306 | 302 | + fold, | ||
307 | 303 | + /* the type of the next variable */ | ||
308 | 304 | + row_cache_chain_t*, | ||
309 | 305 | + /* auxiliary variable */ | ||
310 | 306 | + chain, | ||
311 | 307 | + /* assertion on every traversed item */ | ||
312 | 308 | + , | ||
313 | 309 | + /* this determines if we have found the lock */ | ||
314 | 310 | + (chain->value!=NULL && chain->value->tree_id == index->id && chain->value->table_id == index->table->id && | ||
315 | 311 | + cmp_rec_rec(rec,chain->value->rec,rec_offsets,rec_get_offsets(chain->value->rec, index, offsets, ULINT_UNDEFINED, &heap),index)==0)); | ||
316 | 312 | + | ||
317 | 313 | + if (UNIV_LIKELY_NULL(heap)) { | ||
318 | 314 | + mem_heap_free(heap); | ||
319 | 315 | + } | ||
320 | 316 | + if (chain == NULL) { | ||
321 | 317 | + return(NULL); | ||
322 | 318 | + } | ||
323 | 319 | + /* else */ | ||
324 | 320 | + return(chain->value); | ||
325 | 321 | + | ||
326 | 322 | +} | ||
327 | 323 | + | ||
328 | 324 | + | ||
329 | 325 | + | ||
330 | 326 | + | ||
331 | 327 | +row_cache_value_t* insert_row_cache_value(const ulint fold , row_cache_value_t* value){ | ||
332 | 328 | + HASH_INSERT( | ||
333 | 329 | + /* the type used in the hash chain */ | ||
334 | 330 | + row_cache_chain_t, | ||
335 | 331 | + /* hash_chain->"next" */ | ||
336 | 332 | + next, | ||
337 | 333 | + /* the hash table */ | ||
338 | 334 | + innodb_row_cache->row_cache, | ||
339 | 335 | + /* fold */ | ||
340 | 336 | + fold, | ||
341 | 337 | + /* add this data to the hash */ | ||
342 | 338 | + &value->chain); | ||
343 | 339 | + return value; | ||
344 | 340 | +} | ||
345 | 341 | + | ||
346 | 342 | + | ||
347 | 343 | +void delete_row_cache_value(row_cache_value_t* value){ | ||
348 | 344 | + HASH_DELETE( | ||
349 | 345 | + row_cache_chain_t, | ||
350 | 346 | + next, | ||
351 | 347 | + innodb_row_cache->row_cache, | ||
352 | 348 | + value->fold, | ||
353 | 349 | + (&value->chain)); | ||
354 | 350 | +} | ||
355 | 351 | + | ||
356 | 352 | +void row_cache_enter_mutex_by_no(const ulint no){ | ||
357 | 353 | + mutex_enter(hash_get_nth_mutex(innodb_row_cache->row_cache, no)); | ||
358 | 354 | +} | ||
359 | 355 | + | ||
360 | 356 | +void row_cache_exit_mutex_by_no(const ulint no){ | ||
361 | 357 | + mutex_exit(hash_get_nth_mutex(innodb_row_cache->row_cache, no)); | ||
362 | 358 | +} | ||
363 | 359 | + | ||
364 | 360 | +void row_cache_enter_mutex( const ulint fold ) | ||
365 | 361 | +{ | ||
366 | 362 | + hash_mutex_enter(innodb_row_cache->row_cache , fold); | ||
367 | 363 | +} | ||
368 | 364 | + | ||
369 | 365 | +ulint row_cache_enter_mutex_nowait( const ulint fold ) | ||
370 | 366 | +{ | ||
371 | 367 | + //hash_mutex_enter(innodb_row_cache->row_cache , fold); | ||
372 | 368 | + return mutex_enter_nowait(hash_get_mutex(innodb_row_cache->row_cache, fold)); | ||
373 | 369 | +} | ||
374 | 370 | + | ||
375 | 371 | +void row_cache_exit_mutex( const ulint fold ) | ||
376 | 372 | +{ | ||
377 | 373 | + hash_mutex_exit(innodb_row_cache->row_cache , fold); | ||
378 | 374 | +} | ||
379 | 375 | + | ||
380 | 376 | +int row_cache_own_mutex(const ulint fold1 , const ulint fold2){ | ||
381 | 377 | + return hash_get_mutex_no(innodb_row_cache->row_cache , fold1) == hash_get_mutex_no(innodb_row_cache->row_cache , fold2); | ||
382 | 378 | +} | ||
383 | 379 | + | ||
384 | 380 | +ulint row_cache_get_mutex_no( const ulint fold ) | ||
385 | 381 | +{ | ||
386 | 382 | + return hash_get_mutex_no(innodb_row_cache->row_cache , fold); | ||
387 | 383 | +} | ||
388 | 384 | |||
389 | 385 | Property changes on: storage/innobase/cache/row0cache0hash.c | ||
390 | 386 | ___________________________________________________________________ | ||
391 | 387 | Added: svn:mime-type | ||
392 | 388 | + text/plain | ||
393 | 389 | |||
394 | 390 | Index: storage/innobase/cache/row0cache0filter.c | ||
395 | 391 | =================================================================== | ||
396 | 392 | --- storage/innobase/cache/row0cache0filter.c (revision 0) | ||
397 | 393 | +++ storage/innobase/cache/row0cache0filter.c (revision 724) | ||
398 | 394 | @@ -0,0 +1,295 @@ | ||
399 | 395 | +/******************************************************************** | ||
400 | 396 | + created: 2011/05/31 | ||
401 | 397 | + created: 31:5:2011 11:40 | ||
402 | 398 | + file base: row0cache0filter | ||
403 | 399 | + file ext: c | ||
404 | 400 | + author: wentong@taobao.com | ||
405 | 401 | + | ||
406 | 402 | + purpose: | ||
407 | 403 | + *********************************************************************/ | ||
408 | 404 | + | ||
409 | 405 | +#include "row0cache0filter.h" | ||
410 | 406 | +#include "ut0rbt.h" | ||
411 | 407 | +#include "sync0rw.h" | ||
412 | 408 | +#include "ut0lst.h" | ||
413 | 409 | +#include "hash0hash.h" | ||
414 | 410 | +#include "ut0rnd.h" | ||
415 | 411 | +#include "dict0mem.h" | ||
416 | 412 | + | ||
417 | 413 | +#define FILTER_HASH_CELL_NUM 1000 | ||
418 | 414 | + | ||
419 | 415 | +typedef struct row_cache_filter_index_chain row_cache_filter_index_chain_t; | ||
420 | 416 | +typedef struct row_cache_filter_index_value row_cache_filter_index_value_t; | ||
421 | 417 | + | ||
422 | 418 | +struct row_cache_filter_index_chain { | ||
423 | 419 | + row_cache_filter_index_value_t* value; | ||
424 | 420 | + row_cache_filter_index_chain_t* next; | ||
425 | 421 | +}; | ||
426 | 422 | + | ||
427 | 423 | +struct row_cache_filter_index_value { | ||
428 | 424 | + index_id_t tree_id; | ||
429 | 425 | + row_cache_filter_index_chain_t chain; | ||
430 | 426 | +}; | ||
431 | 427 | + | ||
432 | 428 | +UNIV_INTERN char* innodb_row_cache_index = NULL; | ||
433 | 429 | + | ||
434 | 430 | +static char innodb_row_cache_index_r[INDEX_CONFIG_LEN + 1]; | ||
435 | 431 | + | ||
436 | 432 | +static ibool has_filter = FALSE; | ||
437 | 433 | + | ||
438 | 434 | +static hash_table_t* filtered_in_index; | ||
439 | 435 | + | ||
440 | 436 | +static hash_table_t* filtered_out_index; | ||
441 | 437 | + | ||
442 | 438 | +#ifdef UNIV_PFS_RWLOCK | ||
443 | 439 | +/* Key to register btr_search_sys with performance schema */ | ||
444 | 440 | + | ||
445 | 441 | +UNIV_INTERN mysql_pfs_key_t row_cache_filter_lock_key; | ||
446 | 442 | +#endif /* UNIV_PFS_RWLOCK */ | ||
447 | 443 | + | ||
448 | 444 | +static rw_lock_t filter_lock; | ||
449 | 445 | + | ||
450 | 446 | +void reset_filter(); | ||
451 | 447 | +static int wild_case_compare_wl(const char *str, const char *wildstr, | ||
452 | 448 | + const size_t length); | ||
453 | 449 | +static void free_hash_table_elem(hash_table_t* table); | ||
454 | 450 | + | ||
455 | 451 | +void init_row_cache_filter(my_bool innodb_row_cache_on) { | ||
456 | 452 | + if (innodb_row_cache_on) { | ||
457 | 453 | + rw_lock_create(row_cache_filter_lock_key, &filter_lock, SYNC_MEM_HASH); | ||
458 | 454 | + filtered_in_index = hash_create(FILTER_HASH_CELL_NUM); | ||
459 | 455 | + filtered_out_index = hash_create(FILTER_HASH_CELL_NUM); | ||
460 | 456 | + reset_filter(); | ||
461 | 457 | + } | ||
462 | 458 | +} | ||
463 | 459 | + | ||
464 | 460 | +void deinit_row_cache_filter(my_bool innodb_row_cache_on) { | ||
465 | 461 | + if (innodb_row_cache_on) { | ||
466 | 462 | + reset_filter(); | ||
467 | 463 | + hash_table_free(filtered_in_index); | ||
468 | 464 | + hash_table_free(filtered_out_index); | ||
469 | 465 | + rw_lock_free(&filter_lock); | ||
470 | 466 | + } | ||
471 | 467 | +} | ||
472 | 468 | + | ||
473 | 469 | +void reset_filter() { | ||
474 | 470 | + size_t len = 0; | ||
475 | 471 | + if (innodb_row_cache_index) { | ||
476 | 472 | + len = strlen(innodb_row_cache_index); | ||
477 | 473 | + } | ||
478 | 474 | + len = len > INDEX_CONFIG_LEN ? INDEX_CONFIG_LEN : len; | ||
479 | 475 | + rw_lock_x_lock(&filter_lock); | ||
480 | 476 | + memset(innodb_row_cache_index_r, 0, INDEX_CONFIG_LEN + 1); | ||
481 | 477 | + if (len) { | ||
482 | 478 | + ut_memcpy(innodb_row_cache_index_r, innodb_row_cache_index, len); | ||
483 | 479 | + } | ||
484 | 480 | + innodb_row_cache_index_r[len] = '\0'; | ||
485 | 481 | + innodb_row_cache_index = innodb_row_cache_index_r; | ||
486 | 482 | + | ||
487 | 483 | + free_hash_table_elem(filtered_in_index); | ||
488 | 484 | + free_hash_table_elem(filtered_out_index); | ||
489 | 485 | + if (!len) { | ||
490 | 486 | + has_filter = FALSE; | ||
491 | 487 | + } else { | ||
492 | 488 | + has_filter = TRUE; | ||
493 | 489 | + } | ||
494 | 490 | + rw_lock_x_unlock(&filter_lock); | ||
495 | 491 | +} | ||
496 | 492 | + | ||
497 | 493 | +static void add_result_to_index_hash(ulint fold, index_id_t tree_id, | ||
498 | 494 | + my_bool is_in_filter) { | ||
499 | 495 | + hash_table_t* table; | ||
500 | 496 | + row_cache_filter_index_value_t* value; | ||
501 | 497 | + rw_lock_x_lock(&filter_lock); | ||
502 | 498 | + if (is_in_filter) { | ||
503 | 499 | + table = filtered_in_index; | ||
504 | 500 | + } else { | ||
505 | 501 | + table = filtered_out_index; | ||
506 | 502 | + } | ||
507 | 503 | + value = | ||
508 | 504 | + (row_cache_filter_index_value_t*) mem_alloc(sizeof(row_cache_filter_index_value_t)); | ||
509 | 505 | + if (value) { | ||
510 | 506 | + value->tree_id = tree_id; | ||
511 | 507 | + value->chain.value = value; | ||
512 | 508 | + HASH_INSERT( | ||
513 | 509 | + /* the type used in the hash chain */ | ||
514 | 510 | + row_cache_filter_index_chain_t, | ||
515 | 511 | + /* hash_chain->"next" */ | ||
516 | 512 | + next, | ||
517 | 513 | + /* the hash table */ | ||
518 | 514 | + table, | ||
519 | 515 | + /* fold */ | ||
520 | 516 | + fold, | ||
521 | 517 | + /* add this data to the hash */ | ||
522 | 518 | + &value->chain); | ||
523 | 519 | + } | ||
524 | 520 | + rw_lock_x_unlock(&filter_lock); | ||
525 | 521 | +} | ||
526 | 522 | + | ||
527 | 523 | +static my_bool is_in_filter_index_hash(ulint fold, index_id_t tree_id, | ||
528 | 524 | + my_bool is_in_filter) { | ||
529 | 525 | + hash_table_t* table; | ||
530 | 526 | + row_cache_filter_index_chain_t* chain = NULL; | ||
531 | 527 | + if (is_in_filter) { | ||
532 | 528 | + table = filtered_in_index; | ||
533 | 529 | + } else { | ||
534 | 530 | + table = filtered_out_index; | ||
535 | 531 | + }HASH_SEARCH( | ||
536 | 532 | + /* hash_chain->"next" */ | ||
537 | 533 | + next, | ||
538 | 534 | + /* the hash table */ | ||
539 | 535 | + table, | ||
540 | 536 | + /* fold */ | ||
541 | 537 | + fold, | ||
542 | 538 | + /* the type of the next variable */ | ||
543 | 539 | + row_cache_filter_index_chain_t*, | ||
544 | 540 | + /* auxiliary variable */ | ||
545 | 541 | + chain, | ||
546 | 542 | + /* assertion on every traversed item */ | ||
547 | 543 | + , | ||
548 | 544 | + /* this determines if we have found the lock */ | ||
549 | 545 | + (chain->value!=NULL && chain->value->tree_id == tree_id)); | ||
550 | 546 | + if (chain == NULL) { | ||
551 | 547 | + return (FALSE); | ||
552 | 548 | + } | ||
553 | 549 | + return (TRUE); | ||
554 | 550 | +} | ||
555 | 551 | + | ||
556 | 552 | +static int wild_case_compare_wl(const char *str, const char *wildstr, | ||
557 | 553 | + const size_t length) { | ||
558 | 554 | + const char* start = wildstr; | ||
559 | 555 | + char wild_many = '*'; | ||
560 | 556 | + char wild_one = '?'; | ||
561 | 557 | + char wild_prefix = 0; /* QQ this can potentially cause a SIGSEGV */ | ||
562 | 558 | + int flag; | ||
563 | 559 | + while ((size_t) (wildstr - start) < length) { | ||
564 | 560 | + while ((size_t) (wildstr - start) < length && *wildstr != wild_many | ||
565 | 561 | + && *wildstr != wild_one) { | ||
566 | 562 | + if (*wildstr == wild_prefix | ||
567 | 563 | + && (size_t) (wildstr + 1 - start) < length) | ||
568 | 564 | + wildstr++; | ||
569 | 565 | + if (toupper(*wildstr++) != toupper(*str++)) | ||
570 | 566 | + return (1); | ||
571 | 567 | + } | ||
572 | 568 | + if ((size_t) (wildstr - start) >= length) | ||
573 | 569 | + return (*str != 0); | ||
574 | 570 | + if (*wildstr++ == wild_one) { | ||
575 | 571 | + if (!*str++) | ||
576 | 572 | + return (1); /* One char; skip */ | ||
577 | 573 | + } else { /* Found '*' */ | ||
578 | 574 | + if ((size_t) (wildstr - start) >= length) | ||
579 | 575 | + return (0); /* '*' as last char: OK */ | ||
580 | 576 | + flag = (*wildstr != wild_many && *wildstr != wild_one); | ||
581 | 577 | + do { | ||
582 | 578 | + if (flag) { | ||
583 | 579 | + char cmp; | ||
584 | 580 | + if ((cmp = *wildstr) == wild_prefix | ||
585 | 581 | + && (size_t) (wildstr + 1 - start) < length) | ||
586 | 582 | + cmp = wildstr[1]; | ||
587 | 583 | + cmp = toupper(cmp); | ||
588 | 584 | + while (*str && toupper(*str) != cmp) | ||
589 | 585 | + str++; | ||
590 | 586 | + if (!*str) | ||
591 | 587 | + return (1); | ||
592 | 588 | + } | ||
593 | 589 | + if (wild_case_compare_wl(str, wildstr, | ||
594 | 590 | + length - (wildstr - start)) == 0) | ||
595 | 591 | + return (0); | ||
596 | 592 | + } while (*str++); | ||
597 | 593 | + return (1); | ||
598 | 594 | + } | ||
599 | 595 | + } | ||
600 | 596 | + return (*str != '\0'); | ||
601 | 597 | +} | ||
602 | 598 | + | ||
603 | 599 | +static my_bool compare_index_config(const char* table_name, | ||
604 | 600 | + const char* index_name, const char* config) { | ||
605 | 601 | + char index_sp = ';'; | ||
606 | 602 | + char table_sp = ':'; | ||
607 | 603 | + my_bool is_index_now = FALSE; | ||
608 | 604 | + my_bool is_table_matching = FALSE; | ||
609 | 605 | + my_bool is_index_matching = FALSE; | ||
610 | 606 | + while (*config) { | ||
611 | 607 | + const char* o_config = config; | ||
612 | 608 | + size_t length = 0; | ||
613 | 609 | + while (*config && *config != index_sp && *config != table_sp) { | ||
614 | 610 | + config++; | ||
615 | 611 | + length++; | ||
616 | 612 | + } | ||
617 | 613 | + if (*config == table_sp) { | ||
618 | 614 | + if (!is_index_now) { | ||
619 | 615 | + is_table_matching = !wild_case_compare_wl(table_name, o_config, | ||
620 | 616 | + length); | ||
621 | 617 | + is_index_now = TRUE; | ||
622 | 618 | + } else if (is_table_matching) { | ||
623 | 619 | + is_index_matching = !wild_case_compare_wl(index_name, o_config, | ||
624 | 620 | + length); | ||
625 | 621 | + if (is_index_matching) { | ||
626 | 622 | + return TRUE; | ||
627 | 623 | + } | ||
628 | 624 | + } | ||
629 | 625 | + } else if (!*config || *config == index_sp) { | ||
630 | 626 | + if (is_index_now && is_table_matching) { | ||
631 | 627 | + is_index_matching = !wild_case_compare_wl(index_name, o_config, | ||
632 | 628 | + length); | ||
633 | 629 | + if (is_index_matching) { | ||
634 | 630 | + return TRUE; | ||
635 | 631 | + } | ||
636 | 632 | + } | ||
637 | 633 | + is_index_now = FALSE; | ||
638 | 634 | + is_table_matching = FALSE; | ||
639 | 635 | + is_index_matching = FALSE; | ||
640 | 636 | + } | ||
641 | 637 | + if (*config) { | ||
642 | 638 | + config++; | ||
643 | 639 | + } | ||
644 | 640 | + } | ||
645 | 641 | + return FALSE; | ||
646 | 642 | +} | ||
647 | 643 | + | ||
648 | 644 | +static void free_hash_table_elem(hash_table_t* table) { | ||
649 | 645 | + ulint i; | ||
650 | 646 | + for (i = 0; i < table->n_cells; i++) { | ||
651 | 647 | + row_cache_filter_index_chain_t* chain = | ||
652 | 648 | + (row_cache_filter_index_chain_t*) HASH_GET_FIRST(table,i); | ||
653 | 649 | + while (chain != NULL) { | ||
654 | 650 | + row_cache_filter_index_value_t* value = chain->value; | ||
655 | 651 | + chain = chain->next; | ||
656 | 652 | + mem_free(value); | ||
657 | 653 | + } | ||
658 | 654 | + } | ||
659 | 655 | + hash_table_clear(table); | ||
660 | 656 | +} | ||
661 | 657 | + | ||
662 | 658 | +#define index_id_get_high(id) ((ulint)(((id)>>32)&0xFFFFFFFF)) | ||
663 | 659 | + | ||
664 | 660 | +#define index_id_get_low(id) ((ulint)((id)&0xFFFFFFFF)) | ||
665 | 661 | + | ||
666 | 662 | + | ||
667 | 663 | +my_bool is_index_need_cache(const dict_index_t* index) { | ||
668 | 664 | + const char* table_name = index->table_name; | ||
669 | 665 | + index_id_t index_id = index->id; | ||
670 | 666 | + const char* index_name = index->name; | ||
671 | 667 | + ulint fold = ut_fold_ulint_pair(index_id_get_high(index_id), index_id_get_low(index_id)); | ||
672 | 668 | + my_bool is_in_hash = FALSE; | ||
673 | 669 | + my_bool ret; | ||
674 | 670 | + rw_lock_s_lock(&filter_lock); | ||
675 | 671 | + if (!has_filter) { | ||
676 | 672 | + is_in_hash = TRUE; //didn't need to be cache the result | ||
677 | 673 | + ret = TRUE; | ||
678 | 674 | + } else if (is_in_filter_index_hash(fold, index_id, TRUE)) { | ||
679 | 675 | + is_in_hash = TRUE; | ||
680 | 676 | + ret = TRUE; | ||
681 | 677 | + } else if (is_in_filter_index_hash(fold, index_id, FALSE)) { | ||
682 | 678 | + is_in_hash = TRUE; | ||
683 | 679 | + ret = FALSE; | ||
684 | 680 | + } else { | ||
685 | 681 | + ret = compare_index_config(table_name, index_name, | ||
686 | 682 | + innodb_row_cache_index_r); | ||
687 | 683 | + }rw_lock_s_unlock(&filter_lock); | ||
688 | 684 | + if (!is_in_hash) { | ||
689 | 685 | + add_result_to_index_hash(fold, index_id, ret); | ||
690 | 686 | + } | ||
691 | 687 | + return ret; | ||
692 | 688 | +} | ||
693 | 689 | + | ||
694 | 690 | |||
695 | 691 | Property changes on: storage/innobase/cache/row0cache0filter.c | ||
696 | 692 | ___________________________________________________________________ | ||
697 | 693 | Added: svn:mime-type | ||
698 | 694 | + text/plain | ||
699 | 695 | |||
700 | 696 | Index: storage/innobase/cache/row0cache0mempool.c | ||
701 | 697 | =================================================================== | ||
702 | 698 | --- storage/innobase/cache/row0cache0mempool.c (revision 0) | ||
703 | 699 | +++ storage/innobase/cache/row0cache0mempool.c (revision 724) | ||
704 | 700 | @@ -0,0 +1,800 @@ | ||
705 | 701 | +/******************************************************************** | ||
706 | 702 | + created: 2011/03/23 | ||
707 | 703 | + created: 23:3:2011 15:15 | ||
708 | 704 | + file base: row0cache0mempool | ||
709 | 705 | + file ext: c | ||
710 | 706 | + author: wentong@taobao.com | ||
711 | 707 | + | ||
712 | 708 | + purpose: | ||
713 | 709 | +*********************************************************************/ | ||
714 | 710 | +#include "row0cache0mempool.h" | ||
715 | 711 | +#include "mem0mem.h" | ||
716 | 712 | +#include "sync0sync.h" | ||
717 | 713 | +#include "row0cache0lru.h" | ||
718 | 714 | +#include "mem0pool.h" | ||
719 | 715 | +#include "row0cache.h" | ||
720 | 716 | +#include "ut0lst.h" | ||
721 | 717 | +#include "os0proc.h" | ||
722 | 718 | +#include "srv0srv.h" | ||
723 | 719 | + | ||
724 | 720 | +//copy from mem0pool.c | ||
725 | 721 | + | ||
726 | 722 | +/** The smallest memory area total size */ | ||
727 | 723 | +#define MEM_AREA_MIN_SIZE (2 * MEM_AREA_EXTRA_SIZE) | ||
728 | 724 | + | ||
729 | 725 | +/** Mask used to extract the free bit from area->size */ | ||
730 | 726 | +#define MEM_AREA_FREE 1 | ||
731 | 727 | + | ||
732 | 728 | +/** Data structure for a memory pool. The space is allocated using the buddy | ||
733 | 729 | +algorithm, where free list i contains areas of size 2 to power i. */ | ||
734 | 730 | +struct mem_pool_struct{ | ||
735 | 731 | + byte* buf; /*!< memory pool */ | ||
736 | 732 | + ulint size; /*!< memory common pool size */ | ||
737 | 733 | + ulint reserved; /*!< amount of currently allocated | ||
738 | 734 | + memory */ | ||
739 | 735 | + mutex_t mutex; /*!< mutex protecting this struct */ | ||
740 | 736 | + UT_LIST_BASE_NODE_T(mem_area_t) | ||
741 | 737 | + free_list[64]; /*!< lists of free memory areas: an | ||
742 | 738 | + area is put to the list whose number | ||
743 | 739 | + is the 2-logarithm of the area size */ | ||
744 | 740 | +}; | ||
745 | 741 | + | ||
746 | 742 | +/********************************************************************//** | ||
747 | 743 | +Sets memory area size. */ | ||
748 | 744 | +static | ||
749 | 745 | +void | ||
750 | 746 | +mem_area_set_size_out( | ||
751 | 747 | +/*==============*/ | ||
752 | 748 | + mem_area_t* area, /*!< in: area */ | ||
753 | 749 | + ulint size) /*!< in: size */ | ||
754 | 750 | +{ | ||
755 | 751 | + area->size_and_free = (area->size_and_free & MEM_AREA_FREE) | ||
756 | 752 | + | size; | ||
757 | 753 | +} | ||
758 | 754 | + | ||
759 | 755 | +/********************************************************************//** | ||
760 | 756 | +Sets memory area free bit. */ | ||
761 | 757 | +static | ||
762 | 758 | +void | ||
763 | 759 | +mem_area_set_free_out( | ||
764 | 760 | +/*==============*/ | ||
765 | 761 | +mem_area_t* area, /*!< in: area */ | ||
766 | 762 | +ibool free) /*!< in: free bit value */ | ||
767 | 763 | +{ | ||
768 | 764 | +#if TRUE != MEM_AREA_FREE | ||
769 | 765 | +# error "TRUE != MEM_AREA_FREE" | ||
770 | 766 | +#endif | ||
771 | 767 | + area->size_and_free = (area->size_and_free & ~MEM_AREA_FREE) | ||
772 | 768 | + | free; | ||
773 | 769 | +} | ||
774 | 770 | + | ||
775 | 771 | +/********************************************************************//** | ||
776 | 772 | +Returns memory area free bit. | ||
777 | 773 | + @return TRUE if free */ | ||
778 | 774 | +static | ||
779 | 775 | +ibool | ||
780 | 776 | +mem_area_get_free_out( | ||
781 | 777 | +/*==============*/ | ||
782 | 778 | +mem_area_t* area) /*!< in: area */ | ||
783 | 779 | +{ | ||
784 | 780 | +#if TRUE != MEM_AREA_FREE | ||
785 | 781 | +# error "TRUE != MEM_AREA_FREE" | ||
786 | 782 | +#endif | ||
787 | 783 | + return(area->size_and_free & MEM_AREA_FREE); | ||
788 | 784 | +} | ||
789 | 785 | + | ||
790 | 786 | +/********************************************************************//** | ||
791 | 787 | +Returns memory area size. | ||
792 | 788 | +@return size */ | ||
793 | 789 | +static | ||
794 | 790 | +ulint | ||
795 | 791 | +mem_area_get_size_out( | ||
796 | 792 | +/*==============*/ | ||
797 | 793 | +mem_area_t* area) /*!< in: area */ | ||
798 | 794 | +{ | ||
799 | 795 | + return(area->size_and_free & ~MEM_AREA_FREE); | ||
800 | 796 | +} | ||
801 | 797 | + | ||
802 | 798 | + | ||
803 | 799 | +/********************************************************************//** | ||
804 | 800 | +Gets the buddy of an area, if it exists in pool. | ||
805 | 801 | +@return the buddy, NULL if no buddy in pool */ | ||
806 | 802 | +static | ||
807 | 803 | +mem_area_t* | ||
808 | 804 | +mem_area_get_buddy_out( | ||
809 | 805 | +/*===============*/ | ||
810 | 806 | + mem_area_t* area, /*!< in: memory area */ | ||
811 | 807 | + ulint size, /*!< in: memory area size */ | ||
812 | 808 | + mem_pool_t* pool) /*!< in: memory pool */ | ||
813 | 809 | +{ | ||
814 | 810 | + mem_area_t* buddy; | ||
815 | 811 | + | ||
816 | 812 | + ut_ad(size != 0); | ||
817 | 813 | + | ||
818 | 814 | + if (((((byte*)area) - pool->buf) % (2 * size)) == 0) { | ||
819 | 815 | + | ||
820 | 816 | + /* The buddy is in a higher address */ | ||
821 | 817 | + | ||
822 | 818 | + buddy = (mem_area_t*)(((byte*)area) + size); | ||
823 | 819 | + | ||
824 | 820 | + if ((((byte*)buddy) - pool->buf) + size > pool->size) { | ||
825 | 821 | + | ||
826 | 822 | + /* The buddy is not wholly contained in the pool: | ||
827 | 823 | + there is no buddy */ | ||
828 | 824 | + | ||
829 | 825 | + buddy = NULL; | ||
830 | 826 | + } | ||
831 | 827 | + } else { | ||
832 | 828 | + /* The buddy is in a lower address; NOTE that area cannot | ||
833 | 829 | + be at the pool lower end, because then we would end up to | ||
834 | 830 | + the upper branch in this if-clause: the remainder would be | ||
835 | 831 | + 0 */ | ||
836 | 832 | + | ||
837 | 833 | + buddy = (mem_area_t*)(((byte*)area) - size); | ||
838 | 834 | + } | ||
839 | 835 | + | ||
840 | 836 | + return(buddy); | ||
841 | 837 | +} | ||
842 | 838 | +/********************************************************************//** | ||
843 | 839 | +Fills the specified free list. | ||
844 | 840 | +@return TRUE if we were able to insert a block to the free list */ | ||
845 | 841 | +static | ||
846 | 842 | +ibool | ||
847 | 843 | +mem_pool_fill_free_list( | ||
848 | 844 | +/*====================*/ | ||
849 | 845 | + ulint i, /*!< in: free list index */ | ||
850 | 846 | + mem_pool_t* pool) /*!< in: memory pool */ | ||
851 | 847 | +{ | ||
852 | 848 | + mem_area_t* area; | ||
853 | 849 | + mem_area_t* area2; | ||
854 | 850 | + ibool ret; | ||
855 | 851 | + | ||
856 | 852 | + ut_ad(mutex_own(&(pool->mutex))); | ||
857 | 853 | + | ||
858 | 854 | + if (UNIV_UNLIKELY(i >= 63)) { | ||
859 | 855 | + /* We come here when we have run out of space in the | ||
860 | 856 | + memory pool: */ | ||
861 | 857 | + | ||
862 | 858 | + return(FALSE); | ||
863 | 859 | + } | ||
864 | 860 | + | ||
865 | 861 | + area = UT_LIST_GET_FIRST(pool->free_list[i + 1]); | ||
866 | 862 | + | ||
867 | 863 | + if (area == NULL) { | ||
868 | 864 | + if (UT_LIST_GET_LEN(pool->free_list[i + 1]) > 0) { | ||
869 | 865 | + ut_print_timestamp(stderr); | ||
870 | 866 | + | ||
871 | 867 | + fprintf(stderr, | ||
872 | 868 | + " InnoDB: Error: mem pool free list %lu" | ||
873 | 869 | + " length is %lu\n" | ||
874 | 870 | + "InnoDB: though the list is empty!\n", | ||
875 | 871 | + (ulong) i + 1, | ||
876 | 872 | + (ulong) | ||
877 | 873 | + UT_LIST_GET_LEN(pool->free_list[i + 1])); | ||
878 | 874 | + } | ||
879 | 875 | + | ||
880 | 876 | + ret = mem_pool_fill_free_list(i + 1, pool); | ||
881 | 877 | + | ||
882 | 878 | + if (ret == FALSE) { | ||
883 | 879 | + | ||
884 | 880 | + return(FALSE); | ||
885 | 881 | + } | ||
886 | 882 | + | ||
887 | 883 | + area = UT_LIST_GET_FIRST(pool->free_list[i + 1]); | ||
888 | 884 | + } | ||
889 | 885 | + | ||
890 | 886 | + if (UNIV_UNLIKELY(UT_LIST_GET_LEN(pool->free_list[i + 1]) == 0)) { | ||
891 | 887 | + mem_analyze_corruption(area); | ||
892 | 888 | + | ||
893 | 889 | + ut_error; | ||
894 | 890 | + } | ||
895 | 891 | + | ||
896 | 892 | + UT_LIST_REMOVE(free_list, pool->free_list[i + 1], area); | ||
897 | 893 | + | ||
898 | 894 | + area2 = (mem_area_t*)(((byte*)area) + ut_2_exp(i)); | ||
899 | 895 | + UNIV_MEM_ALLOC(area2, MEM_AREA_EXTRA_SIZE); | ||
900 | 896 | + | ||
901 | 897 | + mem_area_set_size_out(area2, ut_2_exp(i)); | ||
902 | 898 | + mem_area_set_free_out(area2, TRUE); | ||
903 | 899 | + | ||
904 | 900 | + UT_LIST_ADD_FIRST(free_list, pool->free_list[i], area2); | ||
905 | 901 | + | ||
906 | 902 | + mem_area_set_size_out(area, ut_2_exp(i)); | ||
907 | 903 | + | ||
908 | 904 | + UT_LIST_ADD_FIRST(free_list, pool->free_list[i], area); | ||
909 | 905 | + | ||
910 | 906 | + return(TRUE); | ||
911 | 907 | +} | ||
912 | 908 | + | ||
913 | 909 | +/********************************************************************//** | ||
914 | 910 | +Creates a memory pool. | ||
915 | 911 | +@return memory pool */ | ||
916 | 912 | +static | ||
917 | 913 | +mem_pool_t* | ||
918 | 914 | +mem_pool_create_out( | ||
919 | 915 | +/*============*/ | ||
920 | 916 | + ulint size) /*!< in: pool size in bytes */ | ||
921 | 917 | +{ | ||
922 | 918 | + mem_pool_t* pool; | ||
923 | 919 | + mem_area_t* area; | ||
924 | 920 | + ulint i; | ||
925 | 921 | + ulint used; | ||
926 | 922 | + | ||
927 | 923 | + pool = ut_malloc(sizeof(mem_pool_t)); | ||
928 | 924 | + | ||
929 | 925 | + /* We do not set the memory to zero (FALSE) in the pool, | ||
930 | 926 | + but only when allocated at a higher level in mem0mem.c. | ||
931 | 927 | + This is to avoid masking useful Purify warnings. */ | ||
932 | 928 | + | ||
933 | 929 | + pool->buf = ut_malloc_low(size, FALSE, TRUE); | ||
934 | 930 | + /* pool->buf = os_mem_alloc_large(&size); | ||
935 | 931 | + if (pool->buf == NULL) { | ||
936 | 932 | + ut_print_timestamp(stderr); | ||
937 | 933 | + | ||
938 | 934 | + fprintf(stderr, | ||
939 | 935 | + " InnoDB: We now intentionally" | ||
940 | 936 | + " generate a seg fault so that\n" | ||
941 | 937 | + "InnoDB: on Linux we get a stack trace.\n"); | ||
942 | 938 | + | ||
943 | 939 | + if (*ut_mem_null_ptr) ut_mem_null_ptr = 0; | ||
944 | 940 | + } */ | ||
945 | 941 | + pool->size = size; | ||
946 | 942 | + | ||
947 | 943 | + mutex_create(PFS_NOT_INSTRUMENTED, &pool->mutex, SYNC_MEM_POOL); | ||
948 | 944 | + | ||
949 | 945 | + /* Initialize the free lists */ | ||
950 | 946 | + | ||
951 | 947 | + for (i = 0; i < 64; i++) { | ||
952 | 948 | + | ||
953 | 949 | + UT_LIST_INIT(pool->free_list[i]); | ||
954 | 950 | + } | ||
955 | 951 | + | ||
956 | 952 | + used = 0; | ||
957 | 953 | + | ||
958 | 954 | + while (size - used >= MEM_AREA_MIN_SIZE) { | ||
959 | 955 | + | ||
960 | 956 | + i = ut_2_log(size - used); | ||
961 | 957 | + | ||
962 | 958 | + if (ut_2_exp(i) > size - used) { | ||
963 | 959 | + | ||
964 | 960 | + /* ut_2_log rounds upward */ | ||
965 | 961 | + | ||
966 | 962 | + i--; | ||
967 | 963 | + } | ||
968 | 964 | + | ||
969 | 965 | + area = (mem_area_t*)(pool->buf + used); | ||
970 | 966 | + | ||
971 | 967 | + mem_area_set_size_out(area, ut_2_exp(i)); | ||
972 | 968 | + mem_area_set_free_out(area, TRUE); | ||
973 | 969 | + UNIV_MEM_FREE(MEM_AREA_EXTRA_SIZE + (byte*) area, | ||
974 | 970 | + ut_2_exp(i) - MEM_AREA_EXTRA_SIZE); | ||
975 | 971 | + | ||
976 | 972 | + UT_LIST_ADD_FIRST(free_list, pool->free_list[i], area); | ||
977 | 973 | + | ||
978 | 974 | + used = used + ut_2_exp(i); | ||
979 | 975 | + } | ||
980 | 976 | + | ||
981 | 977 | + ut_ad(size >= used); | ||
982 | 978 | + | ||
983 | 979 | + pool->reserved = 0; | ||
984 | 980 | + | ||
985 | 981 | + return(pool); | ||
986 | 982 | +} | ||
987 | 983 | + | ||
988 | 984 | +/********************************************************************//** | ||
989 | 985 | +Frees a memory pool. */ | ||
990 | 986 | +static | ||
991 | 987 | +void | ||
992 | 988 | +mem_pool_free_out( | ||
993 | 989 | +/*==========*/ | ||
994 | 990 | + mem_pool_t* pool) /*!< in, own: memory pool */ | ||
995 | 991 | +{ | ||
996 | 992 | + //os_mem_free_large(pool->buf , pool->size); | ||
997 | 993 | + ut_free(pool->buf); | ||
998 | 994 | + ut_free(pool); | ||
999 | 995 | +} | ||
1000 | 996 | + | ||
1001 | 997 | +/********************************************************************//** | ||
1002 | 998 | +Allocates memory from a pool. NOTE: This low-level function should only be | ||
1003 | 999 | +used in mem0mem.*! | ||
1004 | 1000 | +@return own: allocated memory buffer */ | ||
1005 | 1001 | +static | ||
1006 | 1002 | +void* | ||
1007 | 1003 | +mem_area_alloc_out( | ||
1008 | 1004 | +/*===========*/ | ||
1009 | 1005 | + ulint* psize, /*!< in: requested size in bytes; for optimum | ||
1010 | 1006 | + space usage, the size should be a power of 2 | ||
1011 | 1007 | + minus MEM_AREA_EXTRA_SIZE; | ||
1012 | 1008 | + out: allocated size in bytes (greater than | ||
1013 | 1009 | + or equal to the requested size) */ | ||
1014 | 1010 | + mem_pool_t* pool) /*!< in: memory pool */ | ||
1015 | 1011 | +{ | ||
1016 | 1012 | + mem_area_t* area; | ||
1017 | 1013 | + ulint size; | ||
1018 | 1014 | + ulint n; | ||
1019 | 1015 | + ibool ret; | ||
1020 | 1016 | + | ||
1021 | 1017 | + | ||
1022 | 1018 | + size = *psize; | ||
1023 | 1019 | + n = ut_2_log(ut_max(size + MEM_AREA_EXTRA_SIZE, MEM_AREA_MIN_SIZE)); | ||
1024 | 1020 | + | ||
1025 | 1021 | + mutex_enter(&(pool->mutex)); | ||
1026 | 1022 | + | ||
1027 | 1023 | + | ||
1028 | 1024 | + area = UT_LIST_GET_FIRST(pool->free_list[n]); | ||
1029 | 1025 | + | ||
1030 | 1026 | + if (area == NULL) { | ||
1031 | 1027 | + ret = mem_pool_fill_free_list(n, pool); | ||
1032 | 1028 | + | ||
1033 | 1029 | + if (ret == FALSE) { | ||
1034 | 1030 | + /* Out of memory in memory pool: we try to allocate | ||
1035 | 1031 | + from the operating system with the regular malloc: */ | ||
1036 | 1032 | + | ||
1037 | 1033 | + mutex_exit(&(pool->mutex)); | ||
1038 | 1034 | + | ||
1039 | 1035 | + return(NULL); | ||
1040 | 1036 | + } | ||
1041 | 1037 | + | ||
1042 | 1038 | + area = UT_LIST_GET_FIRST(pool->free_list[n]); | ||
1043 | 1039 | + } | ||
1044 | 1040 | + | ||
1045 | 1041 | + if (!mem_area_get_free_out(area)) { | ||
1046 | 1042 | + fprintf(stderr, | ||
1047 | 1043 | + "InnoDB: Error: Removing element from mem pool" | ||
1048 | 1044 | + " free list %lu though the\n" | ||
1049 | 1045 | + "InnoDB: element is not marked free!\n", | ||
1050 | 1046 | + (ulong) n); | ||
1051 | 1047 | + | ||
1052 | 1048 | + mem_analyze_corruption(area); | ||
1053 | 1049 | + | ||
1054 | 1050 | + /* Try to analyze a strange assertion failure reported at | ||
1055 | 1051 | + mysql@lists.mysql.com where the free bit IS 1 in the | ||
1056 | 1052 | + hex dump above */ | ||
1057 | 1053 | + | ||
1058 | 1054 | + if (mem_area_get_free_out(area)) { | ||
1059 | 1055 | + fprintf(stderr, | ||
1060 | 1056 | + "InnoDB: Probably a race condition" | ||
1061 | 1057 | + " because now the area is marked free!\n"); | ||
1062 | 1058 | + } | ||
1063 | 1059 | + | ||
1064 | 1060 | + ut_error; | ||
1065 | 1061 | + } | ||
1066 | 1062 | + | ||
1067 | 1063 | + if (UT_LIST_GET_LEN(pool->free_list[n]) == 0) { | ||
1068 | 1064 | + fprintf(stderr, | ||
1069 | 1065 | + "InnoDB: Error: Removing element from mem pool" | ||
1070 | 1066 | + " free list %lu\n" | ||
1071 | 1067 | + "InnoDB: though the list length is 0!\n", | ||
1072 | 1068 | + (ulong) n); | ||
1073 | 1069 | + mem_analyze_corruption(area); | ||
1074 | 1070 | + | ||
1075 | 1071 | + ut_error; | ||
1076 | 1072 | + } | ||
1077 | 1073 | + | ||
1078 | 1074 | + ut_ad(mem_area_get_size_out(area) == ut_2_exp(n)); | ||
1079 | 1075 | + | ||
1080 | 1076 | + mem_area_set_free_out(area, FALSE); | ||
1081 | 1077 | + | ||
1082 | 1078 | + UT_LIST_REMOVE(free_list, pool->free_list[n], area); | ||
1083 | 1079 | + | ||
1084 | 1080 | + pool->reserved += mem_area_get_size_out(area); | ||
1085 | 1081 | + | ||
1086 | 1082 | + mutex_exit(&(pool->mutex)); | ||
1087 | 1083 | + | ||
1088 | 1084 | + ut_ad(mem_pool_validate(pool)); | ||
1089 | 1085 | + | ||
1090 | 1086 | + *psize = ut_2_exp(n) - MEM_AREA_EXTRA_SIZE; | ||
1091 | 1087 | + UNIV_MEM_ALLOC(MEM_AREA_EXTRA_SIZE + (byte*)area, *psize); | ||
1092 | 1088 | + | ||
1093 | 1089 | + return((void*)(MEM_AREA_EXTRA_SIZE + ((byte*)area))); | ||
1094 | 1090 | +} | ||
1095 | 1091 | + | ||
1096 | 1092 | +/********************************************************************//** | ||
1097 | 1093 | +Frees memory to a pool. */ | ||
1098 | 1094 | +static | ||
1099 | 1095 | +void | ||
1100 | 1096 | +mem_area_free_out( | ||
1101 | 1097 | +/*==========*/ | ||
1102 | 1098 | + void* ptr, /*!< in, own: pointer to allocated memory | ||
1103 | 1099 | + buffer */ | ||
1104 | 1100 | + mem_pool_t* pool) /*!< in: memory pool */ | ||
1105 | 1101 | +{ | ||
1106 | 1102 | + mem_area_t* area; | ||
1107 | 1103 | + mem_area_t* buddy; | ||
1108 | 1104 | + void* new_ptr; | ||
1109 | 1105 | + ulint size; | ||
1110 | 1106 | + ulint n; | ||
1111 | 1107 | + | ||
1112 | 1108 | + | ||
1113 | 1109 | + | ||
1114 | 1110 | + /* It may be that the area was really allocated from the OS with | ||
1115 | 1111 | + regular malloc: check if ptr points within our memory pool */ | ||
1116 | 1112 | + | ||
1117 | 1113 | + if ((byte*)ptr < pool->buf || (byte*)ptr >= pool->buf + pool->size) { | ||
1118 | 1114 | + ut_free(ptr); | ||
1119 | 1115 | + | ||
1120 | 1116 | + return; | ||
1121 | 1117 | + } | ||
1122 | 1118 | + | ||
1123 | 1119 | + area = (mem_area_t*) (((byte*)ptr) - MEM_AREA_EXTRA_SIZE); | ||
1124 | 1120 | + | ||
1125 | 1121 | + if (mem_area_get_free_out(area)) { | ||
1126 | 1122 | + fprintf(stderr, | ||
1127 | 1123 | + "InnoDB: Error: Freeing element to mem pool" | ||
1128 | 1124 | + " free list though the\n" | ||
1129 | 1125 | + "InnoDB: element is marked free!\n"); | ||
1130 | 1126 | + | ||
1131 | 1127 | + mem_analyze_corruption(area); | ||
1132 | 1128 | + ut_error; | ||
1133 | 1129 | + } | ||
1134 | 1130 | + | ||
1135 | 1131 | + size = mem_area_get_size_out(area); | ||
1136 | 1132 | + UNIV_MEM_FREE(ptr, size - MEM_AREA_EXTRA_SIZE); | ||
1137 | 1133 | + | ||
1138 | 1134 | + if (size == 0) { | ||
1139 | 1135 | + fprintf(stderr, | ||
1140 | 1136 | + "InnoDB: Error: Mem area size is 0. Possibly a" | ||
1141 | 1137 | + " memory overrun of the\n" | ||
1142 | 1138 | + "InnoDB: previous allocated area!\n"); | ||
1143 | 1139 | + | ||
1144 | 1140 | + mem_analyze_corruption(area); | ||
1145 | 1141 | + ut_error; | ||
1146 | 1142 | + } | ||
1147 | 1143 | + | ||
1148 | 1144 | +#ifdef UNIV_LIGHT_MEM_DEBUG | ||
1149 | 1145 | + if (((byte*)area) + size < pool->buf + pool->size) { | ||
1150 | 1146 | + | ||
1151 | 1147 | + ulint next_size; | ||
1152 | 1148 | + | ||
1153 | 1149 | + next_size = mem_area_get_size_out( | ||
1154 | 1150 | + (mem_area_t*)(((byte*)area) + size)); | ||
1155 | 1151 | + if (UNIV_UNLIKELY(!next_size || !ut_is_2pow(next_size))) { | ||
1156 | 1152 | + fprintf(stderr, | ||
1157 | 1153 | + "InnoDB: Error: Memory area size %lu," | ||
1158 | 1154 | + " next area size %lu not a power of 2!\n" | ||
1159 | 1155 | + "InnoDB: Possibly a memory overrun of" | ||
1160 | 1156 | + " the buffer being freed here.\n", | ||
1161 | 1157 | + (ulong) size, (ulong) next_size); | ||
1162 | 1158 | + mem_analyze_corruption(area); | ||
1163 | 1159 | + | ||
1164 | 1160 | + ut_error; | ||
1165 | 1161 | + } | ||
1166 | 1162 | + } | ||
1167 | 1163 | +#endif | ||
1168 | 1164 | + buddy = mem_area_get_buddy_out(area, size, pool); | ||
1169 | 1165 | + | ||
1170 | 1166 | + n = ut_2_log(size); | ||
1171 | 1167 | + | ||
1172 | 1168 | + mutex_enter(&(pool->mutex)); | ||
1173 | 1169 | + | ||
1174 | 1170 | + | ||
1175 | 1171 | + if (buddy && mem_area_get_free_out(buddy) | ||
1176 | 1172 | + && (size == mem_area_get_size_out(buddy))) { | ||
1177 | 1173 | + | ||
1178 | 1174 | + /* The buddy is in a free list */ | ||
1179 | 1175 | + | ||
1180 | 1176 | + if ((byte*)buddy < (byte*)area) { | ||
1181 | 1177 | + new_ptr = ((byte*)buddy) + MEM_AREA_EXTRA_SIZE; | ||
1182 | 1178 | + | ||
1183 | 1179 | + mem_area_set_size_out(buddy, 2 * size); | ||
1184 | 1180 | + mem_area_set_free_out(buddy, FALSE); | ||
1185 | 1181 | + } else { | ||
1186 | 1182 | + new_ptr = ptr; | ||
1187 | 1183 | + | ||
1188 | 1184 | + mem_area_set_size_out(area, 2 * size); | ||
1189 | 1185 | + } | ||
1190 | 1186 | + | ||
1191 | 1187 | + /* Remove the buddy from its free list and merge it to area */ | ||
1192 | 1188 | + | ||
1193 | 1189 | + UT_LIST_REMOVE(free_list, pool->free_list[n], buddy); | ||
1194 | 1190 | + | ||
1195 | 1191 | + pool->reserved += ut_2_exp(n); | ||
1196 | 1192 | + | ||
1197 | 1193 | + mutex_exit(&(pool->mutex)); | ||
1198 | 1194 | + | ||
1199 | 1195 | + mem_area_free_out(new_ptr, pool); | ||
1200 | 1196 | + | ||
1201 | 1197 | + return; | ||
1202 | 1198 | + } else { | ||
1203 | 1199 | + UT_LIST_ADD_FIRST(free_list, pool->free_list[n], area); | ||
1204 | 1200 | + | ||
1205 | 1201 | + mem_area_set_free_out(area, TRUE); | ||
1206 | 1202 | + | ||
1207 | 1203 | + ut_ad(pool->reserved >= size); | ||
1208 | 1204 | + | ||
1209 | 1205 | + pool->reserved -= size; | ||
1210 | 1206 | + } | ||
1211 | 1207 | + | ||
1212 | 1208 | + mutex_exit(&(pool->mutex)); | ||
1213 | 1209 | + | ||
1214 | 1210 | + ut_ad(mem_pool_validate(pool)); | ||
1215 | 1211 | +} | ||
1216 | 1212 | + | ||
1217 | 1213 | +//end copy | ||
1218 | 1214 | + | ||
1219 | 1215 | +UNIV_INTERN my_bool innodb_row_cache_use_sys_malloc = FALSE; | ||
1220 | 1216 | + | ||
1221 | 1217 | +UNIV_INTERN llong innodb_row_cache_mem_pool_size = 1024 * 1024L; //default is 1M | ||
1222 | 1218 | + | ||
1223 | 1219 | +UNIV_INTERN llong innodb_row_cache_additional_mem_pool_size = 1024 * 1024L;//default is 1M | ||
1224 | 1220 | + | ||
1225 | 1221 | +//mem pool | ||
1226 | 1222 | +static mem_pool_t** row_cache_mem_pool = NULL; | ||
1227 | 1223 | + | ||
1228 | 1224 | +//system malloc stat | ||
1229 | 1225 | +static llong* sys_malloc_mem_size = NULL; | ||
1230 | 1226 | + | ||
1231 | 1227 | +static llong* sys_malloc_mem_used = NULL; | ||
1232 | 1228 | + | ||
1233 | 1229 | +static ROW_CACHE_VALUE_LIST_BASE *innodb_row_cache_value_mem_list = NULL; | ||
1234 | 1230 | + | ||
1235 | 1231 | +static row_cache_value_t *innodb_row_cache_value_mem_pool = NULL; | ||
1236 | 1232 | + | ||
1237 | 1233 | +static ROW_CACHE_VALUE_QUEUE_LIST_BASE *innodb_row_cache_queue_mem_list = NULL; | ||
1238 | 1234 | + | ||
1239 | 1235 | +static row_cache_value_queue_t *innodb_row_cache_queue_mem_pool = NULL; | ||
1240 | 1236 | + | ||
1241 | 1237 | +void init_row_cache_mem_pool(my_bool innodb_row_cache_on) | ||
1242 | 1238 | +{ | ||
1243 | 1239 | + if(!innodb_row_cache_on){ | ||
1244 | 1240 | + innodb_row_cache_mem_pool_size = 1; | ||
1245 | 1241 | + } | ||
1246 | 1242 | + if (sizeof(ulint) == 4) { | ||
1247 | 1243 | + if (innodb_row_cache_mem_pool_size > UINT_MAX32) { | ||
1248 | 1244 | + ut_print_timestamp(stderr); | ||
1249 | 1245 | + fprintf(stderr, | ||
1250 | 1246 | + "[Error]innodb_row_cache_mem_pool_size can't be over 4GB" | ||
1251 | 1247 | + " on 32-bit systems\n"); | ||
1252 | 1248 | + } | ||
1253 | 1249 | + } | ||
1254 | 1250 | + if (innodb_row_cache_mutex_num > 0){ | ||
1255 | 1251 | + long long mem_pool_size = 0; | ||
1256 | 1252 | + ulint i; | ||
1257 | 1253 | + //init row cache value mem pool | ||
1258 | 1254 | + ulint row_cache_value_mem_list_size = 0; | ||
1259 | 1255 | + ulint row_cache_mem_pool_value_num = innodb_row_cache_additional_mem_pool_size / sizeof(row_cache_value_t); | ||
1260 | 1256 | + ulint row_cache_max_queue_num; | ||
1261 | 1257 | + if (row_cache_mem_pool_value_num>0){ | ||
1262 | 1258 | + row_cache_value_mem_list_size = innodb_row_cache_mutex_num * sizeof(ROW_CACHE_VALUE_LIST_BASE); | ||
1263 | 1259 | + //create row cache mem list | ||
1264 | 1260 | + innodb_row_cache_value_mem_list = (ROW_CACHE_VALUE_LIST_BASE*) ut_malloc(row_cache_value_mem_list_size); | ||
1265 | 1261 | + innodb_row_cache_value_mem_pool = (row_cache_value_t*) ut_malloc(row_cache_mem_pool_value_num * sizeof(row_cache_value_t)); | ||
1266 | 1262 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1267 | 1263 | + { | ||
1268 | 1264 | + UT_LIST_INIT(*(innodb_row_cache_value_mem_list+i)); | ||
1269 | 1265 | + } | ||
1270 | 1266 | + for (i = 0 ; i < row_cache_mem_pool_value_num ; i++) | ||
1271 | 1267 | + { | ||
1272 | 1268 | + ROW_CACHE_VALUE_LIST_BASE* value_list = innodb_row_cache_value_mem_list + (i % innodb_row_cache_mutex_num); | ||
1273 | 1269 | + row_cache_value_t* value = innodb_row_cache_value_mem_pool + i; | ||
1274 | 1270 | + memset(value , 0 , sizeof(row_cache_value_t)); | ||
1275 | 1271 | + UT_LIST_ADD_FIRST(list,*value_list,value); | ||
1276 | 1272 | + } | ||
1277 | 1273 | + } | ||
1278 | 1274 | + //create row cache queue mem list | ||
1279 | 1275 | + innodb_row_cache_queue_mem_list = (ROW_CACHE_VALUE_QUEUE_LIST_BASE*) ut_malloc(innodb_row_cache_mutex_num * sizeof(ROW_CACHE_VALUE_QUEUE_LIST_BASE)); | ||
1280 | 1276 | + row_cache_max_queue_num = srv_thread_concurrency * 2 * innodb_row_cache_mutex_num; | ||
1281 | 1277 | + innodb_row_cache_queue_mem_pool = (row_cache_value_queue_t*) ut_malloc(row_cache_max_queue_num * sizeof(row_cache_value_queue_t)); | ||
1282 | 1278 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1283 | 1279 | + { | ||
1284 | 1280 | + UT_LIST_INIT(*(innodb_row_cache_queue_mem_list+i)); | ||
1285 | 1281 | + } | ||
1286 | 1282 | + for (i = 0 ; i < row_cache_max_queue_num ; i++) | ||
1287 | 1283 | + { | ||
1288 | 1284 | + ROW_CACHE_VALUE_QUEUE_LIST_BASE* list_base = innodb_row_cache_queue_mem_list + (i % innodb_row_cache_mutex_num); | ||
1289 | 1285 | + row_cache_value_queue_t* value = innodb_row_cache_queue_mem_pool + i; | ||
1290 | 1286 | + memset(value , 0 , sizeof(row_cache_value_queue_t)); | ||
1291 | 1287 | + UT_LIST_ADD_FIRST(list,*list_base,value); | ||
1292 | 1288 | + } | ||
1293 | 1289 | + | ||
1294 | 1290 | + if(!innodb_row_cache_use_sys_malloc){ | ||
1295 | 1291 | + //init general mem pool | ||
1296 | 1292 | + row_cache_mem_pool = (mem_pool_t**) ut_malloc(innodb_row_cache_mutex_num * sizeof(mem_pool_t*)); | ||
1297 | 1293 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1298 | 1294 | + { | ||
1299 | 1295 | + row_cache_mem_pool[i] = mem_pool_create_out(innodb_row_cache_mem_pool_size / innodb_row_cache_mutex_num); | ||
1300 | 1296 | + mem_pool_size += row_cache_mem_pool[i]->size; | ||
1301 | 1297 | + } | ||
1302 | 1298 | + //set to real alloc size! | ||
1303 | 1299 | + innodb_row_cache_mem_pool_size = mem_pool_size; | ||
1304 | 1300 | + }else{ | ||
1305 | 1301 | + //init system malloc mem stat | ||
1306 | 1302 | + sys_malloc_mem_size = (llong*) ut_malloc(innodb_row_cache_mutex_num * sizeof(llong)); | ||
1307 | 1303 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1308 | 1304 | + { | ||
1309 | 1305 | + sys_malloc_mem_size[i] = innodb_row_cache_mem_pool_size / innodb_row_cache_mutex_num; | ||
1310 | 1306 | + } | ||
1311 | 1307 | + sys_malloc_mem_used = (llong*) ut_malloc(innodb_row_cache_mutex_num * sizeof(llong)); | ||
1312 | 1308 | + memset(sys_malloc_mem_used , 0 , innodb_row_cache_mutex_num * sizeof(llong)); | ||
1313 | 1309 | + } | ||
1314 | 1310 | + } | ||
1315 | 1311 | +} | ||
1316 | 1312 | + | ||
1317 | 1313 | +void deinit_row_cache_mem_pool(){ | ||
1318 | 1314 | + if (innodb_row_cache_mutex_num > 0){ | ||
1319 | 1315 | + if(!innodb_row_cache_use_sys_malloc){ | ||
1320 | 1316 | + ulint i; | ||
1321 | 1317 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++){ | ||
1322 | 1318 | + //mutex_free(&row_cache_mem_pool[i]->mutex); //free in sync_close() | ||
1323 | 1319 | + mem_pool_free_out(row_cache_mem_pool[i]); | ||
1324 | 1320 | + } | ||
1325 | 1321 | + ut_free(row_cache_mem_pool); | ||
1326 | 1322 | + row_cache_mem_pool=NULL; | ||
1327 | 1323 | + }else{ | ||
1328 | 1324 | + ut_free(sys_malloc_mem_size); | ||
1329 | 1325 | + ut_free(sys_malloc_mem_used); | ||
1330 | 1326 | + } | ||
1331 | 1327 | + } | ||
1332 | 1328 | + if(innodb_row_cache_value_mem_pool){ | ||
1333 | 1329 | + ut_free(innodb_row_cache_value_mem_pool); | ||
1334 | 1330 | + innodb_row_cache_value_mem_pool=NULL; | ||
1335 | 1331 | + } | ||
1336 | 1332 | + if(innodb_row_cache_value_mem_list){ | ||
1337 | 1333 | + ut_free(innodb_row_cache_value_mem_list); | ||
1338 | 1334 | + innodb_row_cache_value_mem_list=NULL; | ||
1339 | 1335 | + } | ||
1340 | 1336 | + if(innodb_row_cache_queue_mem_pool){ | ||
1341 | 1337 | + ut_free(innodb_row_cache_queue_mem_pool); | ||
1342 | 1338 | + innodb_row_cache_queue_mem_pool=NULL; | ||
1343 | 1339 | + } | ||
1344 | 1340 | + if(innodb_row_cache_queue_mem_list){ | ||
1345 | 1341 | + ut_free(innodb_row_cache_queue_mem_list); | ||
1346 | 1342 | + innodb_row_cache_queue_mem_list=NULL; | ||
1347 | 1343 | + } | ||
1348 | 1344 | +} | ||
1349 | 1345 | + | ||
1350 | 1346 | +static mem_pool_t* get_current_mem_pool(const ulint fold){ | ||
1351 | 1347 | + return row_cache_mem_pool[row_cache_get_mutex_no(fold)]; | ||
1352 | 1348 | +} | ||
1353 | 1349 | + | ||
1354 | 1350 | +void* ca_malloc_low(ulint n , const ulint used_fold) | ||
1355 | 1351 | +{ | ||
1356 | 1352 | + void* ret = NULL; | ||
1357 | 1353 | + if(!innodb_row_cache_use_sys_malloc){ | ||
1358 | 1354 | + mem_pool_t* mem_pool = get_current_mem_pool(used_fold); | ||
1359 | 1355 | + ret=mem_area_alloc_out(&n,mem_pool); | ||
1360 | 1356 | + if(ret==NULL){ | ||
1361 | 1357 | + free_from_lru(n,used_fold); | ||
1362 | 1358 | + ret=mem_area_alloc_out(&n,mem_pool); | ||
1363 | 1359 | + } | ||
1364 | 1360 | + }else{ | ||
1365 | 1361 | + ulint no = row_cache_get_mutex_no(used_fold); | ||
1366 | 1362 | + if(sys_malloc_mem_size[no] - sys_malloc_mem_used[no] > n){ | ||
1367 | 1363 | + ret = malloc(n); | ||
1368 | 1364 | + } | ||
1369 | 1365 | + if(ret==NULL){ | ||
1370 | 1366 | + free_from_lru(n,used_fold); | ||
1371 | 1367 | + ret=malloc(n); | ||
1372 | 1368 | + } | ||
1373 | 1369 | + if(ret){ | ||
1374 | 1370 | + sys_malloc_mem_used[no] += n; | ||
1375 | 1371 | + } | ||
1376 | 1372 | + } | ||
1377 | 1373 | + return ret; | ||
1378 | 1374 | +} | ||
1379 | 1375 | + | ||
1380 | 1376 | +void ca_free_low( void* ptr , const ulint size ,const ulint used_fold) | ||
1381 | 1377 | +{ | ||
1382 | 1378 | + if(!innodb_row_cache_use_sys_malloc){ | ||
1383 | 1379 | + mem_area_free_out(ptr,get_current_mem_pool(used_fold)); | ||
1384 | 1380 | + }else{ | ||
1385 | 1381 | + ulint no = row_cache_get_mutex_no(used_fold); | ||
1386 | 1382 | + free(ptr); | ||
1387 | 1383 | + sys_malloc_mem_used[no] -= size; | ||
1388 | 1384 | + } | ||
1389 | 1385 | +} | ||
1390 | 1386 | + | ||
1391 | 1387 | +static ROW_CACHE_VALUE_LIST_BASE* get_current_value_list(const ulint fold){ | ||
1392 | 1388 | + if(innodb_row_cache_value_mem_list){ | ||
1393 | 1389 | + return innodb_row_cache_value_mem_list + row_cache_get_mutex_no(fold); | ||
1394 | 1390 | + } | ||
1395 | 1391 | + return NULL; | ||
1396 | 1392 | +} | ||
1397 | 1393 | + | ||
1398 | 1394 | +row_cache_value_t* ca_malloc_for_value( const ulint used_fold ) | ||
1399 | 1395 | +{ | ||
1400 | 1396 | + row_cache_value_t* value = NULL; | ||
1401 | 1397 | + ROW_CACHE_VALUE_LIST_BASE* value_list = get_current_value_list(used_fold); | ||
1402 | 1398 | + if(value_list && UT_LIST_GET_LEN(*value_list) > 0){ | ||
1403 | 1399 | + value = UT_LIST_GET_FIRST(*value_list); | ||
1404 | 1400 | + UT_LIST_REMOVE(list,*value_list,value); | ||
1405 | 1401 | + memset(value , 0 , sizeof(row_cache_value_t)); | ||
1406 | 1402 | + //set value mean it come from value's mem pool | ||
1407 | 1403 | + onBit(value->flag,FLAG_VALUE_IS_FROM_VALUE_POOL); | ||
1408 | 1404 | + }else{ | ||
1409 | 1405 | + value = (row_cache_value_t*) ca_malloc(sizeof(row_cache_value_t) , used_fold); | ||
1410 | 1406 | + if(value){ | ||
1411 | 1407 | + memset(value , 0 , sizeof(row_cache_value_t)); | ||
1412 | 1408 | + } | ||
1413 | 1409 | + } | ||
1414 | 1410 | + return value; | ||
1415 | 1411 | +} | ||
1416 | 1412 | + | ||
1417 | 1413 | +void ca_free_for_value( row_cache_value_t* value, const ulint used_fold ) | ||
1418 | 1414 | +{ | ||
1419 | 1415 | + if(isValueFromValuePool(value->flag)){ | ||
1420 | 1416 | + ROW_CACHE_VALUE_LIST_BASE* value_list = get_current_value_list(used_fold); | ||
1421 | 1417 | + UT_LIST_ADD_FIRST(list,*value_list,value); | ||
1422 | 1418 | + }else{ | ||
1423 | 1419 | + ca_free(value,sizeof(row_cache_value_t),used_fold); | ||
1424 | 1420 | + } | ||
1425 | 1421 | +} | ||
1426 | 1422 | + | ||
1427 | 1423 | +static ROW_CACHE_VALUE_QUEUE_LIST_BASE* get_current_value_queue_list(const ulint fold){ | ||
1428 | 1424 | + if(innodb_row_cache_queue_mem_list){ | ||
1429 | 1425 | + return innodb_row_cache_queue_mem_list + row_cache_get_mutex_no(fold); | ||
1430 | 1426 | + } | ||
1431 | 1427 | + return NULL; | ||
1432 | 1428 | +} | ||
1433 | 1429 | + | ||
1434 | 1430 | + | ||
1435 | 1431 | +row_cache_value_queue_t* ca_malloc_for_queue( const ulint used_fold ) | ||
1436 | 1432 | +{ | ||
1437 | 1433 | + row_cache_value_queue_t* value = NULL; | ||
1438 | 1434 | + ROW_CACHE_VALUE_QUEUE_LIST_BASE* list_base = get_current_value_queue_list(used_fold); | ||
1439 | 1435 | + if(list_base && UT_LIST_GET_LEN(*list_base) > 0){ | ||
1440 | 1436 | + value = UT_LIST_GET_FIRST(*list_base); | ||
1441 | 1437 | + UT_LIST_REMOVE(list,*list_base,value); | ||
1442 | 1438 | + memset(value , 0 , sizeof(row_cache_value_queue_t)); | ||
1443 | 1439 | + }else{ | ||
1444 | 1440 | + value = (row_cache_value_queue_t*) ca_malloc(sizeof(row_cache_value_queue_t) , used_fold); | ||
1445 | 1441 | + if(value){ | ||
1446 | 1442 | + memset(value , 0 , sizeof(row_cache_value_queue_t)); | ||
1447 | 1443 | + } | ||
1448 | 1444 | + } | ||
1449 | 1445 | + return value; | ||
1450 | 1446 | + | ||
1451 | 1447 | +} | ||
1452 | 1448 | + | ||
1453 | 1449 | +void ca_free_for_queue( row_cache_value_queue_t* value, const ulint used_fold ) | ||
1454 | 1450 | +{ | ||
1455 | 1451 | + ROW_CACHE_VALUE_QUEUE_LIST_BASE* list_base = get_current_value_queue_list(used_fold); | ||
1456 | 1452 | + UT_LIST_ADD_FIRST(list,*list_base,value); | ||
1457 | 1453 | +} | ||
1458 | 1454 | + | ||
1459 | 1455 | + | ||
1460 | 1456 | +ulint row_cache_mem_pool_used() | ||
1461 | 1457 | +{ | ||
1462 | 1458 | + ulint ret = 0; | ||
1463 | 1459 | + ulint i; | ||
1464 | 1460 | + if(!innodb_row_cache_use_sys_malloc){ | ||
1465 | 1461 | + if(row_cache_mem_pool){ | ||
1466 | 1462 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1467 | 1463 | + { | ||
1468 | 1464 | + ret += mem_pool_get_reserved(row_cache_mem_pool[i]); | ||
1469 | 1465 | + } | ||
1470 | 1466 | + } | ||
1471 | 1467 | + }else{ | ||
1472 | 1468 | + if(sys_malloc_mem_used){ | ||
1473 | 1469 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1474 | 1470 | + { | ||
1475 | 1471 | + ret += sys_malloc_mem_used[i]; | ||
1476 | 1472 | + } | ||
1477 | 1473 | + } | ||
1478 | 1474 | + } | ||
1479 | 1475 | + return ret; | ||
1480 | 1476 | +} | ||
1481 | 1477 | + | ||
1482 | 1478 | +ulint row_cache_get_value_free_count(){ | ||
1483 | 1479 | + ulint ret = 0; | ||
1484 | 1480 | + ulint i; | ||
1485 | 1481 | + if(innodb_row_cache_value_mem_list){ | ||
1486 | 1482 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1487 | 1483 | + { | ||
1488 | 1484 | + ret += UT_LIST_GET_LEN(innodb_row_cache_value_mem_list[i]); | ||
1489 | 1485 | + } | ||
1490 | 1486 | + } | ||
1491 | 1487 | + return ret; | ||
1492 | 1488 | +} | ||
1493 | 1489 | + | ||
1494 | 1490 | +ulint row_cache_get_queue_free_count(){ | ||
1495 | 1491 | + ulint ret = 0; | ||
1496 | 1492 | + ulint i; | ||
1497 | 1493 | + if(innodb_row_cache_queue_mem_list){ | ||
1498 | 1494 | + for (i = 0 ; i < innodb_row_cache_mutex_num ; i++) | ||
1499 | 1495 | + { | ||
1500 | 1496 | + ret += UT_LIST_GET_LEN(innodb_row_cache_queue_mem_list[i]); | ||
1501 | 1497 | + } | ||
1502 | 1498 | + } | ||
1503 | 1499 | + return ret; | ||
1504 | 1500 | +} | ||
1505 | 1501 | |||
1506 | 1502 | Property changes on: storage/innobase/cache/row0cache0mempool.c | ||
1507 | 1503 | ___________________________________________________________________ | ||
1508 | 1504 | Added: svn:mime-type | ||
1509 | 1505 | + text/plain | ||
1510 | 1506 | |||
1511 | 1507 | Index: storage/innobase/cache/row0cache.c | ||
1512 | 1508 | =================================================================== | ||
1513 | 1509 | --- storage/innobase/cache/row0cache.c (revision 0) | ||
1514 | 1510 | +++ storage/innobase/cache/row0cache.c (revision 724) | ||
1515 | 1511 | @@ -0,0 +1,385 @@ | ||
1516 | 1512 | +/******************************************************************** | ||
1517 | 1513 | + created: 2011/03/17 | ||
1518 | 1514 | + created: 17:3:2011 16:51 | ||
1519 | 1515 | + file base: row0cache | ||
1520 | 1516 | + file ext: c | ||
1521 | 1517 | + author: wentong@taobao.com | ||
1522 | 1518 | + | ||
1523 | 1519 | + purpose: for row cache | ||
1524 | 1520 | + *********************************************************************/ | ||
1525 | 1521 | +#include "row0cache.h" | ||
1526 | 1522 | +#include "row0cache0mempool.h" | ||
1527 | 1523 | +#include "row0cache0hash.h" | ||
1528 | 1524 | +#include "mtr0mtr.h" | ||
1529 | 1525 | +#include "rem0rec.h" | ||
1530 | 1526 | +#include "row0cache0lru.h" | ||
1531 | 1527 | +#include "trx0types.h" | ||
1532 | 1528 | +#include "page0page.h" | ||
1533 | 1529 | +#include "log0recv.h" | ||
1534 | 1530 | +#include "read0read.h" | ||
1535 | 1531 | +#include "rem0cmp.h" | ||
1536 | 1532 | +#include "row0cache0filter.h" | ||
1537 | 1533 | +UNIV_INTERN my_bool innodb_row_cache_on = FALSE; | ||
1538 | 1534 | + | ||
1539 | 1535 | +static row_cache_stat_t _row_cache_stat; | ||
1540 | 1536 | +UNIV_INTERN row_cache_stat_t* row_cache_stat = &_row_cache_stat; | ||
1541 | 1537 | + | ||
1542 | 1538 | +void init_row_cache() { | ||
1543 | 1539 | + DBUG_ENTER("init_row_cache"); | ||
1544 | 1540 | + memset(row_cache_stat, 0, sizeof(row_cache_stat_t)); | ||
1545 | 1541 | + init_row_cache_hash(innodb_row_cache_on); | ||
1546 | 1542 | + init_innodb_row_cache_lru(); | ||
1547 | 1543 | + init_row_cache_mem_pool(innodb_row_cache_on); | ||
1548 | 1544 | + init_row_cache_filter(innodb_row_cache_on); | ||
1549 | 1545 | + //init_row_cache_lock_pool(1000);//TODO | ||
1550 | 1546 | + DBUG_VOID_RETURN; | ||
1551 | 1547 | +} | ||
1552 | 1548 | + | ||
1553 | 1549 | +void deinit_row_cache() { | ||
1554 | 1550 | + DBUG_ENTER("deinit_row_cache"); | ||
1555 | 1551 | + deinit_innodb_row_cache_lru(); | ||
1556 | 1552 | + deinit_row_cache_hash(innodb_row_cache_on); | ||
1557 | 1553 | + deinit_row_cache_mem_pool(); | ||
1558 | 1554 | + /*deinit_row_cache_filter(innodb_row_cache_on);*/ // deinitde in innobase_shutdown_for_mysql*/ | ||
1559 | 1555 | + DBUG_VOID_RETURN; | ||
1560 | 1556 | +} | ||
1561 | 1557 | + | ||
1562 | 1558 | +static row_cache_value_t* create_row_cache_value(const rec_t* rec, | ||
1563 | 1559 | + const ulint* offsets, ulint fold, const dict_index_t* index, | ||
1564 | 1560 | + ibool is_sec_index) { | ||
1565 | 1561 | + ulint buf_size; | ||
1566 | 1562 | + row_cache_value_t* value = ca_malloc_for_value(fold); | ||
1567 | 1563 | + if (value == NULL) { | ||
1568 | 1564 | + //not enough mem | ||
1569 | 1565 | + ut_print_timestamp(stderr); | ||
1570 | 1566 | + fprintf( | ||
1571 | 1567 | + stderr, | ||
1572 | 1568 | + "[Warnning] malloc row_cache_value_t failded in create_row_cache_value !\n"); | ||
1573 | 1569 | + return NULL; | ||
1574 | 1570 | + } | ||
1575 | 1571 | + buf_size = rec_offs_extra_size(offsets) + rec_offs_data_size(offsets); | ||
1576 | 1572 | + if (is_sec_index) { | ||
1577 | 1573 | + buf_size += sizeof(trx_id_t); | ||
1578 | 1574 | + } | ||
1579 | 1575 | + value->buf = (rec_t*) ca_malloc(buf_size , fold); | ||
1580 | 1576 | + if (value->buf == NULL) { | ||
1581 | 1577 | + //not enough mem | ||
1582 | 1578 | + ut_print_timestamp(stderr); | ||
1583 | 1579 | + fprintf( | ||
1584 | 1580 | + stderr, | ||
1585 | 1581 | + "[Warnning] malloc rec copy buf failded in create_row_cache_value !\n"); | ||
1586 | 1582 | + ca_free_for_value(value, fold); | ||
1587 | 1583 | + return NULL; | ||
1588 | 1584 | + } | ||
1589 | 1585 | + memset(value->buf, 0, buf_size); | ||
1590 | 1586 | + value->fold = fold; | ||
1591 | 1587 | + value->tree_id = index->id; | ||
1592 | 1588 | + value->table_id = index->table->id; | ||
1593 | 1589 | + value->rec = rec_copy(value->buf, rec, offsets); | ||
1594 | 1590 | + if (is_sec_index) { | ||
1595 | 1591 | + trx_id_t* trx_id_in_rec = (trx_id_t*) (value->buf + buf_size | ||
1596 | 1592 | + - sizeof(trx_id_t)); | ||
1597 | 1593 | + *trx_id_in_rec = page_get_max_trx_id(page_align(rec)); | ||
1598 | 1594 | + } | ||
1599 | 1595 | + value->buf_size = buf_size; | ||
1600 | 1596 | + value->ref_num = 0; | ||
1601 | 1597 | + value->chain.value = value; | ||
1602 | 1598 | + return value; | ||
1603 | 1599 | +} | ||
1604 | 1600 | + | ||
1605 | 1601 | +static int update_row_cache_value(row_cache_value_t* value, const rec_t* rec, | ||
1606 | 1602 | + const ulint* offsets, ulint fold, const dict_index_t* index, | ||
1607 | 1603 | + ibool is_sec_index) { | ||
1608 | 1604 | + ulint buf_size; | ||
1609 | 1605 | + ca_free(value->buf, value->buf_size, fold); | ||
1610 | 1606 | + buf_size = rec_offs_extra_size(offsets) + rec_offs_data_size(offsets); | ||
1611 | 1607 | + if (is_sec_index) { | ||
1612 | 1608 | + buf_size += sizeof(trx_id_t); | ||
1613 | 1609 | + } | ||
1614 | 1610 | + value->buf = (rec_t*) ca_malloc(buf_size , fold); | ||
1615 | 1611 | + if (value->buf == NULL) { | ||
1616 | 1612 | + //not enough mem | ||
1617 | 1613 | + ut_print_timestamp(stderr); | ||
1618 | 1614 | + fprintf( | ||
1619 | 1615 | + stderr, | ||
1620 | 1616 | + "[Warnning] malloc rec copy buf failded in create_row_cache_value !\n"); | ||
1621 | 1617 | + return NOT_ENOUGH_MEM; | ||
1622 | 1618 | + } | ||
1623 | 1619 | + memset(value->buf, 0, buf_size); | ||
1624 | 1620 | + value->fold = fold; | ||
1625 | 1621 | + value->tree_id = index->id; | ||
1626 | 1622 | + value->table_id = index->table->id; | ||
1627 | 1623 | + value->rec = rec_copy(value->buf, rec, offsets); | ||
1628 | 1624 | + if (is_sec_index) { | ||
1629 | 1625 | + trx_id_t* trx_id_in_rec = (trx_id_t*) (value->buf + buf_size | ||
1630 | 1626 | + - sizeof(trx_id_t)); | ||
1631 | 1627 | + *trx_id_in_rec = page_get_max_trx_id(page_align(rec)); | ||
1632 | 1628 | + } | ||
1633 | 1629 | + value->buf_size = buf_size; | ||
1634 | 1630 | + //trun off is_removed | ||
1635 | 1631 | + offBit(value->flag, FLAG_VALUE_IS_REMOVED); | ||
1636 | 1632 | + value->ref_num = 0; | ||
1637 | 1633 | + return 0; | ||
1638 | 1634 | +} | ||
1639 | 1635 | + | ||
1640 | 1636 | +void put_rec_to_row_cache(const dtuple_t* tuple, const rec_t* rec, /*!< in: physical record */ | ||
1641 | 1637 | +const ulint* offsets, /*!< in: array returned by rec_get_offsets() */ | ||
1642 | 1638 | +const dict_index_t* index, ibool is_sec_index) { | ||
1643 | 1639 | + row_cache_value_t* value; | ||
1644 | 1640 | + ulint fold; | ||
1645 | 1641 | + if (!innodb_row_cache_on) { | ||
1646 | 1642 | + return; | ||
1647 | 1643 | + } | ||
1648 | 1644 | + if (!is_index_need_cache(index)) { | ||
1649 | 1645 | + return; | ||
1650 | 1646 | + } | ||
1651 | 1647 | + //if rec is deleted ,it can,t be put into row cache | ||
1652 | 1648 | + if (rec_get_deleted_flag(rec, dict_table_is_comp(index->table))) { | ||
1653 | 1649 | + return; | ||
1654 | 1650 | + } | ||
1655 | 1651 | + fold = calc_fold_by_rec(rec, offsets, dict_index_get_n_unique(index), | ||
1656 | 1652 | + index->id); | ||
1657 | 1653 | + row_cache_enter_mutex(fold); | ||
1658 | 1654 | + value = search_row_cache_value(tuple, index, fold); | ||
1659 | 1655 | + if (value == NULL) { | ||
1660 | 1656 | + value = create_row_cache_value(rec, offsets, fold, index, is_sec_index); | ||
1661 | 1657 | + if (value != NULL) { | ||
1662 | 1658 | + insert_row_cache_value(fold, value); | ||
1663 | 1659 | + add_row_cache_value_to_lru(value); | ||
1664 | 1660 | + } | ||
1665 | 1661 | + } else { | ||
1666 | 1662 | + if (isValueRemoved(value->flag) && value->ref_num == 0) { | ||
1667 | 1663 | + //can be overwrite | ||
1668 | 1664 | + if (update_row_cache_value(value, rec, offsets, fold, index, | ||
1669 | 1665 | + is_sec_index) == NOT_ENOUGH_MEM) { | ||
1670 | 1666 | + delete_row_cache_value(value); | ||
1671 | 1667 | + remove_row_cache_value_from_lru(value); | ||
1672 | 1668 | + ca_free_for_value(value, fold); //value->buf already free in update_row_cache_value() | ||
1673 | 1669 | + } else { | ||
1674 | 1670 | + make_row_cache_value_first_from_lru(value); | ||
1675 | 1671 | + } | ||
1676 | 1672 | + } | ||
1677 | 1673 | + } | ||
1678 | 1674 | + row_cache_exit_mutex(fold); | ||
1679 | 1675 | +} | ||
1680 | 1676 | + | ||
1681 | 1677 | +static int add_row_cache_value_to_mtr(mtr_t* mtr, row_cache_value_t* value) { | ||
1682 | 1678 | + row_cache_value_queue_t* row_cache_value_queue; | ||
1683 | 1679 | + row_cache_value_queue = (row_cache_value_queue_t*) ca_malloc_for_queue( | ||
1684 | 1680 | + value->fold); | ||
1685 | 1681 | + if (NULL == row_cache_value_queue) { | ||
1686 | 1682 | + ut_print_timestamp(stderr); | ||
1687 | 1683 | + fprintf( | ||
1688 | 1684 | + stderr, | ||
1689 | 1685 | + "[Warnning] malloc row_cache_value_queue_t failded in add_row_cache_value_to_mtr!\n"); | ||
1690 | 1686 | + return NOT_ENOUGH_MEM; | ||
1691 | 1687 | + } | ||
1692 | 1688 | + row_cache_value_queue->value = value; | ||
1693 | 1689 | + UT_LIST_ADD_LAST(list, mtr->row_cache_value_queue_base, | ||
1694 | 1690 | + row_cache_value_queue); | ||
1695 | 1691 | + return 0; | ||
1696 | 1692 | +} | ||
1697 | 1693 | + | ||
1698 | 1694 | +void release_row_cache_value_in_mtr(mtr_t* mtr) { | ||
1699 | 1695 | + row_cache_value_queue_t* row_cache_value_queue; | ||
1700 | 1696 | + if (!innodb_row_cache_on) { | ||
1701 | 1697 | + return; | ||
1702 | 1698 | + } | ||
1703 | 1699 | + while (mtr->row_cache_value_queue_base.count != 0) { | ||
1704 | 1700 | + ulint fold; | ||
1705 | 1701 | + row_cache_value_queue = mtr->row_cache_value_queue_base.end; | ||
1706 | 1702 | + //release Reference | ||
1707 | 1703 | + fold = row_cache_value_queue->value->fold; | ||
1708 | 1704 | + row_cache_enter_mutex(fold); | ||
1709 | 1705 | + row_cache_value_queue->value->ref_num--; | ||
1710 | 1706 | + UT_LIST_REMOVE(list, mtr->row_cache_value_queue_base, | ||
1711 | 1707 | + row_cache_value_queue); | ||
1712 | 1708 | + ca_free_for_queue(row_cache_value_queue, fold); | ||
1713 | 1709 | + row_cache_exit_mutex(fold); | ||
1714 | 1710 | + } | ||
1715 | 1711 | +} | ||
1716 | 1712 | + | ||
1717 | 1713 | +static void free_row_cache_value(row_cache_value_t* value) { | ||
1718 | 1714 | + ulint fold = value->fold; | ||
1719 | 1715 | + ca_free(value->buf, value->buf_size, fold); | ||
1720 | 1716 | + ca_free_for_value(value, fold); | ||
1721 | 1717 | +} | ||
1722 | 1718 | + | ||
1723 | 1719 | +int contain_row_cache_low(const ulint fold, const dtuple_t* tuple, | ||
1724 | 1720 | + const dict_index_t* index) { | ||
1725 | 1721 | + row_cache_value_t* value; | ||
1726 | 1722 | + int ret = 0; | ||
1727 | 1723 | + if (!innodb_row_cache_on) { | ||
1728 | 1724 | + return ret; | ||
1729 | 1725 | + } | ||
1730 | 1726 | + if (!is_index_need_cache(index)) { | ||
1731 | 1727 | + return ret; | ||
1732 | 1728 | + } | ||
1733 | 1729 | + row_cache_enter_mutex(fold); | ||
1734 | 1730 | + value = search_row_cache_value(tuple, index, fold); | ||
1735 | 1731 | + if (value != NULL && !isValueRemoved(value->flag)) { | ||
1736 | 1732 | + ret = 1; | ||
1737 | 1733 | + } | ||
1738 | 1734 | + row_cache_exit_mutex(fold); | ||
1739 | 1735 | + return ret; | ||
1740 | 1736 | +} | ||
1741 | 1737 | + | ||
1742 | 1738 | +rec_t* get_from_row_cache_low(const ulint fold, const dtuple_t* tuple, | ||
1743 | 1739 | + const dict_index_t* index, mtr_t* mtr) { | ||
1744 | 1740 | + row_cache_value_t* value; | ||
1745 | 1741 | + if (!innodb_row_cache_on) { | ||
1746 | 1742 | + return NULL; | ||
1747 | 1743 | + } | ||
1748 | 1744 | + if (!is_index_need_cache(index)) { | ||
1749 | 1745 | + return NULL; | ||
1750 | 1746 | + } | ||
1751 | 1747 | + row_cache_stat->n_get++; | ||
1752 | 1748 | + row_cache_enter_mutex(fold); | ||
1753 | 1749 | + value = search_row_cache_value(tuple, index, fold); | ||
1754 | 1750 | + if (value != NULL) { | ||
1755 | 1751 | + if (isValueRemoved(value->flag) && value->ref_num == 0) { | ||
1756 | 1752 | + //can be free | ||
1757 | 1753 | + delete_row_cache_value(value); | ||
1758 | 1754 | + remove_row_cache_value_from_lru(value); | ||
1759 | 1755 | + free_row_cache_value(value); | ||
1760 | 1756 | + value = NULL; | ||
1761 | 1757 | + } else if (!isValueRemoved(value->flag)) { | ||
1762 | 1758 | + //can be read | ||
1763 | 1759 | + if (add_row_cache_value_to_mtr(mtr, value) == 0) { | ||
1764 | 1760 | + make_row_cache_value_first_from_lru(value); | ||
1765 | 1761 | + value->ref_num++; | ||
1766 | 1762 | + } else { | ||
1767 | 1763 | + //not have enough mem | ||
1768 | 1764 | + value = NULL; | ||
1769 | 1765 | + } | ||
1770 | 1766 | + | ||
1771 | 1767 | + } else { | ||
1772 | 1768 | + // is removed can't be read | ||
1773 | 1769 | + value = NULL; | ||
1774 | 1770 | + } | ||
1775 | 1771 | + } | ||
1776 | 1772 | + row_cache_exit_mutex(fold); | ||
1777 | 1773 | + if (value != NULL) { | ||
1778 | 1774 | + row_cache_stat->geted++; | ||
1779 | 1775 | + return value->rec; | ||
1780 | 1776 | + } | ||
1781 | 1777 | + return NULL; | ||
1782 | 1778 | +} | ||
1783 | 1779 | + | ||
1784 | 1780 | +void remove_from_row_cache_low(const ulint fold, const rec_t* rec, | ||
1785 | 1781 | + const ulint* offsets, const dtuple_t* tuple, dict_index_t* index) { | ||
1786 | 1782 | + row_cache_value_t* value = NULL; | ||
1787 | 1783 | + if (!innodb_row_cache_on) { | ||
1788 | 1784 | + return; | ||
1789 | 1785 | + } | ||
1790 | 1786 | + row_cache_enter_mutex(fold); | ||
1791 | 1787 | + if (rec && offsets) { | ||
1792 | 1788 | + value = search_row_cache_value_with_rec(rec, offsets, index, fold); | ||
1793 | 1789 | + } else if (tuple) { | ||
1794 | 1790 | + value = search_row_cache_value(tuple, index, fold); | ||
1795 | 1791 | + } | ||
1796 | 1792 | + if (value != NULL) { | ||
1797 | 1793 | + if (value->ref_num == 0) { | ||
1798 | 1794 | + //can be free | ||
1799 | 1795 | + delete_row_cache_value(value); | ||
1800 | 1796 | + remove_row_cache_value_from_lru(value); | ||
1801 | 1797 | + free_row_cache_value(value); | ||
1802 | 1798 | + value = NULL; | ||
1803 | 1799 | + } else { | ||
1804 | 1800 | + //using just set is_removed = 1 | ||
1805 | 1801 | + onBit(value->flag, FLAG_VALUE_IS_REMOVED); | ||
1806 | 1802 | + } | ||
1807 | 1803 | + } | ||
1808 | 1804 | + row_cache_exit_mutex(fold); | ||
1809 | 1805 | +} | ||
1810 | 1806 | + | ||
1811 | 1807 | +ulint calc_fold_by_rec(const rec_t* rec, /*!< in: the physical record */ | ||
1812 | 1808 | +const ulint* offsets, /*!< in: array returned by rec_get_offsets() */ | ||
1813 | 1809 | +ulint n_fields, /*!< in: number of complete fields to fold */ | ||
1814 | 1810 | +index_id_t tree_id) { | ||
1815 | 1811 | + return rec_fold(rec, offsets, n_fields, 0, tree_id); | ||
1816 | 1812 | +} | ||
1817 | 1813 | + | ||
1818 | 1814 | +ulint lock_sec_rec_in_row_cache_cons_read_sees(const rec_t* rec, | ||
1819 | 1815 | + const ulint* offsets, const read_view_t* view) { | ||
1820 | 1816 | + trx_id_t* max_trx_id; | ||
1821 | 1817 | + if (recv_recovery_is_on()) { | ||
1822 | 1818 | + | ||
1823 | 1819 | + return (FALSE); | ||
1824 | 1820 | + } | ||
1825 | 1821 | + | ||
1826 | 1822 | + max_trx_id = (trx_id_t*) (rec + rec_offs_data_size(offsets)); | ||
1827 | 1823 | + ut_ad((*max_trx_id)!=0); | ||
1828 | 1824 | + | ||
1829 | 1825 | + return (*max_trx_id) < (view->up_limit_id); | ||
1830 | 1826 | +// return(ut_dulint_cmp( *max_trx_id, view->up_limit_id) < 0); | ||
1831 | 1827 | +} | ||
1832 | 1828 | + | ||
1833 | 1829 | +void row_cache_refresh_stats() { | ||
1834 | 1830 | + row_cache_stat->last_printout_time = time(NULL); | ||
1835 | 1831 | + row_cache_stat->old_geted = row_cache_stat->geted; | ||
1836 | 1832 | + row_cache_stat->old_n_get = row_cache_stat->n_get; | ||
1837 | 1833 | + row_cache_lru_stat->old_n_add = row_cache_lru_stat->n_add; | ||
1838 | 1834 | + row_cache_lru_stat->old_n_evict = row_cache_lru_stat->n_evict; | ||
1839 | 1835 | + row_cache_lru_stat->old_n_make_first = row_cache_lru_stat->n_make_first; | ||
1840 | 1836 | +} | ||
1841 | 1837 | + | ||
1842 | 1838 | +void print_row_cache_stats(FILE* file) { | ||
1843 | 1839 | + time_t current_time; | ||
1844 | 1840 | + double time_elapsed; | ||
1845 | 1841 | + ulint n_gets_diff = 0; | ||
1846 | 1842 | + unsigned long long n_geted_diff = 0; | ||
1847 | 1843 | + unsigned long long mem_pool_used = row_cache_mem_pool_used(); | ||
1848 | 1844 | + | ||
1849 | 1845 | + fputs("----------------------\n" | ||
1850 | 1846 | + "ROW CACHE\n" | ||
1851 | 1847 | + "----------------------\n", file); | ||
1852 | 1848 | + fprintf(file, "Total memory allocated " ULINTPF | ||
1853 | 1849 | + "; used " ULINTPF " (" ULINTPF " / 1000)" | ||
1854 | 1850 | + "; additional pool allocated " ULINTPF | ||
1855 | 1851 | + "; Total LRU count " ULINTPF"\n", (ulong) innodb_row_cache_mem_pool_size, | ||
1856 | 1852 | + (ulong) mem_pool_used, | ||
1857 | 1853 | + (ulong) (1000 * mem_pool_used / innodb_row_cache_mem_pool_size), | ||
1858 | 1854 | + (ulong) innodb_row_cache_additional_mem_pool_size, | ||
1859 | 1855 | + (ulong) get_row_cache_lru_count()); | ||
1860 | 1856 | + | ||
1861 | 1857 | + fprintf(file, "Free Value Count " ULINTPF | ||
1862 | 1858 | + "; Free Queue Count " ULINTPF "\n", | ||
1863 | 1859 | + (ulong) row_cache_get_value_free_count(), | ||
1864 | 1860 | + (ulong) row_cache_get_queue_free_count()); | ||
1865 | 1861 | + | ||
1866 | 1862 | + current_time = time(NULL); | ||
1867 | 1863 | + time_elapsed = 0.001 | ||
1868 | 1864 | + + difftime(current_time, row_cache_stat->last_printout_time); | ||
1869 | 1865 | + | ||
1870 | 1866 | + n_geted_diff = row_cache_stat->geted - row_cache_stat->old_geted; | ||
1871 | 1867 | + n_gets_diff = row_cache_stat->n_get - row_cache_stat->old_n_get; | ||
1872 | 1868 | + | ||
1873 | 1869 | + fprintf( | ||
1874 | 1870 | + file, | ||
1875 | 1871 | + "Row total add " ULINTPF " , %.2f add/s \n" | ||
1876 | 1872 | + "Row total make first " ULINTPF " , %.2f mf/s \n" | ||
1877 | 1873 | + "Row total evict " ULINTPF " , %.2f evict/s \n" | ||
1878 | 1874 | + "Row read from cache " ULINTPF ", %.2f read/s \n" | ||
1879 | 1875 | + "Row get from cache " ULINTPF ", %.2f get/s \n", | ||
1880 | 1876 | + (ulong) row_cache_lru_stat->n_add, | ||
1881 | 1877 | + (row_cache_lru_stat->n_add - row_cache_lru_stat->old_n_add) | ||
1882 | 1878 | + / time_elapsed, | ||
1883 | 1879 | + (ulong) row_cache_lru_stat->n_make_first, | ||
1884 | 1880 | + (row_cache_lru_stat->n_make_first | ||
1885 | 1881 | + - row_cache_lru_stat->old_n_make_first) / time_elapsed, | ||
1886 | 1882 | + (ulong) row_cache_lru_stat->n_evict, | ||
1887 | 1883 | + (row_cache_lru_stat->n_evict - row_cache_lru_stat->old_n_evict) | ||
1888 | 1884 | + / time_elapsed, (ulong) row_cache_stat->n_get, | ||
1889 | 1885 | + n_gets_diff / time_elapsed, (ulong) row_cache_stat->geted, | ||
1890 | 1886 | + n_geted_diff / time_elapsed); | ||
1891 | 1887 | + | ||
1892 | 1888 | + if (n_gets_diff) { | ||
1893 | 1889 | + fprintf(file, "Row cache hit rate %lu / 1000 \n", | ||
1894 | 1890 | + (ulong) (1000 * n_geted_diff) / n_gets_diff); | ||
1895 | 1891 | + } else { | ||
1896 | 1892 | + fputs("No row cache gets since the last printout\n", file); | ||
1897 | 1893 | + } | ||
1898 | 1894 | + | ||
1899 | 1895 | + row_cache_refresh_stats(); | ||
1900 | 1896 | +} | ||
1901 | 1897 | |||
1902 | 1898 | Property changes on: storage/innobase/cache/row0cache.c | ||
1903 | 1899 | ___________________________________________________________________ | ||
1904 | 1900 | Added: svn:mime-type | ||
1905 | 1901 | + text/plain | ||
1906 | 1902 | |||
1907 | 1903 | Index: storage/innobase/srv/srv0srv.c | ||
1908 | 1904 | =================================================================== | ||
1909 | 1905 | --- storage/innobase/srv/srv0srv.c (revision 498) | ||
1910 | 1906 | +++ storage/innobase/srv/srv0srv.c (revision 724) | ||
1911 | 1907 | @@ -86,6 +86,9 @@ | ||
1912 | 1908 | #include "read0read.h" | ||
1913 | 1909 | #include "mysql/plugin.h" | ||
1914 | 1910 | #include "mysql/service_thd_wait.h" | ||
1915 | 1911 | +#include "row0cache.h" /* for row cache*/ | ||
1916 | 1912 | +#include "row0cache0lru.h" | ||
1917 | 1913 | +#include "row0cache0mempool.h" | ||
1918 | 1914 | |||
1919 | 1915 | /* prototypes of new functions added to ha_innodb.cc for kill_idle_transaction */ | ||
1920 | 1916 | ibool innobase_thd_is_idle(const void* thd); | ||
1921 | 1917 | @@ -1139,6 +1142,8 @@ | ||
1922 | 1918 | |||
1923 | 1919 | /* Initialize some INFORMATION SCHEMA internal structures */ | ||
1924 | 1920 | trx_i_s_cache_init(trx_i_s_cache); | ||
1925 | 1921 | + | ||
1926 | 1922 | + init_row_cache(); | ||
1927 | 1923 | } | ||
1928 | 1924 | |||
1929 | 1925 | /*********************************************************************//** | ||
1930 | 1926 | @@ -1162,6 +1167,8 @@ | ||
1931 | 1927 | srv_mysql_table = NULL; | ||
1932 | 1928 | |||
1933 | 1929 | trx_i_s_cache_free(trx_i_s_cache); | ||
1934 | 1930 | + | ||
1935 | 1931 | + deinit_row_cache(); | ||
1936 | 1932 | } | ||
1937 | 1933 | |||
1938 | 1934 | /*********************************************************************//** | ||
1939 | 1935 | @@ -1967,6 +1974,8 @@ | ||
1940 | 1936 | |||
1941 | 1937 | buf_refresh_io_stats_all(); | ||
1942 | 1938 | |||
1943 | 1939 | + row_cache_refresh_stats(); | ||
1944 | 1940 | + | ||
1945 | 1941 | srv_n_rows_inserted_old = srv_n_rows_inserted; | ||
1946 | 1942 | srv_n_rows_updated_old = srv_n_rows_updated; | ||
1947 | 1943 | srv_n_rows_deleted_old = srv_n_rows_deleted; | ||
1948 | 1944 | @@ -2160,6 +2169,8 @@ | ||
1949 | 1945 | |||
1950 | 1946 | buf_print_io(file); | ||
1951 | 1947 | |||
1952 | 1948 | + print_row_cache_stats(file); | ||
1953 | 1949 | + | ||
1954 | 1950 | fputs("--------------\n" | ||
1955 | 1951 | "ROW OPERATIONS\n" | ||
1956 | 1952 | "--------------\n", file); | ||
1957 | 1953 | @@ -2471,6 +2482,19 @@ | ||
1958 | 1954 | export_vars.innodb_rows_deleted = srv_n_rows_deleted; | ||
1959 | 1955 | export_vars.innodb_truncated_status_writes = srv_truncated_status_writes; | ||
1960 | 1956 | |||
1961 | 1957 | + | ||
1962 | 1958 | + export_vars.innodb_row_cache_n_get = row_cache_stat->n_get; | ||
1963 | 1959 | + export_vars.innodb_row_cache_geted = row_cache_stat->geted; | ||
1964 | 1960 | + | ||
1965 | 1961 | + export_vars.innodb_row_cache_lru_n_add = row_cache_lru_stat->n_add; | ||
1966 | 1962 | + export_vars.innodb_row_cache_lru_n_make_first = row_cache_lru_stat->n_make_first; | ||
1967 | 1963 | + export_vars.innodb_row_cache_lru_n_evict = row_cache_lru_stat->n_evict; | ||
1968 | 1964 | + export_vars.innodb_row_cache_lru_count = get_row_cache_lru_count(); | ||
1969 | 1965 | + | ||
1970 | 1966 | + export_vars.innodb_row_cache_mem_pool_size = innodb_row_cache_mem_pool_size; | ||
1971 | 1967 | + export_vars.innodb_row_cache_mem_pool_used = row_cache_mem_pool_used(); | ||
1972 | 1968 | + | ||
1973 | 1969 | + | ||
1974 | 1970 | mutex_exit(&srv_innodb_monitor_mutex); | ||
1975 | 1971 | } | ||
1976 | 1972 | |||
1977 | 1973 | Index: storage/innobase/srv/srv0start.c | ||
1978 | 1974 | =================================================================== | ||
1979 | 1975 | --- storage/innobase/srv/srv0start.c (revision 498) | ||
1980 | 1976 | +++ storage/innobase/srv/srv0start.c (revision 724) | ||
1981 | 1977 | @@ -62,6 +62,8 @@ | ||
1982 | 1978 | #include "ibuf0ibuf.h" | ||
1983 | 1979 | #include "srv0start.h" | ||
1984 | 1980 | #include "srv0srv.h" | ||
1985 | 1981 | +#include "row0cache.h" | ||
1986 | 1982 | +#include "row0cache0filter.h" | ||
1987 | 1983 | #ifndef UNIV_HOTBACKUP | ||
1988 | 1984 | # include "os0proc.h" | ||
1989 | 1985 | # include "sync0sync.h" | ||
1990 | 1986 | @@ -2390,6 +2392,7 @@ | ||
1991 | 1987 | /* 3. Free all InnoDB's own mutexes and the os_fast_mutexes inside | ||
1992 | 1988 | them */ | ||
1993 | 1989 | os_aio_free(); | ||
1994 | 1990 | + deinit_row_cache_filter(innodb_row_cache_on); | ||
1995 | 1991 | sync_close(); | ||
1996 | 1992 | srv_free(); | ||
1997 | 1993 | fil_close(); | ||
1998 | 1994 | Index: storage/innobase/CMakeLists.txt | ||
1999 | 1995 | =================================================================== | ||
2000 | 1996 | --- storage/innobase/CMakeLists.txt (revision 498) | ||
2001 | 1997 | +++ storage/innobase/CMakeLists.txt (revision 724) | ||
2002 | 1998 | @@ -245,7 +245,9 @@ | ||
2003 | 1999 | trx/trx0sys.c trx/trx0trx.c trx/trx0undo.c | ||
2004 | 2000 | usr/usr0sess.c | ||
2005 | 2001 | ut/ut0byte.c ut/ut0dbg.c ut/ut0list.c ut/ut0mem.c ut/ut0rbt.c ut/ut0rnd.c | ||
2006 | 2002 | - ut/ut0ut.c ut/ut0vec.c ut/ut0wqueue.c ut/ut0bh.c) | ||
2007 | 2003 | + ut/ut0ut.c ut/ut0vec.c ut/ut0wqueue.c ut/ut0bh.c | ||
2008 | 2004 | + cache/row0cache.c cache/row0cache0hash.c cache/row0cache0lru.c cache/row0cache0mempool.c | ||
2009 | 2005 | + cache/row0cache0filter.c) | ||
2010 | 2006 | |||
2011 | 2007 | IF(WITH_INNODB) | ||
2012 | 2008 | # Legacy option | ||
2013 | 2009 | Index: storage/innobase/handler/ha_innodb.cc | ||
2014 | 2010 | =================================================================== | ||
2015 | 2011 | --- storage/innobase/handler/ha_innodb.cc (revision 498) | ||
2016 | 2012 | +++ storage/innobase/handler/ha_innodb.cc (revision 724) | ||
2017 | 2013 | @@ -97,6 +97,10 @@ | ||
2018 | 2014 | #include "ha_prototypes.h" | ||
2019 | 2015 | #include "ut0mem.h" | ||
2020 | 2016 | #include "ibuf0ibuf.h" | ||
2021 | 2017 | +#include "row0cache0mempool.h" | ||
2022 | 2018 | +#include "row0cache0hash.h" | ||
2023 | 2019 | +#include "row0cache0filter.h" | ||
2024 | 2020 | +#include "row0cache0lru.h" | ||
2025 | 2021 | } | ||
2026 | 2022 | |||
2027 | 2023 | #include "ha_innodb.h" | ||
2028 | 2024 | @@ -342,7 +346,8 @@ | ||
2029 | 2025 | {&trx_i_s_cache_lock_key, "trx_i_s_cache_lock", 0}, | ||
2030 | 2026 | {&trx_purge_latch_key, "trx_purge_latch", 0}, | ||
2031 | 2027 | {&index_tree_rw_lock_key, "index_tree_rw_lock", 0}, | ||
2032 | 2028 | - {&dict_table_stats_latch_key, "dict_table_stats", 0} | ||
2033 | 2029 | + {&dict_table_stats_latch_key, "dict_table_stats", 0}, | ||
2034 | 2030 | + {&row_cache_filter_lock_key, "row_cache_filter_lock", 0} | ||
2035 | 2031 | }; | ||
2036 | 2032 | # endif /* UNIV_PFS_RWLOCK */ | ||
2037 | 2033 | |||
2038 | 2034 | @@ -844,6 +849,22 @@ | ||
2039 | 2035 | (char*) &export_vars.innodb_x_lock_spin_rounds, SHOW_LONGLONG}, | ||
2040 | 2036 | {"x_lock_spin_waits", | ||
2041 | 2037 | (char*) &export_vars.innodb_x_lock_spin_waits, SHOW_LONGLONG}, | ||
2042 | 2038 | + {"row_cache_n_get", | ||
2043 | 2039 | + (char*) &export_vars.innodb_row_cache_n_get, SHOW_LONG}, | ||
2044 | 2040 | + {"row_cache_n_geted", | ||
2045 | 2041 | + (char*) &export_vars.innodb_row_cache_geted, SHOW_LONG}, | ||
2046 | 2042 | + {"row_cache_lru_count", | ||
2047 | 2043 | + (char*) &export_vars.innodb_row_cache_lru_count, SHOW_LONG}, | ||
2048 | 2044 | + {"row_cache_lru_n_add", | ||
2049 | 2045 | + (char*) &export_vars.innodb_row_cache_lru_n_add, SHOW_LONG}, | ||
2050 | 2046 | + {"row_cache_lru_n_evict", | ||
2051 | 2047 | + (char*) &export_vars.innodb_row_cache_lru_n_evict, SHOW_LONG}, | ||
2052 | 2048 | + {"row_cache_lru_n_make_first", | ||
2053 | 2049 | + (char*) &export_vars.innodb_row_cache_lru_n_make_first, SHOW_LONG}, | ||
2054 | 2050 | + {"row_cache_mem_pool_size", | ||
2055 | 2051 | + (char*) &export_vars.innodb_row_cache_mem_pool_size, SHOW_LONGLONG}, | ||
2056 | 2052 | + {"row_cache_mem_pool_used", | ||
2057 | 2053 | + (char*) &export_vars.innodb_row_cache_mem_pool_used, SHOW_LONG}, | ||
2058 | 2054 | {NullS, NullS, SHOW_LONG} | ||
2059 | 2055 | }; | ||
2060 | 2056 | |||
2061 | 2057 | @@ -6330,6 +6351,37 @@ | ||
2062 | 2058 | start of a new SQL statement. */ | ||
2063 | 2059 | |||
2064 | 2060 | |||
2065 | 2061 | +UNIV_INTERN | ||
2066 | 2062 | +bool ha_innobase::is_in_cache(const uchar * key_ptr, uint key_len) { | ||
2067 | 2063 | + dict_index_t* index; | ||
2068 | 2064 | + bool ret = false; | ||
2069 | 2065 | + | ||
2070 | 2066 | + DBUG_ENTER("index_read"); | ||
2071 | 2067 | + ut_a(prebuilt->trx == thd_to_trx(user_thd)); | ||
2072 | 2068 | + index = prebuilt->index; | ||
2073 | 2069 | + | ||
2074 | 2070 | + if (UNIV_UNLIKELY(index == NULL) || dict_index_is_corrupted(index)) { | ||
2075 | 2071 | + prebuilt->index_usable = FALSE; | ||
2076 | 2072 | + DBUG_RETURN(ret); | ||
2077 | 2073 | + } | ||
2078 | 2074 | + if (UNIV_UNLIKELY(!prebuilt->index_usable)) { | ||
2079 | 2075 | + DBUG_RETURN(ret); | ||
2080 | 2076 | + } | ||
2081 | 2077 | + | ||
2082 | 2078 | + if (key_ptr) { | ||
2083 | 2079 | + /* Convert the search key value to InnoDB format into | ||
2084 | 2080 | + prebuilt->search_tuple */ | ||
2085 | 2081 | + | ||
2086 | 2082 | + row_sel_convert_mysql_key_to_innobase(prebuilt->search_tuple, | ||
2087 | 2083 | + (byte*) key_val_buff, (ulint) upd_and_key_val_buff_len, index, | ||
2088 | 2084 | + (byte*) key_ptr, (ulint) key_len, prebuilt->trx); | ||
2089 | 2085 | + | ||
2090 | 2086 | + ret = (contain_row_cache(prebuilt->search_tuple,index) == 1); | ||
2091 | 2087 | + } | ||
2092 | 2088 | + | ||
2093 | 2089 | + DBUG_RETURN(ret); | ||
2094 | 2090 | +} | ||
2095 | 2091 | + | ||
2096 | 2092 | /**********************************************************************//** | ||
2097 | 2093 | Positions an index cursor to the index specified in the handle. Fetches the | ||
2098 | 2094 | row if any. | ||
2099 | 2095 | @@ -12556,6 +12608,89 @@ | ||
2100 | 2096 | "e.g. for http://bugs.mysql.com/51325", | ||
2101 | 2097 | NULL, NULL, 0, 0, 1, 0); | ||
2102 | 2098 | |||
2103 | 2099 | + | ||
2104 | 2100 | +static | ||
2105 | 2101 | +void | ||
2106 | 2102 | +innodb_row_cache_index_update( | ||
2107 | 2103 | +/*===========================*/ | ||
2108 | 2104 | +THD* thd, /*!< in: thread handle */ | ||
2109 | 2105 | +struct st_mysql_sys_var* var, /*!< in: pointer to | ||
2110 | 2106 | + system variable */ | ||
2111 | 2107 | +void* var_ptr, /*!< out: where the | ||
2112 | 2108 | + formal string goes */ | ||
2113 | 2109 | +const void* save) /*!< in: immediate result | ||
2114 | 2110 | + from check function */ | ||
2115 | 2111 | +{ | ||
2116 | 2112 | + ut_a(var_ptr != NULL); | ||
2117 | 2113 | + ut_a(save != NULL); | ||
2118 | 2114 | + | ||
2119 | 2115 | + *static_cast<const char**>(var_ptr) = *static_cast<const char*const*>(save); | ||
2120 | 2116 | + reset_filter(); | ||
2121 | 2117 | +} | ||
2122 | 2118 | + | ||
2123 | 2119 | +static | ||
2124 | 2120 | +void | ||
2125 | 2121 | +innodb_row_cache_clean_cache_update( | ||
2126 | 2122 | +/*===========================*/ | ||
2127 | 2123 | +THD* thd, /*!< in: thread handle */ | ||
2128 | 2124 | +struct st_mysql_sys_var* var, /*!< in: pointer to | ||
2129 | 2125 | + system variable */ | ||
2130 | 2126 | +void* var_ptr, /*!< out: where the | ||
2131 | 2127 | + formal string goes */ | ||
2132 | 2128 | +const void* save) /*!< in: immediate result | ||
2133 | 2129 | + from check function */ | ||
2134 | 2130 | +{ | ||
2135 | 2131 | + ut_a(var_ptr != NULL); | ||
2136 | 2132 | + ut_a(save != NULL); | ||
2137 | 2133 | + | ||
2138 | 2134 | + if(*(my_bool*) save){ | ||
2139 | 2135 | + clean_row_cache(); | ||
2140 | 2136 | + } | ||
2141 | 2137 | +} | ||
2142 | 2138 | + | ||
2143 | 2139 | +static MYSQL_SYSVAR_LONGLONG(row_cache_mem_pool_size, innodb_row_cache_mem_pool_size, | ||
2144 | 2140 | + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, | ||
2145 | 2141 | + "The size of the memory buffer InnoDB uses to cache row.", | ||
2146 | 2142 | + NULL, NULL, 1024*1024L, 1024*1024L, LONGLONG_MAX, 0); | ||
2147 | 2143 | + | ||
2148 | 2144 | +static MYSQL_SYSVAR_LONGLONG(row_cache_additional_mem_pool_size, innodb_row_cache_additional_mem_pool_size, | ||
2149 | 2145 | + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, | ||
2150 | 2146 | + "The size of the memory buffer InnoDB uses create buffer for cache row 's struct.", | ||
2151 | 2147 | + NULL, NULL, 1024*1024L, 1024*1024L, LONGLONG_MAX, 0); | ||
2152 | 2148 | + | ||
2153 | 2149 | +static MYSQL_SYSVAR_BOOL(row_cache_on, innodb_row_cache_on, | ||
2154 | 2150 | + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, | ||
2155 | 2151 | + "Enable row cache", | ||
2156 | 2152 | + NULL, NULL, FALSE); | ||
2157 | 2153 | + | ||
2158 | 2154 | +static MYSQL_SYSVAR_ULONG(row_cache_cell_num, innodb_row_cache_cell_num, | ||
2159 | 2155 | + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, | ||
2160 | 2156 | + "Number of row cache 's hash table cell.", | ||
2161 | 2157 | + NULL, NULL, 10000L, 100L, ~0L, 0); | ||
2162 | 2158 | + | ||
2163 | 2159 | +static MYSQL_SYSVAR_UINT(row_cache_mutex_num_shift, innodb_row_cache_mutex_num_shift, | ||
2164 | 2160 | + PLUGIN_VAR_RQCMDARG | PLUGIN_VAR_READONLY, | ||
2165 | 2161 | + "Number of row cache 's hash table mutex number's shift.", | ||
2166 | 2162 | + NULL, NULL, 6, 1, 14, 0); | ||
2167 | 2163 | + | ||
2168 | 2164 | +static MYSQL_SYSVAR_STR(row_cache_index, innodb_row_cache_index, | ||
2169 | 2165 | + PLUGIN_VAR_OPCMDARG, | ||
2170 | 2166 | + "The config of index need to cache.", | ||
2171 | 2167 | + NULL, | ||
2172 | 2168 | + innodb_row_cache_index_update, NULL); | ||
2173 | 2169 | + | ||
2174 | 2170 | +static MYSQL_SYSVAR_BOOL(row_cache_clean_cache, innodb_row_cache_clean_cache, | ||
2175 | 2171 | + PLUGIN_VAR_NOCMDARG, | ||
2176 | 2172 | + "Set ON to Clean cache For DEBUG!", | ||
2177 | 2173 | + NULL, innodb_row_cache_clean_cache_update, FALSE); | ||
2178 | 2174 | + | ||
2179 | 2175 | +static MYSQL_SYSVAR_BOOL(row_cache_use_sys_malloc, innodb_row_cache_use_sys_malloc, | ||
2180 | 2176 | + PLUGIN_VAR_OPCMDARG | PLUGIN_VAR_READONLY, | ||
2181 | 2177 | + "row cache use system malloc (disabled by default)", | ||
2182 | 2178 | + NULL, NULL, FALSE); | ||
2183 | 2179 | + | ||
2184 | 2180 | + | ||
2185 | 2181 | + | ||
2186 | 2182 | static struct st_mysql_sys_var* innobase_system_variables[]= { | ||
2187 | 2183 | MYSQL_SYSVAR(page_size), | ||
2188 | 2184 | MYSQL_SYSVAR(log_block_size), | ||
2189 | 2185 | @@ -12656,6 +12791,14 @@ | ||
2190 | 2186 | MYSQL_SYSVAR(corrupt_table_action), | ||
2191 | 2187 | MYSQL_SYSVAR(lazy_drop_table), | ||
2192 | 2188 | MYSQL_SYSVAR(fake_changes), | ||
2193 | 2189 | + MYSQL_SYSVAR(row_cache_mem_pool_size), | ||
2194 | 2190 | + MYSQL_SYSVAR(row_cache_additional_mem_pool_size), | ||
2195 | 2191 | + MYSQL_SYSVAR(row_cache_on), | ||
2196 | 2192 | + MYSQL_SYSVAR(row_cache_cell_num), | ||
2197 | 2193 | + MYSQL_SYSVAR(row_cache_mutex_num_shift), | ||
2198 | 2194 | + MYSQL_SYSVAR(row_cache_index), | ||
2199 | 2195 | + MYSQL_SYSVAR(row_cache_clean_cache), | ||
2200 | 2196 | + MYSQL_SYSVAR(row_cache_use_sys_malloc), | ||
2201 | 2197 | NULL | ||
2202 | 2198 | }; | ||
2203 | 2199 | |||
2204 | 2200 | Index: storage/innobase/handler/ha_innodb.h | ||
2205 | 2201 | =================================================================== | ||
2206 | 2202 | --- storage/innobase/handler/ha_innodb.h (revision 498) | ||
2207 | 2203 | +++ storage/innobase/handler/ha_innodb.h (revision 724) | ||
2208 | 2204 | @@ -147,6 +147,8 @@ | ||
2209 | 2205 | |||
2210 | 2206 | int index_init(uint index, bool sorted); | ||
2211 | 2207 | int index_end(); | ||
2212 | 2208 | + bool is_in_cache(const uchar * key, | ||
2213 | 2209 | + uint key_len); | ||
2214 | 2210 | int index_read(uchar * buf, const uchar * key, | ||
2215 | 2211 | uint key_len, enum ha_rkey_function find_flag); | ||
2216 | 2212 | int index_read_idx(uchar * buf, uint index, const uchar * key, | ||
2217 | 2213 | Index: storage/innobase/include/row0cache0mempool.h | ||
2218 | 2214 | =================================================================== | ||
2219 | 2215 | --- storage/innobase/include/row0cache0mempool.h (revision 0) | ||
2220 | 2216 | +++ storage/innobase/include/row0cache0mempool.h (revision 724) | ||
2221 | 2217 | @@ -0,0 +1,55 @@ | ||
2222 | 2218 | +/******************************************************************** | ||
2223 | 2219 | + created: 2011/03/23 | ||
2224 | 2220 | + created: 23:3:2011 14:49 | ||
2225 | 2221 | + file base: row0cache0mempool | ||
2226 | 2222 | + file ext: h | ||
2227 | 2223 | + author: wentong@taobao.vom | ||
2228 | 2224 | + | ||
2229 | 2225 | + purpose: | ||
2230 | 2226 | +*********************************************************************/ | ||
2231 | 2227 | + | ||
2232 | 2228 | +#ifndef row0cache0mempool_h__ | ||
2233 | 2229 | +#define row0cache0mempool_h__ | ||
2234 | 2230 | + | ||
2235 | 2231 | +#include "ut0rbt.h" | ||
2236 | 2232 | +#include "row0cache0hash.h" | ||
2237 | 2233 | +#include "row0cache.h" | ||
2238 | 2234 | + | ||
2239 | 2235 | +typedef long long llong; | ||
2240 | 2236 | + | ||
2241 | 2237 | +extern my_bool innodb_row_cache_use_sys_malloc; | ||
2242 | 2238 | + | ||
2243 | 2239 | +extern llong innodb_row_cache_mem_pool_size; | ||
2244 | 2240 | + | ||
2245 | 2241 | +extern llong innodb_row_cache_additional_mem_pool_size; | ||
2246 | 2242 | + | ||
2247 | 2243 | +#define NOT_ENOUGH_MEM 1 | ||
2248 | 2244 | + | ||
2249 | 2245 | +void init_row_cache_mem_pool(my_bool innodb_row_cache_on); | ||
2250 | 2246 | + | ||
2251 | 2247 | +void deinit_row_cache_mem_pool(); | ||
2252 | 2248 | + | ||
2253 | 2249 | +void* ca_malloc_low(ulint n , const ulint used_fold); | ||
2254 | 2250 | + | ||
2255 | 2251 | +void ca_free_low(void* ptr, const ulint size ,const ulint used_fold); | ||
2256 | 2252 | + | ||
2257 | 2253 | +#define ca_malloc(S,F) ca_malloc_low(S,F) | ||
2258 | 2254 | +#define ca_free(P,S,F) ca_free_low(P,S,F) | ||
2259 | 2255 | + | ||
2260 | 2256 | +//#define ca_malloc_for_value(F) ca_malloc(sizeof(row_cache_value_t),F) | ||
2261 | 2257 | +//#define ca_free_for_value(S,F) ca_free(S,F) | ||
2262 | 2258 | + | ||
2263 | 2259 | +//#define ca_malloc_for_queue(F) ca_malloc(sizeof(row_cache_value_queue_t),F) | ||
2264 | 2260 | +//#define ca_free_for_queue(S,F) ca_free(S,F) | ||
2265 | 2261 | + | ||
2266 | 2262 | +row_cache_value_t* ca_malloc_for_value(const ulint used_fold); | ||
2267 | 2263 | +void ca_free_for_value(row_cache_value_t* value, const ulint used_fold); | ||
2268 | 2264 | + | ||
2269 | 2265 | +row_cache_value_queue_t* ca_malloc_for_queue(const ulint used_fold); | ||
2270 | 2266 | +void ca_free_for_queue(row_cache_value_queue_t* value, const ulint used_fold); | ||
2271 | 2267 | + | ||
2272 | 2268 | +ulint row_cache_mem_pool_used(); | ||
2273 | 2269 | +ulint row_cache_get_value_free_count(); | ||
2274 | 2270 | +ulint row_cache_get_queue_free_count(); | ||
2275 | 2271 | + | ||
2276 | 2272 | +#endif // row0cache0mempool_h__ | ||
2277 | 2273 | |||
2278 | 2274 | Property changes on: storage/innobase/include/row0cache0mempool.h | ||
2279 | 2275 | ___________________________________________________________________ | ||
2280 | 2276 | Added: svn:mime-type | ||
2281 | 2277 | + text/plain | ||
2282 | 2278 | |||
2283 | 2279 | Index: storage/innobase/include/srv0srv.h | ||
2284 | 2280 | =================================================================== | ||
2285 | 2281 | --- storage/innobase/include/srv0srv.h (revision 498) | ||
2286 | 2282 | +++ storage/innobase/include/srv0srv.h (revision 724) | ||
2287 | 2283 | @@ -842,6 +842,18 @@ | ||
2288 | 2284 | ib_int64_t innodb_x_lock_os_waits; | ||
2289 | 2285 | ib_int64_t innodb_x_lock_spin_rounds; | ||
2290 | 2286 | ib_int64_t innodb_x_lock_spin_waits; | ||
2291 | 2287 | + | ||
2292 | 2288 | + ulint innodb_row_cache_n_get; | ||
2293 | 2289 | + ulint innodb_row_cache_geted; | ||
2294 | 2290 | + | ||
2295 | 2291 | + ulint innodb_row_cache_lru_n_add; | ||
2296 | 2292 | + ulint innodb_row_cache_lru_n_make_first; | ||
2297 | 2293 | + ulint innodb_row_cache_lru_n_evict; | ||
2298 | 2294 | + ulint innodb_row_cache_lru_count; | ||
2299 | 2295 | + | ||
2300 | 2296 | + ib_int64_t innodb_row_cache_mem_pool_size; | ||
2301 | 2297 | + ib_int64_t innodb_row_cache_mem_pool_used; | ||
2302 | 2298 | + | ||
2303 | 2299 | }; | ||
2304 | 2300 | |||
2305 | 2301 | /** Thread slot in the thread table */ | ||
2306 | 2302 | Index: storage/innobase/include/row0cache0lru.h | ||
2307 | 2303 | =================================================================== | ||
2308 | 2304 | --- storage/innobase/include/row0cache0lru.h (revision 0) | ||
2309 | 2305 | +++ storage/innobase/include/row0cache0lru.h (revision 724) | ||
2310 | 2306 | @@ -0,0 +1,46 @@ | ||
2311 | 2307 | +/******************************************************************** | ||
2312 | 2308 | + created: 2011/03/23 | ||
2313 | 2309 | + created: 23:3:2011 14:48 | ||
2314 | 2310 | + file base: row0cache0lru | ||
2315 | 2311 | + file ext: h | ||
2316 | 2312 | + author: wentong@taobao.com | ||
2317 | 2313 | + | ||
2318 | 2314 | + purpose: | ||
2319 | 2315 | +*********************************************************************/ | ||
2320 | 2316 | + | ||
2321 | 2317 | +#ifndef row0cache0lru_h__ | ||
2322 | 2318 | +#define row0cache0lru_h__ | ||
2323 | 2319 | + | ||
2324 | 2320 | +#include "row0cache0hash.h" | ||
2325 | 2321 | +#include "rem0types.h" | ||
2326 | 2322 | + | ||
2327 | 2323 | +typedef struct struct_row_cache_lru_stat{ | ||
2328 | 2324 | + ulint n_add; | ||
2329 | 2325 | + ulint n_make_first; | ||
2330 | 2326 | + ulint n_evict; | ||
2331 | 2327 | + ulint old_n_add; | ||
2332 | 2328 | + ulint old_n_make_first; | ||
2333 | 2329 | + ulint old_n_evict; | ||
2334 | 2330 | +}row_cache_lru_stat_t; | ||
2335 | 2331 | + | ||
2336 | 2332 | +extern row_cache_lru_stat_t* row_cache_lru_stat; | ||
2337 | 2333 | + | ||
2338 | 2334 | +extern my_bool innodb_row_cache_clean_cache; | ||
2339 | 2335 | + | ||
2340 | 2336 | +void init_innodb_row_cache_lru(); | ||
2341 | 2337 | + | ||
2342 | 2338 | +void deinit_innodb_row_cache_lru(); | ||
2343 | 2339 | + | ||
2344 | 2340 | +void clean_row_cache(); | ||
2345 | 2341 | + | ||
2346 | 2342 | +void add_row_cache_value_to_lru(row_cache_value_t* value); | ||
2347 | 2343 | + | ||
2348 | 2344 | +void make_row_cache_value_first_from_lru(row_cache_value_t* value); | ||
2349 | 2345 | + | ||
2350 | 2346 | +ulint free_from_lru(const ulint size , const ulint used_fold); | ||
2351 | 2347 | + | ||
2352 | 2348 | +void remove_row_cache_value_from_lru(row_cache_value_t* value); | ||
2353 | 2349 | + | ||
2354 | 2350 | +ulint get_row_cache_lru_count(); | ||
2355 | 2351 | + | ||
2356 | 2352 | +#endif // row0cache0lru_h__ | ||
2357 | 2353 | |||
2358 | 2354 | Property changes on: storage/innobase/include/row0cache0lru.h | ||
2359 | 2355 | ___________________________________________________________________ | ||
2360 | 2356 | Added: svn:mime-type | ||
2361 | 2357 | + text/plain | ||
2362 | 2358 | |||
2363 | 2359 | Index: storage/innobase/include/mtr0mtr.h | ||
2364 | 2360 | =================================================================== | ||
2365 | 2361 | --- storage/innobase/include/mtr0mtr.h (revision 498) | ||
2366 | 2362 | +++ storage/innobase/include/mtr0mtr.h (revision 724) | ||
2367 | 2363 | @@ -34,6 +34,7 @@ | ||
2368 | 2364 | #include "ut0byte.h" | ||
2369 | 2365 | #include "mtr0types.h" | ||
2370 | 2366 | #include "page0types.h" | ||
2371 | 2367 | +#include "row0cache.h" | ||
2372 | 2368 | |||
2373 | 2369 | /* Logging modes for a mini-transaction */ | ||
2374 | 2370 | #define MTR_LOG_ALL 21 /* default mode: log all operations | ||
2375 | 2371 | @@ -368,6 +369,7 @@ | ||
2376 | 2372 | #endif | ||
2377 | 2373 | dyn_array_t memo; /*!< memo stack for locks etc. */ | ||
2378 | 2374 | dyn_array_t log; /*!< mini-transaction log */ | ||
2379 | 2375 | + UT_LIST_BASE_NODE_T(row_cache_value_queue_t) row_cache_value_queue_base; /*!< row cache lock queue base*/ | ||
2380 | 2376 | ibool inside_ibuf; | ||
2381 | 2377 | /*!< TRUE if inside ibuf changes */ | ||
2382 | 2378 | ibool modifications; | ||
2383 | 2379 | Index: storage/innobase/include/mtr0mtr.ic | ||
2384 | 2380 | =================================================================== | ||
2385 | 2381 | --- storage/innobase/include/mtr0mtr.ic (revision 498) | ||
2386 | 2382 | +++ storage/innobase/include/mtr0mtr.ic (revision 724) | ||
2387 | 2383 | @@ -41,6 +41,8 @@ | ||
2388 | 2384 | |||
2389 | 2385 | dyn_array_create(&(mtr->memo)); | ||
2390 | 2386 | dyn_array_create(&(mtr->log)); | ||
2391 | 2387 | + | ||
2392 | 2388 | + UT_LIST_INIT(mtr->row_cache_value_queue_base); | ||
2393 | 2389 | |||
2394 | 2390 | mtr->log_mode = MTR_LOG_ALL; | ||
2395 | 2391 | mtr->modifications = FALSE; | ||
2396 | 2392 | Index: storage/innobase/include/row0cache0hash.h | ||
2397 | 2393 | =================================================================== | ||
2398 | 2394 | --- storage/innobase/include/row0cache0hash.h (revision 0) | ||
2399 | 2395 | +++ storage/innobase/include/row0cache0hash.h (revision 724) | ||
2400 | 2396 | @@ -0,0 +1,86 @@ | ||
2401 | 2397 | +/******************************************************************** | ||
2402 | 2398 | + created: 2011/03/24 | ||
2403 | 2399 | + created: 24:3:2011 8:48 | ||
2404 | 2400 | + file base: row0cache0hash | ||
2405 | 2401 | + file ext: h | ||
2406 | 2402 | + author: wentong@taobao.com | ||
2407 | 2403 | + | ||
2408 | 2404 | + purpose: | ||
2409 | 2405 | +*********************************************************************/ | ||
2410 | 2406 | +#ifndef row0cache0hash_h__ | ||
2411 | 2407 | +#define row0cache0hash_h__ | ||
2412 | 2408 | + | ||
2413 | 2409 | +#include "hash0hash.h" | ||
2414 | 2410 | +#include "rem0types.h" | ||
2415 | 2411 | +#include "data0types.h" | ||
2416 | 2412 | +#include "ut0byte.h" | ||
2417 | 2413 | +#include "ut0rbt.h" | ||
2418 | 2414 | +#include "dict0types.h" | ||
2419 | 2415 | + | ||
2420 | 2416 | +//flag and bit handler | ||
2421 | 2417 | +#define onBit(flag,bit) ((flag) |= (bit)) | ||
2422 | 2418 | +#define offBit(flag,bit) ((flag) &= ~(bit)) | ||
2423 | 2419 | +#define testFlag(flag,bit) (((flag) & (bit)) == (bit)) | ||
2424 | 2420 | + | ||
2425 | 2421 | + | ||
2426 | 2422 | +typedef struct row_cache_chain row_cache_chain_t; | ||
2427 | 2423 | +typedef struct row_cache_value row_cache_value_t; | ||
2428 | 2424 | +typedef UT_LIST_BASE_NODE_T(row_cache_value_t) ROW_CACHE_VALUE_LIST_BASE; | ||
2429 | 2425 | + | ||
2430 | 2426 | +struct row_cache_chain{ | ||
2431 | 2427 | + row_cache_value_t* value; | ||
2432 | 2428 | + row_cache_chain_t* next; | ||
2433 | 2429 | +}; | ||
2434 | 2430 | + | ||
2435 | 2431 | +struct row_cache_value{ | ||
2436 | 2432 | + ulint fold; | ||
2437 | 2433 | + index_id_t tree_id; | ||
2438 | 2434 | + table_id_t table_id; | ||
2439 | 2435 | + rec_t* buf; /*!< the real mem*/ | ||
2440 | 2436 | + rec_t* rec; /*!< the physical record */ | ||
2441 | 2437 | + ulint buf_size; | ||
2442 | 2438 | + UT_LIST_NODE_T(row_cache_value_t) list; | ||
2443 | 2439 | + ulint ref_num; /*!< the Reference Number */ | ||
2444 | 2440 | + unsigned char flag; | ||
2445 | 2441 | + row_cache_chain_t chain; | ||
2446 | 2442 | +}; | ||
2447 | 2443 | + | ||
2448 | 2444 | +#define FLAG_VALUE_IS_FROM_VALUE_POOL 1 | ||
2449 | 2445 | +#define FLAG_VALUE_IS_REMOVED (1<<1) | ||
2450 | 2446 | + | ||
2451 | 2447 | +#define isValueFromValuePool(flag) testFlag(flag,FLAG_VALUE_IS_FROM_VALUE_POOL) | ||
2452 | 2448 | +#define isValueRemoved(flag) testFlag(flag,FLAG_VALUE_IS_REMOVED) | ||
2453 | 2449 | + | ||
2454 | 2450 | +typedef struct ha_row_cache{ | ||
2455 | 2451 | + hash_table_t* row_cache; | ||
2456 | 2452 | +}row_cache_t; | ||
2457 | 2453 | + | ||
2458 | 2454 | +extern row_cache_t* innodb_row_cache; | ||
2459 | 2455 | + | ||
2460 | 2456 | +extern unsigned long innodb_row_cache_cell_num; | ||
2461 | 2457 | + | ||
2462 | 2458 | +extern unsigned int innodb_row_cache_mutex_num_shift; | ||
2463 | 2459 | + | ||
2464 | 2460 | +extern ulint innodb_row_cache_mutex_num; | ||
2465 | 2461 | + | ||
2466 | 2462 | +int init_row_cache_hash(my_bool innodb_row_cache_on); | ||
2467 | 2463 | + | ||
2468 | 2464 | +void deinit_row_cache_hash(my_bool innodb_row_cache_on); | ||
2469 | 2465 | + | ||
2470 | 2466 | +row_cache_value_t* search_row_cache_value(const dtuple_t* tuple ,const dict_index_t* index, const ulint fold); | ||
2471 | 2467 | +row_cache_value_t* search_row_cache_value_with_rec(const rec_t* rec, const ulint* rec_offsets, dict_index_t* index, const ulint fold); | ||
2472 | 2468 | +row_cache_value_t* insert_row_cache_value(const ulint fold , row_cache_value_t* value); | ||
2473 | 2469 | +void delete_row_cache_value(row_cache_value_t* value); | ||
2474 | 2470 | + | ||
2475 | 2471 | +ulint row_cache_enter_mutex_nowait(const ulint fold); | ||
2476 | 2472 | +void row_cache_enter_mutex(const ulint fold); | ||
2477 | 2473 | +void row_cache_exit_mutex(const ulint fold); | ||
2478 | 2474 | + | ||
2479 | 2475 | +int row_cache_own_mutex(const ulint fold1 , const ulint fold2); | ||
2480 | 2476 | + | ||
2481 | 2477 | +ulint row_cache_get_mutex_no(const ulint fold); | ||
2482 | 2478 | + | ||
2483 | 2479 | +void row_cache_enter_mutex_by_no(const ulint no); | ||
2484 | 2480 | +void row_cache_exit_mutex_by_no(const ulint no); | ||
2485 | 2481 | + | ||
2486 | 2482 | +#endif // row0cache0hash_h__ | ||
2487 | 2483 | |||
2488 | 2484 | Property changes on: storage/innobase/include/row0cache0hash.h | ||
2489 | 2485 | ___________________________________________________________________ | ||
2490 | 2486 | Added: svn:mime-type | ||
2491 | 2487 | + text/plain | ||
2492 | 2488 | |||
2493 | 2489 | Index: storage/innobase/include/row0cache.h | ||
2494 | 2490 | =================================================================== | ||
2495 | 2491 | --- storage/innobase/include/row0cache.h (revision 0) | ||
2496 | 2492 | +++ storage/innobase/include/row0cache.h (revision 724) | ||
2497 | 2493 | @@ -0,0 +1,89 @@ | ||
2498 | 2494 | +/******************************************************************** | ||
2499 | 2495 | +created: 2011/03/08 | ||
2500 | 2496 | +created: 8:3:2011 10:56 | ||
2501 | 2497 | +file base: row0cache | ||
2502 | 2498 | +file ext: h | ||
2503 | 2499 | +author: wentong@taobao.com | ||
2504 | 2500 | + | ||
2505 | 2501 | +purpose: for row cache | ||
2506 | 2502 | +*********************************************************************/ | ||
2507 | 2503 | +#ifndef row0cache_h_ | ||
2508 | 2504 | +#define row0cache_h_ | ||
2509 | 2505 | + | ||
2510 | 2506 | +#include "univ.i" | ||
2511 | 2507 | +#include "rem0types.h" | ||
2512 | 2508 | +#include "data0types.h" | ||
2513 | 2509 | +#include "data0data.h" | ||
2514 | 2510 | +#include "ut0byte.h" | ||
2515 | 2511 | +#include "mtr0types.h" | ||
2516 | 2512 | +#include "ut0lst.h" | ||
2517 | 2513 | +#include "row0cache0hash.h" | ||
2518 | 2514 | +#include "read0types.h" | ||
2519 | 2515 | + | ||
2520 | 2516 | +extern my_bool innodb_row_cache_on; | ||
2521 | 2517 | + | ||
2522 | 2518 | +typedef struct struct_row_cache_stat{ | ||
2523 | 2519 | + ulint n_get; /*the total get*/ | ||
2524 | 2520 | + ulint geted; /*the get from row cache*/ | ||
2525 | 2521 | + ulint old_n_get; | ||
2526 | 2522 | + ulint old_geted; | ||
2527 | 2523 | + time_t last_printout_time; | ||
2528 | 2524 | +}row_cache_stat_t; | ||
2529 | 2525 | + | ||
2530 | 2526 | +extern row_cache_stat_t* row_cache_stat; | ||
2531 | 2527 | + | ||
2532 | 2528 | + | ||
2533 | 2529 | +#define calc_fold_by_tuple(tuple , n_fields , tree_id) dtuple_fold((tuple) , (n_fields) , 0 , (tree_id)) | ||
2534 | 2530 | +ulint calc_fold_by_rec( | ||
2535 | 2531 | +/*=====*/ | ||
2536 | 2532 | + const rec_t* rec, /*!< in: the physical record */ | ||
2537 | 2533 | + const ulint* offsets, /*!< in: array returned by | ||
2538 | 2534 | + rec_get_offsets() */ | ||
2539 | 2535 | + ulint n_fields, /*!< in: number of complete | ||
2540 | 2536 | + fields to fold */ | ||
2541 | 2537 | + index_id_t tree_id); /*!< in: index tree id */ | ||
2542 | 2538 | + | ||
2543 | 2539 | +typedef struct row_cache_value_queue_struct row_cache_value_queue_t; | ||
2544 | 2540 | + | ||
2545 | 2541 | +struct row_cache_value_queue_struct{ | ||
2546 | 2542 | + row_cache_value_t* value; | ||
2547 | 2543 | + /*!< linear list of dyn blocks: this node is | ||
2548 | 2544 | + used only in the first block */ | ||
2549 | 2545 | + UT_LIST_NODE_T(row_cache_value_queue_t) list; | ||
2550 | 2546 | + /*!< linear list node: used in all blocks */ | ||
2551 | 2547 | +}; | ||
2552 | 2548 | + | ||
2553 | 2549 | +typedef UT_LIST_BASE_NODE_T(row_cache_value_queue_t) ROW_CACHE_VALUE_QUEUE_LIST_BASE; | ||
2554 | 2550 | + | ||
2555 | 2551 | +void init_row_cache(); | ||
2556 | 2552 | + | ||
2557 | 2553 | +void deinit_row_cache(); | ||
2558 | 2554 | + | ||
2559 | 2555 | +void put_rec_to_row_cache(const dtuple_t* tuple, | ||
2560 | 2556 | + const rec_t* rec, /*!< in: physical record */ | ||
2561 | 2557 | + const ulint* offsets, /*!< in: array returned by rec_get_offsets() */ | ||
2562 | 2558 | + const dict_index_t* index, /*!< in: index tree id */ | ||
2563 | 2559 | + ibool is_sec_index); | ||
2564 | 2560 | + | ||
2565 | 2561 | +int contain_row_cache_low(const ulint fold, const dtuple_t* tuple, | ||
2566 | 2562 | + const dict_index_t* index); | ||
2567 | 2563 | + | ||
2568 | 2564 | +rec_t* get_from_row_cache_low(const ulint fold, const dtuple_t* tuple, const dict_index_t* index ,mtr_t* mtr); /*!<in: mtr*/ | ||
2569 | 2565 | + | ||
2570 | 2566 | +void remove_from_row_cache_low(const ulint fold, const rec_t* rec, const ulint* offsets, const dtuple_t* tuple, dict_index_t* index); | ||
2571 | 2567 | + | ||
2572 | 2568 | +void release_row_cache_value_in_mtr(mtr_t* mtr); | ||
2573 | 2569 | + | ||
2574 | 2570 | +ulint lock_sec_rec_in_row_cache_cons_read_sees(const rec_t* rec, const ulint* offsets, const read_view_t* view); | ||
2575 | 2571 | + | ||
2576 | 2572 | +#define contain_row_cache(tuple , index) contain_row_cache_low(calc_fold_by_tuple((tuple) , (dict_index_get_n_unique(index)) , (index->id)), (tuple) ,(index)) | ||
2577 | 2573 | +#define get_from_row_cache(tuple , index , mtr) get_from_row_cache_low(calc_fold_by_tuple((tuple) , (dict_index_get_n_unique(index)) , (index->id)), (tuple) ,(index) ,(mtr) ) | ||
2578 | 2574 | +#define remove_from_row_cache(rec, offsets, index) remove_from_row_cache_low(calc_fold_by_rec((rec), (offsets), (dict_index_get_n_unique(index)), (index->id)),(rec),(offsets) ,NULL,index) | ||
2579 | 2575 | + | ||
2580 | 2576 | +#define remove_from_row_cache_for_tuple(tuple, index) remove_from_row_cache_low(calc_fold_by_tuple((tuple) , (dict_index_get_n_unique(index)) , (index->id)) , NULL,NULL,(tuple),(index)) | ||
2581 | 2577 | + | ||
2582 | 2578 | +void row_cache_refresh_stats(); | ||
2583 | 2579 | + | ||
2584 | 2580 | +void print_row_cache_stats(FILE* file); | ||
2585 | 2581 | + | ||
2586 | 2582 | +#endif | ||
2587 | 2583 | |||
2588 | 2584 | Property changes on: storage/innobase/include/row0cache.h | ||
2589 | 2585 | ___________________________________________________________________ | ||
2590 | 2586 | Added: svn:mime-type | ||
2591 | 2587 | + text/plain | ||
2592 | 2588 | |||
2593 | 2589 | Index: storage/innobase/include/row0cache0filter.h | ||
2594 | 2590 | =================================================================== | ||
2595 | 2591 | --- storage/innobase/include/row0cache0filter.h (revision 0) | ||
2596 | 2592 | +++ storage/innobase/include/row0cache0filter.h (revision 724) | ||
2597 | 2593 | @@ -0,0 +1,33 @@ | ||
2598 | 2594 | +/******************************************************************** | ||
2599 | 2595 | + created: 2011/05/31 | ||
2600 | 2596 | + created: 31:5:2011 11:39 | ||
2601 | 2597 | + file base: row0cache0filter | ||
2602 | 2598 | + file ext: h | ||
2603 | 2599 | + author: wentong@taobao.com | ||
2604 | 2600 | + | ||
2605 | 2601 | + purpose: | ||
2606 | 2602 | +*********************************************************************/ | ||
2607 | 2603 | + | ||
2608 | 2604 | +#ifndef row0cache0filter_h__ | ||
2609 | 2605 | +#define row0cache0filter_h__ | ||
2610 | 2606 | + | ||
2611 | 2607 | +#include "hash0hash.h" | ||
2612 | 2608 | +#include "dict0types.h" | ||
2613 | 2609 | + | ||
2614 | 2610 | +#ifdef UNIV_PFS_RWLOCK | ||
2615 | 2611 | +/* Key to register btr_search_sys with performance schema */ | ||
2616 | 2612 | +extern mysql_pfs_key_t row_cache_filter_lock_key; | ||
2617 | 2613 | +#endif /* UNIV_PFS_RWLOCK */ | ||
2618 | 2614 | + | ||
2619 | 2615 | +#define INDEX_CONFIG_LEN 2048 | ||
2620 | 2616 | + | ||
2621 | 2617 | +extern char* innodb_row_cache_index; | ||
2622 | 2618 | + | ||
2623 | 2619 | +void init_row_cache_filter(my_bool innodb_row_cache_on); | ||
2624 | 2620 | +void deinit_row_cache_filter(my_bool innodb_row_cache_on); | ||
2625 | 2621 | + | ||
2626 | 2622 | +void reset_filter(); | ||
2627 | 2623 | + | ||
2628 | 2624 | +my_bool is_index_need_cache(const dict_index_t* index); | ||
2629 | 2625 | + | ||
2630 | 2626 | +#endif // row0cache0filter_h__ | ||
2631 | 2627 | |||
2632 | 2628 | Property changes on: storage/innobase/include/row0cache0filter.h | ||
2633 | 2629 | ___________________________________________________________________ | ||
2634 | 2630 | Added: svn:mime-type | ||
2635 | 2631 | + text/plain | ||
2636 | 2632 | |||
2637 | 2633 | Index: storage/innobase/row/row0upd.c | ||
2638 | 2634 | =================================================================== | ||
2639 | 2635 | --- storage/innobase/row/row0upd.c (revision 498) | ||
2640 | 2636 | +++ storage/innobase/row/row0upd.c (revision 724) | ||
2641 | 2637 | @@ -32,6 +32,7 @@ | ||
2642 | 2638 | #include "dict0dict.h" | ||
2643 | 2639 | #include "trx0undo.h" | ||
2644 | 2640 | #include "rem0rec.h" | ||
2645 | 2641 | +#include "row0cache.h" | ||
2646 | 2642 | #ifndef UNIV_HOTBACKUP | ||
2647 | 2643 | #include "dict0boot.h" | ||
2648 | 2644 | #include "dict0crea.h" | ||
2649 | 2645 | @@ -1665,6 +1666,12 @@ | ||
2650 | 2646 | } | ||
2651 | 2647 | break; | ||
2652 | 2648 | } | ||
2653 | 2649 | + //TB_HOOK | ||
2654 | 2650 | + if(dict_index_is_unique(index)){ | ||
2655 | 2651 | + ulint offsets_[REC_OFFS_NORMAL_SIZE]; | ||
2656 | 2652 | + ulint* offsets = rec_get_offsets( rec, index, offsets_, ULINT_UNDEFINED, &heap); | ||
2657 | 2653 | + remove_from_row_cache(rec,offsets,index); | ||
2658 | 2654 | + } | ||
2659 | 2655 | |||
2660 | 2656 | btr_pcur_close(&pcur); | ||
2661 | 2657 | mtr_commit(&mtr); | ||
2662 | 2658 | @@ -2199,6 +2206,8 @@ | ||
2663 | 2659 | } | ||
2664 | 2660 | } | ||
2665 | 2661 | |||
2666 | 2662 | + remove_from_row_cache(rec , offsets , index); | ||
2667 | 2663 | + | ||
2668 | 2664 | /* NOTE: the following function calls will also commit mtr */ | ||
2669 | 2665 | |||
2670 | 2666 | if (node->is_delete) { | ||
2671 | 2667 | Index: storage/innobase/row/row0uins.c | ||
2672 | 2668 | =================================================================== | ||
2673 | 2669 | --- storage/innobase/row/row0uins.c (revision 498) | ||
2674 | 2670 | +++ storage/innobase/row/row0uins.c (revision 724) | ||
2675 | 2671 | @@ -45,6 +45,8 @@ | ||
2676 | 2672 | #include "que0que.h" | ||
2677 | 2673 | #include "ibuf0ibuf.h" | ||
2678 | 2674 | #include "log0log.h" | ||
2679 | 2675 | +#include "row0cache.h" | ||
2680 | 2676 | +#include "btr0sea.h" | ||
2681 | 2677 | |||
2682 | 2678 | /************************************************************************* | ||
2683 | 2679 | IMPORTANT NOTE: Any operation that generates redo MUST check that there | ||
2684 | 2680 | @@ -66,11 +68,23 @@ | ||
2685 | 2681 | /*==========================*/ | ||
2686 | 2682 | undo_node_t* node) /*!< in: undo node */ | ||
2687 | 2683 | { | ||
2688 | 2684 | + dict_index_t* index; | ||
2689 | 2685 | + btr_pcur_t* pcur; | ||
2690 | 2686 | + | ||
2691 | 2687 | btr_cur_t* btr_cur; | ||
2692 | 2688 | ibool success; | ||
2693 | 2689 | ulint err; | ||
2694 | 2690 | ulint n_tries = 0; | ||
2695 | 2691 | mtr_t mtr; | ||
2696 | 2692 | + rec_t* rec; | ||
2697 | 2693 | + | ||
2698 | 2694 | + mem_heap_t* heap = NULL; | ||
2699 | 2695 | + ulint offsets_[REC_OFFS_NORMAL_SIZE]; | ||
2700 | 2696 | + ulint* offsets; | ||
2701 | 2697 | + rec_offs_init(offsets_); | ||
2702 | 2698 | + | ||
2703 | 2699 | + pcur = &(node->pcur); | ||
2704 | 2700 | + index = dict_table_get_first_index(node->table); | ||
2705 | 2701 | |||
2706 | 2702 | mtr_start(&mtr); | ||
2707 | 2703 | |||
2708 | 2704 | @@ -78,6 +92,11 @@ | ||
2709 | 2705 | &mtr); | ||
2710 | 2706 | ut_a(success); | ||
2711 | 2707 | |||
2712 | 2708 | + //TB_HOOK | ||
2713 | 2709 | + rec = btr_pcur_get_rec(pcur); | ||
2714 | 2710 | + offsets = rec_get_offsets(rec, index, offsets_, ULINT_UNDEFINED, &heap); | ||
2715 | 2711 | + remove_from_row_cache(rec , offsets,index); | ||
2716 | 2712 | + | ||
2717 | 2713 | if (node->table->id == DICT_INDEXES_ID) { | ||
2718 | 2714 | ut_ad(node->trx->dict_operation_lock_mode == RW_X_LATCH); | ||
2719 | 2715 | |||
2720 | 2716 | @@ -103,7 +122,9 @@ | ||
2721 | 2717 | |||
2722 | 2718 | if (success) { | ||
2723 | 2719 | trx_undo_rec_release(node->trx, node->undo_no); | ||
2724 | 2720 | - | ||
2725 | 2721 | + if (UNIV_LIKELY_NULL(heap)) { | ||
2726 | 2722 | + mem_heap_free(heap); | ||
2727 | 2723 | + } | ||
2728 | 2724 | return(DB_SUCCESS); | ||
2729 | 2725 | } | ||
2730 | 2726 | retry: | ||
2731 | 2727 | @@ -139,6 +160,9 @@ | ||
2732 | 2728 | |||
2733 | 2729 | trx_undo_rec_release(node->trx, node->undo_no); | ||
2734 | 2730 | |||
2735 | 2731 | + if (UNIV_LIKELY_NULL(heap)) { | ||
2736 | 2732 | + mem_heap_free(heap); | ||
2737 | 2733 | + } | ||
2738 | 2734 | return(err); | ||
2739 | 2735 | } | ||
2740 | 2736 | |||
2741 | 2737 | @@ -348,6 +372,10 @@ | ||
2742 | 2738 | transactions. */ | ||
2743 | 2739 | ut_a(trx_is_recv(node->trx)); | ||
2744 | 2740 | } else { | ||
2745 | 2741 | + //TB_HOOK | ||
2746 | 2742 | + if(dict_index_is_unique(node->index)){ | ||
2747 | 2743 | + remove_from_row_cache_for_tuple(entry , node->index); | ||
2748 | 2744 | + } | ||
2749 | 2745 | log_free_check(); | ||
2750 | 2746 | err = row_undo_ins_remove_sec(node->index, entry); | ||
2751 | 2747 | |||
2752 | 2748 | Index: storage/innobase/row/row0sel.c | ||
2753 | 2749 | =================================================================== | ||
2754 | 2750 | --- storage/innobase/row/row0sel.c (revision 498) | ||
2755 | 2751 | +++ storage/innobase/row/row0sel.c (revision 724) | ||
2756 | 2752 | @@ -57,6 +57,7 @@ | ||
2757 | 2753 | #include "read0read.h" | ||
2758 | 2754 | #include "buf0lru.h" | ||
2759 | 2755 | #include "ha_prototypes.h" | ||
2760 | 2756 | +#include "row0cache.h" | ||
2761 | 2757 | |||
2762 | 2758 | /* Maximum number of rows to prefetch; MySQL interface has another parameter */ | ||
2763 | 2759 | #define SEL_MAX_N_PREFETCH 16 | ||
2764 | 2760 | @@ -2904,6 +2905,7 @@ | ||
2765 | 2761 | rec_t* old_vers; | ||
2766 | 2762 | enum db_err err; | ||
2767 | 2763 | trx_t* trx; | ||
2768 | 2764 | + ibool is_get_from_row_cache = FALSE; | ||
2769 | 2765 | |||
2770 | 2766 | *out_rec = NULL; | ||
2771 | 2767 | trx = thr_get_trx(thr); | ||
2772 | 2768 | @@ -2913,20 +2915,31 @@ | ||
2773 | 2769 | |||
2774 | 2770 | clust_index = dict_table_get_first_index(sec_index->table); | ||
2775 | 2771 | |||
2776 | 2772 | - btr_pcur_open_with_no_init(clust_index, prebuilt->clust_ref, | ||
2777 | 2773 | - PAGE_CUR_LE, BTR_SEARCH_LEAF, | ||
2778 | 2774 | - prebuilt->clust_pcur, 0, mtr); | ||
2779 | 2775 | + //TB_HOOK get_from_row_cache | ||
2780 | 2776 | + if(prebuilt->select_lock_type == LOCK_NONE | ||
2781 | 2777 | + && trx->mysql_n_tables_locked == 0){ | ||
2782 | 2778 | + clust_rec = get_from_row_cache(prebuilt->clust_ref,clust_index,mtr); | ||
2783 | 2779 | + if (clust_rec!=NULL) | ||
2784 | 2780 | + { | ||
2785 | 2781 | + is_get_from_row_cache=TRUE; | ||
2786 | 2782 | + } | ||
2787 | 2783 | + } | ||
2788 | 2784 | + if(is_get_from_row_cache == FALSE){ | ||
2789 | 2785 | + btr_pcur_open_with_no_init(clust_index, prebuilt->clust_ref, | ||
2790 | 2786 | + PAGE_CUR_LE, BTR_SEARCH_LEAF, | ||
2791 | 2787 | + prebuilt->clust_pcur, 0, mtr); | ||
2792 | 2788 | |||
2793 | 2789 | - clust_rec = btr_pcur_get_rec(prebuilt->clust_pcur); | ||
2794 | 2790 | + clust_rec = btr_pcur_get_rec(prebuilt->clust_pcur); | ||
2795 | 2791 | + } | ||
2796 | 2792 | |||
2797 | 2793 | prebuilt->clust_pcur->trx_if_known = trx; | ||
2798 | 2794 | |||
2799 | 2795 | /* Note: only if the search ends up on a non-infimum record is the | ||
2800 | 2796 | low_match value the real match to the search tuple */ | ||
2801 | 2797 | |||
2802 | 2798 | - if (!page_rec_is_user_rec(clust_rec) | ||
2803 | 2799 | + if (is_get_from_row_cache == FALSE && (!page_rec_is_user_rec(clust_rec) | ||
2804 | 2800 | || btr_pcur_get_low_match(prebuilt->clust_pcur) | ||
2805 | 2801 | - < dict_index_get_n_unique(clust_index)) { | ||
2806 | 2802 | + < dict_index_get_n_unique(clust_index))) { | ||
2807 | 2803 | |||
2808 | 2804 | /* In a rare case it is possible that no clust rec is found | ||
2809 | 2805 | for a delete-marked secondary index record: if in row0umod.c | ||
2810 | 2806 | @@ -3011,6 +3024,13 @@ | ||
2811 | 2807 | } | ||
2812 | 2808 | |||
2813 | 2809 | clust_rec = old_vers; | ||
2814 | 2810 | + } else { | ||
2815 | 2811 | + //TB_HOOK put_rec_to_row_cache | ||
2816 | 2812 | + if (is_get_from_row_cache == FALSE && !prebuilt->templ_contains_blob | ||
2817 | 2813 | + && !prebuilt->used_in_HANDLER) { | ||
2818 | 2814 | + put_rec_to_row_cache(prebuilt->clust_ref, clust_rec, *offsets, | ||
2819 | 2815 | + clust_index, FALSE); | ||
2820 | 2816 | + } | ||
2821 | 2817 | } | ||
2822 | 2818 | |||
2823 | 2819 | /* If we had to go to an earlier version of row or the | ||
2824 | 2820 | @@ -3356,6 +3376,18 @@ | ||
2825 | 2821 | return(SEL_FOUND); | ||
2826 | 2822 | } | ||
2827 | 2823 | |||
2828 | 2824 | +static void put_rec_to_row_cache_in_sel(row_prebuilt_t* prebuilt, | ||
2829 | 2825 | + ulint direction, ibool unique_search, const rec_t* rec, ulint* offsets, | ||
2830 | 2826 | + dict_index_t* index, ibool is_sec_index) { | ||
2831 | 2827 | + if (prebuilt->select_lock_type == LOCK_NONE && direction == 0 | ||
2832 | 2828 | + && unique_search && !prebuilt->templ_contains_blob | ||
2833 | 2829 | + && !prebuilt->used_in_HANDLER) { | ||
2834 | 2830 | + put_rec_to_row_cache(prebuilt->search_tuple, rec, offsets, index, | ||
2835 | 2831 | + is_sec_index); | ||
2836 | 2832 | + } | ||
2837 | 2833 | +} | ||
2838 | 2834 | + | ||
2839 | 2835 | + | ||
2840 | 2836 | /********************************************************************//** | ||
2841 | 2837 | Searches for rows in the database. This is used in the interface to | ||
2842 | 2838 | MySQL. This function opens a cursor, and also implements fetch next | ||
2843 | 2839 | @@ -3417,6 +3449,7 @@ | ||
2844 | 2840 | ibool same_user_rec; | ||
2845 | 2841 | mtr_t mtr; | ||
2846 | 2842 | mem_heap_t* heap = NULL; | ||
2847 | 2843 | + ibool is_get_from_row_cache = FALSE; | ||
2848 | 2844 | ulint offsets_[REC_OFFS_NORMAL_SIZE]; | ||
2849 | 2845 | ulint* offsets = offsets_; | ||
2850 | 2846 | ibool table_lock_waited = FALSE; | ||
2851 | 2847 | @@ -3929,14 +3962,26 @@ | ||
2852 | 2848 | |||
2853 | 2849 | } else if (dtuple_get_n_fields(search_tuple) > 0) { | ||
2854 | 2850 | |||
2855 | 2851 | - btr_pcur_open_with_no_init(index, search_tuple, mode, | ||
2856 | 2852 | - BTR_SEARCH_LEAF, | ||
2857 | 2853 | - pcur, 0, &mtr); | ||
2858 | 2854 | - | ||
2859 | 2855 | + //TB_HOOK get_from_row_cache | ||
2860 | 2856 | + if(prebuilt->select_lock_type == LOCK_NONE | ||
2861 | 2857 | + && trx->mysql_n_tables_locked == 0 | ||
2862 | 2858 | + && direction == 0 | ||
2863 | 2859 | + && unique_search | ||
2864 | 2860 | + && !prebuilt->used_in_HANDLER){ | ||
2865 | 2861 | + rec = get_from_row_cache(search_tuple,index,&mtr); | ||
2866 | 2862 | + if (rec != NULL) | ||
2867 | 2863 | + { | ||
2868 | 2864 | + is_get_from_row_cache = TRUE; | ||
2869 | 2865 | + } | ||
2870 | 2866 | + } | ||
2871 | 2867 | + if(is_get_from_row_cache == FALSE){ | ||
2872 | 2868 | + btr_pcur_open_with_no_init(index, search_tuple, mode, | ||
2873 | 2869 | + BTR_SEARCH_LEAF, | ||
2874 | 2870 | + pcur, 0, &mtr); | ||
2875 | 2871 | + rec = btr_pcur_get_rec(pcur); | ||
2876 | 2872 | + } | ||
2877 | 2873 | pcur->trx_if_known = trx; | ||
2878 | 2874 | |||
2879 | 2875 | - rec = btr_pcur_get_rec(pcur); | ||
2880 | 2876 | - | ||
2881 | 2877 | if (!moves_up | ||
2882 | 2878 | && !page_rec_is_supremum(rec) | ||
2883 | 2879 | && set_also_gap_locks | ||
2884 | 2880 | @@ -3979,9 +4024,10 @@ | ||
2885 | 2881 | rec_loop: | ||
2886 | 2882 | /*-------------------------------------------------------------*/ | ||
2887 | 2883 | /* PHASE 4: Look for matching records in a loop */ | ||
2888 | 2884 | - | ||
2889 | 2885 | - rec = btr_pcur_get_rec(pcur); | ||
2890 | 2886 | - | ||
2891 | 2887 | + if (is_get_from_row_cache == FALSE) | ||
2892 | 2888 | + { | ||
2893 | 2889 | + rec = btr_pcur_get_rec(pcur); | ||
2894 | 2890 | + } | ||
2895 | 2891 | if (srv_pass_corrupt_table && !rec) { | ||
2896 | 2892 | err = DB_CORRUPTION; | ||
2897 | 2893 | goto lock_wait_or_error; | ||
2898 | 2894 | @@ -3998,122 +4044,123 @@ | ||
2899 | 2895 | rec_print(rec); | ||
2900 | 2896 | */ | ||
2901 | 2897 | #endif /* UNIV_SEARCH_DEBUG */ | ||
2902 | 2898 | - | ||
2903 | 2899 | - if (page_rec_is_infimum(rec)) { | ||
2904 | 2900 | + if(is_get_from_row_cache==FALSE){ | ||
2905 | 2901 | + if (page_rec_is_infimum(rec)) { | ||
2906 | 2902 | |||
2907 | 2903 | - /* The infimum record on a page cannot be in the result set, | ||
2908 | 2904 | - and neither can a record lock be placed on it: we skip such | ||
2909 | 2905 | - a record. */ | ||
2910 | 2906 | + /* The infimum record on a page cannot be in the result set, | ||
2911 | 2907 | + and neither can a record lock be placed on it: we skip such | ||
2912 | 2908 | + a record. */ | ||
2913 | 2909 | |||
2914 | 2910 | - goto next_rec; | ||
2915 | 2911 | - } | ||
2916 | 2912 | + goto next_rec; | ||
2917 | 2913 | + } | ||
2918 | 2914 | |||
2919 | 2915 | - if (page_rec_is_supremum(rec)) { | ||
2920 | 2916 | + if (page_rec_is_supremum(rec)) { | ||
2921 | 2917 | |||
2922 | 2918 | - if (set_also_gap_locks | ||
2923 | 2919 | - && !(srv_locks_unsafe_for_binlog | ||
2924 | 2920 | - || trx->isolation_level <= TRX_ISO_READ_COMMITTED) | ||
2925 | 2921 | - && prebuilt->select_lock_type != LOCK_NONE) { | ||
2926 | 2922 | + if (set_also_gap_locks | ||
2927 | 2923 | + && !(srv_locks_unsafe_for_binlog | ||
2928 | 2924 | + || trx->isolation_level <= TRX_ISO_READ_COMMITTED) | ||
2929 | 2925 | + && prebuilt->select_lock_type != LOCK_NONE) { | ||
2930 | 2926 | |||
2931 | 2927 | - /* Try to place a lock on the index record */ | ||
2932 | 2928 | + /* Try to place a lock on the index record */ | ||
2933 | 2929 | |||
2934 | 2930 | - /* If innodb_locks_unsafe_for_binlog option is used | ||
2935 | 2931 | - or this session is using a READ COMMITTED isolation | ||
2936 | 2932 | - level we do not lock gaps. Supremum record is really | ||
2937 | 2933 | - a gap and therefore we do not set locks there. */ | ||
2938 | 2934 | + /* If innodb_locks_unsafe_for_binlog option is used | ||
2939 | 2935 | + or this session is using a READ COMMITTED isolation | ||
2940 | 2936 | + level we do not lock gaps. Supremum record is really | ||
2941 | 2937 | + a gap and therefore we do not set locks there. */ | ||
2942 | 2938 | |||
2943 | 2939 | - offsets = rec_get_offsets(rec, index, offsets, | ||
2944 | 2940 | - ULINT_UNDEFINED, &heap); | ||
2945 | 2941 | - err = sel_set_rec_lock(btr_pcur_get_block(pcur), | ||
2946 | 2942 | - rec, index, offsets, | ||
2947 | 2943 | - prebuilt->select_lock_type, | ||
2948 | 2944 | - LOCK_ORDINARY, thr); | ||
2949 | 2945 | + offsets = rec_get_offsets(rec, index, offsets, | ||
2950 | 2946 | + ULINT_UNDEFINED, &heap); | ||
2951 | 2947 | + err = sel_set_rec_lock(btr_pcur_get_block(pcur), | ||
2952 | 2948 | + rec, index, offsets, | ||
2953 | 2949 | + prebuilt->select_lock_type, | ||
2954 | 2950 | + LOCK_ORDINARY, thr); | ||
2955 | 2951 | |||
2956 | 2952 | - switch (err) { | ||
2957 | 2953 | - case DB_SUCCESS_LOCKED_REC: | ||
2958 | 2954 | - err = DB_SUCCESS; | ||
2959 | 2955 | - case DB_SUCCESS: | ||
2960 | 2956 | - break; | ||
2961 | 2957 | - default: | ||
2962 | 2958 | - goto lock_wait_or_error; | ||
2963 | 2959 | + switch (err) { | ||
2964 | 2960 | + case DB_SUCCESS_LOCKED_REC: | ||
2965 | 2961 | + err = DB_SUCCESS; | ||
2966 | 2962 | + case DB_SUCCESS: | ||
2967 | 2963 | + break; | ||
2968 | 2964 | + default: | ||
2969 | 2965 | + goto lock_wait_or_error; | ||
2970 | 2966 | + } | ||
2971 | 2967 | } | ||
2972 | 2968 | - } | ||
2973 | 2969 | - /* A page supremum record cannot be in the result set: skip | ||
2974 | 2970 | - it now that we have placed a possible lock on it */ | ||
2975 | 2971 | + /* A page supremum record cannot be in the result set: skip | ||
2976 | 2972 | + it now that we have placed a possible lock on it */ | ||
2977 | 2973 | |||
2978 | 2974 | - goto next_rec; | ||
2979 | 2975 | - } | ||
2980 | 2976 | + goto next_rec; | ||
2981 | 2977 | + } | ||
2982 | 2978 | |||
2983 | 2979 | - /*-------------------------------------------------------------*/ | ||
2984 | 2980 | - /* Do sanity checks in case our cursor has bumped into page | ||
2985 | 2981 | - corruption */ | ||
2986 | 2982 | + /*-------------------------------------------------------------*/ | ||
2987 | 2983 | + /* Do sanity checks in case our cursor has bumped into page | ||
2988 | 2984 | + corruption */ | ||
2989 | 2985 | |||
2990 | 2986 | - if (comp) { | ||
2991 | 2987 | - next_offs = rec_get_next_offs(rec, TRUE); | ||
2992 | 2988 | - if (UNIV_UNLIKELY(next_offs < PAGE_NEW_SUPREMUM)) { | ||
2993 | 2989 | + if (comp) { | ||
2994 | 2990 | + next_offs = rec_get_next_offs(rec, TRUE); | ||
2995 | 2991 | + if (UNIV_UNLIKELY(next_offs < PAGE_NEW_SUPREMUM)) { | ||
2996 | 2992 | |||
2997 | 2993 | - goto wrong_offs; | ||
2998 | 2994 | - } | ||
2999 | 2995 | - } else { | ||
3000 | 2996 | - next_offs = rec_get_next_offs(rec, FALSE); | ||
3001 | 2997 | - if (UNIV_UNLIKELY(next_offs < PAGE_OLD_SUPREMUM)) { | ||
3002 | 2998 | + goto wrong_offs; | ||
3003 | 2999 | + } | ||
3004 | 3000 | + } else { | ||
3005 | 3001 | + next_offs = rec_get_next_offs(rec, FALSE); | ||
3006 | 3002 | + if (UNIV_UNLIKELY(next_offs < PAGE_OLD_SUPREMUM)) { | ||
3007 | 3003 | |||
3008 | 3004 | - goto wrong_offs; | ||
3009 | 3005 | + goto wrong_offs; | ||
3010 | 3006 | + } | ||
3011 | 3007 | } | ||
3012 | 3008 | - } | ||
3013 | 3009 | |||
3014 | 3010 | - if (UNIV_UNLIKELY(next_offs >= UNIV_PAGE_SIZE - PAGE_DIR)) { | ||
3015 | 3011 | + if (UNIV_UNLIKELY(next_offs >= UNIV_PAGE_SIZE - PAGE_DIR)) { | ||
3016 | 3012 | |||
3017 | 3013 | wrong_offs: | ||
3018 | 3014 | - if (srv_pass_corrupt_table && !trx_sys_sys_space(index->table->space)) { | ||
3019 | 3015 | - index->table->is_corrupt = TRUE; | ||
3020 | 3016 | - fil_space_set_corrupt(index->table->space); | ||
3021 | 3017 | - } | ||
3022 | 3018 | + if (srv_pass_corrupt_table && !trx_sys_sys_space(index->table->space)) { | ||
3023 | 3019 | + index->table->is_corrupt = TRUE; | ||
3024 | 3020 | + fil_space_set_corrupt(index->table->space); | ||
3025 | 3021 | + } | ||
3026 | 3022 | |||
3027 | 3023 | - if ((srv_force_recovery == 0 || moves_up == FALSE) | ||
3028 | 3024 | - && srv_pass_corrupt_table <= 1) { | ||
3029 | 3025 | - ut_print_timestamp(stderr); | ||
3030 | 3026 | - buf_page_print(page_align(rec), 0); | ||
3031 | 3027 | - fprintf(stderr, | ||
3032 | 3028 | - "\nInnoDB: rec address %p," | ||
3033 | 3029 | - " buf block fix count %lu\n", | ||
3034 | 3030 | - (void*) rec, (ulong) | ||
3035 | 3031 | - btr_cur_get_block(btr_pcur_get_btr_cur(pcur)) | ||
3036 | 3032 | - ->page.buf_fix_count); | ||
3037 | 3033 | - fprintf(stderr, | ||
3038 | 3034 | - "InnoDB: Index corruption: rec offs %lu" | ||
3039 | 3035 | - " next offs %lu, page no %lu,\n" | ||
3040 | 3036 | - "InnoDB: ", | ||
3041 | 3037 | - (ulong) page_offset(rec), | ||
3042 | 3038 | - (ulong) next_offs, | ||
3043 | 3039 | - (ulong) page_get_page_no(page_align(rec))); | ||
3044 | 3040 | - dict_index_name_print(stderr, trx, index); | ||
3045 | 3041 | - fputs(". Run CHECK TABLE. You may need to\n" | ||
3046 | 3042 | - "InnoDB: restore from a backup, or" | ||
3047 | 3043 | - " dump + drop + reimport the table.\n", | ||
3048 | 3044 | - stderr); | ||
3049 | 3045 | + if ((srv_force_recovery == 0 || moves_up == FALSE) | ||
3050 | 3046 | + && srv_pass_corrupt_table <= 1) { | ||
3051 | 3047 | + ut_print_timestamp(stderr); | ||
3052 | 3048 | + buf_page_print(page_align(rec), 0); | ||
3053 | 3049 | + fprintf(stderr, | ||
3054 | 3050 | + "\nInnoDB: rec address %p," | ||
3055 | 3051 | + " buf block fix count %lu\n", | ||
3056 | 3052 | + (void*) rec, (ulong) | ||
3057 | 3053 | + btr_cur_get_block(btr_pcur_get_btr_cur(pcur)) | ||
3058 | 3054 | + ->page.buf_fix_count); | ||
3059 | 3055 | + fprintf(stderr, | ||
3060 | 3056 | + "InnoDB: Index corruption: rec offs %lu" | ||
3061 | 3057 | + " next offs %lu, page no %lu,\n" | ||
3062 | 3058 | + "InnoDB: ", | ||
3063 | 3059 | + (ulong) page_offset(rec), | ||
3064 | 3060 | + (ulong) next_offs, | ||
3065 | 3061 | + (ulong) page_get_page_no(page_align(rec))); | ||
3066 | 3062 | + dict_index_name_print(stderr, trx, index); | ||
3067 | 3063 | + fputs(". Run CHECK TABLE. You may need to\n" | ||
3068 | 3064 | + "InnoDB: restore from a backup, or" | ||
3069 | 3065 | + " dump + drop + reimport the table.\n", | ||
3070 | 3066 | + stderr); | ||
3071 | 3067 | |||
3072 | 3068 | - err = DB_CORRUPTION; | ||
3073 | 3069 | + err = DB_CORRUPTION; | ||
3074 | 3070 | |||
3075 | 3071 | - goto lock_wait_or_error; | ||
3076 | 3072 | - } else { | ||
3077 | 3073 | - /* The user may be dumping a corrupt table. Jump | ||
3078 | 3074 | - over the corruption to recover as much as possible. */ | ||
3079 | 3075 | + goto lock_wait_or_error; | ||
3080 | 3076 | + } else { | ||
3081 | 3077 | + /* The user may be dumping a corrupt table. Jump | ||
3082 | 3078 | + over the corruption to recover as much as possible. */ | ||
3083 | 3079 | |||
3084 | 3080 | - fprintf(stderr, | ||
3085 | 3081 | - "InnoDB: Index corruption: rec offs %lu" | ||
3086 | 3082 | - " next offs %lu, page no %lu,\n" | ||
3087 | 3083 | - "InnoDB: ", | ||
3088 | 3084 | - (ulong) page_offset(rec), | ||
3089 | 3085 | - (ulong) next_offs, | ||
3090 | 3086 | - (ulong) page_get_page_no(page_align(rec))); | ||
3091 | 3087 | - dict_index_name_print(stderr, trx, index); | ||
3092 | 3088 | - fputs(". We try to skip the rest of the page.\n", | ||
3093 | 3089 | - stderr); | ||
3094 | 3090 | + fprintf(stderr, | ||
3095 | 3091 | + "InnoDB: Index corruption: rec offs %lu" | ||
3096 | 3092 | + " next offs %lu, page no %lu,\n" | ||
3097 | 3093 | + "InnoDB: ", | ||
3098 | 3094 | + (ulong) page_offset(rec), | ||
3099 | 3095 | + (ulong) next_offs, | ||
3100 | 3096 | + (ulong) page_get_page_no(page_align(rec))); | ||
3101 | 3097 | + dict_index_name_print(stderr, trx, index); | ||
3102 | 3098 | + fputs(". We try to skip the rest of the page.\n", | ||
3103 | 3099 | + stderr); | ||
3104 | 3100 | |||
3105 | 3101 | - btr_pcur_move_to_last_on_page(pcur, &mtr); | ||
3106 | 3102 | + btr_pcur_move_to_last_on_page(pcur, &mtr); | ||
3107 | 3103 | |||
3108 | 3104 | - goto next_rec; | ||
3109 | 3105 | + goto next_rec; | ||
3110 | 3106 | + } | ||
3111 | 3107 | } | ||
3112 | 3108 | } | ||
3113 | 3109 | /*-------------------------------------------------------------*/ | ||
3114 | 3110 | @@ -4364,7 +4411,11 @@ | ||
3115 | 3111 | |||
3116 | 3112 | /* Do nothing: we let a non-locking SELECT read the | ||
3117 | 3113 | latest version of the record */ | ||
3118 | 3114 | - | ||
3119 | 3115 | + //TB_HOOK put_rec_to_row_cache | ||
3120 | 3116 | + if (is_get_from_row_cache == FALSE) { | ||
3121 | 3117 | + put_rec_to_row_cache_in_sel(prebuilt, direction, unique_search, | ||
3122 | 3118 | + rec, offsets, index, index != clust_index); | ||
3123 | 3119 | + } | ||
3124 | 3120 | } else if (index == clust_index) { | ||
3125 | 3121 | |||
3126 | 3122 | /* Fetch a previous version of the row if the current | ||
3127 | 3123 | @@ -4390,6 +4441,11 @@ | ||
3128 | 3124 | } | ||
3129 | 3125 | |||
3130 | 3126 | if (old_vers == NULL) { | ||
3131 | 3127 | + if (is_get_from_row_cache == TRUE) { | ||
3132 | 3128 | + //this should be Phantom read | ||
3133 | 3129 | + err = DB_RECORD_NOT_FOUND; | ||
3134 | 3130 | + goto normal_return; | ||
3135 | 3131 | + } | ||
3136 | 3132 | /* The row did not exist yet in | ||
3137 | 3133 | the read view */ | ||
3138 | 3134 | |||
3139 | 3135 | @@ -4397,6 +4453,13 @@ | ||
3140 | 3136 | } | ||
3141 | 3137 | |||
3142 | 3138 | rec = old_vers; | ||
3143 | 3139 | + } else { | ||
3144 | 3140 | + //TB_HOOK put_rec_to_row_cache | ||
3145 | 3141 | + if (is_get_from_row_cache == FALSE) { | ||
3146 | 3142 | + put_rec_to_row_cache_in_sel(prebuilt, direction, | ||
3147 | 3143 | + unique_search, rec, offsets, index, | ||
3148 | 3144 | + index != clust_index); | ||
3149 | 3145 | + } | ||
3150 | 3146 | } | ||
3151 | 3147 | } else { | ||
3152 | 3148 | /* We are looking into a non-clustered index, | ||
3153 | 3149 | @@ -4407,9 +4470,19 @@ | ||
3154 | 3150 | |||
3155 | 3151 | ut_ad(!dict_index_is_clust(index)); | ||
3156 | 3152 | |||
3157 | 3153 | - if (!lock_sec_rec_cons_read_sees( | ||
3158 | 3154 | - rec, trx->read_view)) { | ||
3159 | 3155 | + if ((is_get_from_row_cache == FALSE | ||
3160 | 3156 | + && !lock_sec_rec_cons_read_sees(rec, trx->read_view)) | ||
3161 | 3157 | + || (is_get_from_row_cache == TRUE | ||
3162 | 3158 | + && !lock_sec_rec_in_row_cache_cons_read_sees(rec, | ||
3163 | 3159 | + offsets, trx->read_view))) { | ||
3164 | 3160 | goto requires_clust_rec; | ||
3165 | 3161 | + } else { | ||
3166 | 3162 | + //TB_HOOK put_rec_to_row_cache | ||
3167 | 3163 | + if (is_get_from_row_cache == FALSE) { | ||
3168 | 3164 | + put_rec_to_row_cache_in_sel(prebuilt, direction, | ||
3169 | 3165 | + unique_search, rec, offsets, index, | ||
3170 | 3166 | + index != clust_index); | ||
3171 | 3167 | + } | ||
3172 | 3168 | } | ||
3173 | 3169 | } | ||
3174 | 3170 | } | ||
3175 | 3171 | @@ -4483,7 +4556,13 @@ | ||
3176 | 3172 | if (clust_rec == NULL) { | ||
3177 | 3173 | /* The record did not exist in the read view */ | ||
3178 | 3174 | ut_ad(prebuilt->select_lock_type == LOCK_NONE); | ||
3179 | 3175 | + if(is_get_from_row_cache == TRUE){ | ||
3180 | 3176 | + // when is is_get_from_row_cache ,if clust_rec == NULL mean there is not record which can be | ||
3181 | 3177 | + // read by this trx | ||
3182 | 3178 | + err = DB_RECORD_NOT_FOUND; | ||
3183 | 3179 | |||
3184 | 3180 | + goto normal_return; | ||
3185 | 3181 | + } | ||
3186 | 3182 | goto next_rec; | ||
3187 | 3183 | } | ||
3188 | 3184 | break; | ||
3189 | 3185 | @@ -4573,6 +4652,10 @@ | ||
3190 | 3186 | goto got_row; | ||
3191 | 3187 | } | ||
3192 | 3188 | |||
3193 | 3189 | + if(is_get_from_row_cache == TRUE){ | ||
3194 | 3190 | + goto got_row; | ||
3195 | 3191 | + } | ||
3196 | 3192 | + | ||
3197 | 3193 | goto next_rec; | ||
3198 | 3194 | } else { | ||
3199 | 3195 | if (UNIV_UNLIKELY | ||
3200 | 3196 | @@ -4634,10 +4717,11 @@ | ||
3201 | 3197 | return 'end of file'. Exceptions are locking reads and the MySQL | ||
3202 | 3198 | HANDLER command where the user can move the cursor with PREV or NEXT | ||
3203 | 3199 | even after a unique search. */ | ||
3204 | 3200 | - | ||
3205 | 3201 | - if (!unique_search_from_clust_index | ||
3206 | 3202 | - || prebuilt->select_lock_type != LOCK_NONE | ||
3207 | 3203 | - || prebuilt->used_in_HANDLER) { | ||
3208 | 3204 | + //if get from row_cache ,it is no next or prev | ||
3209 | 3205 | + if (is_get_from_row_cache == FALSE | ||
3210 | 3206 | + && (!unique_search_from_clust_index | ||
3211 | 3207 | + || prebuilt->select_lock_type != LOCK_NONE | ||
3212 | 3208 | + || prebuilt->used_in_HANDLER)) { | ||
3213 | 3209 | |||
3214 | 3210 | /* Inside an update always store the cursor position */ | ||
3215 | 3211 | |||
3216 | 3212 | @@ -4649,6 +4733,7 @@ | ||
3217 | 3213 | goto normal_return; | ||
3218 | 3214 | |||
3219 | 3215 | next_rec: | ||
3220 | 3216 | + ut_ad(is_get_from_row_cache == FALSE); | ||
3221 | 3217 | /* Reset the old and new "did semi-consistent read" flags. */ | ||
3222 | 3218 | if (UNIV_UNLIKELY(prebuilt->row_read_type | ||
3223 | 3219 | == ROW_READ_DID_SEMI_CONSISTENT)) { | ||
3224 | 3220 | Index: storage/innobase/row/row0umod.c | ||
3225 | 3221 | =================================================================== | ||
3226 | 3222 | --- storage/innobase/row/row0umod.c (revision 498) | ||
3227 | 3223 | +++ storage/innobase/row/row0umod.c (revision 724) | ||
3228 | 3224 | @@ -43,6 +43,8 @@ | ||
3229 | 3225 | #include "row0upd.h" | ||
3230 | 3226 | #include "que0que.h" | ||
3231 | 3227 | #include "log0log.h" | ||
3232 | 3228 | +#include "row0cache.h" | ||
3233 | 3229 | +#include "btr0sea.h" | ||
3234 | 3230 | |||
3235 | 3231 | /* Considerations on undoing a modify operation. | ||
3236 | 3232 | (1) Undoing a delete marking: all index records should be found. Some of | ||
3237 | 3233 | @@ -128,6 +130,26 @@ | ||
3238 | 3234 | |||
3239 | 3235 | ut_ad(success); | ||
3240 | 3236 | |||
3241 | 3237 | + //TB_HOOK | ||
3242 | 3238 | + { | ||
3243 | 3239 | + dict_index_t* index; | ||
3244 | 3240 | + rec_t* rec; | ||
3245 | 3241 | + mem_heap_t* heap = NULL; | ||
3246 | 3242 | + ulint offsets_[REC_OFFS_NORMAL_SIZE]; | ||
3247 | 3243 | + ulint* offsets; | ||
3248 | 3244 | + rec_offs_init(offsets_); | ||
3249 | 3245 | + | ||
3250 | 3246 | + index = dict_table_get_first_index(node->table); | ||
3251 | 3247 | + //TB_HOOK | ||
3252 | 3248 | + rec = btr_pcur_get_rec(pcur); | ||
3253 | 3249 | + offsets = rec_get_offsets(rec, index, offsets_, ULINT_UNDEFINED, &heap); | ||
3254 | 3250 | + remove_from_row_cache(rec, offsets, index); | ||
3255 | 3251 | + | ||
3256 | 3252 | + if (UNIV_LIKELY_NULL(heap)) { | ||
3257 | 3253 | + mem_heap_free(heap); | ||
3258 | 3254 | + } | ||
3259 | 3255 | + } | ||
3260 | 3256 | + | ||
3261 | 3257 | if (mode == BTR_MODIFY_LEAF) { | ||
3262 | 3258 | |||
3263 | 3259 | err = btr_cur_optimistic_update(BTR_NO_LOCKING_FLAG | ||
3264 | 3260 | @@ -597,6 +619,10 @@ | ||
3265 | 3261 | transactions. */ | ||
3266 | 3262 | ut_a(thr_is_recv(thr)); | ||
3267 | 3263 | } else { | ||
3268 | 3264 | + //TB_HOOK | ||
3269 | 3265 | + if(dict_index_is_unique(index)){ | ||
3270 | 3266 | + remove_from_row_cache_for_tuple(entry , index); | ||
3271 | 3267 | + } | ||
3272 | 3268 | err = row_undo_mod_del_mark_or_remove_sec( | ||
3273 | 3269 | node, thr, index, entry); | ||
3274 | 3270 | |||
3275 | 3271 | @@ -646,6 +672,10 @@ | ||
3276 | 3272 | entry = row_build_index_entry(node->row, node->ext, | ||
3277 | 3273 | index, heap); | ||
3278 | 3274 | ut_a(entry); | ||
3279 | 3275 | + //TB_HOOK | ||
3280 | 3276 | + if(dict_index_is_unique(index)){ | ||
3281 | 3277 | + remove_from_row_cache_for_tuple(entry , index); | ||
3282 | 3278 | + } | ||
3283 | 3279 | err = row_undo_mod_del_unmark_sec_and_undo_update( | ||
3284 | 3280 | BTR_MODIFY_LEAF, thr, index, entry); | ||
3285 | 3281 | if (err == DB_FAIL) { | ||
3286 | 3282 | @@ -746,7 +776,10 @@ | ||
3287 | 3283 | version of it, if the secondary index record | ||
3288 | 3284 | through which we do the search is | ||
3289 | 3285 | delete-marked. */ | ||
3290 | 3286 | - | ||
3291 | 3287 | + //TB_HOOK | ||
3292 | 3288 | + if(dict_index_is_unique(index)){ | ||
3293 | 3289 | + remove_from_row_cache_for_tuple(entry , index); | ||
3294 | 3290 | + } | ||
3295 | 3291 | err = row_undo_mod_del_mark_or_remove_sec( | ||
3296 | 3292 | node, thr, index, entry); | ||
3297 | 3293 | if (err != DB_SUCCESS) { | ||
3298 | 3294 | Index: sql/handler.h | ||
3299 | 3295 | =================================================================== | ||
3300 | 3296 | --- sql/handler.h (revision 498) | ||
3301 | 3297 | +++ sql/handler.h (revision 724) | ||
3302 | 3298 | @@ -1579,6 +1579,10 @@ | ||
3303 | 3299 | DBUG_ASSERT(FALSE); | ||
3304 | 3300 | return HA_ERR_WRONG_COMMAND; | ||
3305 | 3301 | } | ||
3306 | 3302 | + virtual bool ha_is_in_cache(const uchar * key, key_part_map keypart_map) { | ||
3307 | 3303 | + uint key_len = calculate_key_len(table, active_index, key, keypart_map); | ||
3308 | 3304 | + return is_in_cache(key, key_len); | ||
3309 | 3305 | + } | ||
3310 | 3306 | /** | ||
3311 | 3307 | @brief | ||
3312 | 3308 | Positions an index cursor to the index specified in the handle. Fetches the | ||
3313 | 3309 | @@ -2069,6 +2073,8 @@ | ||
3314 | 3310 | virtual int open(const char *name, int mode, uint test_if_locked)=0; | ||
3315 | 3311 | virtual int index_init(uint idx, bool sorted) { active_index= idx; return 0; } | ||
3316 | 3312 | virtual int index_end() { active_index= MAX_KEY; return 0; } | ||
3317 | 3313 | + virtual bool is_in_cache(const uchar * key, | ||
3318 | 3314 | + uint key_len){return false;} | ||
3319 | 3315 | /** | ||
3320 | 3316 | rnd_init() can be called two times without rnd_end() in between | ||
3321 | 3317 | (it only makes sense if scan=1). |
fix bug for memory leak on shutdown
fix bug for crash on rollback