Merge lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.1 into lp:percona-server/5.1
- BT-16724-xtradb-bmp-requests-5.1
- Merge into 5.1
Status: | Merged |
---|---|
Approved by: | Alexey Kopytov |
Approved revision: | no longer in the source branch. |
Merged at revision: | 519 |
Proposed branch: | lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.1 |
Merge into: | lp:percona-server/5.1 |
Diff against target: |
1079 lines (+612/-43) 23 files modified
Percona-Server/include/mysql_com.h (+4/-0) Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc (+0/-11) Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result (+1/-0) Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result (+1/-0) Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result (+76/-0) Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result (+3/-0) Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result (+1/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test (+3/-1) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test (+3/-1) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt (+1/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test (+200/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt (+1/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test (+7/-0) Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test (+3/-1) Percona-Server/sql/handler.cc (+36/-0) Percona-Server/sql/handler.h (+5/-0) Percona-Server/sql/lex.h (+1/-0) Percona-Server/sql/sql_parse.cc (+55/-3) Percona-Server/sql/sql_yacc.yy (+13/-0) Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc (+53/-0) Percona-Server/storage/innodb_plugin/include/log0online.h (+19/-6) Percona-Server/storage/innodb_plugin/include/sync0sync.h (+1/-0) Percona-Server/storage/innodb_plugin/log/log0online.c (+125/-20) |
To merge this branch: | bzr merge lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.1 |
Related bugs: | |
Related blueprints: |
User requests for XtraDB changed page tracking
(Undefined)
XtraDB Bitmap user requests for 5.1
(Undefined)
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Alexey Kopytov (community) | Approve | ||
Vlad Lesin | Pending | ||
Review via email:
|
This proposal supersedes a proposal from 2012-11-29.
Commit message
Description of the change
Round 4: All review comments have been addressed.
http://
Round 3: Found nasty typo myself while working on dependent branches.
- Fix that typo in log_online_
- Replace all () function declarations with (void) to get compiler diagnostics when an arg is passed by mistake.
- Use compile_time_assert instead of ut_a where applicable.
http://
Round 2: All Vlad's review comments have been addressed.
http://
Implementation of
https:/
Jenkins: http://
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal | # |
Looks good. I have the following remarks:
1) Lines 350-365: Why only "RESET" request is tested here? Would it be reasonable to test the requests when innodb engine is disabled?
2) Line 512 - unnecessary change;
3) Line 686 - wrong comments format.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal | # |
It would be good if blueprint would mention about permissions for the requests.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal | # |
Vlad -
thanks for the review.
> 1) Lines 350-365: Why only "RESET" request is tested here? Would it be
> reasonable to test the requests when innodb engine is disabled?
Other requests are tested with the log tracking disabled too: see lines 304--339. I will consider testing with disabled InnoDB.
> 2) Line 512 - unnecessary change;
Fixed.
> 3) Line 686 - wrong comments format.
Fixed.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal | # |
> It would be good if blueprint would mention about permissions for the
> requests.
Added. This is a good idea, also I found that RESET request is allowed with FLUSH (or is it REFRESH?) privilege, not SUPER, which is a bug.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal | # |
Looks good for me.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal | # |
Laurynas,
Some mixed comments and questions in addition to the new reserved words
problem which you are already aware of:
- do we really want the ALL clause in FLUSH/RESET/PURGE? It is of
course nice when SQL is close to the natural language. But in this
case it just leads to confusion:
- Does FLUSH ALL ... BITMAPS mean I can flush only some bitmap
files?
- Why do other FLUSH statements not require the ALL clause? (i.e. there's no
"FLUSH ALL HOSTS").
- it is even more confusing in case of PURGE ALL ... BITMAPS
BEFORE. ALL in SQL is usually the opposite to "some", but in
this case we want to purge some files.
- wrong SQL in error messages and comments ("PURGE INNODB CHANGED
PAGE BITMAPS BEFORE")
- it is not obvious from the BP that PURGE ... BITMAPS TO is not
implemented
- no error message if RESET ... BITMAPS fails?
- we should make sure we use FLUSH NO_WRITE_TO_BINLOG ... BITMAPS in
innobackupex
- new handlerton "methods" flush_changed_
purge_
counterparts that value is cast to my_bool. Does not present any
problem with the current code, but I'd either wrap their return
values to test() or change the handlerton "methods" return type
from int to my_bool.
- what's the point of resetting srv_checkpoint_
innobase_
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal | # |
Alexey -
Thanks for your review.
> - do we really want the ALL clause in FLUSH/RESET/PURGE? It is of
> course nice when SQL is close to the natural language. But in this
> case it just leads to confusion:
> - Does FLUSH ALL ... BITMAPS mean I can flush only some bitmap
> files?
> - Why do other FLUSH statements not require the ALL clause? (i.e.
> there's no
> "FLUSH ALL HOSTS").
> - it is even more confusing in case of PURGE ALL ... BITMAPS
> BEFORE. ALL in SQL is usually the opposite to "some", but in
> this case we want to purge some files.
IIRC the way "ALL" is used in the parser which comes closest to this usage, especially in 5.5, is that "ALL" would be a wildcard for storage engines that support this request, i.e. "FLUSH INNODB CHANGED PAGE BITMAPS" invokes the InnoDB handler, and "FLUSH ALL ... " invokes all handlers. Of course, the former syntax is not implemented ATM, nor there are current plans to.
Does it makes sense to keep "ALL" as a storage engine wildcard here?
> - wrong SQL in error messages and comments ("PURGE INNODB CHANGED
> PAGE BITMAPS BEFORE")
Ack, will fix according to how the "ALL" discussion goes.
> - it is not obvious from the BP that PURGE ... BITMAPS TO is not
> implemented
Ack, split off a new BP and removed this request from the current one.
> - no error message if RESET ... BITMAPS fails?
It goes through the common RESET error path handling. But I have to check what kind of error will that produce.
> - we should make sure we use FLUSH NO_WRITE_TO_BINLOG ... BITMAPS in
> innobackupex
Yes. In fact, this is *the* reason, why there is no XtraBackup MP yet. Thanks about NO_WRITE_TO_BINLOG.
> - new handlerton "methods" flush_changed_
> purge_changed_
> counterparts that value is cast to my_bool. Does not present any
> problem with the current code, but I'd either wrap their return
> values to test() or change the handlerton "methods" return type
> from int to my_bool.
Ack.
> - what's the point of resetting srv_checkpoint_
> innobase_
Redundant it is.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal | # |
Hi Laurynas,
On Mon, 21 Jan 2013 07:32:21 -0000, Laurynas Biveinis wrote:
> Alexey -
>
> Thanks for your review.
>
>> - do we really want the ALL clause in FLUSH/RESET/PURGE? It is of
>> course nice when SQL is close to the natural language. But in this
>> case it just leads to confusion:
>> - Does FLUSH ALL ... BITMAPS mean I can flush only some bitmap
>> files?
>> - Why do other FLUSH statements not require the ALL clause? (i.e.
>> there's no
>> "FLUSH ALL HOSTS").
>> - it is even more confusing in case of PURGE ALL ... BITMAPS
>> BEFORE. ALL in SQL is usually the opposite to "some", but in
>> this case we want to purge some files.
>
> IIRC the way "ALL" is used in the parser which comes closest to this usage, especially in 5.5, is that "ALL" would be a wildcard for storage engines that support this request, i.e. "FLUSH INNODB CHANGED PAGE BITMAPS" invokes the InnoDB handler, and "FLUSH ALL ... " invokes all handlers. Of course, the former syntax is not implemented ATM, nor there are current plans to.
>
> Does it makes sense to keep "ALL" as a storage engine wildcard here?
>
That would again be inconsistent with "FLUSH TABLES" and "FLUSH ENGINE
LOGS".
And speaking from the realistic perspective, what other storage engines
could possibly have this support implemented in the next few years?
It's all about keeping consistency and avoiding syntax redundancy.
"CHANGED PAGE BITMAP" is also redundant, and introducing 2 new reserved
words for this specific syntax doesn't look good either. Following the
same logic, it should be "FLUSH DES KEY FILE", rather than "FLUSH
DES_KEY_FILE", or "FLUSH USER RESOURCES" (or "FLUSH ALL USER
RESOURCES"), rather than "FLUSH USER_RESOURCES".
I would implement "FLUSH CHANGED_
of the shortcomings being discussed.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal | # |
> I would implement "FLUSH CHANGED_
> of the shortcomings being discussed.
Agreed, this is how I will proceed.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Alexey Kopytov (akopytov) : | # |
Preview Diff
1 | === modified file 'Percona-Server/include/mysql_com.h' |
2 | --- Percona-Server/include/mysql_com.h 2011-11-24 02:01:17 +0000 |
3 | +++ Percona-Server/include/mysql_com.h 2013-01-22 18:26:24 +0000 |
4 | @@ -133,6 +133,10 @@ |
5 | #define REFRESH_DES_KEY_FILE 0x40000L |
6 | #define REFRESH_USER_RESOURCES 0x80000L |
7 | #define REFRESH_QUERY_RESPONSE_TIME 0x100000L /* response time distibution */ |
8 | +#define REFRESH_FLUSH_PAGE_BITMAPS 0x200000L |
9 | +#define REFRESH_RESET_PAGE_BITMAPS 0x400000L |
10 | + |
11 | +#define PURGE_BITMAPS_TO_LSN 1 |
12 | |
13 | #define CLIENT_LONG_PASSWORD 1 /* new more secure passwords */ |
14 | #define CLIENT_FOUND_ROWS 2 /* Found instead of affected rows */ |
15 | |
16 | === removed file 'Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc' |
17 | --- Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc 2012-11-25 09:29:43 +0000 |
18 | +++ Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc 1970-01-01 00:00:00 +0000 |
19 | @@ -1,11 +0,0 @@ |
20 | -# Remove all the InnoDB bitmap files. A temporary measure until RESET user request is implemented |
21 | - |
22 | -let $MYSQLD_DATADIR= `select @@datadir`; |
23 | - |
24 | ---exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect |
25 | ---shutdown_server 10 |
26 | ---source include/wait_until_disconnected.inc |
27 | -remove_files_wildcard $MYSQLD_DATADIR ib_modified_log*; |
28 | ---enable_reconnect |
29 | ---exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect |
30 | ---source include/wait_until_connected_again.inc |
31 | |
32 | === modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result' |
33 | --- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result 2012-11-25 09:29:43 +0000 |
34 | +++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result 2013-01-22 18:26:24 +0000 |
35 | @@ -1,3 +1,4 @@ |
36 | +RESET CHANGED_PAGE_BITMAPS; |
37 | DROP TABLE IF EXISTS t1, t2; |
38 | CREATE TABLE t1 (x INT) ENGINE=InnoDB; |
39 | INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
40 | |
41 | === modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result' |
42 | --- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result 2012-11-25 09:29:43 +0000 |
43 | +++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result 2013-01-22 18:26:24 +0000 |
44 | @@ -1,3 +1,4 @@ |
45 | +RESET CHANGED_PAGE_BITMAPS; |
46 | DROP TABLE IF EXISTS t1, t2; |
47 | CREATE TABLE t1 (x INT) ENGINE=InnoDB; |
48 | INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
49 | |
50 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result' |
51 | --- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result 1970-01-01 00:00:00 +0000 |
52 | +++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result 2013-01-22 18:26:24 +0000 |
53 | @@ -0,0 +1,76 @@ |
54 | +RESET CHANGED_PAGE_BITMAPS; |
55 | +DROP TABLE IF EXISTS t1; |
56 | +DELETE FROM mysql.user WHERE USER='mysqltest_1'; |
57 | +FLUSH PRIVILEGES; |
58 | +CREATE TABLE t1 (x INT) ENGINE=InnoDB; |
59 | +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
60 | +FLUSH CHANGED_PAGE_BITMAPS; |
61 | +Before RESET: |
62 | +ib_modified_log_1 |
63 | +ib_modified_log_2 |
64 | +RESET CHANGED_PAGE_BITMAPS; |
65 | +After RESET: |
66 | +ib_modified_log_1 |
67 | +RESET CHANGED_PAGE_BITMAPS; |
68 | +After 2nd RESET |
69 | +ib_modified_log_1 |
70 | +After RESETs and restart: |
71 | +ib_modified_log_1 |
72 | +ib_modified_log_2 |
73 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1; |
74 | +After PURGE ... BEFORE 1: |
75 | +ib_modified_log_2 |
76 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
77 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
78 | +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
79 | +After PURGE ... BEFORE 100000000 and restart: |
80 | +ib_modified_log_4 |
81 | +ib_modified_log_5 |
82 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 5+5; |
83 | +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '+5' at line 1 |
84 | +PURGE CHANGED_PAGE_BITMAPS BEFORE -12; |
85 | +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-12' at line 1 |
86 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1.2e-2; |
87 | +ERROR 42000: Only integers allowed as number here near '1.2e-2' at line 1 |
88 | +PURGE CHANGED_PAGE_BITMAPS BEFORE t1; |
89 | +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 't1' at line 1 |
90 | +PURGE CHANGED_PAGE_BITMAPS BEFORE "t1"; |
91 | +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"t1"' at line 1 |
92 | +PURGE CHANGED_PAGE_BITMAPS BEFORE TRUE; |
93 | +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TRUE' at line 1 |
94 | +PURGE CHANGED_PAGE_BITMAPS BEFORE NULL; |
95 | +ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'NULL' at line 1 |
96 | +CREATE USER mysqltest_1@localhost; |
97 | +GRANT ALL ON *.* TO mysqltest_1@localhost; |
98 | +REVOKE SUPER ON *.* FROM mysqltest_1@localhost; |
99 | +FLUSH CHANGED_PAGE_BITMAPS; |
100 | +ERROR 42000: Access denied; you need the SUPER privilege for this operation |
101 | +RESET CHANGED_PAGE_BITMAPS; |
102 | +ERROR 42000: Access denied; you need the SUPER privilege for this operation |
103 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
104 | +ERROR 42000: Access denied; you need the SUPER privilege for this operation |
105 | +DROP USER mysqltest_1@localhost; |
106 | +SELECT @@GLOBAL.innodb_track_changed_pages; |
107 | +@@GLOBAL.innodb_track_changed_pages |
108 | +0 |
109 | +FLUSH CHANGED_PAGE_BITMAPS; |
110 | +Before the PURGE with tracking disabled |
111 | +ib_modified_log_4 |
112 | +ib_modified_log_5 |
113 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1; |
114 | +After the PURGE that deletes nothing: |
115 | +ib_modified_log_4 |
116 | +ib_modified_log_5 |
117 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
118 | +After the PURGE that deletes everything: |
119 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
120 | +After the repeated PURGE: |
121 | +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
122 | +Before the RESET while tracking disabled: |
123 | +ib_modified_log_1 |
124 | +ib_modified_log_2 |
125 | +RESET CHANGED_PAGE_BITMAPS; |
126 | +After the RESET with tracking disabled: |
127 | +DROP TABLE t1; |
128 | +CREATE TABLE CHANGED_PAGE_BITMAPS (a INT); |
129 | +DROP TABLE CHANGED_PAGE_BITMAPS; |
130 | |
131 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result' |
132 | --- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result 1970-01-01 00:00:00 +0000 |
133 | +++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result 2013-01-22 18:26:24 +0000 |
134 | @@ -0,0 +1,3 @@ |
135 | +FLUSH CHANGED_PAGE_BITMAPS; |
136 | +RESET CHANGED_PAGE_BITMAPS; |
137 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1; |
138 | |
139 | === modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result' |
140 | --- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 2012-11-25 09:29:43 +0000 |
141 | +++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 2013-01-22 18:26:24 +0000 |
142 | @@ -1,3 +1,4 @@ |
143 | +RESET CHANGED_PAGE_BITMAPS; |
144 | DROP TABLE IF EXISTS T1; |
145 | CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB; |
146 | 1st interval end LSN greater than interval start LSN: |
147 | |
148 | === modified file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test' |
149 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test 2012-11-25 09:29:43 +0000 |
150 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test 2013-01-22 18:26:24 +0000 |
151 | @@ -13,7 +13,9 @@ |
152 | # Valgrind reports useless errors on very fast server shutdowns |
153 | --source include/not_valgrind.inc |
154 | |
155 | ---source include/delete_innodb_bitmaps.inc |
156 | +# Delete any existing bitmaps |
157 | +--source include/restart_mysqld.inc |
158 | +RESET CHANGED_PAGE_BITMAPS; |
159 | |
160 | --disable_warnings |
161 | DROP TABLE IF EXISTS t1, t2; |
162 | |
163 | === modified file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test' |
164 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test 2012-11-25 09:29:43 +0000 |
165 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test 2013-01-22 18:26:24 +0000 |
166 | @@ -2,7 +2,9 @@ |
167 | # percona_changed_page_bmp test, which contains the full functional tests. |
168 | --source include/have_innodb_plugin.inc |
169 | |
170 | ---source include/delete_innodb_bitmaps.inc |
171 | +# Delete any existing bitmaps |
172 | +--source include/restart_mysqld.inc |
173 | +RESET CHANGED_PAGE_BITMAPS; |
174 | |
175 | --disable_warnings |
176 | DROP TABLE IF EXISTS t1, t2; |
177 | |
178 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt' |
179 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt 1970-01-01 00:00:00 +0000 |
180 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt 2013-01-22 18:26:24 +0000 |
181 | @@ -0,0 +1,1 @@ |
182 | +--innodb_track_changed_pages=TRUE |
183 | |
184 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test' |
185 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test 1970-01-01 00:00:00 +0000 |
186 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test 2013-01-22 18:26:24 +0000 |
187 | @@ -0,0 +1,200 @@ |
188 | +# |
189 | +# Tests for changed page tracking bitmaps: user requests. |
190 | +# |
191 | +--source include/have_innodb_plugin.inc |
192 | + |
193 | +# Delete any existing bitmaps |
194 | +--source include/restart_mysqld.inc |
195 | +RESET CHANGED_PAGE_BITMAPS; |
196 | + |
197 | +--source include/count_sessions.inc |
198 | + |
199 | +let $MYSQLD_DATADIR= `select @@datadir`; |
200 | + |
201 | +--disable_warnings |
202 | +DROP TABLE IF EXISTS t1; |
203 | +DELETE FROM mysql.user WHERE USER='mysqltest_1'; |
204 | +FLUSH PRIVILEGES; |
205 | +--enable_warnings |
206 | + |
207 | +CREATE TABLE t1 (x INT) ENGINE=InnoDB; |
208 | +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
209 | + |
210 | +# |
211 | +# Test FLUSH CHANGED_PAGE_BITMAPS. Not much to test due to checkpointing non-determinism, |
212 | +# just see that it does not crash or return an error |
213 | +# |
214 | +FLUSH CHANGED_PAGE_BITMAPS; |
215 | + |
216 | +# |
217 | +# Test that RESET CHANGED_PAGE_BITMAPS works |
218 | +# |
219 | + |
220 | +# Have at least two bitmap files |
221 | +--source include/restart_mysqld.inc |
222 | + |
223 | +--echo Before RESET: |
224 | +--replace_regex /_[[:digit:]]+\.xdb$// |
225 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
226 | + |
227 | +RESET CHANGED_PAGE_BITMAPS; |
228 | + |
229 | +--echo After RESET: |
230 | +--replace_regex /_[[:digit:]]+\.xdb$// |
231 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
232 | + |
233 | +# Test consecutive RESET |
234 | +RESET CHANGED_PAGE_BITMAPS; |
235 | +--echo After 2nd RESET |
236 | +--replace_regex /_[[:digit:]]+\.xdb$// |
237 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
238 | + |
239 | +# Check that the file sequence after RESET starts with 1 again |
240 | +--source include/restart_mysqld.inc |
241 | + |
242 | +--echo After RESETs and restart: |
243 | +--replace_regex /_[[:digit:]]+\.xdb$// |
244 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
245 | + |
246 | +# |
247 | +# Test that PURGE CHANGED_PAGE_BITMAPS BEFORE works. We don't test partial PURGE |
248 | +# because it's hard to extract good LSN value for that. |
249 | +# |
250 | + |
251 | +# PURGE that deletes nothing |
252 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1; |
253 | +--echo After PURGE ... BEFORE 1: |
254 | +--replace_regex /_[[:digit:]]+\.xdb$// |
255 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
256 | + |
257 | +# PURGE that deletes everything |
258 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
259 | +# Test consecutive PURGE |
260 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
261 | + |
262 | +# Check that file sequence continues |
263 | +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
264 | +--source include/restart_mysqld.inc |
265 | +--echo After PURGE ... BEFORE 100000000 and restart: |
266 | +--replace_regex /_[[:digit:]]+\.xdb$// |
267 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
268 | + |
269 | +# |
270 | +# Test malformed PURGE ... BEFORE |
271 | +# |
272 | +--error ER_PARSE_ERROR |
273 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 5+5; |
274 | + |
275 | +--error ER_PARSE_ERROR |
276 | +PURGE CHANGED_PAGE_BITMAPS BEFORE -12; |
277 | + |
278 | +--error ER_PARSE_ERROR |
279 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1.2e-2; |
280 | + |
281 | +--error ER_PARSE_ERROR |
282 | +PURGE CHANGED_PAGE_BITMAPS BEFORE t1; |
283 | + |
284 | +--error ER_PARSE_ERROR |
285 | +PURGE CHANGED_PAGE_BITMAPS BEFORE "t1"; |
286 | + |
287 | +--error ER_PARSE_ERROR |
288 | +PURGE CHANGED_PAGE_BITMAPS BEFORE TRUE; |
289 | + |
290 | +--error ER_PARSE_ERROR |
291 | +PURGE CHANGED_PAGE_BITMAPS BEFORE NULL; |
292 | + |
293 | +# |
294 | +# Test that users without SUPER privilege cannot issue user requests |
295 | +# |
296 | +CREATE USER mysqltest_1@localhost; |
297 | +GRANT ALL ON *.* TO mysqltest_1@localhost; |
298 | +REVOKE SUPER ON *.* FROM mysqltest_1@localhost; |
299 | + |
300 | +connect (conn1,localhost,mysqltest_1,,); |
301 | + |
302 | +--error ER_SPECIFIC_ACCESS_DENIED_ERROR |
303 | +FLUSH CHANGED_PAGE_BITMAPS; |
304 | + |
305 | +--error ER_SPECIFIC_ACCESS_DENIED_ERROR |
306 | +RESET CHANGED_PAGE_BITMAPS; |
307 | + |
308 | +--error ER_SPECIFIC_ACCESS_DENIED_ERROR |
309 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
310 | + |
311 | +connection default; |
312 | +disconnect conn1; |
313 | + |
314 | +--source include/wait_until_count_sessions.inc |
315 | + |
316 | +DROP USER mysqltest_1@localhost; |
317 | + |
318 | +# |
319 | +# Test FLUSH and PURGE requests with log tracking disabled |
320 | +# |
321 | +--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect |
322 | +--shutdown_server 10 |
323 | +--source include/wait_until_disconnected.inc |
324 | +--enable_reconnect |
325 | +--exec echo "restart:--innodb_track_changed_pages=FALSE" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect |
326 | +--source include/wait_until_connected_again.inc |
327 | + |
328 | +SELECT @@GLOBAL.innodb_track_changed_pages; |
329 | + |
330 | +FLUSH CHANGED_PAGE_BITMAPS; |
331 | + |
332 | +# The previous bitmap files should be present |
333 | +--echo Before the PURGE with tracking disabled |
334 | +--replace_regex /_[[:digit:]]+\.xdb$// |
335 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
336 | + |
337 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1; |
338 | + |
339 | +--echo After the PURGE that deletes nothing: |
340 | +--replace_regex /_[[:digit:]]+\.xdb$// |
341 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
342 | + |
343 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
344 | + |
345 | +--echo After the PURGE that deletes everything: |
346 | +--replace_regex /_[[:digit:]]+\.xdb$// |
347 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
348 | + |
349 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000; |
350 | + |
351 | +--echo After the repeated PURGE: |
352 | +--replace_regex /_[[:digit:]]+\.xdb$// |
353 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
354 | + |
355 | +# |
356 | +# Test RESET request with log tracking disabled |
357 | +# |
358 | + |
359 | +# Generate some bitmap data again |
360 | +--source include/restart_mysqld.inc |
361 | +INSERT INTO t1 VALUES (1),(2),(3),(4),(5); |
362 | +--source include/restart_mysqld.inc |
363 | + |
364 | +--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect |
365 | +--shutdown_server 10 |
366 | +--source include/wait_until_disconnected.inc |
367 | +--enable_reconnect |
368 | +--exec echo "restart:--innodb_track_changed_pages=FALSE" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect |
369 | +--source include/wait_until_connected_again.inc |
370 | + |
371 | +--echo Before the RESET while tracking disabled: |
372 | +--replace_regex /_[[:digit:]]+\.xdb$// |
373 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
374 | + |
375 | +RESET CHANGED_PAGE_BITMAPS; |
376 | + |
377 | +--echo After the RESET with tracking disabled: |
378 | +--replace_regex /_[[:digit:]]+\.xdb$// |
379 | +list_files $MYSQLD_DATADIR ib_modified_log*; |
380 | + |
381 | +DROP TABLE t1; |
382 | + |
383 | +# |
384 | +# Test that the new keyword does not preclude its use as identifier |
385 | +# |
386 | +CREATE TABLE CHANGED_PAGE_BITMAPS (a INT); |
387 | +DROP TABLE CHANGED_PAGE_BITMAPS; |
388 | |
389 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt' |
390 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt 1970-01-01 00:00:00 +0000 |
391 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt 2013-01-22 18:26:24 +0000 |
392 | @@ -0,0 +1,1 @@ |
393 | +--skip-innodb |
394 | |
395 | === added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test' |
396 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test 1970-01-01 00:00:00 +0000 |
397 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test 2013-01-22 18:26:24 +0000 |
398 | @@ -0,0 +1,7 @@ |
399 | +# |
400 | +# Test that changed page tracking user requests are parsed without InnoDB |
401 | +# |
402 | +FLUSH CHANGED_PAGE_BITMAPS; |
403 | +RESET CHANGED_PAGE_BITMAPS; |
404 | +PURGE CHANGED_PAGE_BITMAPS BEFORE 1; |
405 | + |
406 | |
407 | === modified file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test' |
408 | --- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 2012-11-25 09:29:43 +0000 |
409 | +++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 2013-01-22 18:26:24 +0000 |
410 | @@ -4,7 +4,9 @@ |
411 | |
412 | --source include/have_innodb_plugin.inc |
413 | |
414 | ---source include/delete_innodb_bitmaps.inc |
415 | +# Delete any existing bitmaps |
416 | +--source include/restart_mysqld.inc |
417 | +RESET CHANGED_PAGE_BITMAPS; |
418 | |
419 | --disable_warnings |
420 | DROP TABLE IF EXISTS T1; |
421 | |
422 | === modified file 'Percona-Server/sql/handler.cc' |
423 | --- Percona-Server/sql/handler.cc 2012-10-16 04:49:59 +0000 |
424 | +++ Percona-Server/sql/handler.cc 2013-01-22 18:26:24 +0000 |
425 | @@ -4636,6 +4636,42 @@ |
426 | return result; |
427 | } |
428 | |
429 | +static my_bool flush_changed_page_bitmaps_handlerton(THD *unused1, |
430 | + plugin_ref plugin, |
431 | + void *unused2) |
432 | +{ |
433 | + handlerton *hton= plugin_data(plugin, handlerton *); |
434 | + |
435 | + if (hton->flush_changed_page_bitmaps == NULL) |
436 | + return FALSE; |
437 | + |
438 | + return hton->flush_changed_page_bitmaps(); |
439 | +} |
440 | + |
441 | +bool ha_flush_changed_page_bitmaps() |
442 | +{ |
443 | + return plugin_foreach(NULL, flush_changed_page_bitmaps_handlerton, |
444 | + MYSQL_STORAGE_ENGINE_PLUGIN, NULL); |
445 | +} |
446 | + |
447 | +static my_bool purge_changed_page_bitmaps_handlerton(THD *unused1, |
448 | + plugin_ref plugin, |
449 | + void *lsn) |
450 | +{ |
451 | + handlerton *hton= plugin_data(plugin, handlerton *); |
452 | + |
453 | + if (hton->purge_changed_page_bitmaps == NULL) |
454 | + return FALSE; |
455 | + |
456 | + return hton->purge_changed_page_bitmaps(*(ulonglong *)lsn); |
457 | +} |
458 | + |
459 | +bool ha_purge_changed_page_bitmaps(ulonglong lsn) |
460 | +{ |
461 | + return plugin_foreach(NULL, purge_changed_page_bitmaps_handlerton, |
462 | + MYSQL_STORAGE_ENGINE_PLUGIN, &lsn); |
463 | +} |
464 | + |
465 | /* |
466 | Function to check if the conditions for row-based binlogging is |
467 | correct for the table. |
468 | |
469 | === modified file 'Percona-Server/sql/handler.h' |
470 | --- Percona-Server/sql/handler.h 2012-01-17 14:05:16 +0000 |
471 | +++ Percona-Server/sql/handler.h 2013-01-22 18:26:24 +0000 |
472 | @@ -693,6 +693,8 @@ |
473 | int (*fill_files_table)(handlerton *hton, THD *thd, |
474 | TABLE_LIST *tables, |
475 | class Item *cond); |
476 | + my_bool (*flush_changed_page_bitmaps)(void); |
477 | + my_bool (*purge_changed_page_bitmaps)(ulonglong lsn); |
478 | uint32 flags; /* global handler flags */ |
479 | /* |
480 | Those handlerton functions below are properly initialized at handler |
481 | @@ -2074,6 +2076,9 @@ |
482 | int ha_prepare(THD *thd); |
483 | int ha_recover(HASH *commit_list); |
484 | |
485 | +bool ha_flush_changed_page_bitmaps(); |
486 | +bool ha_purge_changed_page_bitmaps(ulonglong lsn); |
487 | + |
488 | /* transactions: these functions never call handlerton functions directly */ |
489 | int ha_commit_trans(THD *thd, bool all); |
490 | int ha_autocommit_or_rollback(THD *thd, int error); |
491 | |
492 | === modified file 'Percona-Server/sql/lex.h' |
493 | --- Percona-Server/sql/lex.h 2011-11-24 02:01:23 +0000 |
494 | +++ Percona-Server/sql/lex.h 2013-01-22 18:26:24 +0000 |
495 | @@ -102,6 +102,7 @@ |
496 | { "CHAIN", SYM(CHAIN_SYM)}, |
497 | { "CHANGE", SYM(CHANGE)}, |
498 | { "CHANGED", SYM(CHANGED)}, |
499 | + { "CHANGED_PAGE_BITMAPS", SYM(CHANGED_PAGE_BITMAPS_SYM)}, |
500 | { "CHAR", SYM(CHAR_SYM)}, |
501 | { "CHARACTER", SYM(CHAR_SYM)}, |
502 | { "CHARSET", SYM(CHARSET)}, |
503 | |
504 | === modified file 'Percona-Server/sql/sql_parse.cc' |
505 | --- Percona-Server/sql/sql_parse.cc 2012-08-20 03:14:02 +0000 |
506 | +++ Percona-Server/sql/sql_parse.cc 2013-01-22 18:26:24 +0000 |
507 | @@ -2487,9 +2487,35 @@ |
508 | { |
509 | if (check_global_access(thd, SUPER_ACL)) |
510 | goto error; |
511 | - /* PURGE MASTER LOGS TO 'file' */ |
512 | - res = purge_master_logs(thd, lex->to_log); |
513 | - break; |
514 | + if (lex->type == 0) |
515 | + { |
516 | + /* PURGE MASTER LOGS TO 'file' */ |
517 | + res = purge_master_logs(thd, lex->to_log); |
518 | + break; |
519 | + } |
520 | + if (lex->type == PURGE_BITMAPS_TO_LSN) |
521 | + { |
522 | + /* PURGE CHANGED_PAGE_BITMAPS BEFORE lsn */ |
523 | + ulonglong lsn= 0; |
524 | + Item *it= (Item *)lex->value_list.head(); |
525 | + if ((!it->fixed && it->fix_fields(lex->thd, &it)) || it->check_cols(1) |
526 | + || it->null_value) |
527 | + { |
528 | + my_error(ER_WRONG_ARGUMENTS, MYF(0), |
529 | + "PURGE CHANGED_PAGE_BITMAPS BEFORE"); |
530 | + goto error; |
531 | + } |
532 | + lsn= it->val_uint(); |
533 | + res= ha_purge_changed_page_bitmaps(lsn); |
534 | + if (res) |
535 | + { |
536 | + my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0), |
537 | + "PURGE CHANGED_PAGE_BITMAPS BEFORE"); |
538 | + goto error; |
539 | + } |
540 | + my_ok(thd); |
541 | + break; |
542 | + } |
543 | } |
544 | case SQLCOM_PURGE_BEFORE: |
545 | { |
546 | @@ -7410,6 +7436,32 @@ |
547 | } |
548 | pthread_mutex_unlock(&LOCK_global_user_client_stats); |
549 | } |
550 | + if (options & REFRESH_FLUSH_PAGE_BITMAPS) |
551 | + { |
552 | + if (check_global_access(thd, SUPER_ACL)) |
553 | + result= 1; |
554 | + else |
555 | + { |
556 | + result= ha_flush_changed_page_bitmaps(); |
557 | + if (result) |
558 | + { |
559 | + my_error(ER_UNKNOWN_ERROR, MYF(0), "FLUSH CHANGED_PAGE_BITMAPS"); |
560 | + } |
561 | + } |
562 | + } |
563 | + if (options & REFRESH_RESET_PAGE_BITMAPS) |
564 | + { |
565 | + if (check_global_access(thd, SUPER_ACL)) |
566 | + result= 1; |
567 | + else |
568 | + { |
569 | + result= ha_purge_changed_page_bitmaps(0); |
570 | + if (result) |
571 | + { |
572 | + my_error(ER_UNKNOWN_ERROR, MYF(0), "RESET CHANGED_PAGE_BITMAPS"); |
573 | + } |
574 | + } |
575 | + } |
576 | if (*write_to_binlog != -1) |
577 | *write_to_binlog= tmp_write_to_binlog; |
578 | /* |
579 | |
580 | === modified file 'Percona-Server/sql/sql_yacc.yy' |
581 | --- Percona-Server/sql/sql_yacc.yy 2012-08-20 03:14:02 +0000 |
582 | +++ Percona-Server/sql/sql_yacc.yy 2013-01-22 18:26:24 +0000 |
583 | @@ -753,6 +753,7 @@ |
584 | %token CHAIN_SYM /* SQL-2003-N */ |
585 | %token CHANGE |
586 | %token CHANGED |
587 | +%token CHANGED_PAGE_BITMAPS_SYM /* MYSQL */ |
588 | %token CHARSET |
589 | %token CHAR_SYM /* SQL-2003-R */ |
590 | %token CHECKSUM_SYM |
591 | @@ -10691,6 +10692,8 @@ |
592 | { Lex->type|= REFRESH_DES_KEY_FILE; } |
593 | | RESOURCES |
594 | { Lex->type|= REFRESH_USER_RESOURCES; } |
595 | + | CHANGED_PAGE_BITMAPS_SYM |
596 | + { Lex->type|= REFRESH_FLUSH_PAGE_BITMAPS; } |
597 | ; |
598 | |
599 | opt_table_list: |
600 | @@ -10717,6 +10720,8 @@ |
601 | SLAVE { Lex->type|= REFRESH_SLAVE; } |
602 | | MASTER_SYM { Lex->type|= REFRESH_MASTER; } |
603 | | QUERY_SYM CACHE_SYM { Lex->type|= REFRESH_QUERY_CACHE;} |
604 | + | CHANGED_PAGE_BITMAPS_SYM |
605 | + { Lex->type |= REFRESH_RESET_PAGE_BITMAPS; } |
606 | ; |
607 | |
608 | purge: |
609 | @@ -10732,6 +10737,13 @@ |
610 | |
611 | purge_options: |
612 | master_or_binary LOGS_SYM purge_option |
613 | + | CHANGED_PAGE_BITMAPS_SYM BEFORE_SYM real_ulonglong_num |
614 | + { |
615 | + LEX *lex= Lex; |
616 | + lex->value_list.empty(); |
617 | + lex->value_list.push_front(new Item_uint($3)); |
618 | + lex->type= PURGE_BITMAPS_TO_LSN; |
619 | + } |
620 | ; |
621 | |
622 | purge_option: |
623 | @@ -11807,6 +11819,7 @@ |
624 | | CASCADED {} |
625 | | CHAIN_SYM {} |
626 | | CHANGED {} |
627 | + | CHANGED_PAGE_BITMAPS_SYM {} |
628 | | CIPHER_SYM {} |
629 | | CLIENT_STATS_SYM {} |
630 | | CLIENT_SYM {} |
631 | |
632 | === modified file 'Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc' |
633 | --- Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2013-01-18 03:33:06 +0000 |
634 | +++ Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2013-01-22 18:26:24 +0000 |
635 | @@ -79,6 +79,7 @@ |
636 | #include "row0sel.h" |
637 | #include "row0upd.h" |
638 | #include "log0log.h" |
639 | +#include "log0online.h" |
640 | #include "lock0lock.h" |
641 | #include "dict0crea.h" |
642 | #include "btr0cur.h" |
643 | @@ -315,6 +316,25 @@ |
644 | innobase_alter_table_flags( |
645 | /*=======================*/ |
646 | uint flags); |
647 | +/************************************************************//** |
648 | +Synchronously read and parse the redo log up to the last |
649 | +checkpoint to write the changed page bitmap. |
650 | +@return 0 to indicate success. Current implementation cannot fail. */ |
651 | +static |
652 | +my_bool |
653 | +innobase_flush_changed_page_bitmaps(); |
654 | +/*==================================*/ |
655 | +/************************************************************//** |
656 | +Delete all the bitmap files for data less than the specified LSN. |
657 | +If called with lsn == 0 (i.e. set by RESET request) or |
658 | +IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise |
659 | +continue it. |
660 | +@return 0 to indicate success, 1 for failure. */ |
661 | +static |
662 | +my_bool |
663 | +innobase_purge_changed_page_bitmaps( |
664 | +/*================================*/ |
665 | + ulonglong lsn); /*!< in: LSN to purge files up to */ |
666 | |
667 | static const char innobase_hton_name[]= "InnoDB"; |
668 | |
669 | @@ -2264,6 +2284,10 @@ |
670 | innobase_hton->flags=HTON_NO_FLAGS; |
671 | innobase_hton->release_temporary_latches=innobase_release_temporary_latches; |
672 | innobase_hton->alter_table_flags = innobase_alter_table_flags; |
673 | + innobase_hton->flush_changed_page_bitmaps |
674 | + = innobase_flush_changed_page_bitmaps; |
675 | + innobase_hton->purge_changed_page_bitmaps |
676 | + = innobase_purge_changed_page_bitmaps; |
677 | |
678 | ut_a(DATA_MYSQL_TRUE_VARCHAR == (ulint)MYSQL_TYPE_VARCHAR); |
679 | |
680 | @@ -2845,6 +2869,35 @@ |
681 | | HA_ONLINE_ADD_PK_INDEX_NO_WRITES); |
682 | } |
683 | |
684 | +/************************************************************//** |
685 | +Synchronously read and parse the redo log up to the last |
686 | +checkpoint to write the changed page bitmap. |
687 | +@return 0 to indicate success. Current implementation cannot fail. */ |
688 | +static |
689 | +my_bool |
690 | +innobase_flush_changed_page_bitmaps() |
691 | +/*=================================*/ |
692 | +{ |
693 | + if (srv_track_changed_pages) { |
694 | + log_online_follow_redo_log(); |
695 | + } |
696 | + return FALSE; |
697 | +} |
698 | + |
699 | +/************************************************************//** |
700 | +Delete all the bitmap files for data less than the specified LSN. |
701 | +If called with lsn == IB_ULONGLONG_MAX (i.e. set by RESET request), |
702 | +restart the bitmap file sequence, otherwise continue it. |
703 | +@return 0 to indicate success, 1 for failure. */ |
704 | +static |
705 | +my_bool |
706 | +innobase_purge_changed_page_bitmaps( |
707 | +/*================================*/ |
708 | + ulonglong lsn) /*!< in: LSN to purge files up to */ |
709 | +{ |
710 | + return (my_bool)log_online_purge_changed_page_bitmaps(lsn); |
711 | +} |
712 | + |
713 | /****************************************************************//** |
714 | Copy the current replication position from MySQL to a transaction. */ |
715 | static |
716 | |
717 | === modified file 'Percona-Server/storage/innodb_plugin/include/log0online.h' |
718 | --- Percona-Server/storage/innodb_plugin/include/log0online.h 2012-11-25 09:29:43 +0000 |
719 | +++ Percona-Server/storage/innodb_plugin/include/log0online.h 2013-01-22 18:26:24 +0000 |
720 | @@ -41,23 +41,36 @@ |
721 | Initializes the online log following subsytem. */ |
722 | UNIV_INTERN |
723 | void |
724 | -log_online_read_init(); |
725 | -/*===================*/ |
726 | +log_online_read_init(void); |
727 | +/*=======================*/ |
728 | |
729 | /*********************************************************************//** |
730 | Shuts down the online log following subsystem. */ |
731 | UNIV_INTERN |
732 | void |
733 | -log_online_read_shutdown(); |
734 | -/*=======================*/ |
735 | +log_online_read_shutdown(void); |
736 | +/*===========================*/ |
737 | |
738 | /*********************************************************************//** |
739 | Reads and parses the redo log up to last checkpoint LSN to build the changed |
740 | page bitmap which is then written to disk. */ |
741 | UNIV_INTERN |
742 | void |
743 | -log_online_follow_redo_log(); |
744 | -/*=========================*/ |
745 | +log_online_follow_redo_log(void); |
746 | +/*=============================*/ |
747 | + |
748 | +/************************************************************//** |
749 | +Delete all the bitmap files for data less than the specified LSN. |
750 | +If called with lsn == 0 (i.e. set by RESET request) or |
751 | +IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise |
752 | +continue it. |
753 | + |
754 | +@return FALSE to indicate success, TRUE for failure. */ |
755 | +UNIV_INTERN |
756 | +ibool |
757 | +log_online_purge_changed_page_bitmaps( |
758 | +/*==================================*/ |
759 | + ib_uint64_t lsn); /*!<in: LSN to purge files up to */ |
760 | |
761 | #define LOG_BITMAP_ITERATOR_START_LSN(i) \ |
762 | ((i).start_lsn) |
763 | |
764 | === modified file 'Percona-Server/storage/innodb_plugin/include/sync0sync.h' |
765 | --- Percona-Server/storage/innodb_plugin/include/sync0sync.h 2011-11-24 16:33:30 +0000 |
766 | +++ Percona-Server/storage/innodb_plugin/include/sync0sync.h 2013-01-22 18:26:24 +0000 |
767 | @@ -478,6 +478,7 @@ |
768 | #define SYNC_REC_LOCK 299 |
769 | #define SYNC_TRX_LOCK_HEAP 298 |
770 | #define SYNC_TRX_SYS_HEADER 290 |
771 | +#define SYNC_LOG_ONLINE 175 |
772 | #define SYNC_LOG 170 |
773 | #define SYNC_RECV 168 |
774 | #define SYNC_WORK_QUEUE 162 |
775 | |
776 | === modified file 'Percona-Server/storage/innodb_plugin/log/log0online.c' |
777 | --- Percona-Server/storage/innodb_plugin/log/log0online.c 2013-01-04 07:28:01 +0000 |
778 | +++ Percona-Server/storage/innodb_plugin/log/log0online.c 2013-01-22 18:26:24 +0000 |
779 | @@ -69,6 +69,7 @@ |
780 | both the correct type and the tree does |
781 | not mind its overwrite during |
782 | rbt_next() tree traversal. */ |
783 | + mutex_t mutex; /*!< mutex protecting all the fields.*/ |
784 | }; |
785 | |
786 | /* The log parsing and bitmap output struct instance */ |
787 | @@ -172,6 +173,8 @@ |
788 | byte search_page[MODIFIED_PAGE_BLOCK_SIZE]; |
789 | byte *page_ptr; |
790 | |
791 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
792 | + |
793 | ut_a(space != ULINT_UNDEFINED); |
794 | ut_a(page_no != ULINT_UNDEFINED); |
795 | |
796 | @@ -312,8 +315,8 @@ |
797 | @return the last fully tracked LSN */ |
798 | static |
799 | ib_uint64_t |
800 | -log_online_read_last_tracked_lsn() |
801 | -/*==============================*/ |
802 | +log_online_read_last_tracked_lsn(void) |
803 | +/*==================================*/ |
804 | { |
805 | byte page[MODIFIED_PAGE_BLOCK_SIZE]; |
806 | ibool is_last_page = FALSE; |
807 | @@ -487,8 +490,8 @@ |
808 | Create a new empty bitmap output file. */ |
809 | static |
810 | void |
811 | -log_online_start_bitmap_file() |
812 | -/*==========================*/ |
813 | +log_online_start_bitmap_file(void) |
814 | +/*==============================*/ |
815 | { |
816 | ibool success; |
817 | |
818 | @@ -517,7 +520,10 @@ |
819 | ib_uint64_t next_file_start_lsn) /*!<in: the start LSN name |
820 | part */ |
821 | { |
822 | - os_file_close(log_bmp_sys->out.file); |
823 | + if (log_bmp_sys->out.file != -1) { |
824 | + os_file_close(log_bmp_sys->out.file); |
825 | + log_bmp_sys->out.file = -1; |
826 | + } |
827 | log_bmp_sys->out_seq_num++; |
828 | log_online_make_bitmap_name(next_file_start_lsn); |
829 | log_online_start_bitmap_file(); |
830 | @@ -555,8 +561,8 @@ |
831 | Initialize the online log following subsytem. */ |
832 | UNIV_INTERN |
833 | void |
834 | -log_online_read_init() |
835 | -/*==================*/ |
836 | +log_online_read_init(void) |
837 | +/*======================*/ |
838 | { |
839 | ibool success; |
840 | ib_uint64_t tracking_start_lsn |
841 | @@ -565,13 +571,15 @@ |
842 | os_file_stat_t bitmap_dir_file_info; |
843 | ib_uint64_t last_file_start_lsn = MIN_TRACKED_LSN; |
844 | |
845 | - /* Assert (could be compile-time assert) that bitmap data start and end |
846 | - in a bitmap block is 8-byte aligned */ |
847 | - ut_a(MODIFIED_PAGE_BLOCK_BITMAP % 8 == 0); |
848 | - ut_a(MODIFIED_PAGE_BLOCK_BITMAP_LEN % 8 == 0); |
849 | + /* Bitmap data start and end in a bitmap block must be 8-byte |
850 | + aligned. */ |
851 | + compile_time_assert(MODIFIED_PAGE_BLOCK_BITMAP % 8 == 0); |
852 | + compile_time_assert(MODIFIED_PAGE_BLOCK_BITMAP_LEN % 8 == 0); |
853 | |
854 | log_bmp_sys = ut_malloc(sizeof(*log_bmp_sys)); |
855 | |
856 | + mutex_create(&log_bmp_sys->mutex, SYNC_LOG_ONLINE); |
857 | + |
858 | /* Enumerate existing bitmap files to either open the last one to get |
859 | the last tracked LSN either to find that there are none and start |
860 | tracking from scratch. */ |
861 | @@ -700,12 +708,15 @@ |
862 | Shut down the online log following subsystem. */ |
863 | UNIV_INTERN |
864 | void |
865 | -log_online_read_shutdown() |
866 | -/*======================*/ |
867 | +log_online_read_shutdown(void) |
868 | +/*==========================*/ |
869 | { |
870 | ib_rbt_node_t *free_list_node = log_bmp_sys->page_free_list; |
871 | |
872 | - os_file_close(log_bmp_sys->out.file); |
873 | + if (log_bmp_sys->out.file != -1) { |
874 | + os_file_close(log_bmp_sys->out.file); |
875 | + log_bmp_sys->out.file = -1; |
876 | + } |
877 | |
878 | rbt_free(log_bmp_sys->modified_pages); |
879 | |
880 | @@ -715,6 +726,8 @@ |
881 | free_list_node = next; |
882 | } |
883 | |
884 | + mutex_free(&log_bmp_sys->mutex); |
885 | + |
886 | ut_free(log_bmp_sys); |
887 | } |
888 | |
889 | @@ -758,14 +771,16 @@ |
890 | buffer. */ |
891 | static |
892 | void |
893 | -log_online_parse_redo_log() |
894 | -/*=======================*/ |
895 | +log_online_parse_redo_log(void) |
896 | +/*===========================*/ |
897 | { |
898 | byte *ptr = log_bmp_sys->parse_buf; |
899 | byte *end = log_bmp_sys->parse_buf_end; |
900 | |
901 | ulint len = 0; |
902 | |
903 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
904 | + |
905 | while (ptr != end |
906 | && log_bmp_sys->next_parse_lsn < log_bmp_sys->end_lsn) { |
907 | |
908 | @@ -856,6 +871,8 @@ |
909 | ulint actual_data_len = (end_offset >= start_offset) |
910 | ? end_offset - start_offset : 0; |
911 | |
912 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
913 | + |
914 | ut_memcpy(log_bmp_sys->parse_buf_end, log_block + start_offset, |
915 | actual_data_len); |
916 | |
917 | @@ -880,6 +897,8 @@ |
918 | { |
919 | ulint block_data_len; |
920 | |
921 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
922 | + |
923 | block_data_len = log_block_get_data_len(log_block); |
924 | |
925 | ut_ad(block_data_len % OS_FILE_LOG_BLOCK_SIZE == 0 |
926 | @@ -906,6 +925,8 @@ |
927 | byte* log_block_end = log_bmp_sys->read_buf |
928 | + (block_end_lsn - block_start_lsn); |
929 | |
930 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
931 | + |
932 | mutex_enter(&log_sys->mutex); |
933 | log_group_read_log_seg(LOG_RECOVER, log_bmp_sys->read_buf, |
934 | group, block_start_lsn, block_end_lsn); |
935 | @@ -968,6 +989,8 @@ |
936 | ib_uint64_t block_start_lsn = contiguous_lsn; |
937 | ib_uint64_t block_end_lsn; |
938 | |
939 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
940 | + |
941 | log_bmp_sys->next_parse_lsn = log_bmp_sys->start_lsn; |
942 | log_bmp_sys->parse_buf_end = log_bmp_sys->parse_buf; |
943 | |
944 | @@ -1004,6 +1027,8 @@ |
945 | { |
946 | ibool success; |
947 | |
948 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
949 | + |
950 | success = os_file_write(log_bmp_sys->out.name, log_bmp_sys->out.file, |
951 | block, |
952 | (ulint)(log_bmp_sys->out.offset & 0xFFFFFFFF), |
953 | @@ -1037,12 +1062,14 @@ |
954 | bitmap tree and recycles its nodes to the free list. */ |
955 | static |
956 | void |
957 | -log_online_write_bitmap() |
958 | -/*=====================*/ |
959 | +log_online_write_bitmap(void) |
960 | +/*=========================*/ |
961 | { |
962 | ib_rbt_node_t *bmp_tree_node; |
963 | const ib_rbt_node_t *last_bmp_tree_node; |
964 | |
965 | + ut_ad(mutex_own(&log_bmp_sys->mutex)); |
966 | + |
967 | if (log_bmp_sys->out.offset >= srv_max_bitmap_file_size) { |
968 | log_online_rotate_bitmap_file(log_bmp_sys->start_lsn); |
969 | } |
970 | @@ -1083,18 +1110,21 @@ |
971 | page bitmap which is then written to disk. */ |
972 | UNIV_INTERN |
973 | void |
974 | -log_online_follow_redo_log() |
975 | -/*========================*/ |
976 | +log_online_follow_redo_log(void) |
977 | +/*============================*/ |
978 | { |
979 | ib_uint64_t contiguous_start_lsn; |
980 | log_group_t* group; |
981 | |
982 | + mutex_enter(&log_bmp_sys->mutex); |
983 | + |
984 | /* Grab the LSN of the last checkpoint, we will parse up to it */ |
985 | mutex_enter(&(log_sys->mutex)); |
986 | log_bmp_sys->end_lsn = log_sys->last_checkpoint_lsn; |
987 | mutex_exit(&(log_sys->mutex)); |
988 | |
989 | if (log_bmp_sys->end_lsn == log_bmp_sys->start_lsn) { |
990 | + mutex_exit(&log_bmp_sys->mutex); |
991 | return; |
992 | } |
993 | |
994 | @@ -1116,6 +1146,8 @@ |
995 | log_online_write_bitmap(); |
996 | log_bmp_sys->start_lsn = log_bmp_sys->end_lsn; |
997 | log_set_tracked_lsn(log_bmp_sys->start_lsn); |
998 | + |
999 | + mutex_exit(&log_bmp_sys->mutex); |
1000 | } |
1001 | |
1002 | /*********************************************************************//** |
1003 | @@ -1512,3 +1544,76 @@ |
1004 | |
1005 | return TRUE; |
1006 | } |
1007 | + |
1008 | +/************************************************************//** |
1009 | +Delete all the bitmap files for data less than the specified LSN. |
1010 | +If called with lsn == 0 (i.e. set by RESET request) or |
1011 | +IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise |
1012 | +continue it. |
1013 | + |
1014 | +@return FALSE to indicate success, TRUE for failure. */ |
1015 | +UNIV_INTERN |
1016 | +ibool |
1017 | +log_online_purge_changed_page_bitmaps( |
1018 | +/*==================================*/ |
1019 | + ib_uint64_t lsn) /*!< in: LSN to purge files up to */ |
1020 | +{ |
1021 | + log_online_bitmap_file_range_t bitmap_files; |
1022 | + size_t i; |
1023 | + ibool result = FALSE; |
1024 | + |
1025 | + if (lsn == 0) { |
1026 | + lsn = IB_ULONGLONG_MAX; |
1027 | + } |
1028 | + |
1029 | + if (srv_track_changed_pages) { |
1030 | + /* User requests might happen with both enabled and disabled |
1031 | + tracking */ |
1032 | + mutex_enter(&log_bmp_sys->mutex); |
1033 | + } |
1034 | + |
1035 | + if (!log_online_setup_bitmap_file_range(&bitmap_files, 0, lsn)) { |
1036 | + if (srv_track_changed_pages) { |
1037 | + mutex_exit(&log_bmp_sys->mutex); |
1038 | + } |
1039 | + return TRUE; |
1040 | + } |
1041 | + |
1042 | + if (srv_track_changed_pages && lsn >= log_bmp_sys->end_lsn) { |
1043 | + /* If we have to delete the current output file, close it |
1044 | + first. */ |
1045 | + os_file_close(log_bmp_sys->out.file); |
1046 | + log_bmp_sys->out.file = -1; |
1047 | + } |
1048 | + |
1049 | + for (i = 0; i < bitmap_files.count; i++) { |
1050 | + if (bitmap_files.files[i].seq_num == 0 |
1051 | + || bitmap_files.files[i].start_lsn >= lsn) { |
1052 | + break; |
1053 | + } |
1054 | + if (!os_file_delete_if_exists(bitmap_files.files[i].name)) { |
1055 | + os_file_get_last_error(TRUE); |
1056 | + result = TRUE; |
1057 | + break; |
1058 | + } |
1059 | + } |
1060 | + |
1061 | + if (srv_track_changed_pages) { |
1062 | + if (lsn > log_bmp_sys->end_lsn) { |
1063 | + if (lsn == IB_ULONGLONG_MAX) { |
1064 | + /* RESET restarts the sequence */ |
1065 | + log_bmp_sys->out_seq_num = 0; |
1066 | + log_online_rotate_bitmap_file(0); |
1067 | + } else { |
1068 | + /* PURGE continues the sequence */ |
1069 | + log_online_rotate_bitmap_file |
1070 | + (log_bmp_sys->end_lsn); |
1071 | + } |
1072 | + } |
1073 | + |
1074 | + mutex_exit(&log_bmp_sys->mutex); |
1075 | + } |
1076 | + |
1077 | + free(bitmap_files.files); |
1078 | + return result; |
1079 | +} |
Typo in the id - it's 16274.