Merge lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.1 into lp:percona-server/5.1

Proposed by Laurynas Biveinis
Status: Merged
Approved by: Alexey Kopytov
Approved revision: no longer in the source branch.
Merged at revision: 519
Proposed branch: lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.1
Merge into: lp:percona-server/5.1
Diff against target: 1079 lines (+612/-43)
23 files modified
Percona-Server/include/mysql_com.h (+4/-0)
Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc (+0/-11)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result (+76/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result (+3/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test (+3/-1)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test (+3/-1)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test (+200/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test (+7/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test (+3/-1)
Percona-Server/sql/handler.cc (+36/-0)
Percona-Server/sql/handler.h (+5/-0)
Percona-Server/sql/lex.h (+1/-0)
Percona-Server/sql/sql_parse.cc (+55/-3)
Percona-Server/sql/sql_yacc.yy (+13/-0)
Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc (+53/-0)
Percona-Server/storage/innodb_plugin/include/log0online.h (+19/-6)
Percona-Server/storage/innodb_plugin/include/sync0sync.h (+1/-0)
Percona-Server/storage/innodb_plugin/log/log0online.c (+125/-20)
To merge this branch: bzr merge lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.1
Reviewer Review Type Date Requested Status
Alexey Kopytov (community) Approve
Vlad Lesin Pending
Review via email: mp+144363@code.launchpad.net

This proposal supersedes a proposal from 2012-11-29.

Description of the change

Round 4: All review comments have been addressed.
http://jenkins.percona.com/job/percona-server-5.1-param/506/

Round 3: Found nasty typo myself while working on dependent branches.
- Fix that typo in log_online_purge_changed_page_bitmaps(), RESTART/PURGE tracking restart/continuation part.
- Replace all () function declarations with (void) to get compiler diagnostics when an arg is passed by mistake.
- Use compile_time_assert instead of ut_a where applicable.
http://jenkins.percona.com/job/percona-server-5.1-param/470/

Round 2: All Vlad's review comments have been addressed.
http://jenkins.percona.com/job/percona-server-5.1-param/466/

Implementation of
https://blueprints.launchpad.net/percona-server/+spec/xtradb-bitmap-user-requests

Jenkins: http://jenkins.percona.com/job/percona-server-5.1-param/462/

To post a comment you must log in.
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Typo in the id - it's 16274.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

Looks good. I have the following remarks:

1) Lines 350-365: Why only "RESET" request is tested here? Would it be reasonable to test the requests when innodb engine is disabled?
2) Line 512 - unnecessary change;
3) Line 686 - wrong comments format.

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

It would be good if blueprint would mention about permissions for the requests.

review: Needs Fixing
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Vlad -

thanks for the review.

> 1) Lines 350-365: Why only "RESET" request is tested here? Would it be
> reasonable to test the requests when innodb engine is disabled?

Other requests are tested with the log tracking disabled too: see lines 304--339. I will consider testing with disabled InnoDB.

> 2) Line 512 - unnecessary change;

Fixed.

> 3) Line 686 - wrong comments format.

Fixed.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

> It would be good if blueprint would mention about permissions for the
> requests.

Added. This is a good idea, also I found that RESET request is allowed with FLUSH (or is it REFRESH?) privilege, not SUPER, which is a bug.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

Looks good for me.

review: Approve
Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

Laurynas,

Some mixed comments and questions in addition to the new reserved words
problem which you are already aware of:

   - do we really want the ALL clause in FLUSH/RESET/PURGE? It is of
     course nice when SQL is close to the natural language. But in this
     case it just leads to confusion:
       - Does FLUSH ALL ... BITMAPS mean I can flush only some bitmap
         files?
       - Why do other FLUSH statements not require the ALL clause? (i.e. there's no
        "FLUSH ALL HOSTS").
       - it is even more confusing in case of PURGE ALL ... BITMAPS
         BEFORE. ALL in SQL is usually the opposite to "some", but in
         this case we want to purge some files.
   - wrong SQL in error messages and comments ("PURGE INNODB CHANGED
     PAGE BITMAPS BEFORE")
   - it is not obvious from the BP that PURGE ... BITMAPS TO is not
     implemented
   - no error message if RESET ... BITMAPS fails?
   - we should make sure we use FLUSH NO_WRITE_TO_BINLOG ... BITMAPS in
     innobackupex
   - new handlerton "methods" flush_changed_page_bitmaps() and
     purge_changed_page_bitmaps() return int, but in their _handlerton()
     counterparts that value is cast to my_bool. Does not present any
     problem with the current code, but I'd either wrap their return
     values to test() or change the handlerton "methods" return type
     from int to my_bool.
   - what's the point of resetting srv_checkpoint_completed_event in
     innobase_flush_changed_page_bitmaps()?

review: Needs Fixing
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Alexey -

Thanks for your review.

> - do we really want the ALL clause in FLUSH/RESET/PURGE? It is of
> course nice when SQL is close to the natural language. But in this
> case it just leads to confusion:
> - Does FLUSH ALL ... BITMAPS mean I can flush only some bitmap
> files?
> - Why do other FLUSH statements not require the ALL clause? (i.e.
> there's no
> "FLUSH ALL HOSTS").
> - it is even more confusing in case of PURGE ALL ... BITMAPS
> BEFORE. ALL in SQL is usually the opposite to "some", but in
> this case we want to purge some files.

IIRC the way "ALL" is used in the parser which comes closest to this usage, especially in 5.5, is that "ALL" would be a wildcard for storage engines that support this request, i.e. "FLUSH INNODB CHANGED PAGE BITMAPS" invokes the InnoDB handler, and "FLUSH ALL ... " invokes all handlers. Of course, the former syntax is not implemented ATM, nor there are current plans to.

Does it makes sense to keep "ALL" as a storage engine wildcard here?

> - wrong SQL in error messages and comments ("PURGE INNODB CHANGED
> PAGE BITMAPS BEFORE")

Ack, will fix according to how the "ALL" discussion goes.

> - it is not obvious from the BP that PURGE ... BITMAPS TO is not
> implemented

Ack, split off a new BP and removed this request from the current one.

> - no error message if RESET ... BITMAPS fails?

It goes through the common RESET error path handling. But I have to check what kind of error will that produce.

> - we should make sure we use FLUSH NO_WRITE_TO_BINLOG ... BITMAPS in
> innobackupex

Yes. In fact, this is *the* reason, why there is no XtraBackup MP yet. Thanks about NO_WRITE_TO_BINLOG.

> - new handlerton "methods" flush_changed_page_bitmaps() and
> purge_changed_page_bitmaps() return int, but in their _handlerton()
> counterparts that value is cast to my_bool. Does not present any
> problem with the current code, but I'd either wrap their return
> values to test() or change the handlerton "methods" return type
> from int to my_bool.

Ack.

> - what's the point of resetting srv_checkpoint_completed_event in
> innobase_flush_changed_page_bitmaps()?

Redundant it is.

Revision history for this message
Alexey Kopytov (akopytov) wrote : Posted in a previous version of this proposal

Hi Laurynas,

On Mon, 21 Jan 2013 07:32:21 -0000, Laurynas Biveinis wrote:
> Alexey -
>
> Thanks for your review.
>
>> - do we really want the ALL clause in FLUSH/RESET/PURGE? It is of
>> course nice when SQL is close to the natural language. But in this
>> case it just leads to confusion:
>> - Does FLUSH ALL ... BITMAPS mean I can flush only some bitmap
>> files?
>> - Why do other FLUSH statements not require the ALL clause? (i.e.
>> there's no
>> "FLUSH ALL HOSTS").
>> - it is even more confusing in case of PURGE ALL ... BITMAPS
>> BEFORE. ALL in SQL is usually the opposite to "some", but in
>> this case we want to purge some files.
>
> IIRC the way "ALL" is used in the parser which comes closest to this usage, especially in 5.5, is that "ALL" would be a wildcard for storage engines that support this request, i.e. "FLUSH INNODB CHANGED PAGE BITMAPS" invokes the InnoDB handler, and "FLUSH ALL ... " invokes all handlers. Of course, the former syntax is not implemented ATM, nor there are current plans to.
>
> Does it makes sense to keep "ALL" as a storage engine wildcard here?
>

That would again be inconsistent with "FLUSH TABLES" and "FLUSH ENGINE
LOGS".

And speaking from the realistic perspective, what other storage engines
could possibly have this support implemented in the next few years?

It's all about keeping consistency and avoiding syntax redundancy.
"CHANGED PAGE BITMAP" is also redundant, and introducing 2 new reserved
words for this specific syntax doesn't look good either. Following the
same logic, it should be "FLUSH DES KEY FILE", rather than "FLUSH
DES_KEY_FILE", or "FLUSH USER RESOURCES" (or "FLUSH ALL USER
RESOURCES"), rather than "FLUSH USER_RESOURCES".

I would implement "FLUSH CHANGED_PAGE_BITMAPS", which doesn't have any
of the shortcomings being discussed.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

> I would implement "FLUSH CHANGED_PAGE_BITMAPS", which doesn't have any
> of the shortcomings being discussed.

Agreed, this is how I will proceed.

Revision history for this message
Alexey Kopytov (akopytov) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Percona-Server/include/mysql_com.h'
2--- Percona-Server/include/mysql_com.h 2011-11-24 02:01:17 +0000
3+++ Percona-Server/include/mysql_com.h 2013-01-22 18:26:24 +0000
4@@ -133,6 +133,10 @@
5 #define REFRESH_DES_KEY_FILE 0x40000L
6 #define REFRESH_USER_RESOURCES 0x80000L
7 #define REFRESH_QUERY_RESPONSE_TIME 0x100000L /* response time distibution */
8+#define REFRESH_FLUSH_PAGE_BITMAPS 0x200000L
9+#define REFRESH_RESET_PAGE_BITMAPS 0x400000L
10+
11+#define PURGE_BITMAPS_TO_LSN 1
12
13 #define CLIENT_LONG_PASSWORD 1 /* new more secure passwords */
14 #define CLIENT_FOUND_ROWS 2 /* Found instead of affected rows */
15
16=== removed file 'Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc'
17--- Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc 2012-11-25 09:29:43 +0000
18+++ Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc 1970-01-01 00:00:00 +0000
19@@ -1,11 +0,0 @@
20-# Remove all the InnoDB bitmap files. A temporary measure until RESET user request is implemented
21-
22-let $MYSQLD_DATADIR= `select @@datadir`;
23-
24---exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
25---shutdown_server 10
26---source include/wait_until_disconnected.inc
27-remove_files_wildcard $MYSQLD_DATADIR ib_modified_log*;
28---enable_reconnect
29---exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
30---source include/wait_until_connected_again.inc
31
32=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result'
33--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result 2012-11-25 09:29:43 +0000
34+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp.result 2013-01-22 18:26:24 +0000
35@@ -1,3 +1,4 @@
36+RESET CHANGED_PAGE_BITMAPS;
37 DROP TABLE IF EXISTS t1, t2;
38 CREATE TABLE t1 (x INT) ENGINE=InnoDB;
39 INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
40
41=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result'
42--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result 2012-11-25 09:29:43 +0000
43+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_no_restart.result 2013-01-22 18:26:24 +0000
44@@ -1,3 +1,4 @@
45+RESET CHANGED_PAGE_BITMAPS;
46 DROP TABLE IF EXISTS t1, t2;
47 CREATE TABLE t1 (x INT) ENGINE=InnoDB;
48 INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
49
50=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result'
51--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result 1970-01-01 00:00:00 +0000
52+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests.result 2013-01-22 18:26:24 +0000
53@@ -0,0 +1,76 @@
54+RESET CHANGED_PAGE_BITMAPS;
55+DROP TABLE IF EXISTS t1;
56+DELETE FROM mysql.user WHERE USER='mysqltest_1';
57+FLUSH PRIVILEGES;
58+CREATE TABLE t1 (x INT) ENGINE=InnoDB;
59+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
60+FLUSH CHANGED_PAGE_BITMAPS;
61+Before RESET:
62+ib_modified_log_1
63+ib_modified_log_2
64+RESET CHANGED_PAGE_BITMAPS;
65+After RESET:
66+ib_modified_log_1
67+RESET CHANGED_PAGE_BITMAPS;
68+After 2nd RESET
69+ib_modified_log_1
70+After RESETs and restart:
71+ib_modified_log_1
72+ib_modified_log_2
73+PURGE CHANGED_PAGE_BITMAPS BEFORE 1;
74+After PURGE ... BEFORE 1:
75+ib_modified_log_2
76+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
77+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
78+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
79+After PURGE ... BEFORE 100000000 and restart:
80+ib_modified_log_4
81+ib_modified_log_5
82+PURGE CHANGED_PAGE_BITMAPS BEFORE 5+5;
83+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '+5' at line 1
84+PURGE CHANGED_PAGE_BITMAPS BEFORE -12;
85+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-12' at line 1
86+PURGE CHANGED_PAGE_BITMAPS BEFORE 1.2e-2;
87+ERROR 42000: Only integers allowed as number here near '1.2e-2' at line 1
88+PURGE CHANGED_PAGE_BITMAPS BEFORE t1;
89+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 't1' at line 1
90+PURGE CHANGED_PAGE_BITMAPS BEFORE "t1";
91+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"t1"' at line 1
92+PURGE CHANGED_PAGE_BITMAPS BEFORE TRUE;
93+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TRUE' at line 1
94+PURGE CHANGED_PAGE_BITMAPS BEFORE NULL;
95+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'NULL' at line 1
96+CREATE USER mysqltest_1@localhost;
97+GRANT ALL ON *.* TO mysqltest_1@localhost;
98+REVOKE SUPER ON *.* FROM mysqltest_1@localhost;
99+FLUSH CHANGED_PAGE_BITMAPS;
100+ERROR 42000: Access denied; you need the SUPER privilege for this operation
101+RESET CHANGED_PAGE_BITMAPS;
102+ERROR 42000: Access denied; you need the SUPER privilege for this operation
103+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
104+ERROR 42000: Access denied; you need the SUPER privilege for this operation
105+DROP USER mysqltest_1@localhost;
106+SELECT @@GLOBAL.innodb_track_changed_pages;
107+@@GLOBAL.innodb_track_changed_pages
108+0
109+FLUSH CHANGED_PAGE_BITMAPS;
110+Before the PURGE with tracking disabled
111+ib_modified_log_4
112+ib_modified_log_5
113+PURGE CHANGED_PAGE_BITMAPS BEFORE 1;
114+After the PURGE that deletes nothing:
115+ib_modified_log_4
116+ib_modified_log_5
117+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
118+After the PURGE that deletes everything:
119+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
120+After the repeated PURGE:
121+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
122+Before the RESET while tracking disabled:
123+ib_modified_log_1
124+ib_modified_log_2
125+RESET CHANGED_PAGE_BITMAPS;
126+After the RESET with tracking disabled:
127+DROP TABLE t1;
128+CREATE TABLE CHANGED_PAGE_BITMAPS (a INT);
129+DROP TABLE CHANGED_PAGE_BITMAPS;
130
131=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result'
132--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result 1970-01-01 00:00:00 +0000
133+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_page_bmp_requests_no_innodb.result 2013-01-22 18:26:24 +0000
134@@ -0,0 +1,3 @@
135+FLUSH CHANGED_PAGE_BITMAPS;
136+RESET CHANGED_PAGE_BITMAPS;
137+PURGE CHANGED_PAGE_BITMAPS BEFORE 1;
138
139=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result'
140--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 2012-11-25 09:29:43 +0000
141+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 2013-01-22 18:26:24 +0000
142@@ -1,3 +1,4 @@
143+RESET CHANGED_PAGE_BITMAPS;
144 DROP TABLE IF EXISTS T1;
145 CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB;
146 1st interval end LSN greater than interval start LSN:
147
148=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test'
149--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test 2012-11-25 09:29:43 +0000
150+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp.test 2013-01-22 18:26:24 +0000
151@@ -13,7 +13,9 @@
152 # Valgrind reports useless errors on very fast server shutdowns
153 --source include/not_valgrind.inc
154
155---source include/delete_innodb_bitmaps.inc
156+# Delete any existing bitmaps
157+--source include/restart_mysqld.inc
158+RESET CHANGED_PAGE_BITMAPS;
159
160 --disable_warnings
161 DROP TABLE IF EXISTS t1, t2;
162
163=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test'
164--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test 2012-11-25 09:29:43 +0000
165+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_no_restart.test 2013-01-22 18:26:24 +0000
166@@ -2,7 +2,9 @@
167 # percona_changed_page_bmp test, which contains the full functional tests.
168 --source include/have_innodb_plugin.inc
169
170---source include/delete_innodb_bitmaps.inc
171+# Delete any existing bitmaps
172+--source include/restart_mysqld.inc
173+RESET CHANGED_PAGE_BITMAPS;
174
175 --disable_warnings
176 DROP TABLE IF EXISTS t1, t2;
177
178=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt'
179--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt 1970-01-01 00:00:00 +0000
180+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests-master.opt 2013-01-22 18:26:24 +0000
181@@ -0,0 +1,1 @@
182+--innodb_track_changed_pages=TRUE
183
184=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test'
185--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test 1970-01-01 00:00:00 +0000
186+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests.test 2013-01-22 18:26:24 +0000
187@@ -0,0 +1,200 @@
188+#
189+# Tests for changed page tracking bitmaps: user requests.
190+#
191+--source include/have_innodb_plugin.inc
192+
193+# Delete any existing bitmaps
194+--source include/restart_mysqld.inc
195+RESET CHANGED_PAGE_BITMAPS;
196+
197+--source include/count_sessions.inc
198+
199+let $MYSQLD_DATADIR= `select @@datadir`;
200+
201+--disable_warnings
202+DROP TABLE IF EXISTS t1;
203+DELETE FROM mysql.user WHERE USER='mysqltest_1';
204+FLUSH PRIVILEGES;
205+--enable_warnings
206+
207+CREATE TABLE t1 (x INT) ENGINE=InnoDB;
208+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
209+
210+#
211+# Test FLUSH CHANGED_PAGE_BITMAPS. Not much to test due to checkpointing non-determinism,
212+# just see that it does not crash or return an error
213+#
214+FLUSH CHANGED_PAGE_BITMAPS;
215+
216+#
217+# Test that RESET CHANGED_PAGE_BITMAPS works
218+#
219+
220+# Have at least two bitmap files
221+--source include/restart_mysqld.inc
222+
223+--echo Before RESET:
224+--replace_regex /_[[:digit:]]+\.xdb$//
225+list_files $MYSQLD_DATADIR ib_modified_log*;
226+
227+RESET CHANGED_PAGE_BITMAPS;
228+
229+--echo After RESET:
230+--replace_regex /_[[:digit:]]+\.xdb$//
231+list_files $MYSQLD_DATADIR ib_modified_log*;
232+
233+# Test consecutive RESET
234+RESET CHANGED_PAGE_BITMAPS;
235+--echo After 2nd RESET
236+--replace_regex /_[[:digit:]]+\.xdb$//
237+list_files $MYSQLD_DATADIR ib_modified_log*;
238+
239+# Check that the file sequence after RESET starts with 1 again
240+--source include/restart_mysqld.inc
241+
242+--echo After RESETs and restart:
243+--replace_regex /_[[:digit:]]+\.xdb$//
244+list_files $MYSQLD_DATADIR ib_modified_log*;
245+
246+#
247+# Test that PURGE CHANGED_PAGE_BITMAPS BEFORE works. We don't test partial PURGE
248+# because it's hard to extract good LSN value for that.
249+#
250+
251+# PURGE that deletes nothing
252+PURGE CHANGED_PAGE_BITMAPS BEFORE 1;
253+--echo After PURGE ... BEFORE 1:
254+--replace_regex /_[[:digit:]]+\.xdb$//
255+list_files $MYSQLD_DATADIR ib_modified_log*;
256+
257+# PURGE that deletes everything
258+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
259+# Test consecutive PURGE
260+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
261+
262+# Check that file sequence continues
263+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
264+--source include/restart_mysqld.inc
265+--echo After PURGE ... BEFORE 100000000 and restart:
266+--replace_regex /_[[:digit:]]+\.xdb$//
267+list_files $MYSQLD_DATADIR ib_modified_log*;
268+
269+#
270+# Test malformed PURGE ... BEFORE
271+#
272+--error ER_PARSE_ERROR
273+PURGE CHANGED_PAGE_BITMAPS BEFORE 5+5;
274+
275+--error ER_PARSE_ERROR
276+PURGE CHANGED_PAGE_BITMAPS BEFORE -12;
277+
278+--error ER_PARSE_ERROR
279+PURGE CHANGED_PAGE_BITMAPS BEFORE 1.2e-2;
280+
281+--error ER_PARSE_ERROR
282+PURGE CHANGED_PAGE_BITMAPS BEFORE t1;
283+
284+--error ER_PARSE_ERROR
285+PURGE CHANGED_PAGE_BITMAPS BEFORE "t1";
286+
287+--error ER_PARSE_ERROR
288+PURGE CHANGED_PAGE_BITMAPS BEFORE TRUE;
289+
290+--error ER_PARSE_ERROR
291+PURGE CHANGED_PAGE_BITMAPS BEFORE NULL;
292+
293+#
294+# Test that users without SUPER privilege cannot issue user requests
295+#
296+CREATE USER mysqltest_1@localhost;
297+GRANT ALL ON *.* TO mysqltest_1@localhost;
298+REVOKE SUPER ON *.* FROM mysqltest_1@localhost;
299+
300+connect (conn1,localhost,mysqltest_1,,);
301+
302+--error ER_SPECIFIC_ACCESS_DENIED_ERROR
303+FLUSH CHANGED_PAGE_BITMAPS;
304+
305+--error ER_SPECIFIC_ACCESS_DENIED_ERROR
306+RESET CHANGED_PAGE_BITMAPS;
307+
308+--error ER_SPECIFIC_ACCESS_DENIED_ERROR
309+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
310+
311+connection default;
312+disconnect conn1;
313+
314+--source include/wait_until_count_sessions.inc
315+
316+DROP USER mysqltest_1@localhost;
317+
318+#
319+# Test FLUSH and PURGE requests with log tracking disabled
320+#
321+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
322+--shutdown_server 10
323+--source include/wait_until_disconnected.inc
324+--enable_reconnect
325+--exec echo "restart:--innodb_track_changed_pages=FALSE" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
326+--source include/wait_until_connected_again.inc
327+
328+SELECT @@GLOBAL.innodb_track_changed_pages;
329+
330+FLUSH CHANGED_PAGE_BITMAPS;
331+
332+# The previous bitmap files should be present
333+--echo Before the PURGE with tracking disabled
334+--replace_regex /_[[:digit:]]+\.xdb$//
335+list_files $MYSQLD_DATADIR ib_modified_log*;
336+
337+PURGE CHANGED_PAGE_BITMAPS BEFORE 1;
338+
339+--echo After the PURGE that deletes nothing:
340+--replace_regex /_[[:digit:]]+\.xdb$//
341+list_files $MYSQLD_DATADIR ib_modified_log*;
342+
343+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
344+
345+--echo After the PURGE that deletes everything:
346+--replace_regex /_[[:digit:]]+\.xdb$//
347+list_files $MYSQLD_DATADIR ib_modified_log*;
348+
349+PURGE CHANGED_PAGE_BITMAPS BEFORE 100000000;
350+
351+--echo After the repeated PURGE:
352+--replace_regex /_[[:digit:]]+\.xdb$//
353+list_files $MYSQLD_DATADIR ib_modified_log*;
354+
355+#
356+# Test RESET request with log tracking disabled
357+#
358+
359+# Generate some bitmap data again
360+--source include/restart_mysqld.inc
361+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
362+--source include/restart_mysqld.inc
363+
364+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
365+--shutdown_server 10
366+--source include/wait_until_disconnected.inc
367+--enable_reconnect
368+--exec echo "restart:--innodb_track_changed_pages=FALSE" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
369+--source include/wait_until_connected_again.inc
370+
371+--echo Before the RESET while tracking disabled:
372+--replace_regex /_[[:digit:]]+\.xdb$//
373+list_files $MYSQLD_DATADIR ib_modified_log*;
374+
375+RESET CHANGED_PAGE_BITMAPS;
376+
377+--echo After the RESET with tracking disabled:
378+--replace_regex /_[[:digit:]]+\.xdb$//
379+list_files $MYSQLD_DATADIR ib_modified_log*;
380+
381+DROP TABLE t1;
382+
383+#
384+# Test that the new keyword does not preclude its use as identifier
385+#
386+CREATE TABLE CHANGED_PAGE_BITMAPS (a INT);
387+DROP TABLE CHANGED_PAGE_BITMAPS;
388
389=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt'
390--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt 1970-01-01 00:00:00 +0000
391+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb-master.opt 2013-01-22 18:26:24 +0000
392@@ -0,0 +1,1 @@
393+--skip-innodb
394
395=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test'
396--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test 1970-01-01 00:00:00 +0000
397+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_page_bmp_requests_no_innodb.test 2013-01-22 18:26:24 +0000
398@@ -0,0 +1,7 @@
399+#
400+# Test that changed page tracking user requests are parsed without InnoDB
401+#
402+FLUSH CHANGED_PAGE_BITMAPS;
403+RESET CHANGED_PAGE_BITMAPS;
404+PURGE CHANGED_PAGE_BITMAPS BEFORE 1;
405+
406
407=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test'
408--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 2012-11-25 09:29:43 +0000
409+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 2013-01-22 18:26:24 +0000
410@@ -4,7 +4,9 @@
411
412 --source include/have_innodb_plugin.inc
413
414---source include/delete_innodb_bitmaps.inc
415+# Delete any existing bitmaps
416+--source include/restart_mysqld.inc
417+RESET CHANGED_PAGE_BITMAPS;
418
419 --disable_warnings
420 DROP TABLE IF EXISTS T1;
421
422=== modified file 'Percona-Server/sql/handler.cc'
423--- Percona-Server/sql/handler.cc 2012-10-16 04:49:59 +0000
424+++ Percona-Server/sql/handler.cc 2013-01-22 18:26:24 +0000
425@@ -4636,6 +4636,42 @@
426 return result;
427 }
428
429+static my_bool flush_changed_page_bitmaps_handlerton(THD *unused1,
430+ plugin_ref plugin,
431+ void *unused2)
432+{
433+ handlerton *hton= plugin_data(plugin, handlerton *);
434+
435+ if (hton->flush_changed_page_bitmaps == NULL)
436+ return FALSE;
437+
438+ return hton->flush_changed_page_bitmaps();
439+}
440+
441+bool ha_flush_changed_page_bitmaps()
442+{
443+ return plugin_foreach(NULL, flush_changed_page_bitmaps_handlerton,
444+ MYSQL_STORAGE_ENGINE_PLUGIN, NULL);
445+}
446+
447+static my_bool purge_changed_page_bitmaps_handlerton(THD *unused1,
448+ plugin_ref plugin,
449+ void *lsn)
450+{
451+ handlerton *hton= plugin_data(plugin, handlerton *);
452+
453+ if (hton->purge_changed_page_bitmaps == NULL)
454+ return FALSE;
455+
456+ return hton->purge_changed_page_bitmaps(*(ulonglong *)lsn);
457+}
458+
459+bool ha_purge_changed_page_bitmaps(ulonglong lsn)
460+{
461+ return plugin_foreach(NULL, purge_changed_page_bitmaps_handlerton,
462+ MYSQL_STORAGE_ENGINE_PLUGIN, &lsn);
463+}
464+
465 /*
466 Function to check if the conditions for row-based binlogging is
467 correct for the table.
468
469=== modified file 'Percona-Server/sql/handler.h'
470--- Percona-Server/sql/handler.h 2012-01-17 14:05:16 +0000
471+++ Percona-Server/sql/handler.h 2013-01-22 18:26:24 +0000
472@@ -693,6 +693,8 @@
473 int (*fill_files_table)(handlerton *hton, THD *thd,
474 TABLE_LIST *tables,
475 class Item *cond);
476+ my_bool (*flush_changed_page_bitmaps)(void);
477+ my_bool (*purge_changed_page_bitmaps)(ulonglong lsn);
478 uint32 flags; /* global handler flags */
479 /*
480 Those handlerton functions below are properly initialized at handler
481@@ -2074,6 +2076,9 @@
482 int ha_prepare(THD *thd);
483 int ha_recover(HASH *commit_list);
484
485+bool ha_flush_changed_page_bitmaps();
486+bool ha_purge_changed_page_bitmaps(ulonglong lsn);
487+
488 /* transactions: these functions never call handlerton functions directly */
489 int ha_commit_trans(THD *thd, bool all);
490 int ha_autocommit_or_rollback(THD *thd, int error);
491
492=== modified file 'Percona-Server/sql/lex.h'
493--- Percona-Server/sql/lex.h 2011-11-24 02:01:23 +0000
494+++ Percona-Server/sql/lex.h 2013-01-22 18:26:24 +0000
495@@ -102,6 +102,7 @@
496 { "CHAIN", SYM(CHAIN_SYM)},
497 { "CHANGE", SYM(CHANGE)},
498 { "CHANGED", SYM(CHANGED)},
499+ { "CHANGED_PAGE_BITMAPS", SYM(CHANGED_PAGE_BITMAPS_SYM)},
500 { "CHAR", SYM(CHAR_SYM)},
501 { "CHARACTER", SYM(CHAR_SYM)},
502 { "CHARSET", SYM(CHARSET)},
503
504=== modified file 'Percona-Server/sql/sql_parse.cc'
505--- Percona-Server/sql/sql_parse.cc 2012-08-20 03:14:02 +0000
506+++ Percona-Server/sql/sql_parse.cc 2013-01-22 18:26:24 +0000
507@@ -2487,9 +2487,35 @@
508 {
509 if (check_global_access(thd, SUPER_ACL))
510 goto error;
511- /* PURGE MASTER LOGS TO 'file' */
512- res = purge_master_logs(thd, lex->to_log);
513- break;
514+ if (lex->type == 0)
515+ {
516+ /* PURGE MASTER LOGS TO 'file' */
517+ res = purge_master_logs(thd, lex->to_log);
518+ break;
519+ }
520+ if (lex->type == PURGE_BITMAPS_TO_LSN)
521+ {
522+ /* PURGE CHANGED_PAGE_BITMAPS BEFORE lsn */
523+ ulonglong lsn= 0;
524+ Item *it= (Item *)lex->value_list.head();
525+ if ((!it->fixed && it->fix_fields(lex->thd, &it)) || it->check_cols(1)
526+ || it->null_value)
527+ {
528+ my_error(ER_WRONG_ARGUMENTS, MYF(0),
529+ "PURGE CHANGED_PAGE_BITMAPS BEFORE");
530+ goto error;
531+ }
532+ lsn= it->val_uint();
533+ res= ha_purge_changed_page_bitmaps(lsn);
534+ if (res)
535+ {
536+ my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0),
537+ "PURGE CHANGED_PAGE_BITMAPS BEFORE");
538+ goto error;
539+ }
540+ my_ok(thd);
541+ break;
542+ }
543 }
544 case SQLCOM_PURGE_BEFORE:
545 {
546@@ -7410,6 +7436,32 @@
547 }
548 pthread_mutex_unlock(&LOCK_global_user_client_stats);
549 }
550+ if (options & REFRESH_FLUSH_PAGE_BITMAPS)
551+ {
552+ if (check_global_access(thd, SUPER_ACL))
553+ result= 1;
554+ else
555+ {
556+ result= ha_flush_changed_page_bitmaps();
557+ if (result)
558+ {
559+ my_error(ER_UNKNOWN_ERROR, MYF(0), "FLUSH CHANGED_PAGE_BITMAPS");
560+ }
561+ }
562+ }
563+ if (options & REFRESH_RESET_PAGE_BITMAPS)
564+ {
565+ if (check_global_access(thd, SUPER_ACL))
566+ result= 1;
567+ else
568+ {
569+ result= ha_purge_changed_page_bitmaps(0);
570+ if (result)
571+ {
572+ my_error(ER_UNKNOWN_ERROR, MYF(0), "RESET CHANGED_PAGE_BITMAPS");
573+ }
574+ }
575+ }
576 if (*write_to_binlog != -1)
577 *write_to_binlog= tmp_write_to_binlog;
578 /*
579
580=== modified file 'Percona-Server/sql/sql_yacc.yy'
581--- Percona-Server/sql/sql_yacc.yy 2012-08-20 03:14:02 +0000
582+++ Percona-Server/sql/sql_yacc.yy 2013-01-22 18:26:24 +0000
583@@ -753,6 +753,7 @@
584 %token CHAIN_SYM /* SQL-2003-N */
585 %token CHANGE
586 %token CHANGED
587+%token CHANGED_PAGE_BITMAPS_SYM /* MYSQL */
588 %token CHARSET
589 %token CHAR_SYM /* SQL-2003-R */
590 %token CHECKSUM_SYM
591@@ -10691,6 +10692,8 @@
592 { Lex->type|= REFRESH_DES_KEY_FILE; }
593 | RESOURCES
594 { Lex->type|= REFRESH_USER_RESOURCES; }
595+ | CHANGED_PAGE_BITMAPS_SYM
596+ { Lex->type|= REFRESH_FLUSH_PAGE_BITMAPS; }
597 ;
598
599 opt_table_list:
600@@ -10717,6 +10720,8 @@
601 SLAVE { Lex->type|= REFRESH_SLAVE; }
602 | MASTER_SYM { Lex->type|= REFRESH_MASTER; }
603 | QUERY_SYM CACHE_SYM { Lex->type|= REFRESH_QUERY_CACHE;}
604+ | CHANGED_PAGE_BITMAPS_SYM
605+ { Lex->type |= REFRESH_RESET_PAGE_BITMAPS; }
606 ;
607
608 purge:
609@@ -10732,6 +10737,13 @@
610
611 purge_options:
612 master_or_binary LOGS_SYM purge_option
613+ | CHANGED_PAGE_BITMAPS_SYM BEFORE_SYM real_ulonglong_num
614+ {
615+ LEX *lex= Lex;
616+ lex->value_list.empty();
617+ lex->value_list.push_front(new Item_uint($3));
618+ lex->type= PURGE_BITMAPS_TO_LSN;
619+ }
620 ;
621
622 purge_option:
623@@ -11807,6 +11819,7 @@
624 | CASCADED {}
625 | CHAIN_SYM {}
626 | CHANGED {}
627+ | CHANGED_PAGE_BITMAPS_SYM {}
628 | CIPHER_SYM {}
629 | CLIENT_STATS_SYM {}
630 | CLIENT_SYM {}
631
632=== modified file 'Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc'
633--- Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2013-01-18 03:33:06 +0000
634+++ Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2013-01-22 18:26:24 +0000
635@@ -79,6 +79,7 @@
636 #include "row0sel.h"
637 #include "row0upd.h"
638 #include "log0log.h"
639+#include "log0online.h"
640 #include "lock0lock.h"
641 #include "dict0crea.h"
642 #include "btr0cur.h"
643@@ -315,6 +316,25 @@
644 innobase_alter_table_flags(
645 /*=======================*/
646 uint flags);
647+/************************************************************//**
648+Synchronously read and parse the redo log up to the last
649+checkpoint to write the changed page bitmap.
650+@return 0 to indicate success. Current implementation cannot fail. */
651+static
652+my_bool
653+innobase_flush_changed_page_bitmaps();
654+/*==================================*/
655+/************************************************************//**
656+Delete all the bitmap files for data less than the specified LSN.
657+If called with lsn == 0 (i.e. set by RESET request) or
658+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
659+continue it.
660+@return 0 to indicate success, 1 for failure. */
661+static
662+my_bool
663+innobase_purge_changed_page_bitmaps(
664+/*================================*/
665+ ulonglong lsn); /*!< in: LSN to purge files up to */
666
667 static const char innobase_hton_name[]= "InnoDB";
668
669@@ -2264,6 +2284,10 @@
670 innobase_hton->flags=HTON_NO_FLAGS;
671 innobase_hton->release_temporary_latches=innobase_release_temporary_latches;
672 innobase_hton->alter_table_flags = innobase_alter_table_flags;
673+ innobase_hton->flush_changed_page_bitmaps
674+ = innobase_flush_changed_page_bitmaps;
675+ innobase_hton->purge_changed_page_bitmaps
676+ = innobase_purge_changed_page_bitmaps;
677
678 ut_a(DATA_MYSQL_TRUE_VARCHAR == (ulint)MYSQL_TYPE_VARCHAR);
679
680@@ -2845,6 +2869,35 @@
681 | HA_ONLINE_ADD_PK_INDEX_NO_WRITES);
682 }
683
684+/************************************************************//**
685+Synchronously read and parse the redo log up to the last
686+checkpoint to write the changed page bitmap.
687+@return 0 to indicate success. Current implementation cannot fail. */
688+static
689+my_bool
690+innobase_flush_changed_page_bitmaps()
691+/*=================================*/
692+{
693+ if (srv_track_changed_pages) {
694+ log_online_follow_redo_log();
695+ }
696+ return FALSE;
697+}
698+
699+/************************************************************//**
700+Delete all the bitmap files for data less than the specified LSN.
701+If called with lsn == IB_ULONGLONG_MAX (i.e. set by RESET request),
702+restart the bitmap file sequence, otherwise continue it.
703+@return 0 to indicate success, 1 for failure. */
704+static
705+my_bool
706+innobase_purge_changed_page_bitmaps(
707+/*================================*/
708+ ulonglong lsn) /*!< in: LSN to purge files up to */
709+{
710+ return (my_bool)log_online_purge_changed_page_bitmaps(lsn);
711+}
712+
713 /****************************************************************//**
714 Copy the current replication position from MySQL to a transaction. */
715 static
716
717=== modified file 'Percona-Server/storage/innodb_plugin/include/log0online.h'
718--- Percona-Server/storage/innodb_plugin/include/log0online.h 2012-11-25 09:29:43 +0000
719+++ Percona-Server/storage/innodb_plugin/include/log0online.h 2013-01-22 18:26:24 +0000
720@@ -41,23 +41,36 @@
721 Initializes the online log following subsytem. */
722 UNIV_INTERN
723 void
724-log_online_read_init();
725-/*===================*/
726+log_online_read_init(void);
727+/*=======================*/
728
729 /*********************************************************************//**
730 Shuts down the online log following subsystem. */
731 UNIV_INTERN
732 void
733-log_online_read_shutdown();
734-/*=======================*/
735+log_online_read_shutdown(void);
736+/*===========================*/
737
738 /*********************************************************************//**
739 Reads and parses the redo log up to last checkpoint LSN to build the changed
740 page bitmap which is then written to disk. */
741 UNIV_INTERN
742 void
743-log_online_follow_redo_log();
744-/*=========================*/
745+log_online_follow_redo_log(void);
746+/*=============================*/
747+
748+/************************************************************//**
749+Delete all the bitmap files for data less than the specified LSN.
750+If called with lsn == 0 (i.e. set by RESET request) or
751+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
752+continue it.
753+
754+@return FALSE to indicate success, TRUE for failure. */
755+UNIV_INTERN
756+ibool
757+log_online_purge_changed_page_bitmaps(
758+/*==================================*/
759+ ib_uint64_t lsn); /*!<in: LSN to purge files up to */
760
761 #define LOG_BITMAP_ITERATOR_START_LSN(i) \
762 ((i).start_lsn)
763
764=== modified file 'Percona-Server/storage/innodb_plugin/include/sync0sync.h'
765--- Percona-Server/storage/innodb_plugin/include/sync0sync.h 2011-11-24 16:33:30 +0000
766+++ Percona-Server/storage/innodb_plugin/include/sync0sync.h 2013-01-22 18:26:24 +0000
767@@ -478,6 +478,7 @@
768 #define SYNC_REC_LOCK 299
769 #define SYNC_TRX_LOCK_HEAP 298
770 #define SYNC_TRX_SYS_HEADER 290
771+#define SYNC_LOG_ONLINE 175
772 #define SYNC_LOG 170
773 #define SYNC_RECV 168
774 #define SYNC_WORK_QUEUE 162
775
776=== modified file 'Percona-Server/storage/innodb_plugin/log/log0online.c'
777--- Percona-Server/storage/innodb_plugin/log/log0online.c 2013-01-04 07:28:01 +0000
778+++ Percona-Server/storage/innodb_plugin/log/log0online.c 2013-01-22 18:26:24 +0000
779@@ -69,6 +69,7 @@
780 both the correct type and the tree does
781 not mind its overwrite during
782 rbt_next() tree traversal. */
783+ mutex_t mutex; /*!< mutex protecting all the fields.*/
784 };
785
786 /* The log parsing and bitmap output struct instance */
787@@ -172,6 +173,8 @@
788 byte search_page[MODIFIED_PAGE_BLOCK_SIZE];
789 byte *page_ptr;
790
791+ ut_ad(mutex_own(&log_bmp_sys->mutex));
792+
793 ut_a(space != ULINT_UNDEFINED);
794 ut_a(page_no != ULINT_UNDEFINED);
795
796@@ -312,8 +315,8 @@
797 @return the last fully tracked LSN */
798 static
799 ib_uint64_t
800-log_online_read_last_tracked_lsn()
801-/*==============================*/
802+log_online_read_last_tracked_lsn(void)
803+/*==================================*/
804 {
805 byte page[MODIFIED_PAGE_BLOCK_SIZE];
806 ibool is_last_page = FALSE;
807@@ -487,8 +490,8 @@
808 Create a new empty bitmap output file. */
809 static
810 void
811-log_online_start_bitmap_file()
812-/*==========================*/
813+log_online_start_bitmap_file(void)
814+/*==============================*/
815 {
816 ibool success;
817
818@@ -517,7 +520,10 @@
819 ib_uint64_t next_file_start_lsn) /*!<in: the start LSN name
820 part */
821 {
822- os_file_close(log_bmp_sys->out.file);
823+ if (log_bmp_sys->out.file != -1) {
824+ os_file_close(log_bmp_sys->out.file);
825+ log_bmp_sys->out.file = -1;
826+ }
827 log_bmp_sys->out_seq_num++;
828 log_online_make_bitmap_name(next_file_start_lsn);
829 log_online_start_bitmap_file();
830@@ -555,8 +561,8 @@
831 Initialize the online log following subsytem. */
832 UNIV_INTERN
833 void
834-log_online_read_init()
835-/*==================*/
836+log_online_read_init(void)
837+/*======================*/
838 {
839 ibool success;
840 ib_uint64_t tracking_start_lsn
841@@ -565,13 +571,15 @@
842 os_file_stat_t bitmap_dir_file_info;
843 ib_uint64_t last_file_start_lsn = MIN_TRACKED_LSN;
844
845- /* Assert (could be compile-time assert) that bitmap data start and end
846- in a bitmap block is 8-byte aligned */
847- ut_a(MODIFIED_PAGE_BLOCK_BITMAP % 8 == 0);
848- ut_a(MODIFIED_PAGE_BLOCK_BITMAP_LEN % 8 == 0);
849+ /* Bitmap data start and end in a bitmap block must be 8-byte
850+ aligned. */
851+ compile_time_assert(MODIFIED_PAGE_BLOCK_BITMAP % 8 == 0);
852+ compile_time_assert(MODIFIED_PAGE_BLOCK_BITMAP_LEN % 8 == 0);
853
854 log_bmp_sys = ut_malloc(sizeof(*log_bmp_sys));
855
856+ mutex_create(&log_bmp_sys->mutex, SYNC_LOG_ONLINE);
857+
858 /* Enumerate existing bitmap files to either open the last one to get
859 the last tracked LSN either to find that there are none and start
860 tracking from scratch. */
861@@ -700,12 +708,15 @@
862 Shut down the online log following subsystem. */
863 UNIV_INTERN
864 void
865-log_online_read_shutdown()
866-/*======================*/
867+log_online_read_shutdown(void)
868+/*==========================*/
869 {
870 ib_rbt_node_t *free_list_node = log_bmp_sys->page_free_list;
871
872- os_file_close(log_bmp_sys->out.file);
873+ if (log_bmp_sys->out.file != -1) {
874+ os_file_close(log_bmp_sys->out.file);
875+ log_bmp_sys->out.file = -1;
876+ }
877
878 rbt_free(log_bmp_sys->modified_pages);
879
880@@ -715,6 +726,8 @@
881 free_list_node = next;
882 }
883
884+ mutex_free(&log_bmp_sys->mutex);
885+
886 ut_free(log_bmp_sys);
887 }
888
889@@ -758,14 +771,16 @@
890 buffer. */
891 static
892 void
893-log_online_parse_redo_log()
894-/*=======================*/
895+log_online_parse_redo_log(void)
896+/*===========================*/
897 {
898 byte *ptr = log_bmp_sys->parse_buf;
899 byte *end = log_bmp_sys->parse_buf_end;
900
901 ulint len = 0;
902
903+ ut_ad(mutex_own(&log_bmp_sys->mutex));
904+
905 while (ptr != end
906 && log_bmp_sys->next_parse_lsn < log_bmp_sys->end_lsn) {
907
908@@ -856,6 +871,8 @@
909 ulint actual_data_len = (end_offset >= start_offset)
910 ? end_offset - start_offset : 0;
911
912+ ut_ad(mutex_own(&log_bmp_sys->mutex));
913+
914 ut_memcpy(log_bmp_sys->parse_buf_end, log_block + start_offset,
915 actual_data_len);
916
917@@ -880,6 +897,8 @@
918 {
919 ulint block_data_len;
920
921+ ut_ad(mutex_own(&log_bmp_sys->mutex));
922+
923 block_data_len = log_block_get_data_len(log_block);
924
925 ut_ad(block_data_len % OS_FILE_LOG_BLOCK_SIZE == 0
926@@ -906,6 +925,8 @@
927 byte* log_block_end = log_bmp_sys->read_buf
928 + (block_end_lsn - block_start_lsn);
929
930+ ut_ad(mutex_own(&log_bmp_sys->mutex));
931+
932 mutex_enter(&log_sys->mutex);
933 log_group_read_log_seg(LOG_RECOVER, log_bmp_sys->read_buf,
934 group, block_start_lsn, block_end_lsn);
935@@ -968,6 +989,8 @@
936 ib_uint64_t block_start_lsn = contiguous_lsn;
937 ib_uint64_t block_end_lsn;
938
939+ ut_ad(mutex_own(&log_bmp_sys->mutex));
940+
941 log_bmp_sys->next_parse_lsn = log_bmp_sys->start_lsn;
942 log_bmp_sys->parse_buf_end = log_bmp_sys->parse_buf;
943
944@@ -1004,6 +1027,8 @@
945 {
946 ibool success;
947
948+ ut_ad(mutex_own(&log_bmp_sys->mutex));
949+
950 success = os_file_write(log_bmp_sys->out.name, log_bmp_sys->out.file,
951 block,
952 (ulint)(log_bmp_sys->out.offset & 0xFFFFFFFF),
953@@ -1037,12 +1062,14 @@
954 bitmap tree and recycles its nodes to the free list. */
955 static
956 void
957-log_online_write_bitmap()
958-/*=====================*/
959+log_online_write_bitmap(void)
960+/*=========================*/
961 {
962 ib_rbt_node_t *bmp_tree_node;
963 const ib_rbt_node_t *last_bmp_tree_node;
964
965+ ut_ad(mutex_own(&log_bmp_sys->mutex));
966+
967 if (log_bmp_sys->out.offset >= srv_max_bitmap_file_size) {
968 log_online_rotate_bitmap_file(log_bmp_sys->start_lsn);
969 }
970@@ -1083,18 +1110,21 @@
971 page bitmap which is then written to disk. */
972 UNIV_INTERN
973 void
974-log_online_follow_redo_log()
975-/*========================*/
976+log_online_follow_redo_log(void)
977+/*============================*/
978 {
979 ib_uint64_t contiguous_start_lsn;
980 log_group_t* group;
981
982+ mutex_enter(&log_bmp_sys->mutex);
983+
984 /* Grab the LSN of the last checkpoint, we will parse up to it */
985 mutex_enter(&(log_sys->mutex));
986 log_bmp_sys->end_lsn = log_sys->last_checkpoint_lsn;
987 mutex_exit(&(log_sys->mutex));
988
989 if (log_bmp_sys->end_lsn == log_bmp_sys->start_lsn) {
990+ mutex_exit(&log_bmp_sys->mutex);
991 return;
992 }
993
994@@ -1116,6 +1146,8 @@
995 log_online_write_bitmap();
996 log_bmp_sys->start_lsn = log_bmp_sys->end_lsn;
997 log_set_tracked_lsn(log_bmp_sys->start_lsn);
998+
999+ mutex_exit(&log_bmp_sys->mutex);
1000 }
1001
1002 /*********************************************************************//**
1003@@ -1512,3 +1544,76 @@
1004
1005 return TRUE;
1006 }
1007+
1008+/************************************************************//**
1009+Delete all the bitmap files for data less than the specified LSN.
1010+If called with lsn == 0 (i.e. set by RESET request) or
1011+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
1012+continue it.
1013+
1014+@return FALSE to indicate success, TRUE for failure. */
1015+UNIV_INTERN
1016+ibool
1017+log_online_purge_changed_page_bitmaps(
1018+/*==================================*/
1019+ ib_uint64_t lsn) /*!< in: LSN to purge files up to */
1020+{
1021+ log_online_bitmap_file_range_t bitmap_files;
1022+ size_t i;
1023+ ibool result = FALSE;
1024+
1025+ if (lsn == 0) {
1026+ lsn = IB_ULONGLONG_MAX;
1027+ }
1028+
1029+ if (srv_track_changed_pages) {
1030+ /* User requests might happen with both enabled and disabled
1031+ tracking */
1032+ mutex_enter(&log_bmp_sys->mutex);
1033+ }
1034+
1035+ if (!log_online_setup_bitmap_file_range(&bitmap_files, 0, lsn)) {
1036+ if (srv_track_changed_pages) {
1037+ mutex_exit(&log_bmp_sys->mutex);
1038+ }
1039+ return TRUE;
1040+ }
1041+
1042+ if (srv_track_changed_pages && lsn >= log_bmp_sys->end_lsn) {
1043+ /* If we have to delete the current output file, close it
1044+ first. */
1045+ os_file_close(log_bmp_sys->out.file);
1046+ log_bmp_sys->out.file = -1;
1047+ }
1048+
1049+ for (i = 0; i < bitmap_files.count; i++) {
1050+ if (bitmap_files.files[i].seq_num == 0
1051+ || bitmap_files.files[i].start_lsn >= lsn) {
1052+ break;
1053+ }
1054+ if (!os_file_delete_if_exists(bitmap_files.files[i].name)) {
1055+ os_file_get_last_error(TRUE);
1056+ result = TRUE;
1057+ break;
1058+ }
1059+ }
1060+
1061+ if (srv_track_changed_pages) {
1062+ if (lsn > log_bmp_sys->end_lsn) {
1063+ if (lsn == IB_ULONGLONG_MAX) {
1064+ /* RESET restarts the sequence */
1065+ log_bmp_sys->out_seq_num = 0;
1066+ log_online_rotate_bitmap_file(0);
1067+ } else {
1068+ /* PURGE continues the sequence */
1069+ log_online_rotate_bitmap_file
1070+ (log_bmp_sys->end_lsn);
1071+ }
1072+ }
1073+
1074+ mutex_exit(&log_bmp_sys->mutex);
1075+ }
1076+
1077+ free(bitmap_files.files);
1078+ return result;
1079+}

Subscribers

People subscribed via source and target branches