Merge lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.5 into lp:percona-server/5.5

Proposed by Laurynas Biveinis on 2012-11-25
Status: Superseded
Proposed branch: lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.5
Merge into: lp:percona-server/5.5
Prerequisite: lp:~laurynas-biveinis/percona-server/xtradb-multiple-bmp-files-5.5
Diff against target: 1108 lines (+615/-44)
24 files modified
Percona-Server/include/mysql_com.h (+4/-0)
Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc (+0/-11)
Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result (+1/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_no_restart.result (+1/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests.result (+74/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests_no_innodb.result (+3/-0)
Percona-Server/mysql-test/suite/innodb/r/percona_changed_pages.result (+1/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test (+3/-1)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_no_restart.test (+3/-1)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests.test (+194/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb.test (+7/-0)
Percona-Server/mysql-test/suite/innodb/t/percona_changed_pages.test (+3/-1)
Percona-Server/sql/handler.cc (+36/-0)
Percona-Server/sql/handler.h (+5/-0)
Percona-Server/sql/lex.h (+1/-0)
Percona-Server/sql/sql_parse.cc (+37/-4)
Percona-Server/sql/sql_reload.cc (+8/-0)
Percona-Server/sql/sql_yacc.yy (+12/-0)
Percona-Server/storage/innobase/handler/ha_innodb.cc (+55/-0)
Percona-Server/storage/innobase/include/log0online.h (+32/-6)
Percona-Server/storage/innobase/include/sync0sync.h (+2/-0)
Percona-Server/storage/innobase/log/log0online.c (+131/-20)
To merge this branch: bzr merge lp:~laurynas-biveinis/percona-server/BT-16724-xtradb-bmp-requests-5.5
Reviewer Review Type Date Requested Status
Vlad Lesin 2012-11-25 Pending
Review via email: mp+136046@code.launchpad.net

This proposal supersedes a proposal from 2012-11-14.

This proposal has been superseded by a proposal from 2012-11-29.

To post a comment you must log in.
Laurynas Biveinis (laurynas-biveinis) wrote : Posted in a previous version of this proposal

Typo in the id - it's 16274.

Vlad Lesin (vlad-lesin) wrote : Posted in a previous version of this proposal

See comments in 5.1 branch.

review: Needs Fixing
352. By Stewart Smith on 2012-11-27

merge fix for Bug #1059738: make innodb_fake_changes faster -- prefetch sibling pages

353. By Stewart Smith on 2012-11-27

merge: Provide an option to build the binary tarball with debug enabled

354. By Stewart Smith on 2012-11-27

merge handlersocket update

355. By Stewart Smith on 2012-11-27

merge multiple changed page bitmap file support in XtraDB (and associated bug fixes)

356. By Stewart Smith on 2012-11-27

merge fix for Bug #1076215: Unnecessary AHI removal in buf_LRU_mark_space_was_deleted

357. By Stewart Smith on 2012-11-27

empty merge from 5.1: handlersocket update

358. By <email address hidden> on 2012-11-27

Merge lp:~akopytov/percona-server/BT27273-file-format-detection-in-innochecksum

359. By <email address hidden> on 2012-11-28

Merge lp:~stewart/percona-server/bug959198

360. By Hrvoje Matijakovic on 2012-12-10

 * bug fix for Bug #1056603

361. By <email address hidden> on 2012-12-11

Empty merge from Percona Server 5.1

362. By <email address hidden> on 2012-12-12

Empty merge from Percona Server 5.1

363. By <email address hidden> on 2012-12-12

Empty merge from Percona Server 5.1

364. By <email address hidden> on 2012-12-12

Empty merge from Percona Server 5.1

365. By <email address hidden> on 2012-12-12

Merge lp:~akopytov/percona-server/bugs-1039536-1081003-5.5

366. By <email address hidden> on 2012-12-13

Empty merge from Percona Server 5.1

367. By <email address hidden> on 2012-12-13

Merge lp:~gl-az/percona-server/BT-27444-bug1083700

368. By <email address hidden> on 2012-12-13

Merge lp:~akopytov/percona-server/bug1070856-5.5

369. By <email address hidden> on 2012-12-13

Merge lp:~akopytov/percona-server/bug830286-5.5

370. By <email address hidden> on 2012-12-13

Empty merge from Percona Server 5.1

371. By <email address hidden> on 2012-12-14

Empty merge from Percona Server 5.1

372. By <email address hidden> on 2012-12-14

Merge lp:~laurynas-biveinis/percona-server/bug901060-5.5

373. By Stewart Smith on 2012-12-14

Merge innodb_kill_idle_trx test fixes

374. By <email address hidden> on 2012-12-14

Empty merge from Percona Server 5.1

375. By Stewart Smith on 2012-12-16

revert revision 373 - which was a slightly incorrect merge from 5.1 (r372) (which itself shouldn't have been conflicted at all, but for whatever reason bzr decided it should be)

376. By <email address hidden> on 2012-12-17

Merge lp:~percona-core/percona-server/release-5.5.28-29.2

377. By <email address hidden> on 2012-12-18

Merge lp:~vlad-lesin/percona-server/5.5-bug1083377

378. By <email address hidden> on 2012-12-19

Empty merge from Percona Server 5.1

379. By <email address hidden> on 2012-12-19

Merge lp:~stewart/percona-server/5.5-bug1090596/

380. By Stewart Smith on 2012-12-19

merge userstat_bug602047 test fix

381. By <email address hidden> on 2012-12-19

Merge lp:~hrvojem/percona-server/bug1057031-5.5

382. By <email address hidden> on 2012-12-19

Empty merge from Percona Server 5.1

383. By <email address hidden> on 2012-12-20

Merge lp:~laurynas-biveinis/percona-server/BT-16274-bug1087202-1087218-5.5

384. By <email address hidden> on 2012-12-20

Merge lp:~vlad-lesin/percona-server/5.5-bug1049871-injections

385. By <email address hidden> on 2012-12-27

Merge lp:~hrvojem/percona-server/rn-5.1.66-14.2-5.5

386. By <email address hidden> on 2012-12-27

Empty merge from Percona Server 5.1

387. By Stewart Smith on 2013-01-03

Merge lp:~lp-dev-merge-bot/percona-server/staging-5.1 containing lp:~vlad-lesin/percona-server/5.1-bug1049871-injections-gca to 5.5 - one small conflict

388. By Stewart Smith on 2013-01-07

reverse the bad merge from previous revision (r387 - Merge lp:~lp-dev-merge-bot/percona-server/staging-5.1 containing lp:~vlad-lesin/percona-server/5.1-bug1049871-injections-gca to 5.5 - one small conflict
modified). This merge was incorrect and caused rpl.rpl_mdev382 to fail

389. By <email address hidden> on 2013-01-07

Merge lp:~hrvojem/percona-server/bug1070930-5.5

390. By <email address hidden> on 2013-01-07

Merge lp:~laurynas-biveinis/percona-server/BT-16274-bug1087202-10872128-5.5-2

391. By <email address hidden> on 2013-01-08

Merge lp:~hrvojem/percona-server/rn-5.5.28-29.3

392. By <email address hidden> on 2013-01-08

Empty merge from Percona Server 5.1

393. By <email address hidden> on 2013-01-09

Merge lp:~percona-core/percona-server/release-5.5.28-29.3

394. By <email address hidden> on 2013-01-09

Merge lp:~hrvojem/percona-server/rn-5.5.28-29.3-r2

395. By <email address hidden> on 2013-01-09

Merge lp:~hrvojem/percona-server/rn-5.1.66-14.2-r2-5.5

396. By <email address hidden> on 2013-01-09

Empty merge from Percona Server 5.1

397. By <email address hidden> on 2013-01-15

Merge lp:~stewart/percona-server/5.5.29

398. By <email address hidden> on 2013-01-17

Merge lp:~hrvojem/percona-server/bug1092106-5.5

399. By <email address hidden> on 2013-01-17

Empty merge from Percona Server 5.1

400. By <email address hidden> on 2013-01-17

Merge lp:~stewart/percona-server/5.5-5.1.66-14.1-merge

401. By <email address hidden> on 2013-01-18

Empty merge from Percona Server 5.1

402. By Stewart Smith on 2013-01-18

merge 5.1.66-14.2 release branch

403. By <email address hidden> on 2013-01-18

Merge lp:~hrvojem/percona-server/rn-5.1.67-14.3-5.5

404. By <email address hidden> on 2013-01-18

Empty merge from Percona Server 5.1

405. By <email address hidden> on 2013-01-18

Merge lp:~sergei.glushchenko/percona-server/ST-27220-bug1042946

406. By <email address hidden> on 2013-01-19

Empty merge from Percona Server 5.1

407. By <email address hidden> on 2013-01-20

Merge lp:~laurynas-biveinis/percona-server/bug1100159-5.5

408. By <email address hidden> on 2013-01-21

Merge lp:~sergei.glushchenko/percona-server/ps55-bug1017192

409. By <email address hidden> on 2013-01-21

Empty merge from Percona Server 5.1

410. By Laurynas Biveinis on 2013-01-22

Merge user requests for the XtraDB changed page bitmaps from 5.1.

Manual merge with the following changes:
- reload_acl_and_cache() bits go to sql_reload.cc instead of
  sql_parse.cc. Remove check_global_access calls as 5.5 calls it for
  us in advance.

- Before reload_acl_and_cache() call for SQLCOM_FLUSH handling in
  mysql_execute_command, check for SUPER_ACL privilege instead of
  RELOAD_ACL for the bitmap requests.

- Instrument log_bmp_sys->mutex for PFS.

- Re-record the testcases, put the newly-created files in the right
  directories.

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Percona-Server/include/mysql_com.h'
2--- Percona-Server/include/mysql_com.h 2012-05-10 07:49:14 +0000
3+++ Percona-Server/include/mysql_com.h 2012-11-28 19:10:27 +0000
4@@ -152,6 +152,10 @@
5 #define REFRESH_USER_STATS 0x800000L /* Refresh user stats my_hash table */
6 #define REFRESH_CLIENT_STATS 0x1000000L /* Refresh client stats my_hash table */
7 #define REFRESH_THREAD_STATS 0x2000000L /* Refresh thread stats my_hash table */
8+#define REFRESH_FLUSH_PAGE_BITMAPS 0x400000L
9+#define REFRESH_RESET_PAGE_BITMAPS 0x800000L
10+
11+#define PURGE_BITMAPS_TO_LSN 1
12
13 #define CLIENT_LONG_PASSWORD 1 /* new more secure passwords */
14 #define CLIENT_FOUND_ROWS 2 /* Found instead of affected rows */
15
16=== removed file 'Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc'
17--- Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc 2012-11-25 09:30:58 +0000
18+++ Percona-Server/mysql-test/include/delete_innodb_bitmaps.inc 1970-01-01 00:00:00 +0000
19@@ -1,11 +0,0 @@
20-# Remove all the InnoDB bitmap files. A temporary measure until RESET user request is implemented
21-
22-let $MYSQLD_DATADIR= `select @@datadir`;
23-
24---exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
25---shutdown_server 10
26---source include/wait_until_disconnected.inc
27-remove_files_wildcard $MYSQLD_DATADIR ib_modified_log*;
28---enable_reconnect
29---exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
30---source include/wait_until_connected_again.inc
31
32=== modified file 'Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result'
33--- Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result 2012-11-25 09:30:58 +0000
34+++ Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result 2012-11-28 19:10:27 +0000
35@@ -1,3 +1,4 @@
36+RESET ALL CHANGED PAGE BITMAPS;
37 DROP TABLE IF EXISTS t1, t2;
38 CREATE TABLE t1 (x INT) ENGINE=InnoDB;
39 INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
40
41=== modified file 'Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_no_restart.result'
42--- Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_no_restart.result 2012-11-25 09:30:58 +0000
43+++ Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_no_restart.result 2012-11-28 19:10:27 +0000
44@@ -1,3 +1,4 @@
45+RESET ALL CHANGED PAGE BITMAPS;
46 DROP TABLE IF EXISTS t1, t2;
47 CREATE TABLE t1 (x INT) ENGINE=InnoDB;
48 INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
49
50=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests.result'
51--- Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests.result 1970-01-01 00:00:00 +0000
52+++ Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests.result 2012-11-28 19:10:27 +0000
53@@ -0,0 +1,74 @@
54+RESET ALL CHANGED PAGE BITMAPS;
55+DROP TABLE IF EXISTS t1;
56+DELETE FROM mysql.user WHERE USER='mysqltest_1';
57+FLUSH PRIVILEGES;
58+CREATE TABLE t1 (x INT) ENGINE=InnoDB;
59+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
60+FLUSH ALL CHANGED PAGE BITMAPS;
61+Before RESET:
62+ib_modified_log_1
63+ib_modified_log_2
64+RESET ALL CHANGED PAGE BITMAPS;
65+After RESET:
66+ib_modified_log_1
67+RESET ALL CHANGED PAGE BITMAPS;
68+After 2nd RESET
69+ib_modified_log_1
70+After RESETs and restart:
71+ib_modified_log_1
72+ib_modified_log_2
73+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1;
74+After PURGE ... BEFORE 1:
75+ib_modified_log_2
76+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
77+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
78+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
79+After PURGE ... BEFORE 100000000 and restart:
80+ib_modified_log_4
81+ib_modified_log_5
82+PURGE ALL CHANGED PAGE BITMAPS BEFORE 5+5;
83+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '+5' at line 1
84+PURGE ALL CHANGED PAGE BITMAPS BEFORE -12;
85+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-12' at line 1
86+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1.2e-2;
87+ERROR 42000: Only integers allowed as number here near '1.2e-2' at line 1
88+PURGE ALL CHANGED PAGE BITMAPS BEFORE t1;
89+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 't1' at line 1
90+PURGE ALL CHANGED PAGE BITMAPS BEFORE "t1";
91+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"t1"' at line 1
92+PURGE ALL CHANGED PAGE BITMAPS BEFORE TRUE;
93+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TRUE' at line 1
94+PURGE ALL CHANGED PAGE BITMAPS BEFORE NULL;
95+ERROR 42000: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'NULL' at line 1
96+CREATE USER mysqltest_1@localhost;
97+GRANT ALL ON *.* TO mysqltest_1@localhost;
98+REVOKE SUPER ON *.* FROM mysqltest_1@localhost;
99+FLUSH ALL CHANGED PAGE BITMAPS;
100+ERROR 42000: Access denied; you need (at least one of) the SUPER privilege(s) for this operation
101+RESET ALL CHANGED PAGE BITMAPS;
102+ERROR 42000: Access denied; you need (at least one of) the SUPER privilege(s) for this operation
103+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
104+ERROR 42000: Access denied; you need (at least one of) the SUPER privilege(s) for this operation
105+DROP USER mysqltest_1@localhost;
106+SELECT @@GLOBAL.innodb_track_changed_pages;
107+@@GLOBAL.innodb_track_changed_pages
108+0
109+FLUSH ALL CHANGED PAGE BITMAPS;
110+Before the PURGE with tracking disabled
111+ib_modified_log_4
112+ib_modified_log_5
113+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1;
114+After the PURGE that deletes nothing:
115+ib_modified_log_4
116+ib_modified_log_5
117+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
118+After the PURGE that deletes everything:
119+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
120+After the repeated PURGE:
121+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
122+Before the RESET while tracking disabled:
123+ib_modified_log_1
124+ib_modified_log_2
125+RESET ALL CHANGED PAGE BITMAPS;
126+After the RESET with tracking disabled:
127+DROP TABLE t1;
128
129=== added file 'Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests_no_innodb.result'
130--- Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests_no_innodb.result 1970-01-01 00:00:00 +0000
131+++ Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp_requests_no_innodb.result 2012-11-28 19:10:27 +0000
132@@ -0,0 +1,3 @@
133+FLUSH ALL CHANGED PAGE BITMAPS;
134+RESET ALL CHANGED PAGE BITMAPS;
135+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1;
136
137=== modified file 'Percona-Server/mysql-test/suite/innodb/r/percona_changed_pages.result'
138--- Percona-Server/mysql-test/suite/innodb/r/percona_changed_pages.result 2012-11-25 09:30:58 +0000
139+++ Percona-Server/mysql-test/suite/innodb/r/percona_changed_pages.result 2012-11-28 19:10:27 +0000
140@@ -1,3 +1,4 @@
141+RESET ALL CHANGED PAGE BITMAPS;
142 DROP TABLE IF EXISTS T1;
143 CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB;
144 1st interval end LSN greater than interval start LSN:
145
146=== modified file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test'
147--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test 2012-11-25 09:30:58 +0000
148+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test 2012-11-28 19:10:27 +0000
149@@ -13,7 +13,9 @@
150 # Valgrind reports useless errors on very fast server shutdowns
151 --source include/not_valgrind.inc
152
153---source include/delete_innodb_bitmaps.inc
154+# Delete any existing bitmaps
155+--source include/restart_mysqld.inc
156+RESET ALL CHANGED PAGE BITMAPS;
157
158 --disable_warnings
159 DROP TABLE IF EXISTS t1, t2;
160
161=== modified file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_no_restart.test'
162--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_no_restart.test 2012-11-25 09:30:58 +0000
163+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_no_restart.test 2012-11-28 19:10:27 +0000
164@@ -2,7 +2,9 @@
165 # percona_changed_page_bmp test, which contains the full functional tests.
166 --source include/have_innodb.inc
167
168---source include/delete_innodb_bitmaps.inc
169+# Delete any existing bitmaps
170+--source include/restart_mysqld.inc
171+RESET ALL CHANGED PAGE BITMAPS;
172
173 --disable_warnings
174 DROP TABLE IF EXISTS t1, t2;
175
176=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests-master.opt'
177--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests-master.opt 1970-01-01 00:00:00 +0000
178+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests-master.opt 2012-11-28 19:10:27 +0000
179@@ -0,0 +1,1 @@
180+--innodb_track_changed_pages=TRUE
181
182=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests.test'
183--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests.test 1970-01-01 00:00:00 +0000
184+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests.test 2012-11-28 19:10:27 +0000
185@@ -0,0 +1,194 @@
186+#
187+# Tests for changed page tracking bitmaps: user requests.
188+#
189+--source include/have_innodb.inc
190+
191+# Delete any existing bitmaps
192+--source include/restart_mysqld.inc
193+RESET ALL CHANGED PAGE BITMAPS;
194+
195+--source include/count_sessions.inc
196+
197+let $MYSQLD_DATADIR= `select @@datadir`;
198+
199+--disable_warnings
200+DROP TABLE IF EXISTS t1;
201+DELETE FROM mysql.user WHERE USER='mysqltest_1';
202+FLUSH PRIVILEGES;
203+--enable_warnings
204+
205+CREATE TABLE t1 (x INT) ENGINE=InnoDB;
206+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
207+
208+#
209+# Test FLUSH ALL CHANGED PAGE BITMAPS. Not much to test due to checkpointing non-determinism,
210+# just see that it does not crash or return an error
211+#
212+FLUSH ALL CHANGED PAGE BITMAPS;
213+
214+#
215+# Test that RESET ALL CHANGED PAGE BITMAPS works
216+#
217+
218+# Have at least two bitmap files
219+--source include/restart_mysqld.inc
220+
221+--echo Before RESET:
222+--replace_regex /_[[:digit:]]+\.xdb$//
223+list_files $MYSQLD_DATADIR ib_modified_log*;
224+
225+RESET ALL CHANGED PAGE BITMAPS;
226+
227+--echo After RESET:
228+--replace_regex /_[[:digit:]]+\.xdb$//
229+list_files $MYSQLD_DATADIR ib_modified_log*;
230+
231+# Test consecutive RESET
232+RESET ALL CHANGED PAGE BITMAPS;
233+--echo After 2nd RESET
234+--replace_regex /_[[:digit:]]+\.xdb$//
235+list_files $MYSQLD_DATADIR ib_modified_log*;
236+
237+# Check that the file sequence after RESET starts with 1 again
238+--source include/restart_mysqld.inc
239+
240+--echo After RESETs and restart:
241+--replace_regex /_[[:digit:]]+\.xdb$//
242+list_files $MYSQLD_DATADIR ib_modified_log*;
243+
244+#
245+# Test that PURGE ALL CHANGED PAGE BITMAPS BEFORE works. We don't test partial PURGE
246+# because it's hard to extract good LSN value for that.
247+#
248+
249+# PURGE that deletes nothing
250+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1;
251+--echo After PURGE ... BEFORE 1:
252+--replace_regex /_[[:digit:]]+\.xdb$//
253+list_files $MYSQLD_DATADIR ib_modified_log*;
254+
255+# PURGE that deletes everything
256+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
257+# Test consecutive PURGE
258+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
259+
260+# Check that file sequence continues
261+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
262+--source include/restart_mysqld.inc
263+--echo After PURGE ... BEFORE 100000000 and restart:
264+--replace_regex /_[[:digit:]]+\.xdb$//
265+list_files $MYSQLD_DATADIR ib_modified_log*;
266+
267+#
268+# Test malformed PURGE ... BEFORE
269+#
270+--error ER_PARSE_ERROR
271+PURGE ALL CHANGED PAGE BITMAPS BEFORE 5+5;
272+
273+--error ER_PARSE_ERROR
274+PURGE ALL CHANGED PAGE BITMAPS BEFORE -12;
275+
276+--error ER_PARSE_ERROR
277+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1.2e-2;
278+
279+--error ER_PARSE_ERROR
280+PURGE ALL CHANGED PAGE BITMAPS BEFORE t1;
281+
282+--error ER_PARSE_ERROR
283+PURGE ALL CHANGED PAGE BITMAPS BEFORE "t1";
284+
285+--error ER_PARSE_ERROR
286+PURGE ALL CHANGED PAGE BITMAPS BEFORE TRUE;
287+
288+--error ER_PARSE_ERROR
289+PURGE ALL CHANGED PAGE BITMAPS BEFORE NULL;
290+
291+#
292+# Test that non-privileged users cannot issue user requests
293+#
294+CREATE USER mysqltest_1@localhost;
295+GRANT ALL ON *.* TO mysqltest_1@localhost;
296+REVOKE SUPER ON *.* FROM mysqltest_1@localhost;
297+
298+connect (conn1,localhost,mysqltest_1,,);
299+
300+--error ER_SPECIFIC_ACCESS_DENIED_ERROR
301+FLUSH ALL CHANGED PAGE BITMAPS;
302+
303+--error ER_SPECIFIC_ACCESS_DENIED_ERROR
304+RESET ALL CHANGED PAGE BITMAPS;
305+
306+--error ER_SPECIFIC_ACCESS_DENIED_ERROR
307+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
308+
309+connection default;
310+disconnect conn1;
311+
312+--source include/wait_until_count_sessions.inc
313+
314+DROP USER mysqltest_1@localhost;
315+
316+#
317+# Test FLUSH and PURGE requests with log tracking disabled
318+#
319+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
320+--shutdown_server 10
321+--source include/wait_until_disconnected.inc
322+--enable_reconnect
323+--exec echo "restart:--innodb_track_changed_pages=FALSE" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
324+--source include/wait_until_connected_again.inc
325+
326+SELECT @@GLOBAL.innodb_track_changed_pages;
327+
328+FLUSH ALL CHANGED PAGE BITMAPS;
329+
330+# The previous bitmap files should be present
331+--echo Before the PURGE with tracking disabled
332+--replace_regex /_[[:digit:]]+\.xdb$//
333+list_files $MYSQLD_DATADIR ib_modified_log*;
334+
335+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1;
336+
337+--echo After the PURGE that deletes nothing:
338+--replace_regex /_[[:digit:]]+\.xdb$//
339+list_files $MYSQLD_DATADIR ib_modified_log*;
340+
341+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
342+
343+--echo After the PURGE that deletes everything:
344+--replace_regex /_[[:digit:]]+\.xdb$//
345+list_files $MYSQLD_DATADIR ib_modified_log*;
346+
347+PURGE ALL CHANGED PAGE BITMAPS BEFORE 100000000;
348+
349+--echo After the repeated PURGE:
350+--replace_regex /_[[:digit:]]+\.xdb$//
351+list_files $MYSQLD_DATADIR ib_modified_log*;
352+
353+#
354+# Test RESET request with log tracking disabled
355+#
356+
357+# Generate some bitmap data again
358+--source include/restart_mysqld.inc
359+INSERT INTO t1 VALUES (1),(2),(3),(4),(5);
360+--source include/restart_mysqld.inc
361+
362+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
363+--shutdown_server 10
364+--source include/wait_until_disconnected.inc
365+--enable_reconnect
366+--exec echo "restart:--innodb_track_changed_pages=FALSE" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
367+--source include/wait_until_connected_again.inc
368+
369+--echo Before the RESET while tracking disabled:
370+--replace_regex /_[[:digit:]]+\.xdb$//
371+list_files $MYSQLD_DATADIR ib_modified_log*;
372+
373+RESET ALL CHANGED PAGE BITMAPS;
374+
375+--echo After the RESET with tracking disabled:
376+--replace_regex /_[[:digit:]]+\.xdb$//
377+list_files $MYSQLD_DATADIR ib_modified_log*;
378+
379+DROP TABLE t1;
380
381=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb-master.opt'
382--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb-master.opt 1970-01-01 00:00:00 +0000
383+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb-master.opt 2012-11-28 19:10:27 +0000
384@@ -0,0 +1,1 @@
385+--skip-innodb
386
387=== added file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb.test'
388--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb.test 1970-01-01 00:00:00 +0000
389+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp_requests_no_innodb.test 2012-11-28 19:10:27 +0000
390@@ -0,0 +1,7 @@
391+#
392+# Test that changed page tracking user requests are parsed without InnoDB
393+#
394+FLUSH ALL CHANGED PAGE BITMAPS;
395+RESET ALL CHANGED PAGE BITMAPS;
396+PURGE ALL CHANGED PAGE BITMAPS BEFORE 1;
397+
398
399=== modified file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_pages.test'
400--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_pages.test 2012-11-25 09:30:58 +0000
401+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_pages.test 2012-11-28 19:10:27 +0000
402@@ -4,7 +4,9 @@
403
404 --source include/have_innodb.inc
405
406---source include/delete_innodb_bitmaps.inc
407+# Delete any existing bitmaps
408+--source include/restart_mysqld.inc
409+RESET ALL CHANGED PAGE BITMAPS;
410
411 --disable_warnings
412 DROP TABLE IF EXISTS T1;
413
414=== modified file 'Percona-Server/sql/handler.cc'
415--- Percona-Server/sql/handler.cc 2012-10-17 03:47:45 +0000
416+++ Percona-Server/sql/handler.cc 2012-11-28 19:10:27 +0000
417@@ -5096,6 +5096,42 @@
418 return result;
419 }
420
421+static my_bool flush_changed_page_bitmaps_handlerton(THD *unused1,
422+ plugin_ref plugin,
423+ void *unused2)
424+{
425+ handlerton *hton= plugin_data(plugin, handlerton *);
426+
427+ if (hton->flush_changed_page_bitmaps == NULL)
428+ return FALSE;
429+
430+ return hton->flush_changed_page_bitmaps();
431+}
432+
433+bool ha_flush_changed_page_bitmaps()
434+{
435+ return plugin_foreach(NULL, flush_changed_page_bitmaps_handlerton,
436+ MYSQL_STORAGE_ENGINE_PLUGIN, NULL);
437+}
438+
439+static my_bool purge_changed_page_bitmaps_handlerton(THD *unused1,
440+ plugin_ref plugin,
441+ void *lsn)
442+{
443+ handlerton *hton= plugin_data(plugin, handlerton *);
444+
445+ if (hton->purge_changed_page_bitmaps == NULL)
446+ return FALSE;
447+
448+ return hton->purge_changed_page_bitmaps(*(ulonglong *)lsn);
449+}
450+
451+bool ha_purge_changed_page_bitmaps(ulonglong lsn)
452+{
453+ return plugin_foreach(NULL, purge_changed_page_bitmaps_handlerton,
454+ MYSQL_STORAGE_ENGINE_PLUGIN, &lsn);
455+}
456+
457 /*
458 Function to check if the conditions for row-based binlogging is
459 correct for the table.
460
461=== modified file 'Percona-Server/sql/handler.h'
462--- Percona-Server/sql/handler.h 2012-10-11 15:03:12 +0000
463+++ Percona-Server/sql/handler.h 2012-11-28 19:10:27 +0000
464@@ -842,6 +842,8 @@
465 int (*fill_is_table)(handlerton *hton, THD *thd, TABLE_LIST *tables,
466 class Item *cond,
467 enum enum_schema_tables);
468+ int (*flush_changed_page_bitmaps)();
469+ int (*purge_changed_page_bitmaps)(ulonglong lsn);
470 uint32 flags; /* global handler flags */
471 /*
472 Those handlerton functions below are properly initialized at handler
473@@ -2388,6 +2390,9 @@
474 int ha_prepare(THD *thd);
475 int ha_recover(HASH *commit_list);
476
477+bool ha_flush_changed_page_bitmaps();
478+bool ha_purge_changed_page_bitmaps(ulonglong lsn);
479+
480 /* transactions: these functions never call handlerton functions directly */
481 int ha_enable_transaction(THD *thd, bool on);
482
483
484=== modified file 'Percona-Server/sql/lex.h'
485--- Percona-Server/sql/lex.h 2012-05-10 07:49:14 +0000
486+++ Percona-Server/sql/lex.h 2012-11-28 19:10:27 +0000
487@@ -86,6 +86,7 @@
488 { "BINARY", SYM(BINARY)},
489 { "BINLOG", SYM(BINLOG_SYM)},
490 { "BIT", SYM(BIT_SYM)},
491+ { "BITMAPS", SYM(BITMAPS_SYM)},
492 { "BLOB", SYM(BLOB_SYM)},
493 { "BLOCK", SYM(BLOCK_SYM)},
494 { "BOOL", SYM(BOOL_SYM)},
495
496=== modified file 'Percona-Server/sql/sql_parse.cc'
497--- Percona-Server/sql/sql_parse.cc 2012-09-17 13:08:32 +0000
498+++ Percona-Server/sql/sql_parse.cc 2012-11-28 19:10:27 +0000
499@@ -2320,9 +2320,35 @@
500 {
501 if (check_global_access(thd, SUPER_ACL))
502 goto error;
503- /* PURGE MASTER LOGS TO 'file' */
504- res = purge_master_logs(thd, lex->to_log);
505- break;
506+ if (lex->type == 0)
507+ {
508+ /* PURGE MASTER LOGS TO 'file' */
509+ res = purge_master_logs(thd, lex->to_log);
510+ break;
511+ }
512+ if (lex->type == PURGE_BITMAPS_TO_LSN)
513+ {
514+ /* PURGE INNODB CHANGED PAGE BITMAPS BEFORE lsn */
515+ ulonglong lsn= 0;
516+ Item *it= (Item *)lex->value_list.head();
517+ if ((!it->fixed && it->fix_fields(lex->thd, &it)) || it->check_cols(1)
518+ || it->null_value)
519+ {
520+ my_error(ER_WRONG_ARGUMENTS, MYF(0),
521+ "PURGE INNODB CHANGED PAGE BITMAPS BEFORE");
522+ goto error;
523+ }
524+ lsn= it->val_uint();
525+ res= ha_purge_changed_page_bitmaps(lsn);
526+ if (res)
527+ {
528+ my_error(ER_LOG_PURGE_UNKNOWN_ERR, MYF(0),
529+ "PURGE INNODB CHANGED PAGE BITMAPS BEFORE");
530+ goto error;
531+ }
532+ my_ok(thd);
533+ break;
534+ }
535 }
536 case SQLCOM_PURGE_BEFORE:
537 {
538@@ -3782,7 +3808,14 @@
539 case SQLCOM_FLUSH:
540 {
541 int write_to_binlog;
542- if (check_global_access(thd,RELOAD_ACL))
543+
544+ if (lex->type & REFRESH_FLUSH_PAGE_BITMAPS
545+ || lex->type & REFRESH_RESET_PAGE_BITMAPS)
546+ {
547+ if (check_global_access(thd, SUPER_ACL))
548+ goto error;
549+ }
550+ else if (check_global_access(thd, RELOAD_ACL))
551 goto error;
552
553 if (first_table && lex->type & REFRESH_READ_LOCK)
554
555=== modified file 'Percona-Server/sql/sql_reload.cc'
556--- Percona-Server/sql/sql_reload.cc 2012-04-18 23:26:01 +0000
557+++ Percona-Server/sql/sql_reload.cc 2012-11-28 19:10:27 +0000
558@@ -362,6 +362,14 @@
559 }
560 mysql_mutex_unlock(&LOCK_global_user_client_stats);
561 }
562+ if (options & REFRESH_FLUSH_PAGE_BITMAPS)
563+ {
564+ result= ha_flush_changed_page_bitmaps();
565+ }
566+ if (options & REFRESH_RESET_PAGE_BITMAPS)
567+ {
568+ result= ha_purge_changed_page_bitmaps(0);
569+ }
570 if (*write_to_binlog != -1)
571 *write_to_binlog= tmp_write_to_binlog;
572 /*
573
574=== modified file 'Percona-Server/sql/sql_yacc.yy'
575--- Percona-Server/sql/sql_yacc.yy 2012-09-17 13:08:32 +0000
576+++ Percona-Server/sql/sql_yacc.yy 2012-11-28 19:10:27 +0000
577@@ -840,6 +840,7 @@
578 %token BIT_OR /* MYSQL-FUNC */
579 %token BIT_SYM /* MYSQL-FUNC */
580 %token BIT_XOR /* MYSQL-FUNC */
581+%token BITMAPS_SYM /* MYSQL */
582 %token BLOB_SYM /* SQL-2003-R */
583 %token BLOCK_SYM
584 %token BOOLEAN_SYM /* SQL-2003-R */
585@@ -11449,6 +11450,8 @@
586 { Lex->type|= REFRESH_DES_KEY_FILE; }
587 | RESOURCES
588 { Lex->type|= REFRESH_USER_RESOURCES; }
589+ | ALL CHANGED PAGE_SYM BITMAPS_SYM
590+ { Lex->type|= REFRESH_FLUSH_PAGE_BITMAPS; }
591 ;
592
593 opt_table_list:
594@@ -11476,6 +11479,8 @@
595 slave_reset_options { }
596 | MASTER_SYM { Lex->type|= REFRESH_MASTER; }
597 | QUERY_SYM CACHE_SYM { Lex->type|= REFRESH_QUERY_CACHE;}
598+ | ALL CHANGED PAGE_SYM BITMAPS_SYM
599+ { Lex->type |= REFRESH_RESET_PAGE_BITMAPS; }
600 ;
601
602 slave_reset_options:
603@@ -11496,6 +11501,13 @@
604
605 purge_options:
606 master_or_binary LOGS_SYM purge_option
607+ | ALL CHANGED PAGE_SYM BITMAPS_SYM BEFORE_SYM real_ulonglong_num
608+ {
609+ LEX *lex= Lex;
610+ lex->value_list.empty();
611+ lex->value_list.push_front(new Item_uint($6));
612+ lex->type= PURGE_BITMAPS_TO_LSN;
613+ }
614 ;
615
616 purge_option:
617
618=== modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc'
619--- Percona-Server/storage/innobase/handler/ha_innodb.cc 2012-11-25 09:30:58 +0000
620+++ Percona-Server/storage/innobase/handler/ha_innodb.cc 2012-11-28 19:10:27 +0000
621@@ -85,6 +85,7 @@
622 #include "row0sel.h"
623 #include "row0upd.h"
624 #include "log0log.h"
625+#include "log0online.h"
626 #include "lock0lock.h"
627 #include "dict0crea.h"
628 #include "btr0cur.h"
629@@ -297,6 +298,7 @@
630 {&ibuf_pessimistic_insert_mutex_key,
631 "ibuf_pessimistic_insert_mutex", 0},
632 {&kernel_mutex_key, "kernel_mutex", 0},
633+ {&log_bmp_sys_mutex_key, "log_bmp_sys_mutex", 0},
634 {&log_sys_mutex_key, "log_sys_mutex", 0},
635 # ifdef UNIV_MEM_DEBUG
636 {&mem_hash_mutex_key, "mem_hash_mutex", 0},
637@@ -438,6 +440,25 @@
638 innobase_alter_table_flags(
639 /*=======================*/
640 uint flags);
641+/************************************************************//**
642+Synchronously read and parse the redo log up to the last
643+checkpoint to write the changed page bitmap.
644+@return 0 to indicate success. Current implementation cannot fail. */
645+static
646+int
647+innobase_flush_changed_page_bitmaps();
648+/*==================================*/
649+/************************************************************//**
650+Delete all the bitmap files for data less than the specified LSN.
651+If called with lsn == 0 (i.e. set by RESET request) or
652+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
653+continue it.
654+@return 0 to indicate success, 1 for failure. */
655+static
656+int
657+innobase_purge_changed_page_bitmaps(
658+/*================================*/
659+ ulonglong lsn); /*!< in: LSN to purge files up to */
660
661 static const char innobase_hton_name[]= "InnoDB";
662
663@@ -2651,6 +2672,10 @@
664 innobase_hton->flags=HTON_NO_FLAGS;
665 innobase_hton->release_temporary_latches=innobase_release_temporary_latches;
666 innobase_hton->alter_table_flags = innobase_alter_table_flags;
667+ innobase_hton->flush_changed_page_bitmaps
668+ = innobase_flush_changed_page_bitmaps;
669+ innobase_hton->purge_changed_page_bitmaps
670+ = innobase_purge_changed_page_bitmaps;
671
672 ut_a(DATA_MYSQL_TRUE_VARCHAR == (ulint)MYSQL_TYPE_VARCHAR);
673
674@@ -3270,6 +3295,36 @@
675 | HA_INPLACE_ADD_PK_INDEX_NO_READ_WRITE);
676 }
677
678+/************************************************************//**
679+Synchronously read and parse the redo log up to the last
680+checkpoint to write the changed page bitmap.
681+@return 0 to indicate success. Current implementation cannot fail. */
682+static
683+int
684+innobase_flush_changed_page_bitmaps()
685+/*=================================*/
686+{
687+ if (srv_track_changed_pages) {
688+ os_event_reset(srv_checkpoint_completed_event);
689+ log_online_follow_redo_log();
690+ }
691+ return 0;
692+}
693+
694+/************************************************************//**
695+Delete all the bitmap files for data less than the specified LSN.
696+If called with lsn == IB_ULONGLONG_MAX (i.e. set by RESET request),
697+restart the bitmap file sequence, otherwise continue it.
698+@return 0 to indicate success, 1 for failure. */
699+static
700+int
701+innobase_purge_changed_page_bitmaps(
702+/*================================*/
703+ ulonglong lsn) /*!< in: LSN to purge files up to */
704+{
705+ return log_online_purge_changed_page_bitmaps(lsn);
706+}
707+
708 /****************************************************************//**
709 Copy the current replication position from MySQL to a transaction. */
710 static
711
712=== modified file 'Percona-Server/storage/innobase/include/log0online.h'
713--- Percona-Server/storage/innobase/include/log0online.h 2012-11-25 09:29:43 +0000
714+++ Percona-Server/storage/innobase/include/log0online.h 2012-11-28 19:10:27 +0000
715@@ -41,23 +41,49 @@
716 Initializes the online log following subsytem. */
717 UNIV_INTERN
718 void
719-log_online_read_init();
720-/*===================*/
721+log_online_read_init(void);
722+/*=======================*/
723
724 /*********************************************************************//**
725 Shuts down the online log following subsystem. */
726 UNIV_INTERN
727 void
728-log_online_read_shutdown();
729-/*=======================*/
730+log_online_read_shutdown(void);
731+/*===========================*/
732
733 /*********************************************************************//**
734 Reads and parses the redo log up to last checkpoint LSN to build the changed
735 page bitmap which is then written to disk. */
736 UNIV_INTERN
737 void
738-log_online_follow_redo_log();
739-/*=========================*/
740+log_online_follow_redo_log(void);
741+/*=============================*/
742+
743+/************************************************************//**
744+Delete all the bitmap files for data less than the specified LSN.
745+If called with lsn == 0 (i.e. set by RESET request) or
746+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
747+continue it.
748+
749+@return FALSE to indicate success, TRUE for failure. */
750+UNIV_INTERN
751+ibool
752+log_online_purge_changed_page_bitmaps(
753+/*==================================*/
754+ ib_uint64_t lsn); /*!<in: LSN to purge files up to */
755+
756+/************************************************************//**
757+Delete all the bitmap files for data less than the specified LSN.
758+If called with lsn == 0 (i.e. set by RESET request) or
759+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
760+continue it.
761+
762+@return FALSE to indicate success, TRUE for failure. */
763+UNIV_INTERN
764+ibool
765+log_online_purge_changed_page_bitmaps(
766+/*==================================*/
767+ ib_uint64_t lsn); /*!<in: LSN to purge files up to */
768
769 #define LOG_BITMAP_ITERATOR_START_LSN(i) \
770 ((i).start_lsn)
771
772=== modified file 'Percona-Server/storage/innobase/include/sync0sync.h'
773--- Percona-Server/storage/innobase/include/sync0sync.h 2012-05-10 07:49:14 +0000
774+++ Percona-Server/storage/innobase/include/sync0sync.h 2012-11-28 19:10:27 +0000
775@@ -89,6 +89,7 @@
776 extern mysql_pfs_key_t ibuf_bitmap_mutex_key;
777 extern mysql_pfs_key_t ibuf_mutex_key;
778 extern mysql_pfs_key_t ibuf_pessimistic_insert_mutex_key;
779+extern mysql_pfs_key_t log_bmp_sys_mutex_key;
780 extern mysql_pfs_key_t log_sys_mutex_key;
781 extern mysql_pfs_key_t log_flush_order_mutex_key;
782 extern mysql_pfs_key_t kernel_mutex_key;
783@@ -672,6 +673,7 @@
784 #define SYNC_TRX_LOCK_HEAP 298
785 #define SYNC_TRX_SYS_HEADER 290
786 #define SYNC_PURGE_QUEUE 200
787+#define SYNC_LOG_ONLINE 175
788 #define SYNC_LOG 170
789 #define SYNC_LOG_FLUSH_ORDER 156
790 #define SYNC_RECV 168
791
792=== modified file 'Percona-Server/storage/innobase/log/log0online.c'
793--- Percona-Server/storage/innobase/log/log0online.c 2012-11-25 09:30:58 +0000
794+++ Percona-Server/storage/innobase/log/log0online.c 2012-11-28 19:10:27 +0000
795@@ -36,6 +36,11 @@
796
797 enum { FOLLOW_SCAN_SIZE = 4 * (UNIV_PAGE_SIZE_MAX) };
798
799+#ifdef UNIV_PFS_MUTEX
800+/* Key to register log_bmp_sys->mutex with PFS */
801+UNIV_INTERN mysql_pfs_key_t log_bmp_sys_mutex_key;
802+#endif /* UNIV_PFS_MUTEX */
803+
804 /** Log parsing and bitmap output data structure */
805 struct log_bitmap_struct {
806 byte read_buf[FOLLOW_SCAN_SIZE];
807@@ -69,6 +74,7 @@
808 both the correct type and the tree does
809 not mind its overwrite during
810 rbt_next() tree traversal. */
811+ mutex_t mutex; /*!< mutex protecting all the fields.*/
812 };
813
814 /* The log parsing and bitmap output struct instance */
815@@ -172,6 +178,8 @@
816 byte search_page[MODIFIED_PAGE_BLOCK_SIZE];
817 byte *page_ptr;
818
819+ ut_ad(mutex_own(&log_bmp_sys->mutex));
820+
821 ut_a(space != ULINT_UNDEFINED);
822 ut_a(page_no != ULINT_UNDEFINED);
823
824@@ -312,8 +320,8 @@
825 @return the last fully tracked LSN */
826 static
827 ib_uint64_t
828-log_online_read_last_tracked_lsn()
829-/*==============================*/
830+log_online_read_last_tracked_lsn(void)
831+/*==================================*/
832 {
833 byte page[MODIFIED_PAGE_BLOCK_SIZE];
834 ibool is_last_page = FALSE;
835@@ -487,8 +495,8 @@
836 Create a new empty bitmap output file. */
837 static
838 void
839-log_online_start_bitmap_file()
840-/*==========================*/
841+log_online_start_bitmap_file(void)
842+/*==============================*/
843 {
844 ibool success;
845
846@@ -518,7 +526,10 @@
847 ib_uint64_t next_file_start_lsn) /*!<in: the start LSN name
848 part */
849 {
850- os_file_close(log_bmp_sys->out.file);
851+ if (log_bmp_sys->out.file != -1) {
852+ os_file_close(log_bmp_sys->out.file);
853+ log_bmp_sys->out.file = -1;
854+ }
855 log_bmp_sys->out_seq_num++;
856 log_online_make_bitmap_name(next_file_start_lsn);
857 log_online_start_bitmap_file();
858@@ -556,8 +567,8 @@
859 Initialize the online log following subsytem. */
860 UNIV_INTERN
861 void
862-log_online_read_init()
863-/*==================*/
864+log_online_read_init(void)
865+/*======================*/
866 {
867 ibool success;
868 ib_uint64_t tracking_start_lsn
869@@ -566,13 +577,16 @@
870 os_file_stat_t bitmap_dir_file_info;
871 ib_uint64_t last_file_start_lsn = MIN_TRACKED_LSN;
872
873- /* Assert (could be compile-time assert) that bitmap data start and end
874- in a bitmap block is 8-byte aligned */
875- ut_a(MODIFIED_PAGE_BLOCK_BITMAP % 8 == 0);
876- ut_a(MODIFIED_PAGE_BLOCK_BITMAP_LEN % 8 == 0);
877+ /* Bitmap data start and end in a bitmap block must be 8-byte
878+ aligned. */
879+ compile_time_assert(MODIFIED_PAGE_BLOCK_BITMAP % 8 == 0);
880+ compile_time_assert(MODIFIED_PAGE_BLOCK_BITMAP_LEN % 8 == 0);
881
882 log_bmp_sys = ut_malloc(sizeof(*log_bmp_sys));
883
884+ mutex_create(log_bmp_sys_mutex_key, &log_bmp_sys->mutex,
885+ SYNC_LOG_ONLINE);
886+
887 /* Enumerate existing bitmap files to either open the last one to get
888 the last tracked LSN either to find that there are none and start
889 tracking from scratch. */
890@@ -701,12 +715,15 @@
891 Shut down the online log following subsystem. */
892 UNIV_INTERN
893 void
894-log_online_read_shutdown()
895-/*======================*/
896+log_online_read_shutdown(void)
897+/*==========================*/
898 {
899 ib_rbt_node_t *free_list_node = log_bmp_sys->page_free_list;
900
901- os_file_close(log_bmp_sys->out.file);
902+ if (log_bmp_sys->out.file != -1) {
903+ os_file_close(log_bmp_sys->out.file);
904+ log_bmp_sys->out.file = -1;
905+ }
906
907 rbt_free(log_bmp_sys->modified_pages);
908
909@@ -716,6 +733,8 @@
910 free_list_node = next;
911 }
912
913+ mutex_free(&log_bmp_sys->mutex);
914+
915 ut_free(log_bmp_sys);
916 }
917
918@@ -759,14 +778,16 @@
919 buffer. */
920 static
921 void
922-log_online_parse_redo_log()
923-/*=======================*/
924+log_online_parse_redo_log(void)
925+/*===========================*/
926 {
927 byte *ptr = log_bmp_sys->parse_buf;
928 byte *end = log_bmp_sys->parse_buf_end;
929
930 ulint len = 0;
931
932+ ut_ad(mutex_own(&log_bmp_sys->mutex));
933+
934 while (ptr != end
935 && log_bmp_sys->next_parse_lsn < log_bmp_sys->end_lsn) {
936
937@@ -857,6 +878,8 @@
938 ulint actual_data_len = (end_offset >= start_offset)
939 ? end_offset - start_offset : 0;
940
941+ ut_ad(mutex_own(&log_bmp_sys->mutex));
942+
943 ut_memcpy(log_bmp_sys->parse_buf_end, log_block + start_offset,
944 actual_data_len);
945
946@@ -881,6 +904,8 @@
947 {
948 ulint block_data_len;
949
950+ ut_ad(mutex_own(&log_bmp_sys->mutex));
951+
952 block_data_len = log_block_get_data_len(log_block);
953
954 ut_ad(block_data_len % OS_FILE_LOG_BLOCK_SIZE == 0
955@@ -907,6 +932,8 @@
956 byte* log_block_end = log_bmp_sys->read_buf
957 + (block_end_lsn - block_start_lsn);
958
959+ ut_ad(mutex_own(&log_bmp_sys->mutex));
960+
961 mutex_enter(&log_sys->mutex);
962 log_group_read_log_seg(LOG_RECOVER, log_bmp_sys->read_buf,
963 group, block_start_lsn, block_end_lsn);
964@@ -969,6 +996,8 @@
965 ib_uint64_t block_start_lsn = contiguous_lsn;
966 ib_uint64_t block_end_lsn;
967
968+ ut_ad(mutex_own(&log_bmp_sys->mutex));
969+
970 log_bmp_sys->next_parse_lsn = log_bmp_sys->start_lsn;
971 log_bmp_sys->parse_buf_end = log_bmp_sys->parse_buf;
972
973@@ -1005,6 +1034,8 @@
974 {
975 ibool success;
976
977+ ut_ad(mutex_own(&log_bmp_sys->mutex));
978+
979 success = os_file_write(log_bmp_sys->out.name, log_bmp_sys->out.file,
980 block,
981 (ulint)(log_bmp_sys->out.offset & 0xFFFFFFFF),
982@@ -1038,12 +1069,14 @@
983 bitmap tree and recycles its nodes to the free list. */
984 static
985 void
986-log_online_write_bitmap()
987-/*=====================*/
988+log_online_write_bitmap(void)
989+/*=========================*/
990 {
991 ib_rbt_node_t *bmp_tree_node;
992 const ib_rbt_node_t *last_bmp_tree_node;
993
994+ ut_ad(mutex_own(&log_bmp_sys->mutex));
995+
996 if (log_bmp_sys->out.offset >= srv_max_bitmap_file_size) {
997 log_online_rotate_bitmap_file(log_bmp_sys->start_lsn);
998 }
999@@ -1084,18 +1117,21 @@
1000 page bitmap which is then written to disk. */
1001 UNIV_INTERN
1002 void
1003-log_online_follow_redo_log()
1004-/*========================*/
1005+log_online_follow_redo_log(void)
1006+/*============================*/
1007 {
1008 ib_uint64_t contiguous_start_lsn;
1009 log_group_t* group;
1010
1011+ mutex_enter(&log_bmp_sys->mutex);
1012+
1013 /* Grab the LSN of the last checkpoint, we will parse up to it */
1014 mutex_enter(&(log_sys->mutex));
1015 log_bmp_sys->end_lsn = log_sys->last_checkpoint_lsn;
1016 mutex_exit(&(log_sys->mutex));
1017
1018 if (log_bmp_sys->end_lsn == log_bmp_sys->start_lsn) {
1019+ mutex_exit(&log_bmp_sys->mutex);
1020 return;
1021 }
1022
1023@@ -1117,6 +1153,8 @@
1024 log_online_write_bitmap();
1025 log_bmp_sys->start_lsn = log_bmp_sys->end_lsn;
1026 log_set_tracked_lsn(log_bmp_sys->start_lsn);
1027+
1028+ mutex_exit(&log_bmp_sys->mutex);
1029 }
1030
1031 /*********************************************************************//**
1032@@ -1514,3 +1552,76 @@
1033
1034 return TRUE;
1035 }
1036+
1037+/************************************************************//**
1038+Delete all the bitmap files for data less than the specified LSN.
1039+If called with lsn == 0 (i.e. set by RESET request) or
1040+IB_ULONGLONG_MAX, restart the bitmap file sequence, otherwise
1041+continue it.
1042+
1043+@return FALSE to indicate success, TRUE for failure. */
1044+UNIV_INTERN
1045+ibool
1046+log_online_purge_changed_page_bitmaps(
1047+/*==================================*/
1048+ ib_uint64_t lsn) /*!< in: LSN to purge files up to */
1049+{
1050+ log_online_bitmap_file_range_t bitmap_files;
1051+ size_t i;
1052+ ibool result = FALSE;
1053+
1054+ if (lsn == 0) {
1055+ lsn = IB_ULONGLONG_MAX;
1056+ }
1057+
1058+ if (srv_track_changed_pages) {
1059+ /* User requests might happen with both enabled and disabled
1060+ tracking */
1061+ mutex_enter(&log_bmp_sys->mutex);
1062+ }
1063+
1064+ if (!log_online_setup_bitmap_file_range(&bitmap_files, 0, lsn)) {
1065+ if (srv_track_changed_pages) {
1066+ mutex_exit(&log_bmp_sys->mutex);
1067+ }
1068+ return TRUE;
1069+ }
1070+
1071+ if (srv_track_changed_pages && lsn >= log_bmp_sys->end_lsn) {
1072+ /* If we have to delete the current output file, close it
1073+ first. */
1074+ os_file_close(log_bmp_sys->out.file);
1075+ log_bmp_sys->out.file = -1;
1076+ }
1077+
1078+ for (i = 0; i < bitmap_files.count; i++) {
1079+ if (bitmap_files.files[i].seq_num == 0
1080+ || bitmap_files.files[i].start_lsn >= lsn) {
1081+ break;
1082+ }
1083+ if (!os_file_delete_if_exists(bitmap_files.files[i].name)) {
1084+ os_file_get_last_error(TRUE);
1085+ result = TRUE;
1086+ break;
1087+ }
1088+ }
1089+
1090+ if (srv_track_changed_pages) {
1091+ if (lsn > log_bmp_sys->end_lsn) {
1092+ if (lsn == IB_ULONGLONG_MAX) {
1093+ /* RESET restarts the sequence */
1094+ log_bmp_sys->out_seq_num = 0;
1095+ log_online_rotate_bitmap_file(0);
1096+ } else {
1097+ /* PURGE continues the sequence */
1098+ log_online_rotate_bitmap_file
1099+ (log_bmp_sys->end_lsn);
1100+ }
1101+ }
1102+
1103+ mutex_exit(&log_bmp_sys->mutex);
1104+ }
1105+
1106+ free(bitmap_files.files);
1107+ return result;
1108+}

Subscribers

People subscribed via source and target branches