Merge lp:~vlad-lesin/percona-server/i_s-innodb-changed-pages into lp:percona-server/5.1

Proposed by Vlad Lesin on 2012-08-02
Status: Merged
Approved by: Laurynas Biveinis on 2012-08-14
Approved revision: no longer in the source branch.
Merged at revision: 464
Proposed branch: lp:~vlad-lesin/percona-server/i_s-innodb-changed-pages
Merge into: lp:percona-server/5.1
Diff against target: 1010 lines (+703/-10)
19 files modified
Percona-Server/mysql-test/r/information_schema.result (+8/-4)
Percona-Server/mysql-test/r/information_schema_db.result (+2/-1)
Percona-Server/mysql-test/r/mysqlshow.result (+4/-2)
Percona-Server/mysql-test/r/percona_server_variables_debug.result (+1/-0)
Percona-Server/mysql-test/r/percona_server_variables_release.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result (+41/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result (+2/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test (+103/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test (+8/-0)
Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc (+10/-1)
Percona-Server/storage/innodb_plugin/handler/i_s.cc (+294/-1)
Percona-Server/storage/innodb_plugin/handler/i_s.h (+1/-0)
Percona-Server/storage/innodb_plugin/include/log0online.h (+60/-0)
Percona-Server/storage/innodb_plugin/include/srv0srv.h (+3/-0)
Percona-Server/storage/innodb_plugin/log/log0online.c (+160/-1)
Percona-Server/storage/innodb_plugin/srv/srv0srv.c (+2/-0)
To merge this branch: bzr merge lp:~vlad-lesin/percona-server/i_s-innodb-changed-pages
Reviewer Review Type Date Requested Status
Laurynas Biveinis (community) 2012-08-02 Approve on 2012-08-14
Review via email: mp+117869@code.launchpad.net

Description of the change

This is implementation if INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table.
Blueprint for this feature can be found here:

https://blueprints.launchpad.net/percona-server/+spec/innodb-changed-pages-table

This merge proposal contains fixes of previous proposals and optimization of the table forming which allows to avoid full bitmap file scanning for certain condition templates(WHERE clauses).

This code was tested locally and all tests passed except "main.partition_rename_longfilename" which fails in the both with and without patch causes.

To post a comment you must log in.

     Diff line 281 query could be written as SELECT MAX(end_lsn) AS
     end_lsn FROM I_S....;

     The same comments re. server restart from
     https://code.launchpad.net/~vlad-lesin/percona-server/i_s-innodb-log-tracking-status/+merge/117871
     apply here too.

     srv_changed_pages_limit is declared in srv0srv.h, there is no
     need for separate extern declaration in i_s.cc.

     I'd name the max_lsn_struct to lsn_range_struct because that's
     what it is - or would get rid of it altogether, passing *min_lsn,
     *max_lsn to the get_max_lsn_from_and_condition does not seem like
     a big deal. Likewise I'd rename get_max_lsn_from_and_condition()
     to limit_lsn_range_from_condition()

     The range start and end values are not updated correctly: they
     are both compared with their old values and updated if they are
     less. So if we have if start_lsn > 100 && start_lsn > 50, 50 will
     be chosen instead of 100. The way to do this is to start with
     start_lsn of zero and compare if it's greater than the old
     value, and end_lsn of +infinity, compared if it's smaller than the old
     value.

     I am not sure if the condition range limiting is applied
     correctly in the main fill loop: if
     (LOG_BITMAP_ITERATOR_START_LSN(i) > max_lsn.start || ...) break;
     In case we have requested an interval that has a checkpoint in
     the middle, this check will only output the rows up to that
     checkpoint and then ...START_LSN(i) > max_lsn.start will become
     true. I think currently (before we have multiple bitmap files)
     the only limitation we can do here is
     LOG_BTIMAP_ITERATOR_START_LSN > max_lsn.end

     It is also possible to check if the condition range limitation
     resulted in empty range of start_lsn > end_lsn and not to
     anything then.

     Line 481: "expression".
     Line 486: not sure what " if fill max_lsn with the minimum
     values." means.
     Line 543: s/cause/case

review: Needs Fixing
Vlad Lesin (vlad-lesin) wrote :
Download full text (4.2 KiB)

> Diff line 281 query could be written as SELECT MAX(end_lsn) AS
> end_lsn FROM I_S....;

Done.

> The same comments re. server restart from
> https://code.launchpad.net/~vlad-lesin/percona-server/i_s-innodb-log-
> tracking-status/+merge/117871
> apply here too.

Done.

> srv_changed_pages_limit is declared in srv0srv.h, there is no
> need for separate extern declaration in i_s.cc.

Done.

> I'd name the max_lsn_struct to lsn_range_struct because that's
> what it is - or would get rid of it altogether, passing *min_lsn,
> *max_lsn to the get_max_lsn_from_and_condition does not seem like
> a big deal. Likewise I'd rename get_max_lsn_from_and_condition()
> to limit_lsn_range_from_condition()

Done.

> The range start and end values are not updated correctly: they
> are both compared with their old values and updated if they are
> less. So if we have if start_lsn > 100 && start_lsn > 50, 50 will
> be chosen instead of 100. The way to do this is to start with
> start_lsn of zero and compare if it's greater than the old
> value, and end_lsn of +infinity, compared if it's smaller than the old
> value.

I think the current logic is right for "AND" operation and "less(or equal)" comparisons. The original task was to avoid full bitmap files scanning if we reach some limit value from condition. That's why only upper limits from "less(or equal)" comparisons are found. It finds the minimum value because if one of the "AND" operands is "false" the result of the whole operation is "false" too. For example for our expression "start_lsn < 100 && start_lsn < 50" there is no need to scan bitmaps further if start_lsn reaches 50 because the result of "AND" operation will be "false". For "start_lsn > 100 && start_lsn > 50 && start_lsn < 150 && 200 >= start_lsn" only last two comparisons will be parsed and "150" value will be a result because there is no sense to parse "greater(or equal)" comparisons as we can't calculate an offset of start bitmap block in a file. So any operand of "AND" is ignored if it is not a "(start|end)_lsn (<|<=) some_numeric_value" or "some_numeric_value (>=|>) (start|end)_lsn". And any conditions except "AND" and comparison itself are currently ignored.

> I am not sure if the condition range limiting is applied
> correctly in the main fill loop: if
> (LOG_BITMAP_ITERATOR_START_LSN(i) > max_lsn.start || ...) break;
> In case we have requested an interval that has a checkpoint in
> the middle, this check will only output the rows up to that
> checkpoint and then ...START_LSN(i) > max_lsn.start will become
> true. I think currently (before we have multiple bitmap files)
> the only limitation we can do here is
> LOG_BTIMAP_ITERATOR_START_LSN > max_lsn.end

I'm not sure I understood you right. The function just stop scanning bitmap files if condition becomes "false" for certain condition pattern. The condition for "AND" operation becomes "false" if at least one operand is "false".
The condition:
                if (LOG_BITMAP_ITERATOR_START_LSN(i) > lsn_range.start ||
                    LOG_BI...

Read more...

Vlad -

You are right, I misunderstood the purpose of max start and max end lsn struct, I thought that it is meant to limit the lower bound of the read interval too, although this information cannot be usefully applied yet. In this case I am not sure we need to have the max start lsn at all and we can have a single end_lsn limit variable for simpler checks, where start_lsn is handled as follows: suppose the condition is start_lsn < 100, this means we have to read all blocks with start_lsn < 100. Which is equivalent to reading all the blocks with end_lsn <= 99, or just end_lsn < 100. And we know that have read all such blocks when we read a block where start_lsn >= 100. Condition LOG_BITMAP_ITERATOR_START_LSN(i) > lsn_range.end would handle this. Does this look correct to you?

A check (whatever its final form is) in lines 627--629 should simply become a part of the while condition.

Should the "regular" ICP condition check (lines 659--660) be moved before table->field[i]->store calls?

A few too long lines in the header comment of limit_lsn_range_from_condition(): lines 476, 477, 484 and 491.

Misaligned comment starts in lines 737--747.

The log0online.h function declarations do not follow InnoDB style for their arguments. The log0online.c definitions follow them and have expanded comments, so probably should be re-copy-pasted to the header file. The header comments should be "Do something" instead of "Does something." The log_bitmap_iterator_t *i arg in all functions should be commented as "in/out" instead of "out" because it is required that i is allocated beforehand.

review: Needs Fixing
Vlad Lesin (vlad-lesin) wrote :

> Vlad -
>
> You are right, I misunderstood the purpose of max start and max end lsn
> struct, I thought that it is meant to limit the lower bound of the read
> interval too, although this information cannot be usefully applied yet. In
> this case I am not sure we need to have the max start lsn at all and we can
> have a single end_lsn limit variable for simpler checks, where start_lsn is
> handled as follows: suppose the condition is start_lsn < 100, this means we
> have to read all blocks with start_lsn < 100. Which is equivalent to reading
> all the blocks with end_lsn <= 99, or just end_lsn < 100.
> And we know that
> have read all such blocks when we read a block where start_lsn >= 100.
> Condition LOG_BITMAP_ITERATOR_START_LSN(i) > lsn_range.end would handle this.
> Does this look correct to you?

Yes, that's a good idea, thanks. I changed limit_lsn_range_from_condition() so it finds maximum lsn value at all, doesn't matter if it's start or end lsn.

> A check (whatever its final form is) in lines 627--629 should simply become a
> part of the while condition.
Done.

> Should the "regular" ICP condition check (lines 659--660) be moved before
> table->field[i]->store calls?
No, it shouldn't. Because "cond->val_int()" use stored field values to calculate condition value. For example if condition is "page_id = 100" val_int() will use the table->field[1] value to calculate if condition for current iteration is true or false.

> A few too long lines in the header comment of
> limit_lsn_range_from_condition(): lines 476, 477, 484 and 491.
>
> Misaligned comment starts in lines 737--747.
Done.

> The log0online.h function declarations do not follow InnoDB style for their
> arguments. The log0online.c definitions follow them and have expanded
> comments, so probably should be re-copy-pasted to the header file.
Done.

> The header
> comments should be "Do something" instead of "Does something."
If so there will not be uniformity because the other functions in "log0online.h" has "Does something" comments.

> The
> log_bitmap_iterator_t *i arg in all functions should be commented as "in/out"
> instead of "out" because it is required that i is allocated beforehand.
Done.

Vlad -

The code looks good, I have some other minor comments:

     Lines 461--462 and 464--465 should be exchanged for a bit
     better comment readability (IMHO).

     The function return value on line 588 is documented
     incorrectly. It should not be a separate line with an /*!<out:
     ... */ comment but rather a "@return desription" tag at the end of
     the header comment, grep InnoDB for "@return" for examples.

     I am not sure what to do about the comment block in 610--630. It
     is very descriptive and it is good to explain why the condition
     it describes must look the way it does. But it breaks the code
     flow and readability. Perhaps extract
     LOG_BITMAP_ITERATOR_START_LSN(i) <= max_lsn into a separate
     (static inline) function and make that comment its header comment?

review: Needs Fixing
Vlad Lesin (vlad-lesin) wrote :

> Vlad -
>
> The code looks good, I have some other minor comments:
>
> Lines 461--462 and 464--465 should be exchanged for a bit
> better comment readability (IMHO).
>
Done.

> The function return value on line 588 is documented
> incorrectly. It should not be a separate line with an /*!<out:
> ... */ comment but rather a "@return desription" tag at the end of
> the header comment, grep InnoDB for "@return" for examples.
>
I saw the both styles. For example, see i_s_innodb_buffer_pool_pages_fill() and i_s_innodb_buffer_pool_pages_*() at all. So I just got one of them as a template. So as "@return" style is more readable I fixed it.

> I am not sure what to do about the comment block in 610--630. It
> is very descriptive and it is good to explain why the condition
> it describes must look the way it does. But it breaks the code
> flow and readability. Perhaps extract
> LOG_BITMAP_ITERATOR_START_LSN(i) <= max_lsn into a separate
> (static inline) function and make that comment its header comment?
I think this comment will be out of context. Could you look at the padded version of the comment please? Maybe it looks readable enough. If no I will replace it in separate function.

Very minor comments:

     Line 283: uppercase MIN()
     Line 339: redundant --innodb
     Line 494: s/filles/filled or maybe just "will be set to"

No need for another MP, just re-push with these fixes.

review: Approve
458. By Vlad Lesin on 2012-08-15

This is implementation if INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table.
Blueprint for this feature can be found here:

https://blueprints.launchpad.net/percona-server/+spec/innodb-changed-pages-table

The table is generated from bitmap file which is written by log tracking thread
which parses and writes bitmap blocks after checkpoint happens. Thus the table
contain only information that was written by log tracking thread, it doesn't
contain realtime data.

Information schema tables are in-memory tables, so it can require a lot of
memory to store the whole INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table. There
are two mechanisms to limit the size of the table. The first is condition
pushdown. It means the table will contain only data which corresponds to
condition in "WHERE" SQL clause. And the second is innodb_changed_pages_limit
system variable which limits the number of rows in the table(0 - unlimited).

Vlad Lesin (vlad-lesin) wrote :

> Very minor comments:
>
> Line 283: uppercase MIN()
> Line 339: redundant --innodb
> Line 494: s/filles/filled or maybe just "will be set to"
>
> No need for another MP, just re-push with these fixes.

Done.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'Percona-Server/mysql-test/r/information_schema.result'
2--- Percona-Server/mysql-test/r/information_schema.result 2012-06-14 05:39:43 +0000
3+++ Percona-Server/mysql-test/r/information_schema.result 2012-08-15 14:14:06 +0000
4@@ -81,7 +81,7 @@
5 VIEWS
6 INNODB_BUFFER_POOL_PAGES_INDEX
7 INNODB_RSEG
8-INNODB_LOCKS
9+INNODB_CHANGED_PAGES
10 INNODB_BUFFER_POOL_PAGES
11 INNODB_TABLE_STATS
12 INNODB_TRX
13@@ -96,6 +96,7 @@
14 INNODB_SYS_INDEXES
15 INNODB_BUFFER_POOL_PAGES_BLOB
16 INNODB_CMPMEM_RESET
17+INNODB_LOCKS
18 columns_priv
19 db
20 event
21@@ -907,7 +908,7 @@
22 flush privileges;
23 SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA;
24 table_schema count(*)
25-information_schema 53
26+information_schema 54
27 mysql 22
28 create table t1 (i int, j int);
29 create trigger trg1 before insert on t1 for each row
30@@ -1332,7 +1333,7 @@
31 VIEWS TABLE_SCHEMA
32 INNODB_BUFFER_POOL_PAGES_INDEX index_id
33 INNODB_RSEG rseg_id
34-INNODB_LOCKS lock_id
35+INNODB_CHANGED_PAGES space_id
36 INNODB_BUFFER_POOL_PAGES page_type
37 INNODB_TABLE_STATS table_schema
38 INNODB_TRX trx_id
39@@ -1347,6 +1348,7 @@
40 INNODB_SYS_INDEXES TABLE_ID
41 INNODB_BUFFER_POOL_PAGES_BLOB space_id
42 INNODB_CMPMEM_RESET page_size
43+INNODB_LOCKS lock_id
44 SELECT t.table_name, c1.column_name
45 FROM information_schema.tables t
46 INNER JOIN
47@@ -1400,7 +1402,7 @@
48 VIEWS TABLE_SCHEMA
49 INNODB_BUFFER_POOL_PAGES_INDEX index_id
50 INNODB_RSEG rseg_id
51-INNODB_LOCKS lock_id
52+INNODB_CHANGED_PAGES space_id
53 INNODB_BUFFER_POOL_PAGES page_type
54 INNODB_TABLE_STATS table_schema
55 INNODB_TRX trx_id
56@@ -1415,6 +1417,7 @@
57 INNODB_SYS_INDEXES TABLE_ID
58 INNODB_BUFFER_POOL_PAGES_BLOB space_id
59 INNODB_CMPMEM_RESET page_size
60+INNODB_LOCKS lock_id
61 SELECT MAX(table_name) FROM information_schema.tables WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test');
62 MAX(table_name)
63 XTRADB_ADMIN_COMMAND
64@@ -1497,6 +1500,7 @@
65 INNODB_BUFFER_POOL_PAGES information_schema.INNODB_BUFFER_POOL_PAGES 1
66 INNODB_BUFFER_POOL_PAGES_BLOB information_schema.INNODB_BUFFER_POOL_PAGES_BLOB 1
67 INNODB_BUFFER_POOL_PAGES_INDEX information_schema.INNODB_BUFFER_POOL_PAGES_INDEX 1
68+INNODB_CHANGED_PAGES information_schema.INNODB_CHANGED_PAGES 1
69 INNODB_CMP information_schema.INNODB_CMP 1
70 INNODB_CMPMEM information_schema.INNODB_CMPMEM 1
71 INNODB_CMPMEM_RESET information_schema.INNODB_CMPMEM_RESET 1
72
73=== modified file 'Percona-Server/mysql-test/r/information_schema_db.result'
74--- Percona-Server/mysql-test/r/information_schema_db.result 2012-06-14 05:39:43 +0000
75+++ Percona-Server/mysql-test/r/information_schema_db.result 2012-08-15 14:14:06 +0000
76@@ -43,7 +43,7 @@
77 VIEWS
78 INNODB_BUFFER_POOL_PAGES_INDEX
79 INNODB_RSEG
80-INNODB_LOCKS
81+INNODB_CHANGED_PAGES
82 INNODB_BUFFER_POOL_PAGES
83 INNODB_TABLE_STATS
84 INNODB_TRX
85@@ -58,6 +58,7 @@
86 INNODB_SYS_INDEXES
87 INNODB_BUFFER_POOL_PAGES_BLOB
88 INNODB_CMPMEM_RESET
89+INNODB_LOCKS
90 show tables from INFORMATION_SCHEMA like 'T%';
91 Tables_in_information_schema (T%)
92 TABLES
93
94=== modified file 'Percona-Server/mysql-test/r/mysqlshow.result'
95--- Percona-Server/mysql-test/r/mysqlshow.result 2012-06-14 05:39:43 +0000
96+++ Percona-Server/mysql-test/r/mysqlshow.result 2012-08-15 14:14:06 +0000
97@@ -117,7 +117,7 @@
98 | VIEWS |
99 | INNODB_BUFFER_POOL_PAGES_INDEX |
100 | INNODB_RSEG |
101-| INNODB_LOCKS |
102+| INNODB_CHANGED_PAGES |
103 | INNODB_BUFFER_POOL_PAGES |
104 | INNODB_TABLE_STATS |
105 | INNODB_TRX |
106@@ -132,6 +132,7 @@
107 | INNODB_SYS_INDEXES |
108 | INNODB_BUFFER_POOL_PAGES_BLOB |
109 | INNODB_CMPMEM_RESET |
110+| INNODB_LOCKS |
111 +---------------------------------------+
112 Database: INFORMATION_SCHEMA
113 +---------------------------------------+
114@@ -175,7 +176,7 @@
115 | VIEWS |
116 | INNODB_BUFFER_POOL_PAGES_INDEX |
117 | INNODB_RSEG |
118-| INNODB_LOCKS |
119+| INNODB_CHANGED_PAGES |
120 | INNODB_BUFFER_POOL_PAGES |
121 | INNODB_TABLE_STATS |
122 | INNODB_TRX |
123@@ -190,6 +191,7 @@
124 | INNODB_SYS_INDEXES |
125 | INNODB_BUFFER_POOL_PAGES_BLOB |
126 | INNODB_CMPMEM_RESET |
127+| INNODB_LOCKS |
128 +---------------------------------------+
129 Wildcard: inf_rmation_schema
130 +--------------------+
131
132=== modified file 'Percona-Server/mysql-test/r/percona_server_variables_debug.result'
133--- Percona-Server/mysql-test/r/percona_server_variables_debug.result 2012-06-14 09:16:03 +0000
134+++ Percona-Server/mysql-test/r/percona_server_variables_debug.result 2012-08-15 14:14:06 +0000
135@@ -86,6 +86,7 @@
136 INNODB_BUFFER_POOL_SHM_CHECKSUM
137 INNODB_BUFFER_POOL_SHM_KEY
138 INNODB_BUFFER_POOL_SIZE
139+INNODB_CHANGED_PAGES_LIMIT
140 INNODB_CHANGE_BUFFERING
141 INNODB_CHECKPOINT_AGE_TARGET
142 INNODB_CHECKSUMS
143
144=== modified file 'Percona-Server/mysql-test/r/percona_server_variables_release.result'
145--- Percona-Server/mysql-test/r/percona_server_variables_release.result 2012-06-14 09:16:03 +0000
146+++ Percona-Server/mysql-test/r/percona_server_variables_release.result 2012-08-15 14:14:06 +0000
147@@ -84,6 +84,7 @@
148 INNODB_BUFFER_POOL_SHM_CHECKSUM
149 INNODB_BUFFER_POOL_SHM_KEY
150 INNODB_BUFFER_POOL_SIZE
151+INNODB_CHANGED_PAGES_LIMIT
152 INNODB_CHANGE_BUFFERING
153 INNODB_CHECKPOINT_AGE_TARGET
154 INNODB_CHECKSUMS
155
156=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result'
157--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 1970-01-01 00:00:00 +0000
158+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 2012-08-15 14:14:06 +0000
159@@ -0,0 +1,41 @@
160+SELECT @@innodb_track_changed_pages;
161+@@innodb_track_changed_pages
162+1
163+SET @OLD_INNODB_CHANGED_PAGES_LIMIT = @@GLOBAL.INNODB_CHANGED_PAGES_LIMIT;
164+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
165+CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB;
166+SET @t1_space_id =
167+(SELECT SPACE
168+FROM INFORMATION_SCHEMA.INNODB_SYS_TABLES
169+WHERE
170+INFORMATION_SCHEMA.INNODB_SYS_TABLES.SCHEMA='test' AND
171+INFORMATION_SCHEMA.INNODB_SYS_TABLES.NAME='T1');
172+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
173+SELECT COUNT(DISTINCT PAGE_ID) >= (1024000/16384)
174+FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
175+WHERE SPACE_ID = @t1_space_id;
176+COUNT(DISTINCT PAGE_ID) >= (1024000/16384)
177+1
178+SELECT MAX(PAGE_ID) < (3*1024000/16384)
179+FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
180+WHERE SPACE_ID = @t1_space_id;
181+MAX(PAGE_ID) < (3*1024000/16384)
182+1
183+SELECT COUNT(*)
184+FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
185+WHERE START_LSN >= END_LSN;
186+COUNT(*)
187+0
188+SELECT COUNT(*) = @cond_test_pages_count
189+FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
190+WHERE
191+END_LSN > (@cond_test_lsn - 1) AND
192+END_LSN < (@cond_test_lsn + 1);
193+COUNT(*) = @cond_test_pages_count
194+1
195+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 1;
196+SELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
197+COUNT(*)
198+1
199+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = @OLD_INNODB_CHANGED_PAGES_LIMIT;
200+DROP TABLE T1;
201
202=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result'
203--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result 1970-01-01 00:00:00 +0000
204+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result 2012-08-15 14:14:06 +0000
205@@ -0,0 +1,2 @@
206+SELECT * FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
207+space_id page_id start_lsn end_lsn
208
209=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result'
210--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result 2012-06-15 05:57:51 +0000
211+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result 2012-08-15 14:14:06 +0000
212@@ -3,6 +3,7 @@
213 INNODB_BUFFER_POOL_PAGES
214 INNODB_BUFFER_POOL_PAGES_BLOB
215 INNODB_BUFFER_POOL_PAGES_INDEX
216+INNODB_CHANGED_PAGES
217 INNODB_CMP
218 INNODB_CMPMEM
219 INNODB_CMPMEM_RESET
220
221=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt'
222--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt 1970-01-01 00:00:00 +0000
223+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt 2012-08-15 14:14:06 +0000
224@@ -0,0 +1,1 @@
225+--innodb_track_changed_pages=TRUE --innodb_log_file_size=1M --innodb-file-per-table
226
227=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test'
228--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 1970-01-01 00:00:00 +0000
229+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 2012-08-15 14:14:06 +0000
230@@ -0,0 +1,103 @@
231+###########################################
232+# Test for I_S.INNODB_CHANGED_PAGES table #
233+###########################################
234+
235+--source include/have_innodb_plugin.inc
236+
237+SELECT @@innodb_track_changed_pages;
238+
239+SET @OLD_INNODB_CHANGED_PAGES_LIMIT = @@GLOBAL.INNODB_CHANGED_PAGES_LIMIT;
240+
241+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
242+
243+CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB;
244+SET @t1_space_id =
245+ (SELECT SPACE
246+ FROM INFORMATION_SCHEMA.INNODB_SYS_TABLES
247+ WHERE
248+ INFORMATION_SCHEMA.INNODB_SYS_TABLES.SCHEMA='test' AND
249+ INFORMATION_SCHEMA.INNODB_SYS_TABLES.NAME='T1');
250+
251+--disable_query_log
252+--disable_result_log
253+
254+#########################################################################
255+# The maximum log size is 1MB. Each row occupies at least 256 bytes. #
256+# Each iteration inserts 100 rows. They occupies at least 25KB. To be #
257+# sure that tracking log thread wrote at least 1M/16K pages we need to #
258+# exceed maximum log size twice. That means we should do at least #
259+# 2M/25K = 80 iterations. #
260+#########################################################################
261+--let $i=80
262+
263+while ($i)
264+{
265+--dec $i
266+INSERT INTO T1 (F1) VALUES
267+("1"), ("2"), ("3"), ("4"), ("5"), ("6"), ("7"), ("8"), ("9"), ("10"),
268+("11"), ("12"), ("13"), ("14"), ("15"), ("16"), ("17"), ("18"), ("19"), ("20"),
269+("21"), ("22"), ("23"), ("24"), ("25"), ("26"), ("27"), ("28"), ("29"), ("30"),
270+("31"), ("32"), ("33"), ("34"), ("35"), ("36"), ("37"), ("38"), ("39"), ("40"),
271+("41"), ("42"), ("43"), ("44"), ("45"), ("46"), ("47"), ("48"), ("49"), ("50"),
272+("51"), ("52"), ("53"), ("54"), ("55"), ("56"), ("57"), ("58"), ("59"), ("60"),
273+("61"), ("62"), ("63"), ("64"), ("65"), ("66"), ("67"), ("68"), ("69"), ("70"),
274+("71"), ("72"), ("73"), ("74"), ("75"), ("76"), ("77"), ("78"), ("79"), ("80"),
275+("81"), ("82"), ("83"), ("84"), ("85"), ("86"), ("87"), ("88"), ("89"), ("90"),
276+("91"), ("92"), ("93"), ("94"), ("95"), ("96"), ("97"), ("98"), ("99"), ("100");
277+}
278+
279+###################################################################
280+# Gather data for condition pushdown testing not using conditions #
281+###################################################################
282+SET @cond_test_lsn=
283+ (SELECT MIN(end_lsn)
284+ FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES);
285+
286+SET @cond_test_pages_count=
287+ (SELECT count(*)
288+ FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
289+ GROUP BY end_lsn
290+ ORDER BY end_lsn
291+ LIMIT 1);
292+
293+--enable_query_log
294+--enable_result_log
295+
296+###############################################################
297+# Check if the number of changed pages is greater than 1M/16K #
298+###############################################################
299+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
300+SELECT COUNT(DISTINCT PAGE_ID) >= (1024000/16384)
301+ FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
302+ WHERE SPACE_ID = @t1_space_id;
303+#############################################################
304+# Check if the maximum page id is less than resonable limit #
305+#############################################################
306+SELECT MAX(PAGE_ID) < (3*1024000/16384)
307+ FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
308+ WHERE SPACE_ID = @t1_space_id;
309+########################################################
310+# The records where START_LSN >= END_LSN should absent #
311+########################################################
312+SELECT COUNT(*)
313+ FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
314+ WHERE START_LSN >= END_LSN;
315+
316+######################################################
317+# Check condition if pushdown doesn't break anything #
318+######################################################
319+SELECT COUNT(*) = @cond_test_pages_count
320+ FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
321+ WHERE
322+ END_LSN > (@cond_test_lsn - 1) AND
323+ END_LSN < (@cond_test_lsn + 1);
324+
325+################################################
326+# Check how limit for maximum rows count works #
327+################################################
328+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 1;
329+SELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
330+
331+SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = @OLD_INNODB_CHANGED_PAGES_LIMIT;
332+
333+DROP TABLE T1;
334
335=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt'
336--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt 1970-01-01 00:00:00 +0000
337+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt 2012-08-15 14:14:06 +0000
338@@ -0,0 +1,1 @@
339+--innodb_track_changed_pages=false
340
341=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test'
342--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test 1970-01-01 00:00:00 +0000
343+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test 2012-08-15 14:14:06 +0000
344@@ -0,0 +1,8 @@
345+###############################################################################
346+# Test for empty I_S.INNODB_CHANGED_PAGES table. The table should be empty if#
347+# innodb_track_changed_pages is false. #
348+###############################################################################
349+
350+--source include/have_innodb_plugin.inc
351+
352+SELECT * FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
353
354=== modified file 'Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc'
355--- Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2012-07-02 02:04:45 +0000
356+++ Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2012-08-15 14:14:06 +0000
357@@ -11934,6 +11934,13 @@
358 "Track the redo log for changed pages and output a changed page bitmap",
359 NULL, NULL, FALSE);
360
361+static MYSQL_SYSVAR_ULONGLONG(changed_pages_limit, srv_changed_pages_limit,
362+ PLUGIN_VAR_RQCMDARG,
363+ "The maximum number of rows for "
364+ "INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table, "
365+ "0 - unlimited",
366+ NULL, NULL, 1000000, 0, ~0ULL, 0);
367+
368 #if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG
369 static MYSQL_SYSVAR_UINT(change_buffering_debug, ibuf_debug,
370 PLUGIN_VAR_RQCMDARG,
371@@ -12181,6 +12188,7 @@
372 MYSQL_SYSVAR(use_sys_malloc),
373 MYSQL_SYSVAR(change_buffering),
374 MYSQL_SYSVAR(track_changed_pages),
375+ MYSQL_SYSVAR(changed_pages_limit),
376 #if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG
377 MYSQL_SYSVAR(change_buffering_debug),
378 #endif /* UNIV_DEBUG || UNIV_IBUF_DEBUG */
379@@ -12230,7 +12238,8 @@
380 i_s_innodb_admin_command,
381 i_s_innodb_sys_tables,
382 i_s_innodb_sys_indexes,
383-i_s_innodb_sys_stats
384+i_s_innodb_sys_stats,
385+i_s_innodb_changed_pages
386 mysql_declare_plugin_end;
387
388 /** @brief Initialize the default value of innodb_commit_concurrency.
389
390=== modified file 'Percona-Server/storage/innodb_plugin/handler/i_s.cc'
391--- Percona-Server/storage/innodb_plugin/handler/i_s.cc 2012-06-15 05:57:51 +0000
392+++ Percona-Server/storage/innodb_plugin/handler/i_s.cc 2012-08-15 14:14:06 +0000
393@@ -22,8 +22,15 @@
394
395 Created July 18, 2007 Vasil Dimov
396 *******************************************************/
397+#ifndef MYSQL_SERVER
398+#define MYSQL_SERVER /* For Item_* classes */
399+#include <mysql_priv.h>
400+/* Prevent influence of this definition to other headers */
401+#undef MYSQL_SERVER
402+#else
403+#include <mysql_priv.h>
404+#endif //MYSQL_SERVER
405
406-#include <mysql_priv.h>
407 #include <mysqld_error.h>
408
409 #include <m_ctype.h>
410@@ -47,6 +54,7 @@
411 #include "dict0dict.h" /* for dict_sys */
412 #include "btr0pcur.h"
413 #include "buf0lru.h" /* for XTRA_LRU_[DUMP/RESTORE] */
414+#include "log0online.h"
415 }
416
417 static const char plugin_author[] = "Innobase Oy";
418@@ -3639,3 +3647,288 @@
419 STRUCT_FLD(system_vars, NULL),
420 STRUCT_FLD(__reserved1, NULL)
421 };
422+
423+static ST_FIELD_INFO i_s_innodb_changed_pages_info[] =
424+{
425+ {STRUCT_FLD(field_name, "space_id"),
426+ STRUCT_FLD(field_length, MY_INT32_NUM_DECIMAL_DIGITS),
427+ STRUCT_FLD(field_type, MYSQL_TYPE_LONG),
428+ STRUCT_FLD(value, 0),
429+ STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
430+ STRUCT_FLD(old_name, ""),
431+ STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
432+
433+ {STRUCT_FLD(field_name, "page_id"),
434+ STRUCT_FLD(field_length, MY_INT32_NUM_DECIMAL_DIGITS),
435+ STRUCT_FLD(field_type, MYSQL_TYPE_LONG),
436+ STRUCT_FLD(value, 0),
437+ STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
438+ STRUCT_FLD(old_name, ""),
439+ STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
440+
441+ {STRUCT_FLD(field_name, "start_lsn"),
442+ STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS),
443+ STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG),
444+ STRUCT_FLD(value, 0),
445+ STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
446+ STRUCT_FLD(old_name, ""),
447+ STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
448+
449+ {STRUCT_FLD(field_name, "end_lsn"),
450+ STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS),
451+ STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG),
452+ STRUCT_FLD(value, 0),
453+ STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
454+ STRUCT_FLD(old_name, ""),
455+ STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
456+
457+ END_OF_ST_FIELD_INFO
458+};
459+
460+/***********************************************************************
461+ This function parses condition and gets upper bounds for start and end LSN's
462+ if condition corresponds to certain pattern.
463+
464+ We can't know right position to avoid scanning bitmap files from the beginning
465+ to the lower bound. But we can stop scanning bitmap files if we reach upper bound.
466+
467+ It's expected the most used queries will be like the following:
468+
469+ SELECT * FROM INNODB_CHANGED_PAGES WHERE START_LSN > num1 AND start_lsn < num2;
470+
471+ That's why the pattern is:
472+
473+ pattern: comp | and_comp;
474+ comp: lsn < int_num | lsn <= int_num | int_num > lsn | int_num >= lsn;
475+ lsn: start_lsn | end_lsn;
476+ and_comp: some_expression AND some_expression | some_expression AND and_comp;
477+ some_expression: comp | any_other_expression;
478+
479+ Suppose the condition is start_lsn < 100, this means we have to read all
480+ blocks with start_lsn < 100. Which is equivalent to reading all the blocks
481+ with end_lsn <= 99, or just end_lsn < 100. That's why it's enough to find
482+ maximum lsn value, doesn't matter if this is start or end lsn and compare
483+ it with "start_lsn" field.
484+
485+ Example:
486+
487+ SELECT * FROM INNODB_CHANGED_PAGES
488+ WHERE
489+ start_lsn > 10 AND
490+ end_lsn <= 1111 AND
491+ 555 > end_lsn AND
492+ page_id = 100;
493+
494+ max_lsn will be set to 555.
495+*/
496+static
497+void
498+limit_lsn_range_from_condition(
499+/*===========================*/
500+ TABLE* table, /*!<in: table */
501+ COND* cond, /*!<in: condition */
502+ ib_uint64_t* max_lsn) /*!<in/out: maximum LSN
503+ (must be initialized with maximum
504+ available value) */
505+{
506+ if (cond->type() != Item::COND_ITEM &&
507+ cond->type() != Item::FUNC_ITEM)
508+ return;
509+
510+ switch (((Item_func*) cond)->functype())
511+ {
512+ case Item_func::COND_AND_FUNC:
513+ {
514+ List_iterator<Item> li(*((Item_cond*) cond)->
515+ argument_list());
516+ Item *item;
517+ while ((item= li++))
518+ limit_lsn_range_from_condition(table,
519+ item,
520+ max_lsn);
521+ break;
522+ }
523+ case Item_func::LT_FUNC:
524+ case Item_func::LE_FUNC:
525+ case Item_func::GT_FUNC:
526+ case Item_func::GE_FUNC:
527+ {
528+ Item *left;
529+ Item *right;
530+ Item_field *item_field;
531+ ib_uint64_t tmp_result;
532+
533+ /*
534+ a <= b equals to b >= a that's why we just exchange
535+ "left" and "right" in the case of ">" or ">="
536+ function
537+ */
538+ if (((Item_func*) cond)->functype() ==
539+ Item_func::LT_FUNC ||
540+ ((Item_func*) cond)->functype() ==
541+ Item_func::LE_FUNC)
542+ {
543+ left = ((Item_func*) cond)->arguments()[0];
544+ right = ((Item_func*) cond)->arguments()[1];
545+ } else {
546+ left = ((Item_func*) cond)->arguments()[1];
547+ right = ((Item_func*) cond)->arguments()[0];
548+ }
549+
550+ if (!left || !right)
551+ return;
552+ if (left->type() != Item::FIELD_ITEM)
553+ return;
554+ if (right->type() != Item::INT_ITEM)
555+ return;
556+
557+ item_field = (Item_field*)left;
558+
559+ if (/* START_LSN */
560+ table->field[2] != item_field->field &&
561+ /* END_LSN */
562+ table->field[3] != item_field->field)
563+ {
564+ return;
565+ }
566+
567+ /* Check if the current field belongs to our table */
568+ if (table != item_field->field->table)
569+ return;
570+
571+ tmp_result = right->val_int();
572+ if (tmp_result < *max_lsn)
573+ *max_lsn = tmp_result;
574+
575+ break;
576+ }
577+ default:;
578+ }
579+
580+}
581+
582+/***********************************************************************
583+Fill the dynamic table information_schema.innodb_changed_pages.
584+@return 0 on success, 1 on failure */
585+static
586+int
587+i_s_innodb_changed_pages_fill(
588+/*==========================*/
589+ THD* thd, /*!<in: thread */
590+ TABLE_LIST* tables, /*!<in/out: tables to fill */
591+ COND* cond) /*!<in: condition */
592+{
593+ TABLE* table = (TABLE *) tables->table;
594+ log_bitmap_iterator_t i;
595+ ib_uint64_t output_rows_num = 0UL;
596+ ib_uint64_t max_lsn = ~0ULL;
597+
598+ if (!srv_track_changed_pages)
599+ return 0;
600+
601+ if (!log_online_bitmap_iterator_init(&i))
602+ return 1;
603+
604+ if (cond)
605+ limit_lsn_range_from_condition(table, cond, &max_lsn);
606+
607+ while(log_online_bitmap_iterator_next(&i) &&
608+ (!srv_changed_pages_limit ||
609+ output_rows_num < srv_changed_pages_limit) &&
610+ /*
611+ There is no need to compare both start LSN and end LSN fields
612+ with maximum value. It's enough to compare only start LSN.
613+ Example:
614+
615+ max_lsn = 100
616+ \\\\\\\\\\\\\\\\\\\\\\\\\|\\\\\\\\ - Query 1
617+ I------I I-------I I-------------I I----I
618+ ////////////////// | - Query 2
619+ 1 2 3 4
620+
621+ Query 1:
622+ SELECT * FROM INNODB_CHANGED_PAGES WHERE start_lsn < 100
623+ will select 1,2,3 bitmaps
624+ Query 2:
625+ SELECT * FROM INNODB_CHANGED_PAGES WHERE end_lsn < 100
626+ will select 1,2 bitmaps
627+
628+ The condition start_lsn <= 100 will be false after reading
629+ 1,2,3 bitmaps which suits for both cases.
630+ */
631+ LOG_BITMAP_ITERATOR_START_LSN(i) <= max_lsn)
632+ {
633+ if (!LOG_BITMAP_ITERATOR_PAGE_CHANGED(i))
634+ continue;
635+
636+ /* SPACE_ID */
637+ table->field[0]->store(
638+ LOG_BITMAP_ITERATOR_SPACE_ID(i));
639+ /* PAGE_ID */
640+ table->field[1]->store(
641+ LOG_BITMAP_ITERATOR_PAGE_NUM(i));
642+ /* START_LSN */
643+ table->field[2]->store(
644+ LOG_BITMAP_ITERATOR_START_LSN(i));
645+ /* END_LSN */
646+ table->field[3]->store(
647+ LOG_BITMAP_ITERATOR_END_LSN(i));
648+
649+ /*
650+ I_S tables are in-memory tables. If bitmap file is big enough
651+ a lot of memory can be used to store the table. But the size
652+ of used memory can be diminished if we store only data which
653+ corresponds to some conditions (in WHERE sql clause). Here
654+ conditions are checked for the field values stored above.
655+
656+ Conditions are checked twice. The first is here (during table
657+ generation) and the second during query execution. Maybe it
658+ makes sense to use some flag in THD object to avoid double
659+ checking.
660+ */
661+ if (cond && !cond->val_int())
662+ continue;
663+
664+ if (schema_table_store_record(thd, table))
665+ {
666+ log_online_bitmap_iterator_release(&i);
667+ return 1;
668+ }
669+
670+ ++output_rows_num;
671+ }
672+
673+ log_online_bitmap_iterator_release(&i);
674+ return 0;
675+}
676+
677+static
678+int
679+i_s_innodb_changed_pages_init(
680+/*==========================*/
681+ void* p)
682+{
683+ DBUG_ENTER("i_s_innodb_changed_pages_init");
684+ ST_SCHEMA_TABLE* schema = (ST_SCHEMA_TABLE*) p;
685+
686+ schema->fields_info = i_s_innodb_changed_pages_info;
687+ schema->fill_table = i_s_innodb_changed_pages_fill;
688+
689+ DBUG_RETURN(0);
690+}
691+
692+UNIV_INTERN struct st_mysql_plugin i_s_innodb_changed_pages =
693+{
694+ STRUCT_FLD(type, MYSQL_INFORMATION_SCHEMA_PLUGIN),
695+ STRUCT_FLD(info, &i_s_info),
696+ STRUCT_FLD(name, "INNODB_CHANGED_PAGES"),
697+ STRUCT_FLD(author, "Percona"),
698+ STRUCT_FLD(descr, "InnoDB CHANGED_PAGES table"),
699+ STRUCT_FLD(license, PLUGIN_LICENSE_GPL),
700+ STRUCT_FLD(init, i_s_innodb_changed_pages_init),
701+ STRUCT_FLD(deinit, i_s_common_deinit),
702+ STRUCT_FLD(version, 0x0100 /* 1.0 */),
703+ STRUCT_FLD(status_vars, NULL),
704+ STRUCT_FLD(system_vars, NULL),
705+ STRUCT_FLD(__reserved1, NULL)
706+};
707
708=== modified file 'Percona-Server/storage/innodb_plugin/handler/i_s.h'
709--- Percona-Server/storage/innodb_plugin/handler/i_s.h 2012-06-14 05:39:43 +0000
710+++ Percona-Server/storage/innodb_plugin/handler/i_s.h 2012-08-15 14:14:06 +0000
711@@ -43,5 +43,6 @@
712 extern struct st_mysql_plugin i_s_innodb_sys_tables;
713 extern struct st_mysql_plugin i_s_innodb_sys_indexes;
714 extern struct st_mysql_plugin i_s_innodb_sys_stats;
715+extern struct st_mysql_plugin i_s_innodb_changed_pages;
716
717 #endif /* i_s_h */
718
719=== modified file 'Percona-Server/storage/innodb_plugin/include/log0online.h'
720--- Percona-Server/storage/innodb_plugin/include/log0online.h 2012-06-14 09:16:03 +0000
721+++ Percona-Server/storage/innodb_plugin/include/log0online.h 2012-08-15 14:14:06 +0000
722@@ -25,6 +25,7 @@
723 #define log0online_h
724
725 #include "univ.i"
726+#include "os0file.h"
727
728 /*********************************************************************//**
729 Initializes the online log following subsytem. */
730@@ -48,4 +49,63 @@
731 log_online_follow_redo_log();
732 /*=========================*/
733
734+/** The iterator through all bits of changed pages bitmap blocks */
735+struct log_bitmap_iterator_struct
736+{
737+ char in_name[FN_REFLEN]; /*!< the file name for bitmap
738+ input */
739+ os_file_t in; /*!< the bitmap input file */
740+ ib_uint64_t in_offset; /*!< the next write position in the
741+ bitmap output file */
742+ ib_uint32_t bit_offset; /*!< bit offset inside of bitmap
743+ block*/
744+ ib_uint64_t start_lsn; /*!< Start lsn of the block */
745+ ib_uint64_t end_lsn; /*!< End lsn of the block */
746+ ib_uint32_t space_id; /*!< Block space id */
747+ ib_uint32_t first_page_id; /*!< First block page id */
748+ ibool changed; /*!< true if current page was changed */
749+ byte* page; /*!< Bitmap block */
750+};
751+
752+typedef struct log_bitmap_iterator_struct log_bitmap_iterator_t;
753+
754+#define LOG_BITMAP_ITERATOR_START_LSN(i) \
755+ ((i).start_lsn)
756+#define LOG_BITMAP_ITERATOR_END_LSN(i) \
757+ ((i).end_lsn)
758+#define LOG_BITMAP_ITERATOR_SPACE_ID(i) \
759+ ((i).space_id)
760+#define LOG_BITMAP_ITERATOR_PAGE_NUM(i) \
761+ ((i).first_page_id + (i).bit_offset)
762+#define LOG_BITMAP_ITERATOR_PAGE_CHANGED(i) \
763+ ((i).changed)
764+
765+/*********************************************************************//**
766+Initializes log bitmap iterator.
767+@return TRUE if the iterator is initialized OK, FALSE otherwise. */
768+UNIV_INTERN
769+ibool
770+log_online_bitmap_iterator_init(
771+/*============================*/
772+ log_bitmap_iterator_t *i); /*!<in/out: iterator */
773+
774+/*********************************************************************//**
775+Releases log bitmap iterator. */
776+UNIV_INTERN
777+void
778+log_online_bitmap_iterator_release(
779+/*===============================*/
780+ log_bitmap_iterator_t *i); /*!<in/out: iterator */
781+
782+/*********************************************************************//**
783+Iterates through bits of saved bitmap blocks.
784+Sequentially reads blocks from bitmap file(s) and interates through
785+their bits. Ignores blocks with wrong checksum.
786+@return TRUE if iteration is successful, FALSE if all bits are iterated. */
787+UNIV_INTERN
788+ibool
789+log_online_bitmap_iterator_next(
790+/*============================*/
791+ log_bitmap_iterator_t *i); /*!<in/out: iterator */
792+
793 #endif
794
795=== modified file 'Percona-Server/storage/innodb_plugin/include/srv0srv.h'
796--- Percona-Server/storage/innodb_plugin/include/srv0srv.h 2012-06-14 09:16:03 +0000
797+++ Percona-Server/storage/innodb_plugin/include/srv0srv.h 2012-08-15 14:14:06 +0000
798@@ -136,6 +136,9 @@
799
800 extern my_bool srv_track_changed_pages;
801
802+extern
803+ulonglong srv_changed_pages_limit;
804+
805 extern ibool srv_auto_extend_last_data_file;
806 extern ulint srv_last_file_size_max;
807 extern char** srv_log_group_home_dirs;
808
809=== modified file 'Percona-Server/storage/innodb_plugin/log/log0online.c'
810--- Percona-Server/storage/innodb_plugin/log/log0online.c 2012-06-14 09:16:03 +0000
811+++ Percona-Server/storage/innodb_plugin/log/log0online.c 2012-08-15 14:14:06 +0000
812@@ -29,7 +29,6 @@
813 #include "log0recv.h"
814 #include "mach0data.h"
815 #include "mtr0log.h"
816-#include "os0file.h"
817 #include "srv0srv.h"
818 #include "srv0start.h"
819 #include "trx0sys.h"
820@@ -84,6 +83,10 @@
821 which case the first LSN of actual log records will be this. */
822 #define MIN_TRACKED_LSN ((LOG_START_LSN) + (LOG_BLOCK_HDR_SIZE))
823
824+/* Tests if num bit of bitmap is set */
825+#define IS_BIT_SET(bitmap, num) \
826+ (*((bitmap) + ((num) >> 3)) & (1UL << ((num) & 7UL)))
827+
828 /** The bitmap file block size in bytes. All writes will be multiples of this.
829 */
830 enum {
831@@ -117,6 +120,7 @@
832 enum { MODIFIED_PAGE_BLOCK_BITMAP_LEN
833 = MODIFIED_PAGE_BLOCK_UNUSED_2 - MODIFIED_PAGE_BLOCK_BITMAP };
834
835+
836 /****************************************************************//**
837 Provide a comparisson function for the RB-tree tree (space,
838 block_start_page) pairs. Actual implementation does not matter as
839@@ -922,3 +926,158 @@
840 log_bmp_sys->start_lsn = log_bmp_sys->end_lsn;
841 log_set_tracked_lsn(log_bmp_sys->start_lsn);
842 }
843+
844+/*********************************************************************//**
845+Initializes log bitmap iterator.
846+@return TRUE if the iterator is initialized OK, FALSE otherwise. */
847+UNIV_INTERN
848+ibool
849+log_online_bitmap_iterator_init(
850+/*============================*/
851+ log_bitmap_iterator_t *i) /*!<in/out: iterator */
852+{
853+ ibool success;
854+
855+ ut_a(i);
856+ ut_snprintf(i->in_name, FN_REFLEN, "%s%s%d", srv_data_home,
857+ modified_page_stem, 1);
858+ i->in_offset = 0;
859+ /*
860+ Set up bit offset out of the reasonable limit
861+ to intiate reading block from file in
862+ log_online_bitmap_iterator_next()
863+ */
864+ i->bit_offset = MODIFIED_PAGE_BLOCK_BITMAP_LEN;
865+ i->in =
866+ os_file_create_simple_no_error_handling(
867+ i->in_name,
868+ OS_FILE_OPEN,
869+ OS_FILE_READ_ONLY,
870+ &success);
871+
872+ if (!success) {
873+ /* The following call prints an error message */
874+ os_file_get_last_error(TRUE);
875+ fprintf(stderr,
876+ "InnoDB: Error: Cannot open \'%s\'\n",
877+ i->in_name);
878+ return FALSE;
879+ }
880+
881+ i->page = ut_malloc(MODIFIED_PAGE_BLOCK_SIZE);
882+
883+ i->start_lsn = i->end_lsn = 0;
884+ i->space_id = 0;
885+ i->first_page_id = 0;
886+ i->changed = FALSE;
887+
888+ return TRUE;
889+}
890+
891+/*********************************************************************//**
892+Releases log bitmap iterator. */
893+UNIV_INTERN
894+void
895+log_online_bitmap_iterator_release(
896+/*===============================*/
897+ log_bitmap_iterator_t *i) /*!<in/out: iterator */
898+{
899+ ut_a(i);
900+ os_file_close(i->in);
901+ ut_free(i->page);
902+}
903+
904+/*********************************************************************//**
905+Iterates through bits of saved bitmap blocks.
906+Sequentially reads blocks from bitmap file(s) and interates through
907+their bits. Ignores blocks with wrong checksum.
908+@return TRUE if iteration is successful, FALSE if all bits are iterated. */
909+UNIV_INTERN
910+ibool
911+log_online_bitmap_iterator_next(
912+/*============================*/
913+ log_bitmap_iterator_t *i) /*!<in/out: iterator */
914+{
915+ ulint offset_low;
916+ ulint offset_high;
917+ ulint size_low;
918+ ulint size_high;
919+ ulint checksum = 0;
920+ ulint actual_checksum = !checksum;
921+
922+ ibool success;
923+
924+ ut_a(i);
925+
926+ if (i->bit_offset < MODIFIED_PAGE_BLOCK_BITMAP_LEN)
927+ {
928+ ++i->bit_offset;
929+ i->changed =
930+ IS_BIT_SET(i->page + MODIFIED_PAGE_BLOCK_BITMAP,
931+ i->bit_offset);
932+ return TRUE;
933+ }
934+
935+ while (checksum != actual_checksum)
936+ {
937+ success = os_file_get_size(i->in,
938+ &size_low,
939+ &size_high);
940+ if (!success) {
941+ os_file_get_last_error(TRUE);
942+ fprintf(stderr,
943+ "InnoDB: Warning: can't get size of "
944+ "page bitmap file \'%s\'\n",
945+ i->in_name);
946+ return FALSE;
947+ }
948+
949+ if (i->in_offset >=
950+ (ib_uint64_t)(size_low) +
951+ ((ib_uint64_t)(size_high) << 32))
952+ return FALSE;
953+
954+ offset_high = (ulint)(i->in_offset >> 32);
955+ offset_low = (ulint)(i->in_offset & 0xFFFFFFFF);
956+
957+ success = os_file_read(
958+ i->in,
959+ i->page,
960+ offset_low,
961+ offset_high,
962+ MODIFIED_PAGE_BLOCK_SIZE);
963+
964+ if (!success) {
965+ os_file_get_last_error(TRUE);
966+ fprintf(stderr,
967+ "InnoDB: Warning: failed reading "
968+ "changed page bitmap file \'%s\'\n",
969+ i->in_name);
970+ return FALSE;
971+ }
972+
973+ checksum = mach_read_from_4(
974+ i->page + MODIFIED_PAGE_BLOCK_CHECKSUM);
975+
976+ actual_checksum = log_online_calc_checksum(i->page);
977+
978+ i->in_offset += MODIFIED_PAGE_BLOCK_SIZE;
979+ }
980+
981+ i->start_lsn =
982+ mach_read_ull(i->page + MODIFIED_PAGE_START_LSN);
983+ i->end_lsn =
984+ mach_read_ull(i->page + MODIFIED_PAGE_END_LSN);
985+ i->space_id =
986+ mach_read_from_4(i->page + MODIFIED_PAGE_SPACE_ID);
987+ i->first_page_id =
988+ mach_read_from_4(i->page + MODIFIED_PAGE_1ST_PAGE_ID);
989+ i->bit_offset =
990+ 0;
991+ i->changed =
992+ IS_BIT_SET(i->page + MODIFIED_PAGE_BLOCK_BITMAP,
993+ i->bit_offset);
994+
995+ return TRUE;
996+}
997+
998
999=== modified file 'Percona-Server/storage/innodb_plugin/srv/srv0srv.c'
1000--- Percona-Server/storage/innodb_plugin/srv/srv0srv.c 2012-06-14 09:16:03 +0000
1001+++ Percona-Server/storage/innodb_plugin/srv/srv0srv.c 2012-08-15 14:14:06 +0000
1002@@ -164,6 +164,8 @@
1003
1004 UNIV_INTERN my_bool srv_track_changed_pages = TRUE;
1005
1006+UNIV_INTERN ulonglong srv_changed_pages_limit = 0;
1007+
1008 /* if TRUE, then we auto-extend the last data file */
1009 UNIV_INTERN ibool srv_auto_extend_last_data_file = FALSE;
1010 /* if != 0, this tells the max size auto-extending may increase the

Subscribers

People subscribed via source and target branches