Merge lp:~vlad-lesin/percona-server/i_s-innodb-changed-pages into lp:percona-server/5.1

Proposed by Vlad Lesin
Status: Merged
Approved by: Laurynas Biveinis
Approved revision: no longer in the source branch.
Merged at revision: 464
Proposed branch: lp:~vlad-lesin/percona-server/i_s-innodb-changed-pages
Merge into: lp:percona-server/5.1
Diff against target: 1010 lines (+703/-10)
19 files modified
Percona-Server/mysql-test/r/information_schema.result (+8/-4)
Percona-Server/mysql-test/r/information_schema_db.result (+2/-1)
Percona-Server/mysql-test/r/mysqlshow.result (+4/-2)
Percona-Server/mysql-test/r/percona_server_variables_debug.result (+1/-0)
Percona-Server/mysql-test/r/percona_server_variables_release.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result (+41/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result (+2/-0)
Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test (+103/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt (+1/-0)
Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test (+8/-0)
Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc (+10/-1)
Percona-Server/storage/innodb_plugin/handler/i_s.cc (+294/-1)
Percona-Server/storage/innodb_plugin/handler/i_s.h (+1/-0)
Percona-Server/storage/innodb_plugin/include/log0online.h (+60/-0)
Percona-Server/storage/innodb_plugin/include/srv0srv.h (+3/-0)
Percona-Server/storage/innodb_plugin/log/log0online.c (+160/-1)
Percona-Server/storage/innodb_plugin/srv/srv0srv.c (+2/-0)
To merge this branch: bzr merge lp:~vlad-lesin/percona-server/i_s-innodb-changed-pages
Reviewer Review Type Date Requested Status
Laurynas Biveinis (community) Approve
Review via email: mp+117869@code.launchpad.net

Description of the change

This is implementation if INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table.
Blueprint for this feature can be found here:

https://blueprints.launchpad.net/percona-server/+spec/innodb-changed-pages-table

This merge proposal contains fixes of previous proposals and optimization of the table forming which allows to avoid full bitmap file scanning for certain condition templates(WHERE clauses).

This code was tested locally and all tests passed except "main.partition_rename_longfilename" which fails in the both with and without patch causes.

To post a comment you must log in.
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

     Diff line 281 query could be written as SELECT MAX(end_lsn) AS
     end_lsn FROM I_S....;

     The same comments re. server restart from
     https://code.launchpad.net/~vlad-lesin/percona-server/i_s-innodb-log-tracking-status/+merge/117871
     apply here too.

     srv_changed_pages_limit is declared in srv0srv.h, there is no
     need for separate extern declaration in i_s.cc.

     I'd name the max_lsn_struct to lsn_range_struct because that's
     what it is - or would get rid of it altogether, passing *min_lsn,
     *max_lsn to the get_max_lsn_from_and_condition does not seem like
     a big deal. Likewise I'd rename get_max_lsn_from_and_condition()
     to limit_lsn_range_from_condition()

     The range start and end values are not updated correctly: they
     are both compared with their old values and updated if they are
     less. So if we have if start_lsn > 100 && start_lsn > 50, 50 will
     be chosen instead of 100. The way to do this is to start with
     start_lsn of zero and compare if it's greater than the old
     value, and end_lsn of +infinity, compared if it's smaller than the old
     value.

     I am not sure if the condition range limiting is applied
     correctly in the main fill loop: if
     (LOG_BITMAP_ITERATOR_START_LSN(i) > max_lsn.start || ...) break;
     In case we have requested an interval that has a checkpoint in
     the middle, this check will only output the rows up to that
     checkpoint and then ...START_LSN(i) > max_lsn.start will become
     true. I think currently (before we have multiple bitmap files)
     the only limitation we can do here is
     LOG_BTIMAP_ITERATOR_START_LSN > max_lsn.end

     It is also possible to check if the condition range limitation
     resulted in empty range of start_lsn > end_lsn and not to
     anything then.

     Line 481: "expression".
     Line 486: not sure what " if fill max_lsn with the minimum
     values." means.
     Line 543: s/cause/case

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :
Download full text (4.2 KiB)

> Diff line 281 query could be written as SELECT MAX(end_lsn) AS
> end_lsn FROM I_S....;

Done.

> The same comments re. server restart from
> https://code.launchpad.net/~vlad-lesin/percona-server/i_s-innodb-log-
> tracking-status/+merge/117871
> apply here too.

Done.

> srv_changed_pages_limit is declared in srv0srv.h, there is no
> need for separate extern declaration in i_s.cc.

Done.

> I'd name the max_lsn_struct to lsn_range_struct because that's
> what it is - or would get rid of it altogether, passing *min_lsn,
> *max_lsn to the get_max_lsn_from_and_condition does not seem like
> a big deal. Likewise I'd rename get_max_lsn_from_and_condition()
> to limit_lsn_range_from_condition()

Done.

> The range start and end values are not updated correctly: they
> are both compared with their old values and updated if they are
> less. So if we have if start_lsn > 100 && start_lsn > 50, 50 will
> be chosen instead of 100. The way to do this is to start with
> start_lsn of zero and compare if it's greater than the old
> value, and end_lsn of +infinity, compared if it's smaller than the old
> value.

I think the current logic is right for "AND" operation and "less(or equal)" comparisons. The original task was to avoid full bitmap files scanning if we reach some limit value from condition. That's why only upper limits from "less(or equal)" comparisons are found. It finds the minimum value because if one of the "AND" operands is "false" the result of the whole operation is "false" too. For example for our expression "start_lsn < 100 && start_lsn < 50" there is no need to scan bitmaps further if start_lsn reaches 50 because the result of "AND" operation will be "false". For "start_lsn > 100 && start_lsn > 50 && start_lsn < 150 && 200 >= start_lsn" only last two comparisons will be parsed and "150" value will be a result because there is no sense to parse "greater(or equal)" comparisons as we can't calculate an offset of start bitmap block in a file. So any operand of "AND" is ignored if it is not a "(start|end)_lsn (<|<=) some_numeric_value" or "some_numeric_value (>=|>) (start|end)_lsn". And any conditions except "AND" and comparison itself are currently ignored.

> I am not sure if the condition range limiting is applied
> correctly in the main fill loop: if
> (LOG_BITMAP_ITERATOR_START_LSN(i) > max_lsn.start || ...) break;
> In case we have requested an interval that has a checkpoint in
> the middle, this check will only output the rows up to that
> checkpoint and then ...START_LSN(i) > max_lsn.start will become
> true. I think currently (before we have multiple bitmap files)
> the only limitation we can do here is
> LOG_BTIMAP_ITERATOR_START_LSN > max_lsn.end

I'm not sure I understood you right. The function just stop scanning bitmap files if condition becomes "false" for certain condition pattern. The condition for "AND" operation becomes "false" if at least one operand is "false".
The condition:
                if (LOG_BITMAP_ITERATOR_START_LSN(i) > lsn_range.start ||
                    LOG_BI...

Read more...

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Vlad -

You are right, I misunderstood the purpose of max start and max end lsn struct, I thought that it is meant to limit the lower bound of the read interval too, although this information cannot be usefully applied yet. In this case I am not sure we need to have the max start lsn at all and we can have a single end_lsn limit variable for simpler checks, where start_lsn is handled as follows: suppose the condition is start_lsn < 100, this means we have to read all blocks with start_lsn < 100. Which is equivalent to reading all the blocks with end_lsn <= 99, or just end_lsn < 100. And we know that have read all such blocks when we read a block where start_lsn >= 100. Condition LOG_BITMAP_ITERATOR_START_LSN(i) > lsn_range.end would handle this. Does this look correct to you?

A check (whatever its final form is) in lines 627--629 should simply become a part of the while condition.

Should the "regular" ICP condition check (lines 659--660) be moved before table->field[i]->store calls?

A few too long lines in the header comment of limit_lsn_range_from_condition(): lines 476, 477, 484 and 491.

Misaligned comment starts in lines 737--747.

The log0online.h function declarations do not follow InnoDB style for their arguments. The log0online.c definitions follow them and have expanded comments, so probably should be re-copy-pasted to the header file. The header comments should be "Do something" instead of "Does something." The log_bitmap_iterator_t *i arg in all functions should be commented as "in/out" instead of "out" because it is required that i is allocated beforehand.

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> Vlad -
>
> You are right, I misunderstood the purpose of max start and max end lsn
> struct, I thought that it is meant to limit the lower bound of the read
> interval too, although this information cannot be usefully applied yet. In
> this case I am not sure we need to have the max start lsn at all and we can
> have a single end_lsn limit variable for simpler checks, where start_lsn is
> handled as follows: suppose the condition is start_lsn < 100, this means we
> have to read all blocks with start_lsn < 100. Which is equivalent to reading
> all the blocks with end_lsn <= 99, or just end_lsn < 100.
> And we know that
> have read all such blocks when we read a block where start_lsn >= 100.
> Condition LOG_BITMAP_ITERATOR_START_LSN(i) > lsn_range.end would handle this.
> Does this look correct to you?

Yes, that's a good idea, thanks. I changed limit_lsn_range_from_condition() so it finds maximum lsn value at all, doesn't matter if it's start or end lsn.

> A check (whatever its final form is) in lines 627--629 should simply become a
> part of the while condition.
Done.

> Should the "regular" ICP condition check (lines 659--660) be moved before
> table->field[i]->store calls?
No, it shouldn't. Because "cond->val_int()" use stored field values to calculate condition value. For example if condition is "page_id = 100" val_int() will use the table->field[1] value to calculate if condition for current iteration is true or false.

> A few too long lines in the header comment of
> limit_lsn_range_from_condition(): lines 476, 477, 484 and 491.
>
> Misaligned comment starts in lines 737--747.
Done.

> The log0online.h function declarations do not follow InnoDB style for their
> arguments. The log0online.c definitions follow them and have expanded
> comments, so probably should be re-copy-pasted to the header file.
Done.

> The header
> comments should be "Do something" instead of "Does something."
If so there will not be uniformity because the other functions in "log0online.h" has "Does something" comments.

> The
> log_bitmap_iterator_t *i arg in all functions should be commented as "in/out"
> instead of "out" because it is required that i is allocated beforehand.
Done.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Vlad -

The code looks good, I have some other minor comments:

     Lines 461--462 and 464--465 should be exchanged for a bit
     better comment readability (IMHO).

     The function return value on line 588 is documented
     incorrectly. It should not be a separate line with an /*!<out:
     ... */ comment but rather a "@return desription" tag at the end of
     the header comment, grep InnoDB for "@return" for examples.

     I am not sure what to do about the comment block in 610--630. It
     is very descriptive and it is good to explain why the condition
     it describes must look the way it does. But it breaks the code
     flow and readability. Perhaps extract
     LOG_BITMAP_ITERATOR_START_LSN(i) <= max_lsn into a separate
     (static inline) function and make that comment its header comment?

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> Vlad -
>
> The code looks good, I have some other minor comments:
>
> Lines 461--462 and 464--465 should be exchanged for a bit
> better comment readability (IMHO).
>
Done.

> The function return value on line 588 is documented
> incorrectly. It should not be a separate line with an /*!<out:
> ... */ comment but rather a "@return desription" tag at the end of
> the header comment, grep InnoDB for "@return" for examples.
>
I saw the both styles. For example, see i_s_innodb_buffer_pool_pages_fill() and i_s_innodb_buffer_pool_pages_*() at all. So I just got one of them as a template. So as "@return" style is more readable I fixed it.

> I am not sure what to do about the comment block in 610--630. It
> is very descriptive and it is good to explain why the condition
> it describes must look the way it does. But it breaks the code
> flow and readability. Perhaps extract
> LOG_BITMAP_ITERATOR_START_LSN(i) <= max_lsn into a separate
> (static inline) function and make that comment its header comment?
I think this comment will be out of context. Could you look at the padded version of the comment please? Maybe it looks readable enough. If no I will replace it in separate function.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Very minor comments:

     Line 283: uppercase MIN()
     Line 339: redundant --innodb
     Line 494: s/filles/filled or maybe just "will be set to"

No need for another MP, just re-push with these fixes.

review: Approve
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> Very minor comments:
>
> Line 283: uppercase MIN()
> Line 339: redundant --innodb
> Line 494: s/filles/filled or maybe just "will be set to"
>
> No need for another MP, just re-push with these fixes.

Done.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'Percona-Server/mysql-test/r/information_schema.result'
--- Percona-Server/mysql-test/r/information_schema.result 2012-06-14 05:39:43 +0000
+++ Percona-Server/mysql-test/r/information_schema.result 2012-08-15 14:14:06 +0000
@@ -81,7 +81,7 @@
81VIEWS81VIEWS
82INNODB_BUFFER_POOL_PAGES_INDEX82INNODB_BUFFER_POOL_PAGES_INDEX
83INNODB_RSEG83INNODB_RSEG
84INNODB_LOCKS84INNODB_CHANGED_PAGES
85INNODB_BUFFER_POOL_PAGES85INNODB_BUFFER_POOL_PAGES
86INNODB_TABLE_STATS86INNODB_TABLE_STATS
87INNODB_TRX87INNODB_TRX
@@ -96,6 +96,7 @@
96INNODB_SYS_INDEXES96INNODB_SYS_INDEXES
97INNODB_BUFFER_POOL_PAGES_BLOB97INNODB_BUFFER_POOL_PAGES_BLOB
98INNODB_CMPMEM_RESET98INNODB_CMPMEM_RESET
99INNODB_LOCKS
99columns_priv100columns_priv
100db101db
101event102event
@@ -907,7 +908,7 @@
907flush privileges;908flush privileges;
908SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA;909SELECT table_schema, count(*) FROM information_schema.TABLES WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test', 'mysqltest') AND table_name<>'ndb_binlog_index' AND table_name<>'ndb_apply_status' GROUP BY TABLE_SCHEMA;
909table_schema count(*)910table_schema count(*)
910information_schema 53911information_schema 54
911mysql 22912mysql 22
912create table t1 (i int, j int);913create table t1 (i int, j int);
913create trigger trg1 before insert on t1 for each row914create trigger trg1 before insert on t1 for each row
@@ -1332,7 +1333,7 @@
1332VIEWS TABLE_SCHEMA1333VIEWS TABLE_SCHEMA
1333INNODB_BUFFER_POOL_PAGES_INDEX index_id1334INNODB_BUFFER_POOL_PAGES_INDEX index_id
1334INNODB_RSEG rseg_id1335INNODB_RSEG rseg_id
1335INNODB_LOCKS lock_id1336INNODB_CHANGED_PAGES space_id
1336INNODB_BUFFER_POOL_PAGES page_type1337INNODB_BUFFER_POOL_PAGES page_type
1337INNODB_TABLE_STATS table_schema1338INNODB_TABLE_STATS table_schema
1338INNODB_TRX trx_id1339INNODB_TRX trx_id
@@ -1347,6 +1348,7 @@
1347INNODB_SYS_INDEXES TABLE_ID1348INNODB_SYS_INDEXES TABLE_ID
1348INNODB_BUFFER_POOL_PAGES_BLOB space_id1349INNODB_BUFFER_POOL_PAGES_BLOB space_id
1349INNODB_CMPMEM_RESET page_size1350INNODB_CMPMEM_RESET page_size
1351INNODB_LOCKS lock_id
1350SELECT t.table_name, c1.column_name1352SELECT t.table_name, c1.column_name
1351FROM information_schema.tables t1353FROM information_schema.tables t
1352INNER JOIN1354INNER JOIN
@@ -1400,7 +1402,7 @@
1400VIEWS TABLE_SCHEMA1402VIEWS TABLE_SCHEMA
1401INNODB_BUFFER_POOL_PAGES_INDEX index_id1403INNODB_BUFFER_POOL_PAGES_INDEX index_id
1402INNODB_RSEG rseg_id1404INNODB_RSEG rseg_id
1403INNODB_LOCKS lock_id1405INNODB_CHANGED_PAGES space_id
1404INNODB_BUFFER_POOL_PAGES page_type1406INNODB_BUFFER_POOL_PAGES page_type
1405INNODB_TABLE_STATS table_schema1407INNODB_TABLE_STATS table_schema
1406INNODB_TRX trx_id1408INNODB_TRX trx_id
@@ -1415,6 +1417,7 @@
1415INNODB_SYS_INDEXES TABLE_ID1417INNODB_SYS_INDEXES TABLE_ID
1416INNODB_BUFFER_POOL_PAGES_BLOB space_id1418INNODB_BUFFER_POOL_PAGES_BLOB space_id
1417INNODB_CMPMEM_RESET page_size1419INNODB_CMPMEM_RESET page_size
1420INNODB_LOCKS lock_id
1418SELECT MAX(table_name) FROM information_schema.tables WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test');1421SELECT MAX(table_name) FROM information_schema.tables WHERE table_schema IN ('mysql', 'INFORMATION_SCHEMA', 'test');
1419MAX(table_name)1422MAX(table_name)
1420XTRADB_ADMIN_COMMAND1423XTRADB_ADMIN_COMMAND
@@ -1497,6 +1500,7 @@
1497INNODB_BUFFER_POOL_PAGES information_schema.INNODB_BUFFER_POOL_PAGES 11500INNODB_BUFFER_POOL_PAGES information_schema.INNODB_BUFFER_POOL_PAGES 1
1498INNODB_BUFFER_POOL_PAGES_BLOB information_schema.INNODB_BUFFER_POOL_PAGES_BLOB 11501INNODB_BUFFER_POOL_PAGES_BLOB information_schema.INNODB_BUFFER_POOL_PAGES_BLOB 1
1499INNODB_BUFFER_POOL_PAGES_INDEX information_schema.INNODB_BUFFER_POOL_PAGES_INDEX 11502INNODB_BUFFER_POOL_PAGES_INDEX information_schema.INNODB_BUFFER_POOL_PAGES_INDEX 1
1503INNODB_CHANGED_PAGES information_schema.INNODB_CHANGED_PAGES 1
1500INNODB_CMP information_schema.INNODB_CMP 11504INNODB_CMP information_schema.INNODB_CMP 1
1501INNODB_CMPMEM information_schema.INNODB_CMPMEM 11505INNODB_CMPMEM information_schema.INNODB_CMPMEM 1
1502INNODB_CMPMEM_RESET information_schema.INNODB_CMPMEM_RESET 11506INNODB_CMPMEM_RESET information_schema.INNODB_CMPMEM_RESET 1
15031507
=== modified file 'Percona-Server/mysql-test/r/information_schema_db.result'
--- Percona-Server/mysql-test/r/information_schema_db.result 2012-06-14 05:39:43 +0000
+++ Percona-Server/mysql-test/r/information_schema_db.result 2012-08-15 14:14:06 +0000
@@ -43,7 +43,7 @@
43VIEWS43VIEWS
44INNODB_BUFFER_POOL_PAGES_INDEX44INNODB_BUFFER_POOL_PAGES_INDEX
45INNODB_RSEG45INNODB_RSEG
46INNODB_LOCKS46INNODB_CHANGED_PAGES
47INNODB_BUFFER_POOL_PAGES47INNODB_BUFFER_POOL_PAGES
48INNODB_TABLE_STATS48INNODB_TABLE_STATS
49INNODB_TRX49INNODB_TRX
@@ -58,6 +58,7 @@
58INNODB_SYS_INDEXES58INNODB_SYS_INDEXES
59INNODB_BUFFER_POOL_PAGES_BLOB59INNODB_BUFFER_POOL_PAGES_BLOB
60INNODB_CMPMEM_RESET60INNODB_CMPMEM_RESET
61INNODB_LOCKS
61show tables from INFORMATION_SCHEMA like 'T%';62show tables from INFORMATION_SCHEMA like 'T%';
62Tables_in_information_schema (T%)63Tables_in_information_schema (T%)
63TABLES64TABLES
6465
=== modified file 'Percona-Server/mysql-test/r/mysqlshow.result'
--- Percona-Server/mysql-test/r/mysqlshow.result 2012-06-14 05:39:43 +0000
+++ Percona-Server/mysql-test/r/mysqlshow.result 2012-08-15 14:14:06 +0000
@@ -117,7 +117,7 @@
117| VIEWS |117| VIEWS |
118| INNODB_BUFFER_POOL_PAGES_INDEX |118| INNODB_BUFFER_POOL_PAGES_INDEX |
119| INNODB_RSEG |119| INNODB_RSEG |
120| INNODB_LOCKS |120| INNODB_CHANGED_PAGES |
121| INNODB_BUFFER_POOL_PAGES |121| INNODB_BUFFER_POOL_PAGES |
122| INNODB_TABLE_STATS |122| INNODB_TABLE_STATS |
123| INNODB_TRX |123| INNODB_TRX |
@@ -132,6 +132,7 @@
132| INNODB_SYS_INDEXES |132| INNODB_SYS_INDEXES |
133| INNODB_BUFFER_POOL_PAGES_BLOB |133| INNODB_BUFFER_POOL_PAGES_BLOB |
134| INNODB_CMPMEM_RESET |134| INNODB_CMPMEM_RESET |
135| INNODB_LOCKS |
135+---------------------------------------+136+---------------------------------------+
136Database: INFORMATION_SCHEMA137Database: INFORMATION_SCHEMA
137+---------------------------------------+138+---------------------------------------+
@@ -175,7 +176,7 @@
175| VIEWS |176| VIEWS |
176| INNODB_BUFFER_POOL_PAGES_INDEX |177| INNODB_BUFFER_POOL_PAGES_INDEX |
177| INNODB_RSEG |178| INNODB_RSEG |
178| INNODB_LOCKS |179| INNODB_CHANGED_PAGES |
179| INNODB_BUFFER_POOL_PAGES |180| INNODB_BUFFER_POOL_PAGES |
180| INNODB_TABLE_STATS |181| INNODB_TABLE_STATS |
181| INNODB_TRX |182| INNODB_TRX |
@@ -190,6 +191,7 @@
190| INNODB_SYS_INDEXES |191| INNODB_SYS_INDEXES |
191| INNODB_BUFFER_POOL_PAGES_BLOB |192| INNODB_BUFFER_POOL_PAGES_BLOB |
192| INNODB_CMPMEM_RESET |193| INNODB_CMPMEM_RESET |
194| INNODB_LOCKS |
193+---------------------------------------+195+---------------------------------------+
194Wildcard: inf_rmation_schema196Wildcard: inf_rmation_schema
195+--------------------+197+--------------------+
196198
=== modified file 'Percona-Server/mysql-test/r/percona_server_variables_debug.result'
--- Percona-Server/mysql-test/r/percona_server_variables_debug.result 2012-06-14 09:16:03 +0000
+++ Percona-Server/mysql-test/r/percona_server_variables_debug.result 2012-08-15 14:14:06 +0000
@@ -86,6 +86,7 @@
86INNODB_BUFFER_POOL_SHM_CHECKSUM86INNODB_BUFFER_POOL_SHM_CHECKSUM
87INNODB_BUFFER_POOL_SHM_KEY87INNODB_BUFFER_POOL_SHM_KEY
88INNODB_BUFFER_POOL_SIZE88INNODB_BUFFER_POOL_SIZE
89INNODB_CHANGED_PAGES_LIMIT
89INNODB_CHANGE_BUFFERING90INNODB_CHANGE_BUFFERING
90INNODB_CHECKPOINT_AGE_TARGET91INNODB_CHECKPOINT_AGE_TARGET
91INNODB_CHECKSUMS92INNODB_CHECKSUMS
9293
=== modified file 'Percona-Server/mysql-test/r/percona_server_variables_release.result'
--- Percona-Server/mysql-test/r/percona_server_variables_release.result 2012-06-14 09:16:03 +0000
+++ Percona-Server/mysql-test/r/percona_server_variables_release.result 2012-08-15 14:14:06 +0000
@@ -84,6 +84,7 @@
84INNODB_BUFFER_POOL_SHM_CHECKSUM84INNODB_BUFFER_POOL_SHM_CHECKSUM
85INNODB_BUFFER_POOL_SHM_KEY85INNODB_BUFFER_POOL_SHM_KEY
86INNODB_BUFFER_POOL_SIZE86INNODB_BUFFER_POOL_SIZE
87INNODB_CHANGED_PAGES_LIMIT
87INNODB_CHANGE_BUFFERING88INNODB_CHANGE_BUFFERING
88INNODB_CHECKPOINT_AGE_TARGET89INNODB_CHECKPOINT_AGE_TARGET
89INNODB_CHECKSUMS90INNODB_CHECKSUMS
9091
=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result'
--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages.result 2012-08-15 14:14:06 +0000
@@ -0,0 +1,41 @@
1SELECT @@innodb_track_changed_pages;
2@@innodb_track_changed_pages
31
4SET @OLD_INNODB_CHANGED_PAGES_LIMIT = @@GLOBAL.INNODB_CHANGED_PAGES_LIMIT;
5SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
6CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB;
7SET @t1_space_id =
8(SELECT SPACE
9FROM INFORMATION_SCHEMA.INNODB_SYS_TABLES
10WHERE
11INFORMATION_SCHEMA.INNODB_SYS_TABLES.SCHEMA='test' AND
12INFORMATION_SCHEMA.INNODB_SYS_TABLES.NAME='T1');
13SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
14SELECT COUNT(DISTINCT PAGE_ID) >= (1024000/16384)
15FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
16WHERE SPACE_ID = @t1_space_id;
17COUNT(DISTINCT PAGE_ID) >= (1024000/16384)
181
19SELECT MAX(PAGE_ID) < (3*1024000/16384)
20FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
21WHERE SPACE_ID = @t1_space_id;
22MAX(PAGE_ID) < (3*1024000/16384)
231
24SELECT COUNT(*)
25FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
26WHERE START_LSN >= END_LSN;
27COUNT(*)
280
29SELECT COUNT(*) = @cond_test_pages_count
30FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
31WHERE
32END_LSN > (@cond_test_lsn - 1) AND
33END_LSN < (@cond_test_lsn + 1);
34COUNT(*) = @cond_test_pages_count
351
36SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 1;
37SELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
38COUNT(*)
391
40SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = @OLD_INNODB_CHANGED_PAGES_LIMIT;
41DROP TABLE T1;
042
=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result'
--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result 1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_changed_pages_empty.result 2012-08-15 14:14:06 +0000
@@ -0,0 +1,2 @@
1SELECT * FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
2space_id page_id start_lsn end_lsn
03
=== modified file 'Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result'
--- Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result 2012-06-15 05:57:51 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/r/percona_skip_innodb_i_s.result 2012-08-15 14:14:06 +0000
@@ -3,6 +3,7 @@
3INNODB_BUFFER_POOL_PAGES3INNODB_BUFFER_POOL_PAGES
4INNODB_BUFFER_POOL_PAGES_BLOB4INNODB_BUFFER_POOL_PAGES_BLOB
5INNODB_BUFFER_POOL_PAGES_INDEX5INNODB_BUFFER_POOL_PAGES_INDEX
6INNODB_CHANGED_PAGES
6INNODB_CMP7INNODB_CMP
7INNODB_CMPMEM8INNODB_CMPMEM
8INNODB_CMPMEM_RESET9INNODB_CMPMEM_RESET
910
=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt'
--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt 1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages-master.opt 2012-08-15 14:14:06 +0000
@@ -0,0 +1,1 @@
1--innodb_track_changed_pages=TRUE --innodb_log_file_size=1M --innodb-file-per-table
02
=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test'
--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages.test 2012-08-15 14:14:06 +0000
@@ -0,0 +1,103 @@
1###########################################
2# Test for I_S.INNODB_CHANGED_PAGES table #
3###########################################
4
5--source include/have_innodb_plugin.inc
6
7SELECT @@innodb_track_changed_pages;
8
9SET @OLD_INNODB_CHANGED_PAGES_LIMIT = @@GLOBAL.INNODB_CHANGED_PAGES_LIMIT;
10
11SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
12
13CREATE TABLE T1 (F1 CHAR(255)) ENGINE=INNODB;
14SET @t1_space_id =
15 (SELECT SPACE
16 FROM INFORMATION_SCHEMA.INNODB_SYS_TABLES
17 WHERE
18 INFORMATION_SCHEMA.INNODB_SYS_TABLES.SCHEMA='test' AND
19 INFORMATION_SCHEMA.INNODB_SYS_TABLES.NAME='T1');
20
21--disable_query_log
22--disable_result_log
23
24#########################################################################
25# The maximum log size is 1MB. Each row occupies at least 256 bytes. #
26# Each iteration inserts 100 rows. They occupies at least 25KB. To be #
27# sure that tracking log thread wrote at least 1M/16K pages we need to #
28# exceed maximum log size twice. That means we should do at least #
29# 2M/25K = 80 iterations. #
30#########################################################################
31--let $i=80
32
33while ($i)
34{
35--dec $i
36INSERT INTO T1 (F1) VALUES
37("1"), ("2"), ("3"), ("4"), ("5"), ("6"), ("7"), ("8"), ("9"), ("10"),
38("11"), ("12"), ("13"), ("14"), ("15"), ("16"), ("17"), ("18"), ("19"), ("20"),
39("21"), ("22"), ("23"), ("24"), ("25"), ("26"), ("27"), ("28"), ("29"), ("30"),
40("31"), ("32"), ("33"), ("34"), ("35"), ("36"), ("37"), ("38"), ("39"), ("40"),
41("41"), ("42"), ("43"), ("44"), ("45"), ("46"), ("47"), ("48"), ("49"), ("50"),
42("51"), ("52"), ("53"), ("54"), ("55"), ("56"), ("57"), ("58"), ("59"), ("60"),
43("61"), ("62"), ("63"), ("64"), ("65"), ("66"), ("67"), ("68"), ("69"), ("70"),
44("71"), ("72"), ("73"), ("74"), ("75"), ("76"), ("77"), ("78"), ("79"), ("80"),
45("81"), ("82"), ("83"), ("84"), ("85"), ("86"), ("87"), ("88"), ("89"), ("90"),
46("91"), ("92"), ("93"), ("94"), ("95"), ("96"), ("97"), ("98"), ("99"), ("100");
47}
48
49###################################################################
50# Gather data for condition pushdown testing not using conditions #
51###################################################################
52SET @cond_test_lsn=
53 (SELECT MIN(end_lsn)
54 FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES);
55
56SET @cond_test_pages_count=
57 (SELECT count(*)
58 FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
59 GROUP BY end_lsn
60 ORDER BY end_lsn
61 LIMIT 1);
62
63--enable_query_log
64--enable_result_log
65
66###############################################################
67# Check if the number of changed pages is greater than 1M/16K #
68###############################################################
69SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 0;
70SELECT COUNT(DISTINCT PAGE_ID) >= (1024000/16384)
71 FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
72 WHERE SPACE_ID = @t1_space_id;
73#############################################################
74# Check if the maximum page id is less than resonable limit #
75#############################################################
76SELECT MAX(PAGE_ID) < (3*1024000/16384)
77 FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
78 WHERE SPACE_ID = @t1_space_id;
79########################################################
80# The records where START_LSN >= END_LSN should absent #
81########################################################
82SELECT COUNT(*)
83 FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
84 WHERE START_LSN >= END_LSN;
85
86######################################################
87# Check condition if pushdown doesn't break anything #
88######################################################
89SELECT COUNT(*) = @cond_test_pages_count
90 FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
91 WHERE
92 END_LSN > (@cond_test_lsn - 1) AND
93 END_LSN < (@cond_test_lsn + 1);
94
95################################################
96# Check how limit for maximum rows count works #
97################################################
98SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = 1;
99SELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
100
101SET GLOBAL INNODB_CHANGED_PAGES_LIMIT = @OLD_INNODB_CHANGED_PAGES_LIMIT;
102
103DROP TABLE T1;
0104
=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt'
--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt 1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty-master.opt 2012-08-15 14:14:06 +0000
@@ -0,0 +1,1 @@
1--innodb_track_changed_pages=false
02
=== added file 'Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test'
--- Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test 1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/innodb_plugin/t/percona_changed_pages_empty.test 2012-08-15 14:14:06 +0000
@@ -0,0 +1,8 @@
1###############################################################################
2# Test for empty I_S.INNODB_CHANGED_PAGES table. The table should be empty if#
3# innodb_track_changed_pages is false. #
4###############################################################################
5
6--source include/have_innodb_plugin.inc
7
8SELECT * FROM INFORMATION_SCHEMA.INNODB_CHANGED_PAGES;
09
=== modified file 'Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc'
--- Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2012-07-02 02:04:45 +0000
+++ Percona-Server/storage/innodb_plugin/handler/ha_innodb.cc 2012-08-15 14:14:06 +0000
@@ -11934,6 +11934,13 @@
11934 "Track the redo log for changed pages and output a changed page bitmap",11934 "Track the redo log for changed pages and output a changed page bitmap",
11935 NULL, NULL, FALSE);11935 NULL, NULL, FALSE);
1193611936
11937static MYSQL_SYSVAR_ULONGLONG(changed_pages_limit, srv_changed_pages_limit,
11938 PLUGIN_VAR_RQCMDARG,
11939 "The maximum number of rows for "
11940 "INFORMATION_SCHEMA.INNODB_CHANGED_PAGES table, "
11941 "0 - unlimited",
11942 NULL, NULL, 1000000, 0, ~0ULL, 0);
11943
11937#if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG11944#if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG
11938static MYSQL_SYSVAR_UINT(change_buffering_debug, ibuf_debug,11945static MYSQL_SYSVAR_UINT(change_buffering_debug, ibuf_debug,
11939 PLUGIN_VAR_RQCMDARG,11946 PLUGIN_VAR_RQCMDARG,
@@ -12181,6 +12188,7 @@
12181 MYSQL_SYSVAR(use_sys_malloc),12188 MYSQL_SYSVAR(use_sys_malloc),
12182 MYSQL_SYSVAR(change_buffering),12189 MYSQL_SYSVAR(change_buffering),
12183 MYSQL_SYSVAR(track_changed_pages),12190 MYSQL_SYSVAR(track_changed_pages),
12191 MYSQL_SYSVAR(changed_pages_limit),
12184#if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG12192#if defined UNIV_DEBUG || defined UNIV_IBUF_DEBUG
12185 MYSQL_SYSVAR(change_buffering_debug),12193 MYSQL_SYSVAR(change_buffering_debug),
12186#endif /* UNIV_DEBUG || UNIV_IBUF_DEBUG */12194#endif /* UNIV_DEBUG || UNIV_IBUF_DEBUG */
@@ -12230,7 +12238,8 @@
12230i_s_innodb_admin_command,12238i_s_innodb_admin_command,
12231i_s_innodb_sys_tables,12239i_s_innodb_sys_tables,
12232i_s_innodb_sys_indexes,12240i_s_innodb_sys_indexes,
12233i_s_innodb_sys_stats12241i_s_innodb_sys_stats,
12242i_s_innodb_changed_pages
12234mysql_declare_plugin_end;12243mysql_declare_plugin_end;
1223512244
12236/** @brief Initialize the default value of innodb_commit_concurrency.12245/** @brief Initialize the default value of innodb_commit_concurrency.
1223712246
=== modified file 'Percona-Server/storage/innodb_plugin/handler/i_s.cc'
--- Percona-Server/storage/innodb_plugin/handler/i_s.cc 2012-06-15 05:57:51 +0000
+++ Percona-Server/storage/innodb_plugin/handler/i_s.cc 2012-08-15 14:14:06 +0000
@@ -22,8 +22,15 @@
2222
23Created July 18, 2007 Vasil Dimov23Created July 18, 2007 Vasil Dimov
24*******************************************************/24*******************************************************/
25#ifndef MYSQL_SERVER
26#define MYSQL_SERVER /* For Item_* classes */
27#include <mysql_priv.h>
28/* Prevent influence of this definition to other headers */
29#undef MYSQL_SERVER
30#else
31#include <mysql_priv.h>
32#endif //MYSQL_SERVER
2533
26#include <mysql_priv.h>
27#include <mysqld_error.h>34#include <mysqld_error.h>
2835
29#include <m_ctype.h>36#include <m_ctype.h>
@@ -47,6 +54,7 @@
47#include "dict0dict.h" /* for dict_sys */54#include "dict0dict.h" /* for dict_sys */
48#include "btr0pcur.h"55#include "btr0pcur.h"
49#include "buf0lru.h" /* for XTRA_LRU_[DUMP/RESTORE] */56#include "buf0lru.h" /* for XTRA_LRU_[DUMP/RESTORE] */
57#include "log0online.h"
50}58}
5159
52static const char plugin_author[] = "Innobase Oy";60static const char plugin_author[] = "Innobase Oy";
@@ -3639,3 +3647,288 @@
3639 STRUCT_FLD(system_vars, NULL),3647 STRUCT_FLD(system_vars, NULL),
3640 STRUCT_FLD(__reserved1, NULL)3648 STRUCT_FLD(__reserved1, NULL)
3641};3649};
3650
3651static ST_FIELD_INFO i_s_innodb_changed_pages_info[] =
3652{
3653 {STRUCT_FLD(field_name, "space_id"),
3654 STRUCT_FLD(field_length, MY_INT32_NUM_DECIMAL_DIGITS),
3655 STRUCT_FLD(field_type, MYSQL_TYPE_LONG),
3656 STRUCT_FLD(value, 0),
3657 STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
3658 STRUCT_FLD(old_name, ""),
3659 STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
3660
3661 {STRUCT_FLD(field_name, "page_id"),
3662 STRUCT_FLD(field_length, MY_INT32_NUM_DECIMAL_DIGITS),
3663 STRUCT_FLD(field_type, MYSQL_TYPE_LONG),
3664 STRUCT_FLD(value, 0),
3665 STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
3666 STRUCT_FLD(old_name, ""),
3667 STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
3668
3669 {STRUCT_FLD(field_name, "start_lsn"),
3670 STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS),
3671 STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG),
3672 STRUCT_FLD(value, 0),
3673 STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
3674 STRUCT_FLD(old_name, ""),
3675 STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
3676
3677 {STRUCT_FLD(field_name, "end_lsn"),
3678 STRUCT_FLD(field_length, MY_INT64_NUM_DECIMAL_DIGITS),
3679 STRUCT_FLD(field_type, MYSQL_TYPE_LONGLONG),
3680 STRUCT_FLD(value, 0),
3681 STRUCT_FLD(field_flags, MY_I_S_UNSIGNED),
3682 STRUCT_FLD(old_name, ""),
3683 STRUCT_FLD(open_method, SKIP_OPEN_TABLE)},
3684
3685 END_OF_ST_FIELD_INFO
3686};
3687
3688/***********************************************************************
3689 This function parses condition and gets upper bounds for start and end LSN's
3690 if condition corresponds to certain pattern.
3691
3692 We can't know right position to avoid scanning bitmap files from the beginning
3693 to the lower bound. But we can stop scanning bitmap files if we reach upper bound.
3694
3695 It's expected the most used queries will be like the following:
3696
3697 SELECT * FROM INNODB_CHANGED_PAGES WHERE START_LSN > num1 AND start_lsn < num2;
3698
3699 That's why the pattern is:
3700
3701 pattern: comp | and_comp;
3702 comp: lsn < int_num | lsn <= int_num | int_num > lsn | int_num >= lsn;
3703 lsn: start_lsn | end_lsn;
3704 and_comp: some_expression AND some_expression | some_expression AND and_comp;
3705 some_expression: comp | any_other_expression;
3706
3707 Suppose the condition is start_lsn < 100, this means we have to read all
3708 blocks with start_lsn < 100. Which is equivalent to reading all the blocks
3709 with end_lsn <= 99, or just end_lsn < 100. That's why it's enough to find
3710 maximum lsn value, doesn't matter if this is start or end lsn and compare
3711 it with "start_lsn" field.
3712
3713 Example:
3714
3715 SELECT * FROM INNODB_CHANGED_PAGES
3716 WHERE
3717 start_lsn > 10 AND
3718 end_lsn <= 1111 AND
3719 555 > end_lsn AND
3720 page_id = 100;
3721
3722 max_lsn will be set to 555.
3723*/
3724static
3725void
3726limit_lsn_range_from_condition(
3727/*===========================*/
3728 TABLE* table, /*!<in: table */
3729 COND* cond, /*!<in: condition */
3730 ib_uint64_t* max_lsn) /*!<in/out: maximum LSN
3731 (must be initialized with maximum
3732 available value) */
3733{
3734 if (cond->type() != Item::COND_ITEM &&
3735 cond->type() != Item::FUNC_ITEM)
3736 return;
3737
3738 switch (((Item_func*) cond)->functype())
3739 {
3740 case Item_func::COND_AND_FUNC:
3741 {
3742 List_iterator<Item> li(*((Item_cond*) cond)->
3743 argument_list());
3744 Item *item;
3745 while ((item= li++))
3746 limit_lsn_range_from_condition(table,
3747 item,
3748 max_lsn);
3749 break;
3750 }
3751 case Item_func::LT_FUNC:
3752 case Item_func::LE_FUNC:
3753 case Item_func::GT_FUNC:
3754 case Item_func::GE_FUNC:
3755 {
3756 Item *left;
3757 Item *right;
3758 Item_field *item_field;
3759 ib_uint64_t tmp_result;
3760
3761 /*
3762 a <= b equals to b >= a that's why we just exchange
3763 "left" and "right" in the case of ">" or ">="
3764 function
3765 */
3766 if (((Item_func*) cond)->functype() ==
3767 Item_func::LT_FUNC ||
3768 ((Item_func*) cond)->functype() ==
3769 Item_func::LE_FUNC)
3770 {
3771 left = ((Item_func*) cond)->arguments()[0];
3772 right = ((Item_func*) cond)->arguments()[1];
3773 } else {
3774 left = ((Item_func*) cond)->arguments()[1];
3775 right = ((Item_func*) cond)->arguments()[0];
3776 }
3777
3778 if (!left || !right)
3779 return;
3780 if (left->type() != Item::FIELD_ITEM)
3781 return;
3782 if (right->type() != Item::INT_ITEM)
3783 return;
3784
3785 item_field = (Item_field*)left;
3786
3787 if (/* START_LSN */
3788 table->field[2] != item_field->field &&
3789 /* END_LSN */
3790 table->field[3] != item_field->field)
3791 {
3792 return;
3793 }
3794
3795 /* Check if the current field belongs to our table */
3796 if (table != item_field->field->table)
3797 return;
3798
3799 tmp_result = right->val_int();
3800 if (tmp_result < *max_lsn)
3801 *max_lsn = tmp_result;
3802
3803 break;
3804 }
3805 default:;
3806 }
3807
3808}
3809
3810/***********************************************************************
3811Fill the dynamic table information_schema.innodb_changed_pages.
3812@return 0 on success, 1 on failure */
3813static
3814int
3815i_s_innodb_changed_pages_fill(
3816/*==========================*/
3817 THD* thd, /*!<in: thread */
3818 TABLE_LIST* tables, /*!<in/out: tables to fill */
3819 COND* cond) /*!<in: condition */
3820{
3821 TABLE* table = (TABLE *) tables->table;
3822 log_bitmap_iterator_t i;
3823 ib_uint64_t output_rows_num = 0UL;
3824 ib_uint64_t max_lsn = ~0ULL;
3825
3826 if (!srv_track_changed_pages)
3827 return 0;
3828
3829 if (!log_online_bitmap_iterator_init(&i))
3830 return 1;
3831
3832 if (cond)
3833 limit_lsn_range_from_condition(table, cond, &max_lsn);
3834
3835 while(log_online_bitmap_iterator_next(&i) &&
3836 (!srv_changed_pages_limit ||
3837 output_rows_num < srv_changed_pages_limit) &&
3838 /*
3839 There is no need to compare both start LSN and end LSN fields
3840 with maximum value. It's enough to compare only start LSN.
3841 Example:
3842
3843 max_lsn = 100
3844 \\\\\\\\\\\\\\\\\\\\\\\\\|\\\\\\\\ - Query 1
3845 I------I I-------I I-------------I I----I
3846 ////////////////// | - Query 2
3847 1 2 3 4
3848
3849 Query 1:
3850 SELECT * FROM INNODB_CHANGED_PAGES WHERE start_lsn < 100
3851 will select 1,2,3 bitmaps
3852 Query 2:
3853 SELECT * FROM INNODB_CHANGED_PAGES WHERE end_lsn < 100
3854 will select 1,2 bitmaps
3855
3856 The condition start_lsn <= 100 will be false after reading
3857 1,2,3 bitmaps which suits for both cases.
3858 */
3859 LOG_BITMAP_ITERATOR_START_LSN(i) <= max_lsn)
3860 {
3861 if (!LOG_BITMAP_ITERATOR_PAGE_CHANGED(i))
3862 continue;
3863
3864 /* SPACE_ID */
3865 table->field[0]->store(
3866 LOG_BITMAP_ITERATOR_SPACE_ID(i));
3867 /* PAGE_ID */
3868 table->field[1]->store(
3869 LOG_BITMAP_ITERATOR_PAGE_NUM(i));
3870 /* START_LSN */
3871 table->field[2]->store(
3872 LOG_BITMAP_ITERATOR_START_LSN(i));
3873 /* END_LSN */
3874 table->field[3]->store(
3875 LOG_BITMAP_ITERATOR_END_LSN(i));
3876
3877 /*
3878 I_S tables are in-memory tables. If bitmap file is big enough
3879 a lot of memory can be used to store the table. But the size
3880 of used memory can be diminished if we store only data which
3881 corresponds to some conditions (in WHERE sql clause). Here
3882 conditions are checked for the field values stored above.
3883
3884 Conditions are checked twice. The first is here (during table
3885 generation) and the second during query execution. Maybe it
3886 makes sense to use some flag in THD object to avoid double
3887 checking.
3888 */
3889 if (cond && !cond->val_int())
3890 continue;
3891
3892 if (schema_table_store_record(thd, table))
3893 {
3894 log_online_bitmap_iterator_release(&i);
3895 return 1;
3896 }
3897
3898 ++output_rows_num;
3899 }
3900
3901 log_online_bitmap_iterator_release(&i);
3902 return 0;
3903}
3904
3905static
3906int
3907i_s_innodb_changed_pages_init(
3908/*==========================*/
3909 void* p)
3910{
3911 DBUG_ENTER("i_s_innodb_changed_pages_init");
3912 ST_SCHEMA_TABLE* schema = (ST_SCHEMA_TABLE*) p;
3913
3914 schema->fields_info = i_s_innodb_changed_pages_info;
3915 schema->fill_table = i_s_innodb_changed_pages_fill;
3916
3917 DBUG_RETURN(0);
3918}
3919
3920UNIV_INTERN struct st_mysql_plugin i_s_innodb_changed_pages =
3921{
3922 STRUCT_FLD(type, MYSQL_INFORMATION_SCHEMA_PLUGIN),
3923 STRUCT_FLD(info, &i_s_info),
3924 STRUCT_FLD(name, "INNODB_CHANGED_PAGES"),
3925 STRUCT_FLD(author, "Percona"),
3926 STRUCT_FLD(descr, "InnoDB CHANGED_PAGES table"),
3927 STRUCT_FLD(license, PLUGIN_LICENSE_GPL),
3928 STRUCT_FLD(init, i_s_innodb_changed_pages_init),
3929 STRUCT_FLD(deinit, i_s_common_deinit),
3930 STRUCT_FLD(version, 0x0100 /* 1.0 */),
3931 STRUCT_FLD(status_vars, NULL),
3932 STRUCT_FLD(system_vars, NULL),
3933 STRUCT_FLD(__reserved1, NULL)
3934};
36423935
=== modified file 'Percona-Server/storage/innodb_plugin/handler/i_s.h'
--- Percona-Server/storage/innodb_plugin/handler/i_s.h 2012-06-14 05:39:43 +0000
+++ Percona-Server/storage/innodb_plugin/handler/i_s.h 2012-08-15 14:14:06 +0000
@@ -43,5 +43,6 @@
43extern struct st_mysql_plugin i_s_innodb_sys_tables;43extern struct st_mysql_plugin i_s_innodb_sys_tables;
44extern struct st_mysql_plugin i_s_innodb_sys_indexes;44extern struct st_mysql_plugin i_s_innodb_sys_indexes;
45extern struct st_mysql_plugin i_s_innodb_sys_stats;45extern struct st_mysql_plugin i_s_innodb_sys_stats;
46extern struct st_mysql_plugin i_s_innodb_changed_pages;
4647
47#endif /* i_s_h */48#endif /* i_s_h */
4849
=== modified file 'Percona-Server/storage/innodb_plugin/include/log0online.h'
--- Percona-Server/storage/innodb_plugin/include/log0online.h 2012-06-14 09:16:03 +0000
+++ Percona-Server/storage/innodb_plugin/include/log0online.h 2012-08-15 14:14:06 +0000
@@ -25,6 +25,7 @@
25#define log0online_h25#define log0online_h
2626
27#include "univ.i"27#include "univ.i"
28#include "os0file.h"
2829
29/*********************************************************************//**30/*********************************************************************//**
30Initializes the online log following subsytem. */31Initializes the online log following subsytem. */
@@ -48,4 +49,63 @@
48log_online_follow_redo_log();49log_online_follow_redo_log();
49/*=========================*/50/*=========================*/
5051
52/** The iterator through all bits of changed pages bitmap blocks */
53struct log_bitmap_iterator_struct
54{
55 char in_name[FN_REFLEN]; /*!< the file name for bitmap
56 input */
57 os_file_t in; /*!< the bitmap input file */
58 ib_uint64_t in_offset; /*!< the next write position in the
59 bitmap output file */
60 ib_uint32_t bit_offset; /*!< bit offset inside of bitmap
61 block*/
62 ib_uint64_t start_lsn; /*!< Start lsn of the block */
63 ib_uint64_t end_lsn; /*!< End lsn of the block */
64 ib_uint32_t space_id; /*!< Block space id */
65 ib_uint32_t first_page_id; /*!< First block page id */
66 ibool changed; /*!< true if current page was changed */
67 byte* page; /*!< Bitmap block */
68};
69
70typedef struct log_bitmap_iterator_struct log_bitmap_iterator_t;
71
72#define LOG_BITMAP_ITERATOR_START_LSN(i) \
73 ((i).start_lsn)
74#define LOG_BITMAP_ITERATOR_END_LSN(i) \
75 ((i).end_lsn)
76#define LOG_BITMAP_ITERATOR_SPACE_ID(i) \
77 ((i).space_id)
78#define LOG_BITMAP_ITERATOR_PAGE_NUM(i) \
79 ((i).first_page_id + (i).bit_offset)
80#define LOG_BITMAP_ITERATOR_PAGE_CHANGED(i) \
81 ((i).changed)
82
83/*********************************************************************//**
84Initializes log bitmap iterator.
85@return TRUE if the iterator is initialized OK, FALSE otherwise. */
86UNIV_INTERN
87ibool
88log_online_bitmap_iterator_init(
89/*============================*/
90 log_bitmap_iterator_t *i); /*!<in/out: iterator */
91
92/*********************************************************************//**
93Releases log bitmap iterator. */
94UNIV_INTERN
95void
96log_online_bitmap_iterator_release(
97/*===============================*/
98 log_bitmap_iterator_t *i); /*!<in/out: iterator */
99
100/*********************************************************************//**
101Iterates through bits of saved bitmap blocks.
102Sequentially reads blocks from bitmap file(s) and interates through
103their bits. Ignores blocks with wrong checksum.
104@return TRUE if iteration is successful, FALSE if all bits are iterated. */
105UNIV_INTERN
106ibool
107log_online_bitmap_iterator_next(
108/*============================*/
109 log_bitmap_iterator_t *i); /*!<in/out: iterator */
110
51#endif111#endif
52112
=== modified file 'Percona-Server/storage/innodb_plugin/include/srv0srv.h'
--- Percona-Server/storage/innodb_plugin/include/srv0srv.h 2012-06-14 09:16:03 +0000
+++ Percona-Server/storage/innodb_plugin/include/srv0srv.h 2012-08-15 14:14:06 +0000
@@ -136,6 +136,9 @@
136136
137extern my_bool srv_track_changed_pages;137extern my_bool srv_track_changed_pages;
138138
139extern
140ulonglong srv_changed_pages_limit;
141
139extern ibool srv_auto_extend_last_data_file;142extern ibool srv_auto_extend_last_data_file;
140extern ulint srv_last_file_size_max;143extern ulint srv_last_file_size_max;
141extern char** srv_log_group_home_dirs;144extern char** srv_log_group_home_dirs;
142145
=== modified file 'Percona-Server/storage/innodb_plugin/log/log0online.c'
--- Percona-Server/storage/innodb_plugin/log/log0online.c 2012-06-14 09:16:03 +0000
+++ Percona-Server/storage/innodb_plugin/log/log0online.c 2012-08-15 14:14:06 +0000
@@ -29,7 +29,6 @@
29#include "log0recv.h"29#include "log0recv.h"
30#include "mach0data.h"30#include "mach0data.h"
31#include "mtr0log.h"31#include "mtr0log.h"
32#include "os0file.h"
33#include "srv0srv.h"32#include "srv0srv.h"
34#include "srv0start.h"33#include "srv0start.h"
35#include "trx0sys.h"34#include "trx0sys.h"
@@ -84,6 +83,10 @@
84which case the first LSN of actual log records will be this. */83which case the first LSN of actual log records will be this. */
85#define MIN_TRACKED_LSN ((LOG_START_LSN) + (LOG_BLOCK_HDR_SIZE))84#define MIN_TRACKED_LSN ((LOG_START_LSN) + (LOG_BLOCK_HDR_SIZE))
8685
86/* Tests if num bit of bitmap is set */
87#define IS_BIT_SET(bitmap, num) \
88 (*((bitmap) + ((num) >> 3)) & (1UL << ((num) & 7UL)))
89
87/** The bitmap file block size in bytes. All writes will be multiples of this.90/** The bitmap file block size in bytes. All writes will be multiples of this.
88 */91 */
89enum {92enum {
@@ -117,6 +120,7 @@
117enum { MODIFIED_PAGE_BLOCK_BITMAP_LEN120enum { MODIFIED_PAGE_BLOCK_BITMAP_LEN
118 = MODIFIED_PAGE_BLOCK_UNUSED_2 - MODIFIED_PAGE_BLOCK_BITMAP };121 = MODIFIED_PAGE_BLOCK_UNUSED_2 - MODIFIED_PAGE_BLOCK_BITMAP };
119122
123
120/****************************************************************//**124/****************************************************************//**
121Provide a comparisson function for the RB-tree tree (space,125Provide a comparisson function for the RB-tree tree (space,
122block_start_page) pairs. Actual implementation does not matter as126block_start_page) pairs. Actual implementation does not matter as
@@ -922,3 +926,158 @@
922 log_bmp_sys->start_lsn = log_bmp_sys->end_lsn;926 log_bmp_sys->start_lsn = log_bmp_sys->end_lsn;
923 log_set_tracked_lsn(log_bmp_sys->start_lsn);927 log_set_tracked_lsn(log_bmp_sys->start_lsn);
924}928}
929
930/*********************************************************************//**
931Initializes log bitmap iterator.
932@return TRUE if the iterator is initialized OK, FALSE otherwise. */
933UNIV_INTERN
934ibool
935log_online_bitmap_iterator_init(
936/*============================*/
937 log_bitmap_iterator_t *i) /*!<in/out: iterator */
938{
939 ibool success;
940
941 ut_a(i);
942 ut_snprintf(i->in_name, FN_REFLEN, "%s%s%d", srv_data_home,
943 modified_page_stem, 1);
944 i->in_offset = 0;
945 /*
946 Set up bit offset out of the reasonable limit
947 to intiate reading block from file in
948 log_online_bitmap_iterator_next()
949 */
950 i->bit_offset = MODIFIED_PAGE_BLOCK_BITMAP_LEN;
951 i->in =
952 os_file_create_simple_no_error_handling(
953 i->in_name,
954 OS_FILE_OPEN,
955 OS_FILE_READ_ONLY,
956 &success);
957
958 if (!success) {
959 /* The following call prints an error message */
960 os_file_get_last_error(TRUE);
961 fprintf(stderr,
962 "InnoDB: Error: Cannot open \'%s\'\n",
963 i->in_name);
964 return FALSE;
965 }
966
967 i->page = ut_malloc(MODIFIED_PAGE_BLOCK_SIZE);
968
969 i->start_lsn = i->end_lsn = 0;
970 i->space_id = 0;
971 i->first_page_id = 0;
972 i->changed = FALSE;
973
974 return TRUE;
975}
976
977/*********************************************************************//**
978Releases log bitmap iterator. */
979UNIV_INTERN
980void
981log_online_bitmap_iterator_release(
982/*===============================*/
983 log_bitmap_iterator_t *i) /*!<in/out: iterator */
984{
985 ut_a(i);
986 os_file_close(i->in);
987 ut_free(i->page);
988}
989
990/*********************************************************************//**
991Iterates through bits of saved bitmap blocks.
992Sequentially reads blocks from bitmap file(s) and interates through
993their bits. Ignores blocks with wrong checksum.
994@return TRUE if iteration is successful, FALSE if all bits are iterated. */
995UNIV_INTERN
996ibool
997log_online_bitmap_iterator_next(
998/*============================*/
999 log_bitmap_iterator_t *i) /*!<in/out: iterator */
1000{
1001 ulint offset_low;
1002 ulint offset_high;
1003 ulint size_low;
1004 ulint size_high;
1005 ulint checksum = 0;
1006 ulint actual_checksum = !checksum;
1007
1008 ibool success;
1009
1010 ut_a(i);
1011
1012 if (i->bit_offset < MODIFIED_PAGE_BLOCK_BITMAP_LEN)
1013 {
1014 ++i->bit_offset;
1015 i->changed =
1016 IS_BIT_SET(i->page + MODIFIED_PAGE_BLOCK_BITMAP,
1017 i->bit_offset);
1018 return TRUE;
1019 }
1020
1021 while (checksum != actual_checksum)
1022 {
1023 success = os_file_get_size(i->in,
1024 &size_low,
1025 &size_high);
1026 if (!success) {
1027 os_file_get_last_error(TRUE);
1028 fprintf(stderr,
1029 "InnoDB: Warning: can't get size of "
1030 "page bitmap file \'%s\'\n",
1031 i->in_name);
1032 return FALSE;
1033 }
1034
1035 if (i->in_offset >=
1036 (ib_uint64_t)(size_low) +
1037 ((ib_uint64_t)(size_high) << 32))
1038 return FALSE;
1039
1040 offset_high = (ulint)(i->in_offset >> 32);
1041 offset_low = (ulint)(i->in_offset & 0xFFFFFFFF);
1042
1043 success = os_file_read(
1044 i->in,
1045 i->page,
1046 offset_low,
1047 offset_high,
1048 MODIFIED_PAGE_BLOCK_SIZE);
1049
1050 if (!success) {
1051 os_file_get_last_error(TRUE);
1052 fprintf(stderr,
1053 "InnoDB: Warning: failed reading "
1054 "changed page bitmap file \'%s\'\n",
1055 i->in_name);
1056 return FALSE;
1057 }
1058
1059 checksum = mach_read_from_4(
1060 i->page + MODIFIED_PAGE_BLOCK_CHECKSUM);
1061
1062 actual_checksum = log_online_calc_checksum(i->page);
1063
1064 i->in_offset += MODIFIED_PAGE_BLOCK_SIZE;
1065 }
1066
1067 i->start_lsn =
1068 mach_read_ull(i->page + MODIFIED_PAGE_START_LSN);
1069 i->end_lsn =
1070 mach_read_ull(i->page + MODIFIED_PAGE_END_LSN);
1071 i->space_id =
1072 mach_read_from_4(i->page + MODIFIED_PAGE_SPACE_ID);
1073 i->first_page_id =
1074 mach_read_from_4(i->page + MODIFIED_PAGE_1ST_PAGE_ID);
1075 i->bit_offset =
1076 0;
1077 i->changed =
1078 IS_BIT_SET(i->page + MODIFIED_PAGE_BLOCK_BITMAP,
1079 i->bit_offset);
1080
1081 return TRUE;
1082}
1083
9251084
=== modified file 'Percona-Server/storage/innodb_plugin/srv/srv0srv.c'
--- Percona-Server/storage/innodb_plugin/srv/srv0srv.c 2012-06-14 09:16:03 +0000
+++ Percona-Server/storage/innodb_plugin/srv/srv0srv.c 2012-08-15 14:14:06 +0000
@@ -164,6 +164,8 @@
164164
165UNIV_INTERN my_bool srv_track_changed_pages = TRUE;165UNIV_INTERN my_bool srv_track_changed_pages = TRUE;
166166
167UNIV_INTERN ulonglong srv_changed_pages_limit = 0;
168
167/* if TRUE, then we auto-extend the last data file */169/* if TRUE, then we auto-extend the last data file */
168UNIV_INTERN ibool srv_auto_extend_last_data_file = FALSE;170UNIV_INTERN ibool srv_auto_extend_last_data_file = FALSE;
169/* if != 0, this tells the max size auto-extending may increase the171/* if != 0, this tells the max size auto-extending may increase the

Subscribers

People subscribed via source and target branches