Merge lp:~laurynas-biveinis/percona-server/BT-16274-bug1108613-5.5 into lp:percona-server/5.5

[Laurynas Biveinis]

Merge bug 1108613 fix.
BT 16274
http://jenkins.percona.com/job/percona-server-5.5-param/652/

[Laurynas Biveinis (laurynas-biveinis)]

Please ignore diff lines 38--63 and srv0start.c, or better yet, review the commit 415. The MP diff contains extra changes because I can set only one prerequisite branch.

[George Ormond Lorch III (gl-az)]

Laurynas, can you offer some detail how to recover from this situation if/when it occurs? If this happens I assume that the next attempt at an incremental backup will use a slow incremental. Is there any possibility of a race between XB and PS from the point of detecting srv_track_changed_pages=TRUE, hitting this error, which stops change tracking, then XB attempting to access/use the bitmap which is no longer being updated?

review: Needs Information (g2)

[Laurynas Biveinis (laurynas-biveinis)]

George -

The situation is not recoverable from the bitmap backup point of view for the overwritten LSNs. So it will have to use the slow incremental backup for that time.

I don't see a possibility for a race condition. XB uses the bitmaps only if it has the full data for the required LSN interval and is not concerned with the current tracking state or anything like that. So what would happen is that tracking stops with the resulting bitmap data not covering the desired interval, XB detects this, switches to slow incrementals.

[George Ormond Lorch III (gl-az)]

OK. By 'recovery' I mean making it clear how to get 'back on track' WRT incrementals and bitmap tracking. If it is possible to simply re-enable bitmap tracking after this error and a slow incremental is done, then we should make that process clear. I just want to make sure that if/when this is encountered, it is easily recognized and can be worked around to get the feature functioning again. If though, once this error occurs, one can _never_ use changed page bitmaps again, then I think we have a bigger issue and need to address it somehow.

review: Needs Information (g2)

[Laurynas Biveinis (laurynas-biveinis)]

The current limitation will be documented by Hrvoje and might be lifted by implementing https://blueprints.launchpad.net/percona-server/+spec/online-log-tracking-restart .

[George Ormond Lorch III (gl-az)]

OK, sounds good.

review: Approve (g2)

[Stewart Smith (stewart)]

review: Approve

=== modified file 'Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result'
--- Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result	2013-01-31 11:52:07 +0000
+++ Percona-Server/mysql-test/suite/innodb/r/percona_changed_page_bmp.result	2013-02-05 09:53:25 +0000
@@ -44,4 +44,16 @@
 ib_modified_log_1
 ib_modified_log_2
 7th restart
-DROP TABLE t1, t2;
+ib_modified_log_1
+ib_modified_log_2
+ib_modified_log_3
+8th restart
+RESET CHANGED_PAGE_BITMAPS;
+call mtr.add_suppression("InnoDB: Error: page [0-9]* log sequence number [0-9]*");
+9th restart
+CREATE TABLE t3 (a MEDIUMBLOB) ENGINE=InnoDB;
+call mtr.add_suppression("InnoDB: Error: the age of the oldest untracked record exceeds the log group capacity!");
+call mtr.add_suppression("InnoDB: Error: stopping the log tracking thread at LSN");
+INSERT INTO t3 VALUES (REPEAT('a', 12582912));
+10th restart
+DROP TABLE t1, t2, t3;

=== modified file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp-master.opt'
--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp-master.opt	2013-01-31 11:52:07 +0000
+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp-master.opt	2013-02-05 09:53:25 +0000
@@ -1,1 +1,1 @@
---innodb_track_changed_pages=TRUE --innodb_log_file_size=5M --innodb_file_per_table 
+--innodb_track_changed_pages=TRUE --innodb_log_file_size=5M --innodb_file_per_table --max_allowed_packet=13631488 

=== modified file 'Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test'
--- Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test	2013-01-31 11:52:07 +0000
+++ Percona-Server/mysql-test/suite/innodb/t/percona_changed_page_bmp.test	2013-02-05 09:53:25 +0000
@@ -198,4 +198,43 @@
 --exec echo "restart" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
 --source include/wait_until_connected_again.inc
 
-DROP TABLE t1, t2;
+# TODO: check the tracked LSN range continuity once this info is exposed through
+# INFORMATION_SCHEMA.
+
+file_exists $BITMAP_FILE;
+--replace_regex /_[[:digit:]]+\.xdb$//
+list_files $MYSQLD_DATADIR ib_modified_log*;
+
+#
+# Test for log tracking compatibility with innodb_force_recovery (bug 1083596).
+#
+
+--exec echo "wait" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+--shutdown_server 10
+--source include/wait_until_disconnected.inc
+--enable_reconnect
+--echo 8th restart
+--exec echo "restart:--innodb-force-recovery=6" > $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+--source include/wait_until_connected_again.inc
+
+# innodb_force_recovery desyncs bitmap data and logs
+RESET CHANGED_PAGE_BITMAPS;
+
+call mtr.add_suppression("InnoDB: Error: page [0-9]* log sequence number [0-9]*");
+--echo 9th restart
+--source include/restart_mysqld.inc
+
+#
+# Test that impossible to track log is handled gracefully (bug 1108613)
+#
+CREATE TABLE t3 (a MEDIUMBLOB) ENGINE=InnoDB;
+
+call mtr.add_suppression("InnoDB: Error: the age of the oldest untracked record exceeds the log group capacity!");
+call mtr.add_suppression("InnoDB: Error: stopping the log tracking thread at LSN");
+
+INSERT INTO t3 VALUES (REPEAT('a', 12582912));
+
+--echo 10th restart
+--source include/restart_mysqld.inc
+
+DROP TABLE t1, t2, t3;

=== modified file 'Percona-Server/storage/innobase/log/log0log.c'
--- Percona-Server/storage/innobase/log/log0log.c	2013-01-04 15:24:45 +0000
+++ Percona-Server/storage/innobase/log/log0log.c	2013-02-05 09:53:25 +0000
@@ -492,9 +492,12 @@
 
 		if (tracked_lsn_age >= log->log_group_capacity) {
 
-			fprintf(stderr, " InnoDB: Error: the age of the "
+			fprintf(stderr, "InnoDB: Error: the age of the "
 				"oldest untracked record exceeds the log "
 				"group capacity!\n");
+			fprintf(stderr, "InnoDB: Error: stopping the log "
+				"tracking thread at LSN %llu\n", tracked_lsn);
+			srv_track_changed_pages = FALSE;
 		}
 	}
 

=== modified file 'Percona-Server/storage/innobase/log/log0online.c'
--- Percona-Server/storage/innobase/log/log0online.c	2013-02-05 09:53:25 +0000
+++ Percona-Server/storage/innobase/log/log0online.c	2013-02-05 09:53:25 +0000
@@ -413,8 +413,10 @@
 
 	if (last_tracked_lsn > tracking_start_lsn) {
 		fprintf(stderr,
-			"InnoDB: Error: last tracked LSN is in future.  This "
-			"can be caused by mismatched bitmap files.\n");
+			"InnoDB: Error: last tracked LSN %llu is ahead of "
+			"tracking start LSN %llu.  This can be caused by "
+			"mismatched bitmap files.\n", last_tracked_lsn,
+			tracking_start_lsn);
 		exit(1);
 	}
 

=== modified file 'Percona-Server/storage/innobase/srv/srv0start.c'
--- Percona-Server/storage/innobase/srv/srv0start.c	2012-11-25 09:30:58 +0000
+++ Percona-Server/storage/innobase/srv/srv0start.c	2013-02-05 09:53:25 +0000
@@ -1155,6 +1155,11 @@
 init_log_online(void)
 /*=================*/
 {
+	if (UNIV_UNLIKELY(srv_force_recovery > 0)) {
+		srv_track_changed_pages = FALSE;
+		return;
+	}
+
 	if (srv_track_changed_pages) {
 
 		log_online_read_init();