Merge lp:~hrvojem/percona-server/rn-5.1.65-14.0 into lp:percona-server/5.1

[Hrvoje Matijakovic]

[Alexey Kopytov (akopytov)]

Looks good to me except that bugs #1032482, #1032606, #1040483 and #1036530 are bugs in changed page tracking being introduced in this release. I.e. they never affected any released version and thus, are of no interest to users.

review: Needs Fixing

[Alexey Kopytov (akopytov)]

review: Approve

[Laurynas Biveinis (laurynas-biveinis)]

Review in progress, please don't merge.

review: Needs Fixing

[Laurynas Biveinis (laurynas-biveinis)]

   In Features of RN, swap the changed page tracking and its I_S
   table, as the latter depends on the former.
   Instead "real incremental backups" (are the current incremental
   backups less real?) we have to be more specific and say "faster
   incremental backups that use this information to avoid full data
   scans."
   "This table contains a list of pages marked as ``modified`` by log
   tracking thread." - is an implementation detail. Would be nicer
   something like "a list of modified pages from the modified page
   bitmap files produced by the log tracking thread."
   Bug 1039384 does not apply for 5.1. Its commit only contains
   expanded testcases.
   For bug 686392: s/implemente/implemented, s/rename/renames.
   For bug 686534: the mutex is periodically released in order not to
   block server while the dump is in progress.

   For changed page tracking docs, it's not a "reliability" but rather
   "management" feature IMHO, so should be placed accordingly.
   Instead of "InnoDB/XtraDB" we can just say "XtraDB", unless the
   former is consistent with other docs.

   In general I find the current description focusing too much on
   implementation specifics and not on the users. For introduction,
   "XtraDB now tracks the pages that have changes written to them
   according to the redo log. This information is written out in a
   special changed page bitmap file. This information can be used to
   speed up incremental backups using xb, by removing the need to scan
   whole data files to find the changed pages. This change tracking
   ois done by a new XtraDB worker thread that reads and parses log
   records between checkpoints."

   The current "implementation details" I think would only confuse,
   and use undefined terms like "last page flag". I'd rather omit
   this section for now and wait for confused users first that need
   this info.

   In I_S tables section, "This table contains a list of modified
   pages from the bitmap file data. As these files are generated by
   the log tracking thread parsing the log whenever the checkpoint is
   made, it is not real-time data"

   The start_lsn, end_lsn columns have wrong meanings. They actually
   mean "the change to this page happened at least once between these
   two LSNs". They are also equal to checkpoint LSNs.

review: Needs Fixing

[Laurynas Biveinis (laurynas-biveinis)]

s/using xb/using Percona XtraBackup/ (I was typing in IRC style :), also here and below linkified perhaps.
"that reads and parses log records" - two spaces before "records"

"for implementing faster incremental backups that use this information to avoid full data scans." - "... backups in Percona XtraBackup ...", linkified perhaps too.

For bug 713481, s/LRU messages/error log messages

review: Needs Fixing

[Laurynas Biveinis (laurynas-biveinis)]

My second comment is unaddressed.
And, bug 951588 is me, not Alexey :)

review: Needs Fixing

[Laurynas Biveinis (laurynas-biveinis)]

review: Approve

=== modified file 'doc/source/conf.py'
--- doc/source/conf.py	2012-06-26 11:44:32 +0000
+++ doc/source/conf.py	2012-09-03 09:35:25 +0000
@@ -52,9 +52,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '5.1.63'
+version = '5.1.65'
 # The full version, including alpha/beta/rc tags.
-release = '5.1.63-13.4'
+release = '5.1.65-14.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
@@ -108,6 +108,8 @@
 
 .. |XtraBackup|  replace:: *XtraBackup*
 
+.. |Percona XtraBackup|  replace:: *Percona XtraBackup*
+
 .. |myisamchk|  replace:: *myisamchk*
 
 .. |Percona Server|  replace:: *Percona Server*

=== modified file 'doc/source/diagnostics/index_info_schema_tables.rst'
--- doc/source/diagnostics/index_info_schema_tables.rst	2011-10-07 00:55:06 +0000
+++ doc/source/diagnostics/index_info_schema_tables.rst	2012-09-03 09:35:25 +0000
@@ -28,6 +28,8 @@
 
   * :table:`INNODB_TRX`
 
+  * :table:`INNODB_CHANGED_PAGES`
+
   * :table:`PROCESSLIST`
 
   * :table:`QUERY_RESPONSE_TIME`

=== modified file 'doc/source/diagnostics/innodb_show_lock_names.rst'
--- doc/source/diagnostics/innodb_show_lock_names.rst	2011-10-07 00:55:06 +0000
+++ doc/source/diagnostics/innodb_show_lock_names.rst	2012-09-03 09:35:25 +0000
@@ -4,11 +4,11 @@
  Show Lock Names
 =================
 
-This feature is curently undocumented except for the following example.
+This feature is currently undocumented except for the following example.
 
 Example: ::
 
-  mysql> show mutex status;
+  mysql> SHOW ENGINE INNODB MUTEX;
   +--------+---------------------------+---------------+
   | Type   | Name                      | Status        |
   +--------+---------------------------+---------------+

=== modified file 'doc/source/index.rst'
--- doc/source/index.rst	2012-04-02 09:36:37 +0000
+++ doc/source/index.rst	2012-09-03 09:35:25 +0000
@@ -126,6 +126,7 @@
    management/udf_maatkit
    management/innodb_fake_changes
    management/innodb_kill_idle_trx
+   management/changed_page_tracking
 
 Diagnostics Improvements
 ========================

=== added file 'doc/source/management/changed_page_tracking.rst'
--- doc/source/management/changed_page_tracking.rst	1970-01-01 00:00:00 +0000
+++ doc/source/management/changed_page_tracking.rst	2012-09-03 09:35:25 +0000
@@ -0,0 +1,44 @@
+.. _changed_page_tracking:
+
+=============================
+XtraDB changed page tracking
+=============================
+
+XtraDB now tracks the pages that have changes written to them according to the redo log. This information is written out in a special changed page bitmap file.  This information can be used to speed up incremental backups using `Percona XtraBackup <http://www.percona.com/doc/percona-xtrabackup/>`_, by removing the need to scan whole data files to find the changed pages. This change tracking is done by a new XtraDB worker thread that reads and parses log records between checkpoints. The tracking is controlled by a new read-only server variable :variable:`innodb_track_changed_pages`.
+
+This feature will be used for implementing faster incremental backups that use this information to avoid full data scans in |Percona XtraBackup|.
+
+Additional information in SHOW ENGINE INNODB STATUS
+===================================================
+When log tracking is enabled, the following additional fields are displayed in the LOG section of the ``SHOW ENGINE INNODB STATUS`` output:
+
+ * "Log tracked up to:" displays the LSN up to which all the changes have been parsed and stored as a bitmap on disk by the log tracking thread
+ * "Max tracked LSN age:" displays the maximum limit on how far behind the log tracking thread may be.
+
+INFORMATION_SCHEMA Tables
+=========================
+
+This table contains a list of modified pages from the bitmap file data.  As these files are generated by the log tracking thread parsing the log whenever the checkpoint is made, it is not real-time data.
+
+.. table:: INFORMATION_SCHEMA.INNODB_CHANGED_PAGES
+
+   :column INT(11) space_id: space id of modified page
+   :column INT(11) page_id: id of modified page
+   :column BIGINT(21) start_lsn: start of the interval
+   :column BIGINT(21) end_lsn: end of the interval 
+
+The ``start_lsn`` and the ``end_lsn`` columns denote between which two checkpoints this page was changed at least once. They are also equal to checkpoint LSNs.
+
+System Variables
+================
+
+.. variable:: innodb_track_changed_pages
+
+   :version 5.1.65-14.0: Variable introduced
+   :cli: No
+   :conf: Yes
+   :scope: Global
+   :dyn: No
+   :vartype: Boolean
+   :default: 0 - False
+   :range: 0-1

=== added file 'doc/source/release-notes/Percona-Server-5.1.65-14.0.rst'
--- doc/source/release-notes/Percona-Server-5.1.65-14.0.rst	1970-01-01 00:00:00 +0000
+++ doc/source/release-notes/Percona-Server-5.1.65-14.0.rst	2012-09-03 09:35:25 +0000
@@ -0,0 +1,54 @@
+.. rn:: 5.1.65-14.0
+
+=============================
+ |Percona Server| 5.1.65-14.0
+=============================
+
+Percona is glad to announce the release of |Percona Server| 5.1.65-14.0 on September 3rd, 2012 (Downloads are available from `Percona Server 5.1.65-14.0 downloads <http://www.percona.com/downloads/Percona-Server-5.1/Percona-Server-5.1.65-14.0/>`_ and from the `Percona Software Repositories <http://www.percona.com/docs/wiki/repositories:start>`_).
+
+Based on `MySQL 5.1.65 <http://dev.mysql.com/doc/refman/5.1/en/news-5-1-63.html>`_, including all the bug fixes in it, |Percona Server| 5.1.65-14.0 is now the current stable release in the 5.1 series. All of |Percona|'s software is open-source and free, all the details of the release can be found in the `5.1.65-14.0 milestone at Launchpad <https://launchpad.net/percona-server/+milestone/5.1.65-14.0>`_.
+
+Features
+========
+  
+  |Percona Server| now supports :ref:`changed_page_tracking`. This feature will be used for implementing faster incremental backups that use this information to avoid full data scans.  
+
+  New table :table:`INNODB_CHANGED_PAGES` has been implemented. This table contains a list of modified pages from the modified page bitmap files produced by the log tracking thread.
+
+  HandlerSocket has been upgraded to version 1.1.0.
+ 
+Bug Fixes
+=========
+
+ Loading LRU dump was preventing shutdown. Bug fixed :bug:`712055` (*George Ormond Lorch III*).
+
+ The kill idle transactions feature in |XtraDB| (if enabled) could sometimes cause the server to crash. Bug Fixed: :bug:`871722` (*Stewart Smith*)
+
+ Fixed server assertion error related to buffer pool, only visible in debug builds. Bug fixed :bug:`905334` (*Laurynas Biveinis*).
+ 
+ Querying I_S.GLOBAL_TEMPORARY_TABLES or TEMPORARY_TABLES would crash threads working with temporary tables. Bug fixed :bug:`951588` (*Laurynas Biveinis*).
+
+ A crash could leave behind an InnoDB temporary table with temporary indexes resulting in an unbootable server. Bug fixed :bug:`999147` (*Laurynas Biveinis*).
+
+ Fixed the upstream MySQL bug `#66301 <http://bugs.mysql.com/bug.php?id=66301>`_. Concurrent INSERT ... ON DUPLICATE KEY UPDATE statements on a table with an AUTO_INCREMENT column could result in spurious duplicate key errors (and, as a result, lost data due to some rows being updated rather than inserted) with the default value of innodb_autoinc_lock_mode=1. Bug fixed :bug:`1035225` (*Alexey Kopytov*)
+
+ Since the output file is simply overwritten when dumping the LRU list, we could end up with a partially written dump file in case of a crash, or when making a backup copy of it. Safer approach has been implemented. It now dumps to a temporary file first, and then renames it to the actual dump file. Bug fixed :bug:`686392` (*George Ormond Lorch III*).
+
+ Fixed the issue where LRU dump would hold LRU_list_mutex during the entire dump process. Now the mutex is periodically released in order not to block server while the dump is in progress. Bug fixed :bug:`686534` (*George Ormond Lorch III*).
+
+ Removed error log warnings that occured after enabling :variable:`innodb_use_sys_stats_table` and before ANALYZE TABLE is run for each table. Bug fixed :bug:`890623` (*Alexey Kopytov*).
+
+ A Server acting as a replication slave with the query cache enabled could crash with glibc detected memory corruption. Bug fixed :bug:`915814` (*George Ormond Lorch III*). 
+
+ If the tablespace has been created with MySQL 5.0 or older, importing that table could crash |Percona Server| in some cases. Bug fixed :bug:`1000221` (*Alexey Kopytov*).
+
+ Error log messages are now more verbose for LRU dump. Bug fixed :bug:`713481` (*George Ormond Lorch III*). 
+
+ Fixed issue where :variable:`innodb_blocking_lru_restore` did not take an optional bool argument similar to other bool options. Bug fixed :bug:`881001` (*George Ormond Lorch III*).
+
+ Removed the unneeded lrusort.py script. The server now does this sorting automatically and has done for some time. Bug fixed :bug:`882653` (*Stewart Smith*).
+
+ Server started with :option:`skip-innodb` crashes on `SELECT * FROM INNODB_TABLE_STATS` or `INNODB_INDEX_STATS`. Bug fixed :bug:`896439` (*Stewart Smith*).
+
+ Removed the INFORMATION_SCHEMA table INNODB_PATCHES (actually XTRADB_ENHANCEMENTS) as it was out of date and isn't in 5.5 or later either. Bug fixed :bug:`1009997` (*Stewart Smith*).
+

=== modified file 'doc/source/reliability/show_slave_status_nolock.rst'
--- doc/source/reliability/show_slave_status_nolock.rst	2011-10-07 00:55:06 +0000
+++ doc/source/reliability/show_slave_status_nolock.rst	2012-09-03 09:35:25 +0000
@@ -8,13 +8,14 @@
 
 If a ``STOP SLAVE`` command is given in this situation, it will wait and not complete execution until the long-executing thread has completed its task. If another thread now executes a ``SHOW SLAVE STATUS`` command while the STOP SLAVE command is waiting to complete, the ``SHOW SLAVE STATUS`` command will not be able to execute while the ``STOP SLAVE`` command is waiting.
 
-This features modifies the ``SLOW SLAVE STATUS`` syntax to allow: ::
+This features modifies the ``SHOW SLAVE STATUS`` syntax to allow: ::
 
-  SLOW SLAVE STATUS NOLOCK
+  SHOW SLAVE STATUS NOLOCK
 
 This will display the slave's status as if there were no lock, allowing the user to detect and understand the situation that is occurring.
 
-**NOTE:** The information given when ``NOLOCK`` is used may be slightly inconsistent with the actual situation while the lock is being held.
+.. note:: 
+ The information given when ``NOLOCK`` is used may be slightly inconsistent with the actual situation while the lock is being held.
 
 
 Version Specific Information