Merge lp:~hrvojem/percona-server/bug930571-5.1 into lp:percona-server/5.1

[Hrvoje Matijakovic]

[Stewart Smith (stewart)]

I think the following could be fixed, although it's not going to stop me merging this tree as is:

In "Relocation of the doublewrite buffer", "turned on/off without breaking the compatibility." can drop the "the".

In "doc/source/diagnostics/misc_info_schema_tables.rst", "Only the temporary tables that were explicitly created with `CREATE TEMPORARY TABLE` or `ALTER TABLE` are shown, and not the ones created to process complex queries." should drop the first "the" and "to process complex queries" should be replaced with "during query execution".

In "innodb_fake_changes.rst", "Restarting a slave server in a replication environment or setting up new slave server can cause a replication reads slower. This is happening because replication in |MySQL| is single-threaded and because it needs to read the data before it can execute the queries. The process can be speeded up by having prefetch threads to warm the server: replay statements and then rollback at commit." should be "Replication in |MySQL| is single-threaded and because it needs to read the data before it can execute the queries, this can decrease performance when the full data set is not already in the buffer pool. The fake changes feature allows a process to replay a set of operations against a replication slave to warm the buffer pool for the replication apply thread."

review: Approve

=== added file 'doc/source/compatibility.rst'
--- doc/source/compatibility.rst	1970-01-01 00:00:00 +0000
+++ doc/source/compatibility.rst	2012-04-02 07:10:25 +0000
@@ -0,0 +1,25 @@
+.. _compatibility: 
+
+==============================================================
+Options that make XtraDB tablespaces not compatible with MySQL
+==============================================================
+
+Expanded undo slots
+===================
+
+Enabling :variable:`innodb_extra_undoslots` breaks compatibility with other programs. Specifically, it makes the datafiles unusable for ibbackup or for a MySQL server that is not run with this option.
+
+Fast checksums
+==============
+
+Enabling :variable:`innodb_fast_checksum` will use more CPU-efficient algorithm, based on 4-byte words which can be beneficial for some workloads. Once enabled, turning it off will require table to be dump/imported again, since it will fail to start on data files created when :variable:`innodb_fast_checksums` was enabled.
+
+Page sizes other than 16KiB
+===========================
+
+This is controlled by variable :variable:`innodb_page_size`. Changing the page size for an existing database is not supported. Table will need to be dumped/imported again if compatibility with |MySQL| is required.
+
+Relocation of the doublewrite buffer
+====================================
+
+Variable :variable:`innodb_doublewrite_file` provides an option to put the buffer on a dedicated disk in order to parallelize I/O activity on the buffer and on the tablespace. Only in case of crash recovery this variable cannot be changed, in all other cases it can be turned on/off without breaking compatibility. 
 
=== modified file 'doc/source/diagnostics/misc_info_schema_tables.rst'
--- doc/source/diagnostics/misc_info_schema_tables.rst	2011-09-20 06:59:03 +0000
+++ doc/source/diagnostics/misc_info_schema_tables.rst	2012-04-02 07:10:25 +0000
@@ -98,6 +98,8 @@
 Temporary tables
 ================
 
+ Only temporary tables that were explicitly created with `CREATE TEMPORARY TABLE` or `ALTER TABLE` are shown, and not the ones created during query execution.
+
 .. table:: INFORMATION_SCHEMA.GLOBAL_TEMPORARY_TABLES
 
    :column SESSION_ID: |MySQL| connection id
 
=== modified file 'doc/source/index.rst'
--- doc/source/index.rst	2012-03-16 09:07:44 +0000
+++ doc/source/index.rst	2012-04-02 07:10:25 +0000
@@ -104,6 +104,7 @@
    reliability/log_connection_error
    reliability/error_pad
    reliability/innodb_corrupt_table_action
+   reliability/innodb_recovery_update_relay_log
    reliability/show_slave_status_nolock
 
 Management Improvements
@@ -163,6 +164,7 @@
    development
    trademark-policy
    faq
+   compatibility
    release-notes/release-notes_index
    glossary
 
 
=== modified file 'doc/source/management/innodb_fake_changes.rst'
--- doc/source/management/innodb_fake_changes.rst	2011-10-14 02:52:34 +0000
+++ doc/source/management/innodb_fake_changes.rst	2012-04-02 07:10:25 +0000
@@ -4,7 +4,7 @@
  Support for Fake Changes
 ==========================
 
-When restarting a slave server in a replication environment, the process can be speed up by having prefetch threads to warm the server: replay statements and then rollback at commit.
+Replication in |MySQL| is single-threaded and because it needs to read the data before it can execute the queries, this can decrease performance when the full data set is not already in the buffer pool. The fake changes feature allows a process to replay a set of operations against a replication slave to warm the buffer pool for the replication apply thread.
 
 That makes prefetch simple but has high overhead from locking rows only to undo changes at rollback.
 
@@ -63,3 +63,9 @@
   * It will not prefetch allocate/free, split/merge, ``INODE``, ``XDES`` or other management pages. The same is for extern pages, i.e. large ``BLOB`` s).
 
   * Foreign key constraints are checked (for causing IO), but passed always.
+
+Related Reading
+===============
+
+ * `on MySQL replication prefetching <http://dom.as/2011/12/03/replication-prefetching/>`_
+
 
=== modified file 'doc/source/performance/innodb_fast_checksum.rst'
--- doc/source/performance/innodb_fast_checksum.rst	2011-10-07 00:55:06 +0000
+++ doc/source/performance/innodb_fast_checksum.rst	2012-04-02 07:10:25 +0000
@@ -10,8 +10,7 @@
 
 The original algorithm is checked after the new one, so you can have data pages with old checksums and data pages with new checksums. However in this case, you may experience slow reads from pages having old checksums. If you want to have the entire benefit of this change, you will need to recreate all your |InnoDB| tables, for instance by dumping and reloading all |InnoDB| tables.
 
-Once enabled, turning it off will require table re-creation as well, since it will fail to start on data files created when ``innodb_fast_checksums`` was enabled.
-
+Once enabled, turning it off will require table(s) to be dump/imported, since it will fail to start on data files created when ``innodb_fast_checksums`` was enabled. In this case ALTER TABLE won't work due to its implementation.
 
 System Variables
 ================
 
=== modified file 'doc/source/release-notes/Percona-Server-1.0.3-6.rst'
--- doc/source/release-notes/Percona-Server-1.0.3-6.rst	2011-10-07 00:55:06 +0000
+++ doc/source/release-notes/Percona-Server-1.0.3-6.rst	2012-04-02 07:10:25 +0000
@@ -27,7 +27,7 @@
 Fixed bugs
 ==========
 
-  * |MySQL| Bug: ``#39793 <http://bugs.mysql.com/39793>`_: Foreign keys not constructed when column has a ``#`` in a comment or default value
+  * |MySQL| Bug: `#39793 <http://bugs.mysql.com/39793>`_: Foreign keys not constructed when column has a ``#`` in a comment or default value
 
   * Bug :bug:`395784` in Percona-XtraDB: ``main.innodb_xtradb_bug317074 internal check fails on 5.1.36``
 
 
=== modified file 'doc/source/release-notes/Percona-Server-1.0.4-8.rst'
--- doc/source/release-notes/Percona-Server-1.0.4-8.rst	2011-09-26 22:00:30 +0000
+++ doc/source/release-notes/Percona-Server-1.0.4-8.rst	2012-04-02 07:10:25 +0000
@@ -13,10 +13,10 @@
 Fixed bugs:
 ===========
 
-    * [[https://bugs.launchpad.net/percona-xtradb/+bug/413858|Bugs#413858: Crash from failed assertion in dict0dict.c]]
-
-    * [[https://bugs.launchpad.net/percona-xtradb/+bug/417751|Bug#417751: |XtraDB| crashes on startup on windows]]
-
-    * [[https://code.launchpad.net/~percona-dev/percona-xtradb/fix-import-extern-pages|fix-import-extern-pages]]
+    * :bug:`413858`: Crash from failed assertion in dict0dict.c]]
+
+    * :bug:`417751`: |XtraDB| crashes on startup on windows]]
+
+    * `fix-import-extern-pages <https://code.launchpad.net/~percona-dev/percona-xtradb/fix-import-extern-pages>`_
 
     * Number of |MySQL|-tests are fixed
 
=== modified file 'doc/source/release-notes/Percona-Server-1.0.6-9.rst'
--- doc/source/release-notes/Percona-Server-1.0.6-9.rst	2011-09-26 22:00:30 +0000
+++ doc/source/release-notes/Percona-Server-1.0.6-9.rst	2012-04-02 07:10:25 +0000
@@ -28,8 +28,8 @@
 
 Fixed bugs:
 
-    * Bug :bug:`488315`: rename columns and add unique index cause serious inconsistent with :command:`mysqld`
-
-    * Bug :bug:495342]]: “MySQL 5.1.41 + InnoDB 1.0.6 + XtraDB patches extensions-1.0.6, rev. 127 fails to compile on OpenSolaris”
-
-    * |MySQL| bug `#47621 <http://bugs.mysql.com/47621>`_: change the fix of http://bugs.mysql.com/47621 more reasonable
+    * Bug :bug:`488315`: rename columns and add unique index cause serious inconsistent with :command:`mysqld`.
+
+    * Bug :bug:`495342`: MySQL 5.1.41 + InnoDB 1.0.6 + XtraDB patches extensions-1.0.6, rev. 127 fails to compile on OpenSolaris.
+
+    * |MySQL| bug `#47621 <http://bugs.mysql.com/47621>`_: change the fix of http://bugs.mysql.com/47621 more reasonable.
 
=== modified file 'doc/source/release-notes/Percona-Server-5.1.47-rel11.1.rst'
--- doc/source/release-notes/Percona-Server-5.1.47-rel11.1.rst	2011-09-26 22:00:30 +0000
+++ doc/source/release-notes/Percona-Server-5.1.47-rel11.1.rst	2012-04-02 07:10:25 +0000
@@ -36,7 +36,7 @@
 
   * :bug:`580324` - Security bug in upstream 
 
-  * :bug:`586579` -|Percona Server| 11 fails to compile with ``CFLAGS=-DUNIV_DEBUG`` 
+  * :bug:`586579` - |Percona Server| 11 fails to compile with ``CFLAGS=-DUNIV_DEBUG`` 
 
   * :bug:`569156` - CentOS 5: ``mysql-server`` conflicts with ``mysql-server`` 
 
 
=== modified file 'doc/source/release-notes/Percona-Server-5.1.52-rel11.6.rst'
--- doc/source/release-notes/Percona-Server-5.1.52-rel11.6.rst	2011-09-26 22:00:30 +0000
+++ doc/source/release-notes/Percona-Server-5.1.52-rel11.6.rst	2012-04-02 07:10:25 +0000
@@ -18,7 +18,7 @@
 Bugs Fixed
 ===========
 
-  * Bug :bug:`671764` - ``innochecksum`` wasn``t distributed with RPM and .DEB packages (Aleksandr Kuzminsky)
+  * Bug :bug:`671764` - ``innochecksum`` wasn't distributed with RPM and .DEB packages (Aleksandr Kuzminsky)
 
   * Bug :bug:`673426` - Use of some system variables as command-line options caused a crash or undefined behavior. (*Oleg Tsarev*)
 
 
=== modified file 'doc/source/release-notes/Percona-Server-5.1.53-12.4.rst'
--- doc/source/release-notes/Percona-Server-5.1.53-12.4.rst	2011-10-07 00:55:06 +0000
+++ doc/source/release-notes/Percona-Server-5.1.53-12.4.rst	2012-04-02 07:10:25 +0000
@@ -45,7 +45,7 @@
 
   * Bug :bug:`677407` - The ``innodb_information_schema`` test could fail sporadically due to flawed logic in the ``INFORMATION_SCHEMA.INNODB_LOCKS`` caching mechanism. (contributed by *Kristian Nielsen*) (*Alexey Kopytov*)
 
-  * Bug :bug:`681486` - A dependency between Debian packages ``lib|MySQL|client16`` and ``percona-server-common`` was removed. (*Aleksandr Kuzminsky*)
+  * Bug :bug:`681486` - A dependency between Debian packages ``libmysqlclient16`` and ``percona-server-common`` was removed. (*Aleksandr Kuzminsky*)
 
   * Bug :bug:`693815` - The test ``percona_innodb_buffer_pool_shm`` was failing. It should be run with the ``--big-test`` option. As the buffer pool size used in the test is 128M, the shared memory segment should be increased appropriately in order to run the test successfully.
 
 
=== modified file 'doc/source/release-notes/Percona-Server-5.1.58-12.9.rst'
--- doc/source/release-notes/Percona-Server-5.1.58-12.9.rst	2011-12-03 02:46:52 +0000
+++ doc/source/release-notes/Percona-Server-5.1.58-12.9.rst	2012-04-02 07:10:25 +0000
@@ -23,7 +23,7 @@
 
 Two new collations, ``utf8_general50_ci`` and ``ucs2_general50_ci``, have been to improve compatibility for those upgrading from |MySQL| 5.0 or 5.1 prior to version 5.1.24.
 
-A fix for a |MySQL| bug ([[`#27877 <http://bugs.mysql.com/bug.php?id=27877>`) introduced an incompatible change in collations in |MySQL| 5.1.24. If the following collations were used:
+A fix for a |MySQL| bug `#27877 <http://bugs.mysql.com/bug.php?id=27877>`_ introduced an incompatible change in collations in |MySQL| 5.1.24. If the following collations were used:
 
   * ``utf8_general_ci`` 
   * ``ucs2_general_ci``
@@ -37,7 +37,7 @@
   * ``utf8_general_ci`` to ``utf8_general50_ci``, and 
   * ``ucs2_general_ci`` to ``ucs2_general50_ci``.
 
-Blueprint:[[https://blueprints.launchpad.net/percona-server/+spec/utf8-general50-ci-5.1|utf8-general50-ci-5.1]] (*Alexey Kopytov*).
+Blueprint: `utf8-general50-ci-5.1 <https://blueprints.launchpad.net/percona-server/+spec/utf8-general50-ci-5.1>`_ (*Alexey Kopytov*).
 
 SHM Buffer Pool has been removed
 --------------------------------
 
=== added file 'doc/source/reliability/innodb_recovery_update_relay_log.rst'
--- doc/source/reliability/innodb_recovery_update_relay_log.rst	1970-01-01 00:00:00 +0000
+++ doc/source/reliability/innodb_recovery_update_relay_log.rst	2012-04-02 07:10:25 +0000
@@ -0,0 +1,80 @@
+.. _innodb_recovery_update_relay_log_page:
+
+=============================
+ Crash-Resistant Replication
+=============================
+
+This feature makes replication much more reliable after a crash by making the replica's position relative to the master transactional.
+
+|MySQL| replication normally stores its position in a file that is neither durable nor consistent. Thus, if the replica crashes, it can re-execute committed transactions. This usually causes replication to fail, potentially forcing the replica's data to be re-initialized from the master or from a recent backup.
+
+The improvement in |Percona Server| makes |InnoDB| store the replication position transactionally, and overwrite the usual relay-log.info file upon recovery, so replication restarts from the correct position and does not try to re-execute committed transactions. This change greatly improves the durability of |MySQL| replication. It can be set to activate automatically, so replication “just works” and no intervention is necessary after a crash.
+
+
+Restrictions
+============
+
+When :variable:`innodb_overwrite_relay_log_info` is enabled, you should only update |InnoDB| / |XtraDB| tables, not |MyISAM| tables or other storage engines.
+You should not use relay or binary log filenames longer than 480 characters (normal: up to 512). If longer, the replication position information is not recorded in |InnoDB|.
+
+Example Server Error Log Output
+===============================
+
+Upon crash recovery, the error log on a replica will show information similar to the following: ::
+
+  InnoDB: Starting crash recovery.
+  ....
+  InnoDB: Apply batch completed
+  InnoDB: In a MySQL replication slave the last master binlog file
+  InnoDB: position 0 468, file name gauntlet3-bin.000015
+  InnoDB: and relay log file
+  InnoDB: position 0 617, file name ./gauntlet3-relay-bin.000111
+
+If this feature is enabled, the output will look like the following, with additional lines prefixed with a ``+`` symbol: ::
+
+  ....
+  + InnoDB: Warning: innodb_overwrite_relay_log_info is enabled. Updates of other storage engines may have problem of consistency.
+  + InnoDB: relay-log.info is detected.
+  + InnoDB: relay log: position 429, file name ./gauntlet3-relay-bin.000111
+  + InnoDB: master log: position 280, file name gauntlet3-bin.000015
+  ....
+    InnoDB: Starting crash recovery.
+  ....
+    InnoDB: Apply batch completed
+  + InnoDB: In a MySQL replication slave the last master binlog file
+  + InnoDB: position 0 468, file name gauntlet3-bin.000015
+  + InnoDB: and relay log file
+  + InnoDB: position 0 617, file name ./gauntlet3-relay-bin.000111
+    090205 17:41:31 InnoDB Plugin 1.0.2-3 started; log sequence number 57933
+  + InnoDB: relay-log.info have been overwritten.
+  ....
+    090205 17:41:31 [Note] Slave SQL thread initialized, starting replication in log ``gauntlet3-bin.000015`` at position 468, relay log ``./gauntlet3-relay-bin.000111`` position: 617
+
+In this case, the master log position was overwritten to 468 from 280, so replication will start at position 468 and not repeat the transaction beginning at 280.
+
+System Variables
+================
+
+One new system variable was introduced by this feature.
+
+.. variable:: innodb_overwrite_relay_log_info
+
+     :version 5.5.10-20.1: Renamed.
+     :cli: Yes
+     :conf: Yes
+     :scope: Global
+     :dyn: No
+     :vartype: BOOLEAN
+     :default: FALSE
+     :range: TRUE/FALSE
+
+If set to true, |InnoDB| overwrites :file:`relay-log.info` at crash recovery when the information is different from the record in |InnoDB|.
+
+Other Reading
+=============
+
+  * Another solution for |MySQL| 5.0 is `Google's transactional replication feature <http://code.google.com/p/google-mysql-tools/wiki/TransactionalReplication>`_, but it had some problems and bugs.
+
+  * `Related bug (fixed and re-implemented in this feature) <http://bugs.|MySQL|.com/bug.php?id=34058>`_
+
+  * `A blog post explaining how this feature makes replication more reliable <http://www.|MySQL|performanceblog.com/2009/03/04/making-replication-a-bit-more-reliable/>`_
 
=== modified file 'doc/source/scalability/innodb_expand_undo_slots.rst'
--- doc/source/scalability/innodb_expand_undo_slots.rst	2011-10-07 00:55:06 +0000
+++ doc/source/scalability/innodb_expand_undo_slots.rst	2012-04-02 07:10:25 +0000
@@ -4,7 +4,7 @@
 More Concurrent Transactions Available
 ======================================
 
-|InnoDB| provides a fixed number of 1024 undo slots in its rollback segment, leaving room for 1024 transactions to run in parallel. If all the slots are used any new transaction will fail until a slot is freed, which can cause st:range: behaviors. This change provides a variable to expand the number of undo slots, up to 4072.
+|InnoDB| provides a fixed number of 1024 undo slots in its rollback segment, leaving room for 1024 transactions to run in parallel. If all the slots are used any new transaction will fail until a slot is freed, which can cause strange behaviors. This change provides a variable to expand the number of undo slots, up to 4072.
 
 This option is provided for servers that run out of undo slots. Use it if you find the following warning in the error log: ``Warning: cannot find a free slot for an undo log``.
 
 
=== modified file 'doc/source/scalability/innodb_io.rst'
--- doc/source/scalability/innodb_io.rst	2012-03-09 08:03:43 +0000
+++ doc/source/scalability/innodb_io.rst	2012-04-02 07:10:25 +0000
@@ -188,6 +188,21 @@
 
 This variable changes the size of transaction log records. The default size of 512 bytes is good in most situations. However, setting it to 4096 may be a good optimization with SSD cards. While settings other than 512 and 4096 are possible, as a practical matter these are really the only two that it makes sense to use.
 
+.. variable:: innodb_log_file_size
+
+   :version 1.0.6-10: Introduced
+   :cli: Yes
+   :conf: Yes
+   :scope: Global
+   :dyn: No
+   :type: Numeric
+   :default: 5242880
+   :range: 1048576 .. 4294967295
+
+In upstream |MySQL| the limit for the combined size of log files must be less than 4GB. But in Percona Server it is:
+  * on 32-bit systems: individual log file limit is 4 GB and total log file size limit is 4 GB, i.e. the same as in the upstream server.
+  * on 64-bit systems: both individual log files and total log file size are practically unlimited (the limit is 2^63 - 1 bytes which is 8+ million TB).
+
 .. variable:: innodb_read_ahead
 
      :cli: Yes