Merge lp:~laurynas-biveinis/percona-server/tokudb-non-blocking-optimize into lp:percona-server/5.6

[Laurynas Biveinis]

2nd MP:

Explicitly forbid returning HA_ADMIN_TRY_ALTER from optimize if SE supports online optimize, because it might deadlock.
Rebase on the current trunk.

http://jenkins.percona.com/job/percona-server-5.6-param/489/

1st MP:

Add support for storage engines with online OPTIMIZE, implementing
https://blueprints.launchpad.net/percona-server/+spec/allow-writes-during-table-optimization.

The implementation is a combination of
https://github.com/Tokutek/mysql56/commit/1a321b6ae70f76da3dc76c944014baab2b7ce1ae
and
https://github.com/Tokutek/mysql56/commit/a701cea6de67c713d418a319a6a8486da8e07ff2.

It adds a new handler table flag HA_CAN_WRITE_DURING_OPTIMIZE. While
executing OPTIMIZE, after opening the tables, the presence of this
flag is checked and the MDL lock is downgraded from
MDL_SHARED_NO_READ_WRITE to MDL_SHARED_WRITE, and the closing of any
open instances of this table for other connections is skipped.

Adjust MDL_ticker::downgrade_lock() to allow downgrades from
MDL_SHARED_NO_READ_WRITE.

http://jenkins.percona.com/job/percona-server-5.6-param/468/

[Laurynas Biveinis (laurynas-biveinis)]

https://github.com/Tokutek/mysql56/issues/68

review: Needs Fixing

[Laurynas Biveinis (laurynas-biveinis)]

MySQL 5.6.17 introduced online OPTIMIZE for InnoDB. Perhaps this can be dropped then, I'll review the 5.6.17 implementation, holding this MP until then.

review: Needs Information

[Laurynas Biveinis (laurynas-biveinis)]

5.6.16 online OPTIMIZE is a better replacement.

review: Disapprove

Unmerged revisions

535. By Laurynas Biveinis on 2014-01-21

Add support for storage engines with online OPTIMIZE, implementing
https://blueprints.launchpad.net/percona-server/+spec/allow-writes-during-table-optimization.

The implementation is based on a combination of
https://github.com/Tokutek/mysql56/commit/1a321b6ae70f76da3dc76c944014baab2b7ce1ae
and
https://github.com/Tokutek/mysql56/commit/a701cea6de67c713d418a319a6a8486da8e07ff2

It adds a new handler table flag HA_CAN_WRITE_DURING_OPTIMIZE. While
executing OPTIMIZE, after opening the tables, the presence of this
flag is checked and the MDL lock is downgraded from
MDL_SHARED_NO_READ_WRITE to MDL_SHARED_WRITE, and the closing of any
open instances of this table for other connections is skipped.

Adjust handler::ha_optimize() to assert that HA_ADMIN_TRY_ALTER return
value is incompatible with HA_CAN_WRITE_DURING_OPTIMIZE.

Adjust MDL_ticker::downgrade_lock() to allow downgrades from
MDL_SHARED_NO_READ_WRITE.

=== modified file 'sql/handler.cc'
--- sql/handler.cc	2013-12-23 04:24:10 +0000
+++ sql/handler.cc	2014-01-21 14:15:01 +0000
@@ -4291,11 +4291,20 @@
 int
 handler::ha_optimize(THD* thd, HA_CHECK_OPT* check_opt)
 {
+  int result;
   DBUG_ASSERT(table_share->tmp_table != NO_TMP_TABLE ||
               m_lock_type == F_WRLCK);
   mark_trx_read_write();
 
-  return optimize(thd, check_opt);
+  result= optimize(thd, check_opt);
+  /* Forbid returning HA_ADMIN_TRY_ALTER if handler supports online OPTIMIZE as
+  this combination may result in a deadlock between OPTIMIZE and a concurrent
+  online ALTER: online OPTIMIZE will downgrade its MDL lock to SW, enabling
+  ALTER to MDL lock SU, lock the table TL_READ, and block on the X upgrade,
+  while preventing OPTIMIZE to lock the table TL_WRITE. */
+  DBUG_ASSERT(result != HA_ADMIN_TRY_ALTER
+	      || !(ha_table_flags() & HA_CAN_WRITE_DURING_OPTIMIZE));
+  return result;
 }
 
 

=== modified file 'sql/handler.h'
--- sql/handler.h	2013-12-16 12:54:12 +0000
+++ sql/handler.h	2014-01-21 14:15:01 +0000
@@ -248,6 +248,12 @@
 */
 #define HA_BLOCK_CONST_TABLE          (LL(1) << 42)
 
+/*
+  Storage engine implements online OPTIMIZE and does not return
+  HA_ADMIN_TRY_ALTER for it.
+*/
+#define HA_CAN_WRITE_DURING_OPTIMIZE (LL(1) << 63)
+
 /* bits in index_flags(index_number) for what you can do with index */
 #define HA_READ_NEXT            1       /* TODO really use this flag */
 #define HA_READ_PREV            2       /* supports ::index_prev */

=== modified file 'sql/mdl.cc'
--- sql/mdl.cc	2013-12-05 17:23:10 +0000
+++ sql/mdl.cc	2014-01-21 14:15:01 +0000
@@ -2812,7 +2812,8 @@
 
 
 /**
-  Downgrade an EXCLUSIVE or SHARED_NO_WRITE lock to shared metadata lock.
+  Downgrade an EXCLUSIVE, SHARED_NO_WRITE, or SHARED_NO_READ_WRITE lock to
+  shared metadata lock.
 
   @param type  Type of lock to which exclusive lock should be downgraded.
 */
@@ -2833,7 +2834,8 @@
 
   /* Only allow downgrade from EXCLUSIVE and SHARED_NO_WRITE. */
   DBUG_ASSERT(m_type == MDL_EXCLUSIVE ||
-              m_type == MDL_SHARED_NO_WRITE);
+              m_type == MDL_SHARED_NO_WRITE ||
+              m_type == MDL_SHARED_NO_READ_WRITE);
 
   mysql_prlock_wrlock(&m_lock->m_rwlock);
   /*

=== modified file 'sql/sql_admin.cc'
--- sql/sql_admin.cc	2013-12-05 17:23:10 +0000
+++ sql/sql_admin.cc	2014-01-21 14:15:01 +0000
@@ -544,6 +544,14 @@
       /* purecov: end */
     }
 
+    if (operator_func == &handler::ha_optimize
+        && (table->table->file->ha_table_flags()
+            & HA_CAN_WRITE_DURING_OPTIMIZE)
+        && table->mdl_request.ticket
+        && table->mdl_request.ticket->get_type() == MDL_SHARED_NO_READ_WRITE)
+    {
+      table->mdl_request.ticket->downgrade_lock(MDL_SHARED_WRITE);
+    } else
     /*
       Close all instances of the table to allow MyISAM "repair"
       to rename files.