Merge lp:~laurynas-biveinis/percona-server/atomic-fio-5.5 into lp:percona-server/5.5

[Laurynas Biveinis]

2nd MP:
- all the review comments addressed;
- complete_io label in fil_extend_space_to_desired_space() moved to skip fil_node_complete_io(). That is required only for the case when the node is extended by writing to it, not posix_fallocate() call.

Jenkins at http://jenkins.percona.com/job/percona-server-5.5-param/742/. Still running, but enough slaves completed for confidence. (This is a low-impact MP for common code path).
Note that while submitted branch is a GCA, the Jenkins-tested one is based on trunk. This is to save a staging test run.

Please review this without a 5.6 MP. That one is ready too, but Jenkins testing is postponed to let the more urgent 5.5 Jenkins jobs proceed.

Implement directFS Fusion I/O atomic writes for 5.5.
https://blueprints.launchpad.net/percona-server/+spec/atomic-writes-beta-5.5

http://jenkins.percona.com/job/percona-server-5.5-param/738/

    Implement atomic write support for Fusion I/O storage with directFS
    file system, implementing blueprint
    https://blueprints.launchpad.net/percona-server/+spec/atomic-writes-beta-5.5

    This implementation is based on MariaDB implementation at
    https://mariadb.atlassian.net/browse/MDEV-4338.

    - Add new InnoDB global, read-only option innodb_use_atomic_writes.
    - If this option is enabled, then at InnoDB initialization disable the
      doublewrite buffer if it's enabled and set file flush method to
      O_DIRECT if it's not O_DIRECT or ALL_O_DIRECT.
    - Add new function os_file_set_atomic_writes() that either enables
      atomic writes on a specified file descriptor if a Fusion
      I/O-specific syscall is available, either fails.
    - Call os_file_set_atomic_writes() from os_file_create() on data files
      if atomic writes are enabled.
    - If atomic writes are enabled and posix_fallocate() is available,
      then work around a directFS bug that atomic files fail beyond
      current EOF by:
      - calling os_file_set_size() from fil_extend_space_to_desired_size();
      - calling posix_fallocate() in os_file_set_size().
    - New variable test sys_vars.innodb_use_atomic_writes_basic, re-record
      percona_server_variables_debug and percona_server_variables_release
      tests.

[Alexey Kopytov (akopytov)]

Laurynas,

  - fil_extend_space_to_desired_size() will not work correctly for a
    multi-node ibdata1. The existing code extends the last node, but
    takes the size of other nodes into account by calculating offsets
    as via (start_page_no - file_start_page_no).

    But the new srv_use_posix_fallocate code path just extends the last
    node to the total desired size regardless of other node sizes,
    i.e. overallocate space by always assuming a single-node tablespace.

Minor things (since needs recommitting anyway):

   - spurious comment change on lines 324-247
   - incorrect function calls formatting ("func (...)"):

+ if (ioctl (file, DFS_IOCTL_ATOMIC_WRITE_SET, &atomic_option)) {
+ os_file_handle_error_no_exit (name, "ioctl");
+ os_file_handle_error_no_exit (name, "posix_fallocate");

   - the following doesn't really have any effect, but the convention
     there is to assign variables to their default values, which is
     FALSE for innobase_use_atomic_writes:

+static my_bool innobase_use_atomic_writes = TRUE;

     which is also inconsistent with:

+UNIV_INTERN ibool srv_use_atomic_writes = FALSE;

   - and wrong comment formatting:

+ /* Due to a bug in directFS, using atomics needs
+ * posix_fallocate to extend the file
+ * pwrite() past end of the file won't work
+ */

review: Needs Fixing

[Alexey Kopytov (akopytov)]

review: Approve

[Stewart Smith (stewart)]

review: Approve

=== modified file 'Percona-Server/mysql-test/r/percona_server_variables_debug.result'
--- Percona-Server/mysql-test/r/percona_server_variables_debug.result	2013-05-15 05:05:18 +0000
+++ Percona-Server/mysql-test/r/percona_server_variables_debug.result	2013-05-22 17:19:29 +0000
@@ -178,6 +178,7 @@
 INNODB_TRACK_CHANGED_PAGES
 INNODB_TRX_PURGE_VIEW_UPDATE_ONLY_DEBUG
 INNODB_TRX_RSEG_N_SLOTS_DEBUG
+INNODB_USE_ATOMIC_WRITES
 INNODB_USE_GLOBAL_FLUSH_LOG_AT_TRX_COMMIT
 INNODB_USE_NATIVE_AIO
 INNODB_USE_SYS_MALLOC

=== modified file 'Percona-Server/mysql-test/r/percona_server_variables_release.result'
--- Percona-Server/mysql-test/r/percona_server_variables_release.result	2013-04-04 10:52:07 +0000
+++ Percona-Server/mysql-test/r/percona_server_variables_release.result	2013-05-22 17:19:29 +0000
@@ -170,6 +170,7 @@
 INNODB_THREAD_CONCURRENCY_TIMER_BASED
 INNODB_THREAD_SLEEP_DELAY
 INNODB_TRACK_CHANGED_PAGES
+INNODB_USE_ATOMIC_WRITES
 INNODB_USE_GLOBAL_FLUSH_LOG_AT_TRX_COMMIT
 INNODB_USE_NATIVE_AIO
 INNODB_USE_SYS_MALLOC

=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_use_atomic_writes_basic.result'
--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_use_atomic_writes_basic.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_use_atomic_writes_basic.result	2013-05-22 17:19:29 +0000
@@ -0,0 +1,30 @@
+SELECT @@global.innodb_use_atomic_writes;
+@@global.innodb_use_atomic_writes
+0
+SELECT @@innodb_use_atomic_writes;
+@@innodb_use_atomic_writes
+0
+SELECT COUNT(VARIABLE_VALUE) AS should_be_1
+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES 
+WHERE VARIABLE_NAME='innodb_use_atomic_writes';
+should_be_1
+1
+SELECT COUNT(VARIABLE_VALUE) AS should_be_1
+FROM INFORMATION_SCHEMA.SESSION_VARIABLES 
+WHERE VARIABLE_NAME='innodb_use_atomic_writes';
+should_be_1
+1
+SET @@global.innodb_use_atomic_writes=1;
+ERROR HY000: Variable 'innodb_use_atomic_writes' is a read only variable
+SELECT IF(@@global.innodb_use_atomic_writes, "ON", "OFF") = VARIABLE_VALUE AS should_be_1
+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
+WHERE VARIABLE_NAME='innodb_use_atomic_writes';
+should_be_1
+1
+SELECT @@innodb_use_atomic_writes = @@global.innodb_use_atomic_writes AS should_be_1;
+should_be_1
+1
+SELECT @@local.innodb_use_atomic_writes;
+ERROR HY000: Variable 'innodb_use_atomic_writes' is a GLOBAL variable
+SELECT @@session.innodb_use_atomic_writes;
+ERROR HY000: Variable 'innodb_use_atomic_writes' is a GLOBAL variable

=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_use_atomic_writes_basic.test'
--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_use_atomic_writes_basic.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_use_atomic_writes_basic.test	2013-05-22 17:19:29 +0000
@@ -0,0 +1,29 @@
+# Test that innodb_use_atomic_writes is a global read-only variable
+--source include/have_innodb.inc
+
+SELECT @@global.innodb_use_atomic_writes;
+
+SELECT @@innodb_use_atomic_writes;
+
+SELECT COUNT(VARIABLE_VALUE) AS should_be_1
+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES 
+WHERE VARIABLE_NAME='innodb_use_atomic_writes';
+
+SELECT COUNT(VARIABLE_VALUE) AS should_be_1
+FROM INFORMATION_SCHEMA.SESSION_VARIABLES 
+WHERE VARIABLE_NAME='innodb_use_atomic_writes';
+
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SET @@global.innodb_use_atomic_writes=1;
+
+SELECT IF(@@global.innodb_use_atomic_writes, "ON", "OFF") = VARIABLE_VALUE AS should_be_1
+FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES
+WHERE VARIABLE_NAME='innodb_use_atomic_writes';
+
+SELECT @@innodb_use_atomic_writes = @@global.innodb_use_atomic_writes AS should_be_1;
+
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SELECT @@local.innodb_use_atomic_writes;
+
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SELECT @@session.innodb_use_atomic_writes;

=== modified file 'Percona-Server/storage/innobase/fil/fil0fil.c'
--- Percona-Server/storage/innobase/fil/fil0fil.c	2013-04-04 10:50:07 +0000
+++ Percona-Server/storage/innobase/fil/fil0fil.c	2013-05-22 17:19:29 +0000
@@ -4861,6 +4861,26 @@
 	start_page_no = space->size;
 	file_start_page_no = space->size - node->size;
 
+#ifdef HAVE_POSIX_FALLOCATE
+	if (srv_use_posix_fallocate) {
+		offset_high = (size_after_extend - file_start_page_no)
+			* page_size / (4ULL * 1024 * 1024 * 1024);
+		offset_low = (size_after_extend - file_start_page_no)
+			* page_size % (4ULL * 1024 * 1024 * 1024);
+
+		mutex_exit(&fil_system->mutex);
+		success = os_file_set_size(node->name, node->handle,
+					   offset_low, offset_high);
+		mutex_enter(&fil_system->mutex);
+		if (success) {
+			node->size += (size_after_extend - start_page_no);
+			space->size += (size_after_extend - start_page_no);
+			os_has_said_disk_full = FALSE;
+		}
+		goto complete_io;
+	}
+#endif
+
 	/* Extend at most 64 pages at a time */
 	buf_size = ut_min(64, size_after_extend - start_page_no) * page_size;
 	buf2 = mem_alloc(buf_size + page_size);
@@ -4919,6 +4939,10 @@
 
 	fil_node_complete_io(node, fil_system, OS_FILE_WRITE);
 
+#ifdef HAVE_POSIX_FALLOCATE
+complete_io:
+#endif
+
 	*actual_size = space->size;
 
 #ifndef UNIV_HOTBACKUP

=== modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc'
--- Percona-Server/storage/innobase/handler/ha_innodb.cc	2013-05-21 16:37:02 +0000
+++ Percona-Server/storage/innobase/handler/ha_innodb.cc	2013-05-22 17:19:29 +0000
@@ -186,6 +186,7 @@
 static my_bool	innobase_log_archive			= FALSE;
 static char*	innobase_log_arch_dir			= NULL;
 #endif /* UNIV_LOG_ARCHIVE */
+static my_bool	innobase_use_atomic_writes		= FALSE;
 static my_bool	innobase_use_doublewrite		= TRUE;
 static my_bool	innobase_use_checksums			= TRUE;
 static my_bool	innobase_fast_checksum			= FALSE;
@@ -3118,6 +3119,39 @@
 	srv_kill_idle_transaction = 0;
 #endif
 
+	srv_use_atomic_writes = (ibool) innobase_use_atomic_writes;
+	if (innobase_use_atomic_writes) {
+		fprintf(stderr, "InnoDB: using atomic writes.\n");
+
+		/* Force doublewrite buffer off, atomic writes replace it. */
+		if (srv_use_doublewrite_buf) {
+			fprintf(stderr,
+				"InnoDB: Switching off doublewrite buffer "
+				"because of atomic writes.\n");
+			innobase_use_doublewrite = FALSE;
+			srv_use_doublewrite_buf	= FALSE;
+		}
+
+		/* Force O_DIRECT on Unixes (on Windows writes are always
+		unbuffered)*/
+#ifndef _WIN32
+		if(!innobase_file_flush_method ||
+		   !strstr(innobase_file_flush_method, "O_DIRECT")) {
+			innobase_file_flush_method =
+				srv_file_flush_method_str = (char*)"O_DIRECT";
+			fprintf(stderr,
+				"InnoDB: using O_DIRECT due to atomic "
+				"writes.\n");
+		}
+#endif
+#ifdef HAVE_POSIX_FALLOCATE
+		/* Due to a bug in directFS, using atomics needs
+		posix_fallocate() to extend the file, because pwrite() past the
+		end of the file won't work */
+		srv_use_posix_fallocate = TRUE;
+#endif
+	}
+
 #ifdef HAVE_PSI_INTERFACE
 	/* Register keys with MySQL performance schema */
 	if (PSI_server) {
@@ -12526,6 +12560,15 @@
   "Disable with --skip-innodb-doublewrite.",
   NULL, NULL, TRUE);
 
+static MYSQL_SYSVAR_BOOL(use_atomic_writes, innobase_use_atomic_writes,
+  PLUGIN_VAR_NOCMDARG | PLUGIN_VAR_READONLY,
+  "Prevent partial page writes, via atomic writes (beta). "
+  "The option is used to prevent partial writes in case of a crash/poweroff, "
+  "as faster alternative to doublewrite buffer. "
+  "Currently this option works only "
+  "on Linux only with FusionIO device, and directFS filesystem.",
+  NULL, NULL, FALSE);
+
 static MYSQL_SYSVAR_ULONG(io_capacity, srv_io_capacity,
   PLUGIN_VAR_RQCMDARG,
   "Number of IOPs the server can do. Tunes the background IO rate",
@@ -13180,6 +13223,7 @@
   MYSQL_SYSVAR(doublewrite_file),
   MYSQL_SYSVAR(data_home_dir),
   MYSQL_SYSVAR(doublewrite),
+  MYSQL_SYSVAR(use_atomic_writes),
   MYSQL_SYSVAR(recovery_stats),
   MYSQL_SYSVAR(fast_shutdown),
   MYSQL_SYSVAR(file_io_threads),

=== modified file 'Percona-Server/storage/innobase/include/srv0srv.h'
--- Percona-Server/storage/innobase/include/srv0srv.h	2013-04-27 10:04:14 +0000
+++ Percona-Server/storage/innobase/include/srv0srv.h	2013-05-22 17:19:29 +0000
@@ -246,6 +246,11 @@
 #endif
 
 extern ibool	srv_use_doublewrite_buf;
+extern ibool	srv_use_atomic_writes;
+#ifdef HAVE_POSIX_FALLOCATE
+extern ibool	srv_use_posix_fallocate;
+#endif
+
 extern ibool	srv_use_checksums;
 extern ibool	srv_fast_checksum;
 

=== modified file 'Percona-Server/storage/innobase/os/os0file.c'
--- Percona-Server/storage/innobase/os/os0file.c	2013-04-11 21:03:27 +0000
+++ Percona-Server/storage/innobase/os/os0file.c	2013-05-22 17:19:29 +0000
@@ -62,6 +62,13 @@
 #include <libaio.h>
 #endif
 
+#if defined(UNIV_LINUX) && defined(HAVE_SYS_IOCTL_H)
+# include <sys/ioctl.h>
+# ifndef DFS_IOCTL_ATOMIC_WRITE_SET
+#  define DFS_IOCTL_ATOMIC_WRITE_SET _IOW(0x95, 2, uint)
+# endif
+#endif
+
 /* This specifies the file permissions InnoDB uses when it creates files in
 Unix; the value of os_innodb_umask is initialized in ha_innodb.cc to
 my_umask */
@@ -1367,6 +1374,35 @@
 }
 
 /****************************************************************//**
+Tries to enable the atomic write feature, if available, for the specified file
+handle.
+@return TRUE if success */
+static __attribute__((warn_unused_result))
+ibool
+os_file_set_atomic_writes(
+/*======================*/
+	const char*	name,	/*!< in: name of the file */
+	os_file_t	file)	/*!< in: handle to the file */
+{
+#ifdef DFS_IOCTL_ATOMIC_WRITE_SET
+	int	atomic_option	= 1;
+
+	if (ioctl(file, DFS_IOCTL_ATOMIC_WRITE_SET, &atomic_option)) {
+
+		os_file_handle_error_no_exit(name, "ioctl");
+		return(FALSE);
+	}
+
+	return(TRUE);
+#else
+	fprintf(stderr, "InnoDB: Error: trying to enable atomic writes on "
+		"non-supported platform! Please restart with "
+		"innodb_use_atomic_writes disabled.\n");
+	return(FALSE);
+#endif
+}
+
+/****************************************************************//**
 NOTE! Use the corresponding macro os_file_create(), not directly
 this function!
 Opens an existing file or creates a new.
@@ -1637,6 +1673,14 @@
 	}
 #endif /* USE_FILE_LOCK */
 
+	if (srv_use_atomic_writes && type == OS_DATA_FILE
+	    && os_file_set_atomic_writes(name, file)) {
+
+		*success = FALSE;
+		close(file);
+		file = -1;
+	}
+
 	return(file);
 #endif /* __WIN__ */
 }
@@ -1980,6 +2024,22 @@
 	current_size = 0;
 	desired_size = (ib_int64_t)size + (((ib_int64_t)size_high) << 32);
 
+#ifdef HAVE_POSIX_FALLOCATE
+	if (srv_use_posix_fallocate) {
+
+		if (posix_fallocate(file, current_size, desired_size) == -1) {
+
+			fprintf(stderr, "InnoDB: Error: preallocating file "
+				"space for file \'%s\' failed.  Current size "
+				"%lld, desired size %lld\n",
+				name, current_size, desired_size);
+			os_file_handle_error_no_exit(name, "posix_fallocate");
+			return(FALSE);
+		}
+		return(TRUE);
+	}
+#endif
+
 	/* Write up to 1 megabyte at a time. */
 	buf_size = ut_min(64, (ulint) (desired_size / UNIV_PAGE_SIZE))
 		* UNIV_PAGE_SIZE;

=== modified file 'Percona-Server/storage/innobase/srv/srv0srv.c'
--- Percona-Server/storage/innobase/srv/srv0srv.c	2013-05-21 16:37:02 +0000
+++ Percona-Server/storage/innobase/srv/srv0srv.c	2013-05-22 17:19:29 +0000
@@ -409,6 +409,11 @@
 #endif
 
 UNIV_INTERN ibool	srv_use_doublewrite_buf	= TRUE;
+UNIV_INTERN ibool       srv_use_atomic_writes = FALSE;
+#ifdef HAVE_POSIX_FALLOCATE
+UNIV_INTERN ibool       srv_use_posix_fallocate = FALSE;
+#endif
+
 UNIV_INTERN ibool	srv_use_checksums = TRUE;
 UNIV_INTERN ibool	srv_fast_checksum = FALSE;