Merge lp:~laurynas-biveinis/percona-server/xtradb-thread-priority-flag into lp:percona-server/5.6

[Laurynas Biveinis]

2nd MP:

Thread priority flag, priority mutex, priority free list refill, priority rw latch.

Jenkins http://jenkins.percona.com/job/percona-server-5.6-param/313/. All the yellow dots due to an unfortunate forgotten fprintf(stderr), which fails innodb-log-file-size-1. Will of course remove in the next push.

Additionally tested by making sync_arr_wake_threads_if_sema_free() a no-op, in order to catch any missed wake up events.

Thread priority flag changes from the original MP:

    - UNIV_THREAD_LOCAL defined to __thread under UNIV_LINUX instead
      of __GNUC__, and as no-op elsewhere. The original suggestion to define it as
      0 on other platforms was not used because lvalue is needed.
    - The thread-priority setting system variables are exposed only
      under UNIV_LINUX too.

1st MP:

Implement thread priority flag
(https://blueprints.launchpad.net/percona-server/+spec/xtradb-thread-priority-flag)
that would be available for InnoDB threads to check whether they
should acquire some shared resource with priority or no. The actual
uses of this flag will be in follow-up.

This flag srv_current_thread_priority is implemented through
thread-local storage. There were two other alternatives considered
for its implementation.
1. Passing the flag value from the DECLARE_THREAD functions to its use
   sites through the callstacks. But such callstacks would be very
   deep and would require patching dozens of InnoDB functions to
   include this new arg.
2. pthread_setspecific()/pthread_getspecific(). This would have the
   advantage of being slightly more portable than TLS, but it's more
   complicated to use: 1) the flag would have to be allocated in
   heap. 2) pthread_key_create() calls through pthread_once() would
   be necessary, moreover they'd have to be placed in mysys instead of
   InnoDB and any affected non-mysys-initializing threads in InnoDB
   would have to be converted to initialize mysys. 3)
   pthread_getspecific() calls would have to happen in very hot code
   paths, such as mutex locking/unlocking code.

Thus TLS appears to be the best option.

Means to change the flag value for individual InnoDB utility threads
are provided for UNIV_PERF_DEBUG or UNIV_DEBUG builds through the new
global dynamic variables innodb_priority_purge, innodb_priority_io,
innodb_priority_cleaner, innodb_priority_master. Added sys_vars tests
for them.

http://jenkins.percona.com/job/percona-server-5.6-param/281/

[Alexey Kopytov (akopytov)]

Just make it Linux-specific for now, i.e. so that the system variables only appear in UNIV_LINUX builds, and srv_current_thread_priority is #define'd to 0 otherwise.

Making it truly portable is more complex than just #ifdef __GNUC__, and I will address it later.

But I also don't like reviewing this change in isolation. Please resubmit with the code that actually makes use of srv_current_thread_priority.

review: Needs Resubmitting

[Laurynas Biveinis (laurynas-biveinis)]

Yes, UNIV_LINUX makes more sense than __GNUC__.

Re. isolation. The whole set of changes would be as follows, in the smallest parts:
- This MP.
- Priority mutex implementation.
- Specific buffer pool free list mutex conversion to priority.
- Non-specific mutex conversion to priority: dict_sys, LRU list, rseg, log.
- Priority rwlatch implementation.
- Non-specific rwlatch conversion to priority: fsp, page_hash, AHI, index, purge.

How this should be distributed among MPs? One big one with everything? Are separate commits OK or one big commit too?

Thanks.

[Alexey Kopytov (akopytov)]

1 MP, multiple commits should be fine.

[Laurynas Biveinis (laurynas-biveinis)]

Repushed. Removed fprintf, and fixed a typo in the last rw_lock_higher_prio_waiters_exist() call in rw_lock_s_lock_spin(), where true was incorrectly passed instead of priority_lock.

[Laurynas Biveinis (laurynas-biveinis)]

Jenkins of the repush http://jenkins.percona.com/job/percona-server-5.6-param/314/

[Alexey Kopytov (akopytov)]

review: Approve

=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_cleaner_basic.result'
--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_cleaner_basic.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_cleaner_basic.result	2013-09-26 14:59:13 +0000
@@ -0,0 +1,31 @@
+SET @start_value = @@GLOBAL.innodb_priority_cleaner;
+SELECT @@GLOBAL.innodb_priority_cleaner;
+@@GLOBAL.innodb_priority_cleaner
+0
+SELECT @@SESSION.innodb_priority_cleaner;
+ERROR HY000: Variable 'innodb_priority_cleaner' is a GLOBAL variable
+SET GLOBAL innodb_priority_cleaner='OFF';
+SELECT @@GLOBAL.innodb_priority_cleaner;
+@@GLOBAL.innodb_priority_cleaner
+0
+SET GLOBAL innodb_priority_cleaner='ON';
+SELECT @@GLOBAL.innodb_priority_cleaner;
+@@GLOBAL.innodb_priority_cleaner
+1
+SET GLOBAL innodb_priority_cleaner=0;
+SELECT @@GLOBAL.innodb_priority_cleaner;
+@@GLOBAL.innodb_priority_cleaner
+0
+SET GLOBAL innodb_priority_cleaner=1;
+SELECT @@GLOBAL.innodb_priority_cleaner;
+@@GLOBAL.innodb_priority_cleaner
+1
+SET GLOBAL innodb_priority_cleaner=1.1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_cleaner'
+SET GLOBAL innodb_priority_cleaner=1e1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_cleaner'
+SET GLOBAL innodb_priority_cleaner=2;
+ERROR 42000: Variable 'innodb_priority_cleaner' can't be set to the value of '2'
+SET GLOBAL innodb_priority_cleaner='foo';
+ERROR 42000: Variable 'innodb_priority_cleaner' can't be set to the value of 'foo'
+SET GLOBAL innodb_priority_cleaner = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_io_basic.result'
--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_io_basic.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_io_basic.result	2013-09-26 14:59:13 +0000
@@ -0,0 +1,31 @@
+SET @start_value = @@GLOBAL.innodb_priority_io;
+SELECT @@GLOBAL.innodb_priority_io;
+@@GLOBAL.innodb_priority_io
+0
+SELECT @@SESSION.innodb_priority_io;
+ERROR HY000: Variable 'innodb_priority_io' is a GLOBAL variable
+SET GLOBAL innodb_priority_io='OFF';
+SELECT @@GLOBAL.innodb_priority_io;
+@@GLOBAL.innodb_priority_io
+0
+SET GLOBAL innodb_priority_io='ON';
+SELECT @@GLOBAL.innodb_priority_io;
+@@GLOBAL.innodb_priority_io
+1
+SET GLOBAL innodb_priority_io=0;
+SELECT @@GLOBAL.innodb_priority_io;
+@@GLOBAL.innodb_priority_io
+0
+SET GLOBAL innodb_priority_io=1;
+SELECT @@GLOBAL.innodb_priority_io;
+@@GLOBAL.innodb_priority_io
+1
+SET GLOBAL innodb_priority_io=1.1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_io'
+SET GLOBAL innodb_priority_io=1e1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_io'
+SET GLOBAL innodb_priority_io=2;
+ERROR 42000: Variable 'innodb_priority_io' can't be set to the value of '2'
+SET GLOBAL innodb_priority_io='foo';
+ERROR 42000: Variable 'innodb_priority_io' can't be set to the value of 'foo'
+SET GLOBAL innodb_priority_io = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_master_basic.result'
--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_master_basic.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_master_basic.result	2013-09-26 14:59:13 +0000
@@ -0,0 +1,31 @@
+SET @start_value = @@GLOBAL.innodb_priority_master;
+SELECT @@GLOBAL.innodb_priority_master;
+@@GLOBAL.innodb_priority_master
+0
+SELECT @@SESSION.innodb_priority_master;
+ERROR HY000: Variable 'innodb_priority_master' is a GLOBAL variable
+SET GLOBAL innodb_priority_master='OFF';
+SELECT @@GLOBAL.innodb_priority_master;
+@@GLOBAL.innodb_priority_master
+0
+SET GLOBAL innodb_priority_master='ON';
+SELECT @@GLOBAL.innodb_priority_master;
+@@GLOBAL.innodb_priority_master
+1
+SET GLOBAL innodb_priority_master=0;
+SELECT @@GLOBAL.innodb_priority_master;
+@@GLOBAL.innodb_priority_master
+0
+SET GLOBAL innodb_priority_master=1;
+SELECT @@GLOBAL.innodb_priority_master;
+@@GLOBAL.innodb_priority_master
+1
+SET GLOBAL innodb_priority_master=1.1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_master'
+SET GLOBAL innodb_priority_master=1e1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_master'
+SET GLOBAL innodb_priority_master=2;
+ERROR 42000: Variable 'innodb_priority_master' can't be set to the value of '2'
+SET GLOBAL innodb_priority_master='foo';
+ERROR 42000: Variable 'innodb_priority_master' can't be set to the value of 'foo'
+SET GLOBAL innodb_priority_master = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_purge_basic.result'
--- Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_purge_basic.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/r/innodb_priority_purge_basic.result	2013-09-26 14:59:13 +0000
@@ -0,0 +1,31 @@
+SET @start_value = @@GLOBAL.innodb_priority_purge;
+SELECT @@GLOBAL.innodb_priority_purge;
+@@GLOBAL.innodb_priority_purge
+0
+SELECT @@SESSION.innodb_priority_purge;
+ERROR HY000: Variable 'innodb_priority_purge' is a GLOBAL variable
+SET GLOBAL innodb_priority_purge='OFF';
+SELECT @@GLOBAL.innodb_priority_purge;
+@@GLOBAL.innodb_priority_purge
+0
+SET GLOBAL innodb_priority_purge='ON';
+SELECT @@GLOBAL.innodb_priority_purge;
+@@GLOBAL.innodb_priority_purge
+1
+SET GLOBAL innodb_priority_purge=0;
+SELECT @@GLOBAL.innodb_priority_purge;
+@@GLOBAL.innodb_priority_purge
+0
+SET GLOBAL innodb_priority_purge=1;
+SELECT @@GLOBAL.innodb_priority_purge;
+@@GLOBAL.innodb_priority_purge
+1
+SET GLOBAL innodb_priority_purge=1.1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_purge'
+SET GLOBAL innodb_priority_purge=1e1;
+ERROR 42000: Incorrect argument type to variable 'innodb_priority_purge'
+SET GLOBAL innodb_priority_purge=2;
+ERROR 42000: Variable 'innodb_priority_purge' can't be set to the value of '2'
+SET GLOBAL innodb_priority_purge='foo';
+ERROR 42000: Variable 'innodb_priority_purge' can't be set to the value of 'foo'
+SET GLOBAL innodb_priority_purge = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_cleaner_basic.test'
--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_cleaner_basic.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_cleaner_basic.test	2013-09-26 14:59:13 +0000
@@ -0,0 +1,40 @@
+--source include/have_debug.inc
+--source include/have_innodb.inc
+
+if (`SELECT @@version_compile_os LIKE "Linux" = 0`)
+{
+  skip Needs Linux;
+}
+
+# A dynamic, global variable
+
+SET @start_value = @@GLOBAL.innodb_priority_cleaner;
+
+# Default value
+SELECT @@GLOBAL.innodb_priority_cleaner;
+
+# Global only
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SELECT @@SESSION.innodb_priority_cleaner;
+
+# Correct values
+SET GLOBAL innodb_priority_cleaner='OFF';
+SELECT @@GLOBAL.innodb_priority_cleaner;
+SET GLOBAL innodb_priority_cleaner='ON';
+SELECT @@GLOBAL.innodb_priority_cleaner;
+SET GLOBAL innodb_priority_cleaner=0;
+SELECT @@GLOBAL.innodb_priority_cleaner;
+SET GLOBAL innodb_priority_cleaner=1;
+SELECT @@GLOBAL.innodb_priority_cleaner;
+
+# Incorrect values
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_cleaner=1.1;
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_cleaner=1e1;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_cleaner=2;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_cleaner='foo';
+
+SET GLOBAL innodb_priority_cleaner = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_io_basic.test'
--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_io_basic.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_io_basic.test	2013-09-26 14:59:13 +0000
@@ -0,0 +1,40 @@
+--source include/have_debug.inc
+--source include/have_innodb.inc
+
+if (`SELECT @@version_compile_os LIKE "Linux" = 0`)
+{
+  skip Needs Linux;
+}
+
+# A dynamic, global variable
+
+SET @start_value = @@GLOBAL.innodb_priority_io;
+
+# Default value
+SELECT @@GLOBAL.innodb_priority_io;
+
+# Global only
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SELECT @@SESSION.innodb_priority_io;
+
+# Correct values
+SET GLOBAL innodb_priority_io='OFF';
+SELECT @@GLOBAL.innodb_priority_io;
+SET GLOBAL innodb_priority_io='ON';
+SELECT @@GLOBAL.innodb_priority_io;
+SET GLOBAL innodb_priority_io=0;
+SELECT @@GLOBAL.innodb_priority_io;
+SET GLOBAL innodb_priority_io=1;
+SELECT @@GLOBAL.innodb_priority_io;
+
+# Incorrect values
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_io=1.1;
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_io=1e1;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_io=2;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_io='foo';
+
+SET GLOBAL innodb_priority_io = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_master_basic.test'
--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_master_basic.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_master_basic.test	2013-09-26 14:59:13 +0000
@@ -0,0 +1,40 @@
+--source include/have_debug.inc
+--source include/have_innodb.inc
+
+if (`SELECT @@version_compile_os LIKE "Linux" = 0`)
+{
+  skip Needs Linux;
+}
+
+# A dynamic, global variable
+
+SET @start_value = @@GLOBAL.innodb_priority_master;
+
+# Default value
+SELECT @@GLOBAL.innodb_priority_master;
+
+# Global only
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SELECT @@SESSION.innodb_priority_master;
+
+# Correct values
+SET GLOBAL innodb_priority_master='OFF';
+SELECT @@GLOBAL.innodb_priority_master;
+SET GLOBAL innodb_priority_master='ON';
+SELECT @@GLOBAL.innodb_priority_master;
+SET GLOBAL innodb_priority_master=0;
+SELECT @@GLOBAL.innodb_priority_master;
+SET GLOBAL innodb_priority_master=1;
+SELECT @@GLOBAL.innodb_priority_master;
+
+# Incorrect values
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_master=1.1;
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_master=1e1;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_master=2;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_master='foo';
+
+SET GLOBAL innodb_priority_master = @start_value;

=== added file 'Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_purge_basic.test'
--- Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_purge_basic.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/sys_vars/t/innodb_priority_purge_basic.test	2013-09-26 14:59:13 +0000
@@ -0,0 +1,40 @@
+--source include/have_debug.inc
+--source include/have_innodb.inc
+
+if (`SELECT @@version_compile_os LIKE "Linux" = 0`)
+{
+  skip Needs Linux;
+}
+
+# A dynamic, global variable
+
+SET @start_value = @@GLOBAL.innodb_priority_purge;
+
+# Default value
+SELECT @@GLOBAL.innodb_priority_purge;
+
+# Global only
+--error ER_INCORRECT_GLOBAL_LOCAL_VAR
+SELECT @@SESSION.innodb_priority_purge;
+
+# Correct values
+SET GLOBAL innodb_priority_purge='OFF';
+SELECT @@GLOBAL.innodb_priority_purge;
+SET GLOBAL innodb_priority_purge='ON';
+SELECT @@GLOBAL.innodb_priority_purge;
+SET GLOBAL innodb_priority_purge=0;
+SELECT @@GLOBAL.innodb_priority_purge;
+SET GLOBAL innodb_priority_purge=1;
+SELECT @@GLOBAL.innodb_priority_purge;
+
+# Incorrect values
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_purge=1.1;
+--error ER_WRONG_TYPE_FOR_VAR
+SET GLOBAL innodb_priority_purge=1e1;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_purge=2;
+--error ER_WRONG_VALUE_FOR_VAR
+SET GLOBAL innodb_priority_purge='foo';
+
+SET GLOBAL innodb_priority_purge = @start_value;

=== modified file 'Percona-Server/storage/innobase/btr/btr0sea.cc'
--- Percona-Server/storage/innobase/btr/btr0sea.cc	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/btr/btr0sea.cc	2013-09-26 14:59:13 +0000
@@ -72,7 +72,7 @@
 being updated in-place! We can use fact (1) to perform unique searches to
 indexes. */
 
-UNIV_INTERN rw_lock_t*		btr_search_latch_arr;
+UNIV_INTERN prio_rw_lock_t*	btr_search_latch_arr;
 
 /** padding to prevent other memory update hotspots from residing on
 the same memory cache line */
@@ -184,8 +184,8 @@
 	/* btr_search_index_num is constrained to machine word size for
 	historical reasons. This limitation can be easily removed later. */
 
-	btr_search_latch_arr = (rw_lock_t *)
-		mem_alloc(sizeof(rw_lock_t) * btr_search_index_num);
+	btr_search_latch_arr = (prio_rw_lock_t *)
+		mem_alloc(sizeof(prio_rw_lock_t) * btr_search_index_num);
 
 	btr_search_sys = (btr_search_sys_t*)
 		mem_alloc(sizeof(btr_search_sys_t));

=== modified file 'Percona-Server/storage/innobase/buf/buf0buddy.cc'
--- Percona-Server/storage/innobase/buf/buf0buddy.cc	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/buf/buf0buddy.cc	2013-09-26 14:59:13 +0000
@@ -548,7 +548,7 @@
 	ib_mutex_t*	mutex;
 	ulint		space;
 	ulint		offset;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	ut_ad(mutex_own(&buf_pool->zip_free_mutex));
 	ut_ad(!mutex_own(&buf_pool->zip_mutex));

=== modified file 'Percona-Server/storage/innobase/buf/buf0buf.cc'
--- Percona-Server/storage/innobase/buf/buf0buf.cc	2013-09-25 05:20:37 +0000
+++ Percona-Server/storage/innobase/buf/buf0buf.cc	2013-09-26 14:59:13 +0000
@@ -1640,7 +1640,7 @@
 	buf_page_t*	bpage;
 	ulint		i;
 	buf_pool_t*	buf_pool = buf_pool_get(space, offset);
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
 
@@ -1756,7 +1756,7 @@
 {
 #ifdef UNIV_SYNC_DEBUG
 	/* We must also own the appropriate hash_bucket mutex. */
-	rw_lock_t* hash_lock = buf_page_hash_lock_get(buf_pool, fold);
+	prio_rw_lock_t* hash_lock = buf_page_hash_lock_get(buf_pool, fold);
 	ut_ad(rw_lock_own(hash_lock, RW_LOCK_EX));
 #endif /* UNIV_SYNC_DEBUG */
 
@@ -1782,7 +1782,7 @@
 	buf_page_t*	bpage;
 	buf_pool_t*	buf_pool = buf_pool_get(space, offset);
 	ulint		fold = buf_page_address_fold(space, offset);
-	rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool,
+	prio_rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool,
 							     fold);
 
 	rw_lock_x_lock(hash_lock);
@@ -1828,7 +1828,7 @@
 	buf_page_t*	bpage;
 	buf_pool_t*	buf_pool = buf_pool_get(space, offset);
 	ulint		fold	= buf_page_address_fold(space, offset);
-	rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool,
+	prio_rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool,
 							     fold);
 
 	rw_lock_s_lock(hash_lock);
@@ -1919,7 +1919,7 @@
 {
 	buf_page_t*	bpage;
 	buf_pool_t*	buf_pool = buf_pool_get(space, offset);
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	bpage = buf_page_hash_get_s_locked(buf_pool, space, offset,
 					   &hash_lock);
@@ -1953,7 +1953,7 @@
 {
 	buf_page_t*	bpage;
 	buf_pool_t*	buf_pool = buf_pool_get(space, offset);
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	bpage = buf_page_hash_get_s_locked(buf_pool, space, offset,
 					   &hash_lock);
@@ -2031,7 +2031,7 @@
 {
 	buf_page_t*	bpage;
 	ib_mutex_t*	block_mutex;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 	ibool		discard_attempted = FALSE;
 	ibool		must_read;
 	trx_t*		trx = NULL;
@@ -2505,7 +2505,7 @@
 	unsigned	access_time;
 	ulint		fix_type;
 	ibool		must_read;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 	ib_mutex_t*	block_mutex;
 	buf_page_t*	hash_bpage;
 	ulint		retries = 0;
@@ -3286,7 +3286,7 @@
 	ibool		success;
 	ulint		fix_type;
 	buf_pool_t*	buf_pool = buf_pool_get(space_id, page_no);
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	ut_ad(mtr);
 	ut_ad(mtr->state == MTR_ACTIVE);
@@ -3498,7 +3498,7 @@
 	buf_block_t*	block;
 	buf_page_t*	bpage	= NULL;
 	buf_page_t*	watch_page;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 	mtr_t		mtr;
 	ulint		fold;
 	ibool		lru	= FALSE;
@@ -3751,7 +3751,7 @@
 	ulint		fold;
 	buf_block_t*	free_block	= NULL;
 	buf_pool_t*	buf_pool	= buf_pool_get(space, offset);
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	ut_ad(mtr);
 	ut_ad(mtr->state == MTR_ACTIVE);
@@ -4007,7 +4007,7 @@
 	ibool		ret = TRUE;
 	const ulint	fold = buf_page_address_fold(bpage->space,
 						     bpage->offset);
-	rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
+	prio_rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
 
 	/* First unfix and release lock on the bpage */
 	mutex_enter(&buf_pool->LRU_list_mutex);

=== modified file 'Percona-Server/storage/innobase/buf/buf0flu.cc'
--- Percona-Server/storage/innobase/buf/buf0flu.cc	2013-09-25 05:20:37 +0000
+++ Percona-Server/storage/innobase/buf/buf0flu.cc	2013-09-26 14:59:13 +0000
@@ -1141,7 +1141,7 @@
 	buf_page_t*	bpage;
 	buf_pool_t*	buf_pool = buf_pool_get(space, offset);
 	bool		ret;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 	ib_mutex_t*	block_mutex;
 
 	ut_ad(flush_type == BUF_FLUSH_LRU
@@ -1260,7 +1260,7 @@
 	for (i = low; i < high; i++) {
 
 		buf_page_t*	bpage;
-		rw_lock_t*	hash_lock;
+		prio_rw_lock_t*	hash_lock;
 		ib_mutex_t*	block_mutex;
 
 		if ((count + n_flushed) >= n_to_flush) {
@@ -2452,6 +2452,8 @@
 
 	while (srv_shutdown_state == SRV_SHUTDOWN_NONE) {
 
+		srv_current_thread_priority = srv_cleaner_thread_priority;
+
 		/* The page_cleaner skips sleep if the server is
 		idle and there are no pending IOs in the buffer pool
 		and there is work to do. */

=== modified file 'Percona-Server/storage/innobase/buf/buf0lru.cc'
--- Percona-Server/storage/innobase/buf/buf0lru.cc	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/buf/buf0lru.cc	2013-09-26 14:59:13 +0000
@@ -751,7 +751,7 @@
 	     bpage != NULL;
 	     /* No op */) {
 
-		rw_lock_t*	hash_lock;
+		prio_rw_lock_t*	hash_lock;
 		buf_page_t*	prev_bpage;
 		ib_mutex_t*	block_mutex = NULL;
 
@@ -1187,7 +1187,7 @@
 {
 	buf_block_t*	block;
 
-	mutex_enter(&buf_pool->free_list_mutex);
+	mutex_enter_last(&buf_pool->free_list_mutex);
 
 	block = (buf_block_t*) UT_LIST_GET_LAST(buf_pool->free);
 
@@ -1900,7 +1900,7 @@
 	buf_pool_t*	buf_pool = buf_pool_from_bpage(bpage);
 	const ulint	fold = buf_page_address_fold(bpage->space,
 						     bpage->offset);
-	rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
+	prio_rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
 
 	ib_mutex_t*	block_mutex = buf_page_get_mutex(bpage);
 
@@ -2228,7 +2228,7 @@
 		page_zip_set_size(&block->page.zip, 0);
 	}
 
-	mutex_enter(&buf_pool->free_list_mutex);
+	mutex_enter_first(&buf_pool->free_list_mutex);
 	buf_block_set_state(block, BUF_BLOCK_NOT_USED);
 	UT_LIST_ADD_FIRST(list, buf_pool->free, (&block->page));
 	ut_d(block->page.in_free_list = TRUE);
@@ -2264,7 +2264,7 @@
 	ulint			fold;
 	const buf_page_t*	hashed_bpage;
 	buf_pool_t*		buf_pool = buf_pool_from_bpage(bpage);
-	rw_lock_t*		hash_lock;
+	prio_rw_lock_t*		hash_lock;
 
 	ut_ad(bpage);
 	ut_ad(mutex_own(&buf_pool->LRU_list_mutex));
@@ -2513,7 +2513,7 @@
 #ifdef UNIV_SYNC_DEBUG
 	const ulint	fold = buf_page_address_fold(bpage->space,
 						     bpage->offset);
-	rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
+	prio_rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
 #endif
 	ib_mutex_t*	block_mutex = buf_page_get_mutex(bpage);
 

=== modified file 'Percona-Server/storage/innobase/buf/buf0rea.cc'
--- Percona-Server/storage/innobase/buf/buf0rea.cc	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/buf/buf0rea.cc	2013-09-26 14:59:13 +0000
@@ -65,7 +65,7 @@
 					== BUF_BLOCK_FILE_PAGE);
 	const ulint	fold = buf_page_address_fold(bpage->space,
 						     bpage->offset);
-	rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
+	prio_rw_lock_t*	hash_lock = buf_page_hash_lock_get(buf_pool, fold);
 
 	mutex_enter(&buf_pool->LRU_list_mutex);
 	rw_lock_x_lock(hash_lock);
@@ -347,7 +347,7 @@
 
 	for (i = low; i < high; i++) {
 
-		rw_lock_t*	hash_lock;
+		prio_rw_lock_t*	hash_lock;
 
 		const buf_page_t* bpage =
 			buf_page_hash_get_s_locked(buf_pool, space, i,
@@ -642,7 +642,7 @@
 
 	for (i = low; i < high; i++) {
 
-		rw_lock_t*	hash_lock;
+		prio_rw_lock_t*	hash_lock;
 
 		bpage = buf_page_hash_get_s_locked(buf_pool, space, i,
 						   &hash_lock);

=== modified file 'Percona-Server/storage/innobase/fil/fil0fil.cc'
--- Percona-Server/storage/innobase/fil/fil0fil.cc	2013-09-09 14:20:25 +0000
+++ Percona-Server/storage/innobase/fil/fil0fil.cc	2013-09-26 14:59:13 +0000
@@ -238,7 +238,7 @@
 	hash_node_t	hash;	/*!< hash chain node */
 	hash_node_t	name_hash;/*!< hash chain the name_hash table */
 #ifndef UNIV_HOTBACKUP
-	rw_lock_t	latch;	/*!< latch protecting the file space storage
+	prio_rw_lock_t	latch;	/*!< latch protecting the file space storage
 				allocation */
 #endif /* !UNIV_HOTBACKUP */
 	UT_LIST_NODE_T(fil_space_t) unflushed_spaces;
@@ -545,7 +545,7 @@
 Returns the latch of a file space.
 @return	latch protecting storage allocation */
 UNIV_INTERN
-rw_lock_t*
+prio_rw_lock_t*
 fil_space_get_latch(
 /*================*/
 	ulint	id,	/*!< in: space id */

=== modified file 'Percona-Server/storage/innobase/fsp/fsp0fsp.cc'
--- Percona-Server/storage/innobase/fsp/fsp0fsp.cc	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/fsp/fsp0fsp.cc	2013-09-26 14:59:13 +0000
@@ -2044,7 +2044,7 @@
 	ib_id_t		seg_id;
 	buf_block_t*	block	= 0; /* remove warning */
 	fseg_header_t*	header	= 0; /* remove warning */
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	ibool		success;
 	ulint		n_reserved;
 	ulint		i;
@@ -2212,7 +2212,7 @@
 	ulint		space;
 	ulint		flags;
 	ulint		zip_size;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 
 	space = page_get_space_id(page_align(header));
 	latch = fil_space_get_latch(space, &flags);
@@ -2632,7 +2632,7 @@
 	ulint		space;
 	ulint		flags;
 	ulint		zip_size;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	buf_block_t*	block;
 	ulint		n_reserved;
 
@@ -2749,7 +2749,7 @@
 	mtr_t*	mtr)	/*!< in/out: mini-transaction */
 {
 	fsp_header_t*	space_header;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	ulint		n_free_list_ext;
 	ulint		free_limit;
 	ulint		size;
@@ -2865,7 +2865,7 @@
 	ulint		n_free;
 	ulint		n_free_up;
 	ulint		reserve;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	mtr_t		mtr;
 
 	/* The convoluted mutex acquire is to overcome latching order
@@ -3195,7 +3195,7 @@
 	ulint		flags;
 	ulint		zip_size;
 	fseg_inode_t*	seg_inode;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 
 	latch = fil_space_get_latch(space, &flags);
 	zip_size = fsp_flags_get_zip_size(flags);
@@ -3225,7 +3225,7 @@
 	mtr_t		mtr;
 	ibool		is_free;
 	ulint		flags;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	xdes_t*		descr;
 	ulint		zip_size;
 	fseg_inode_t*	seg_inode;
@@ -3350,7 +3350,7 @@
 	ulint		flags;
 	ulint		zip_size;
 	ulint		header_page;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 
 	space = page_get_space_id(page_align(header));
 	header_page = page_get_page_no(page_align(header));
@@ -3438,7 +3438,7 @@
 	ulint		flags;
 	ulint		zip_size;
 	ulint		page_no;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 
 	space = page_get_space_id(page_align(header));
 
@@ -3755,7 +3755,7 @@
 	fsp_header_t*	header;
 	fseg_inode_t*	seg_inode;
 	page_t*		seg_inode_page;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	ulint		size;
 	ulint		flags;
 	ulint		zip_size;
@@ -4007,7 +4007,7 @@
 	fsp_header_t*	header;
 	fseg_inode_t*	seg_inode;
 	page_t*		seg_inode_page;
-	rw_lock_t*	latch;
+	prio_rw_lock_t*	latch;
 	ulint		flags;
 	ulint		zip_size;
 	ulint		size;

=== modified file 'Percona-Server/storage/innobase/ha/hash0hash.cc'
--- Percona-Server/storage/innobase/ha/hash0hash.cc	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/ha/hash0hash.cc	2013-09-26 14:59:13 +0000
@@ -105,15 +105,15 @@
 void
 hash_mutex_exit_all_but(
 /*====================*/
-	hash_table_t*	table,		/*!< in: hash table */
-	ib_mutex_t*	keep_mutex)	/*!< in: mutex to keep */
+	hash_table_t*		table,		/*!< in: hash table */
+	ib_prio_mutex_t*	keep_mutex)	/*!< in: mutex to keep */
 {
 	ulint	i;
 
 	ut_ad(table->type == HASH_TABLE_SYNC_MUTEX);
 	for (i = 0; i < table->n_sync_obj; i++) {
 
-		ib_mutex_t* mutex = table->sync_obj.mutexes + i;
+		ib_prio_mutex_t* mutex = table->sync_obj.mutexes + i;
 		if (UNIV_LIKELY(keep_mutex != mutex)) {
 			mutex_exit(mutex);
 		}
@@ -132,7 +132,7 @@
 	ulint		fold)	/*!< in: fold */
 {
 
-	rw_lock_t* lock = hash_get_lock(table, fold);
+	prio_rw_lock_t* lock = hash_get_lock(table, fold);
 
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	ut_ad(lock);
@@ -155,7 +155,7 @@
 	ulint		fold)	/*!< in: fold */
 {
 
-	rw_lock_t* lock = hash_get_lock(table, fold);
+	prio_rw_lock_t* lock = hash_get_lock(table, fold);
 
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	ut_ad(lock);
@@ -179,7 +179,7 @@
 	ulint		fold)	/*!< in: fold */
 {
 
-	rw_lock_t* lock = hash_get_lock(table, fold);
+	prio_rw_lock_t* lock = hash_get_lock(table, fold);
 
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	ut_ad(lock);
@@ -200,7 +200,7 @@
 	hash_table_t*	table,	/*!< in: hash table */
 	ulint		fold)	/*!< in: fold */
 {
-	rw_lock_t* lock = hash_get_lock(table, fold);
+	prio_rw_lock_t* lock = hash_get_lock(table, fold);
 
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	ut_ad(lock);
@@ -225,7 +225,7 @@
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	for (i = 0; i < table->n_sync_obj; i++) {
 
-		rw_lock_t* lock = table->sync_obj.rw_locks + i;
+		prio_rw_lock_t* lock = table->sync_obj.rw_locks + i;
 #ifdef UNIV_SYNC_DEBUG
 		ut_ad(!rw_lock_own(lock, RW_LOCK_SHARED));
 		ut_ad(!rw_lock_own(lock, RW_LOCK_EX));
@@ -248,7 +248,7 @@
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	for (i = 0; i < table->n_sync_obj; i++) {
 
-		rw_lock_t* lock = table->sync_obj.rw_locks + i;
+		prio_rw_lock_t* lock = table->sync_obj.rw_locks + i;
 #ifdef UNIV_SYNC_DEBUG
 		ut_ad(rw_lock_own(lock, RW_LOCK_EX));
 #endif /* UNIV_SYNC_DEBUG */
@@ -264,14 +264,14 @@
 hash_unlock_x_all_but(
 /*==================*/
 	hash_table_t*	table,		/*!< in: hash table */
-	rw_lock_t*	keep_lock)	/*!< in: lock to keep */
+	prio_rw_lock_t*	keep_lock)	/*!< in: lock to keep */
 {
 	ulint	i;
 
 	ut_ad(table->type == HASH_TABLE_SYNC_RW_LOCK);
 	for (i = 0; i < table->n_sync_obj; i++) {
 
-		rw_lock_t* lock = table->sync_obj.rw_locks + i;
+		prio_rw_lock_t* lock = table->sync_obj.rw_locks + i;
 #ifdef UNIV_SYNC_DEBUG
 		ut_ad(rw_lock_own(lock, RW_LOCK_EX));
 #endif /* UNIV_SYNC_DEBUG */
@@ -373,8 +373,8 @@
 
 	switch (type) {
 	case HASH_TABLE_SYNC_MUTEX:
-		table->sync_obj.mutexes = static_cast<ib_mutex_t*>(
-			mem_alloc(n_sync_obj * sizeof(ib_mutex_t)));
+		table->sync_obj.mutexes = static_cast<ib_prio_mutex_t*>(
+			mem_alloc(n_sync_obj * sizeof(ib_prio_mutex_t)));
 
 		for (i = 0; i < n_sync_obj; i++) {
 			mutex_create(hash_table_mutex_key,
@@ -384,8 +384,8 @@
 		break;
 
 	case HASH_TABLE_SYNC_RW_LOCK:
-		table->sync_obj.rw_locks = static_cast<rw_lock_t*>(
-			mem_alloc(n_sync_obj * sizeof(rw_lock_t)));
+		table->sync_obj.rw_locks = static_cast<prio_rw_lock_t*>(
+			mem_alloc(n_sync_obj * sizeof(prio_rw_lock_t)));
 
 		for (i = 0; i < n_sync_obj; i++) {
 			rw_lock_create(hash_table_rw_lock_key,

=== modified file 'Percona-Server/storage/innobase/handler/ha_innodb.cc'
--- Percona-Server/storage/innobase/handler/ha_innodb.cc	2013-09-24 13:11:08 +0000
+++ Percona-Server/storage/innobase/handler/ha_innodb.cc	2013-09-26 14:59:13 +0000
@@ -16549,6 +16549,31 @@
   PLUGIN_VAR_OPCMDARG | PLUGIN_VAR_READONLY,
   "Number of pages reserved in doublewrite buffer for batch flushing",
   NULL, NULL, 120, 1, 127, 0);
+
+#ifdef UNIV_LINUX
+
+static MYSQL_SYSVAR_BOOL(priority_purge, srv_purge_thread_priority,
+  PLUGIN_VAR_OPCMDARG,
+  "Make purge coordinator and worker threads acquire shared resources with "
+  "priority", NULL, NULL, FALSE);
+
+static MYSQL_SYSVAR_BOOL(priority_io, srv_io_thread_priority,
+  PLUGIN_VAR_OPCMDARG,
+  "Make I/O threads acquire shared resources with priority",
+   NULL, NULL, FALSE);
+
+static MYSQL_SYSVAR_BOOL(priority_cleaner, srv_cleaner_thread_priority,
+  PLUGIN_VAR_OPCMDARG,
+  "Make buffer pool cleaner thread acquire shared resources with priority",
+  NULL, NULL, FALSE);
+
+static MYSQL_SYSVAR_BOOL(priority_master, srv_master_thread_priority,
+  PLUGIN_VAR_OPCMDARG,
+  "Make buffer pool cleaner thread acquire shared resources with priority",
+   NULL, NULL, FALSE);
+
+#endif /* UNIV_LINUX */
+
 #endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */
 
 static MYSQL_SYSVAR_LONG(buffer_pool_instances, innobase_buffer_pool_instances,
@@ -17210,6 +17235,12 @@
 #if defined UNIV_DEBUG || defined UNIV_PERF_DEBUG
   MYSQL_SYSVAR(page_hash_locks),
   MYSQL_SYSVAR(doublewrite_batch_size),
+#ifdef UNIV_LINUX
+  MYSQL_SYSVAR(priority_purge),
+  MYSQL_SYSVAR(priority_io),
+  MYSQL_SYSVAR(priority_cleaner),
+  MYSQL_SYSVAR(priority_master),
+#endif /* UNIV_LINUX */
 #endif /* defined UNIV_DEBUG || defined UNIV_PERF_DEBUG */
   MYSQL_SYSVAR(print_all_deadlocks),
   MYSQL_SYSVAR(cmp_per_index_enabled),

=== modified file 'Percona-Server/storage/innobase/include/btr0sea.h'
--- Percona-Server/storage/innobase/include/btr0sea.h	2013-09-06 13:40:39 +0000
+++ Percona-Server/storage/innobase/include/btr0sea.h	2013-09-26 14:59:13 +0000
@@ -208,7 +208,7 @@
 Returns the adaptive hash index latch for a given index key.
 @return the adaptive hash index latch for a given index key */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 btr_search_get_latch(
 /*=================*/
 	const dict_index_t*	index)	/*!< in: index */

=== modified file 'Percona-Server/storage/innobase/include/btr0sea.ic'
--- Percona-Server/storage/innobase/include/btr0sea.ic	2013-09-06 13:40:39 +0000
+++ Percona-Server/storage/innobase/include/btr0sea.ic	2013-09-26 14:59:13 +0000
@@ -100,7 +100,7 @@
 Returns the adaptive hash index latch for a given index key.
 @return the adaptive hash index latch for a given index key */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 btr_search_get_latch(
 /*=================*/
 	const dict_index_t*	index)	/*!< in: index */

=== modified file 'Percona-Server/storage/innobase/include/btr0types.h'
--- Percona-Server/storage/innobase/include/btr0types.h	2013-09-06 13:40:39 +0000
+++ Percona-Server/storage/innobase/include/btr0types.h	2013-09-26 14:59:13 +0000
@@ -55,7 +55,7 @@
 Bear in mind (3) and (4) when using the hash indexes.
 */
 
-extern rw_lock_t*	btr_search_latch_arr;
+extern prio_rw_lock_t*	btr_search_latch_arr;
 
 #endif /* UNIV_HOTBACKUP */
 

=== modified file 'Percona-Server/storage/innobase/include/buf0buf.h'
--- Percona-Server/storage/innobase/include/buf0buf.h	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/include/buf0buf.h	2013-09-26 14:59:13 +0000
@@ -1256,7 +1256,7 @@
 	buf_pool_t*	buf_pool,	/*!< buffer pool instance */
 	ulint		space,		/*!< in: space id */
 	ulint		offset,		/*!< in: page number */
-	rw_lock_t**	lock,		/*!< in/out: lock of the page
+	prio_rw_lock_t**	lock,	/*!< in/out: lock of the page
 					hash acquired if bpage is
 					found. NULL otherwise. If NULL
 					is passed then the hash_lock
@@ -1282,7 +1282,7 @@
 	buf_pool_t*	buf_pool,	/*!< buffer pool instance */
 	ulint		space,		/*!< in: space id */
 	ulint		offset,		/*!< in: page number */
-	rw_lock_t**	lock,		/*!< in/out: lock of the page
+	prio_rw_lock_t**	lock,	/*!< in/out: lock of the page
 					hash acquired if bpage is
 					found. NULL otherwise. If NULL
 					is passed then the hash_lock
@@ -1818,8 +1818,8 @@
 					pool instance, protects compressed
 					only pages (of type buf_page_t, not
 					buf_block_t */
-	ib_mutex_t	LRU_list_mutex;
-	ib_mutex_t	free_list_mutex;
+	ib_prio_mutex_t	LRU_list_mutex;
+	ib_prio_mutex_t	free_list_mutex;
 	ib_mutex_t	zip_free_mutex;
 	ib_mutex_t	zip_hash_mutex;
 	ib_mutex_t	flush_state_mutex;	/*!< Flush state protection

=== modified file 'Percona-Server/storage/innobase/include/buf0buf.ic'
--- Percona-Server/storage/innobase/include/buf0buf.ic	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/include/buf0buf.ic	2013-09-26 14:59:13 +0000
@@ -1100,7 +1100,7 @@
 
 #ifdef UNIV_SYNC_DEBUG
 	ulint		hash_fold;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 
 	hash_fold = buf_page_address_fold(space, offset);
 	ut_ad(hash_fold == fold);
@@ -1149,7 +1149,7 @@
 	buf_pool_t*	buf_pool,	/*!< buffer pool instance */
 	ulint		space,		/*!< in: space id */
 	ulint		offset,		/*!< in: page number */
-	rw_lock_t**	lock,		/*!< in/out: lock of the page
+	prio_rw_lock_t**	lock,	/*!< in/out: lock of the page
 					hash acquired if bpage is
 					found. NULL otherwise. If NULL
 					is passed then the hash_lock
@@ -1160,7 +1160,7 @@
 {
 	buf_page_t*	bpage = NULL;
 	ulint		fold;
-	rw_lock_t*	hash_lock;
+	prio_rw_lock_t*	hash_lock;
 	ulint		mode = RW_LOCK_SHARED;
 
 	if (lock != NULL) {
@@ -1232,7 +1232,7 @@
 	buf_pool_t*	buf_pool,	/*!< buffer pool instance */
 	ulint		space,		/*!< in: space id */
 	ulint		offset,		/*!< in: page number */
-	rw_lock_t**	lock,		/*!< in/out: lock of the page
+	prio_rw_lock_t**	lock,	/*!< in/out: lock of the page
 					hash acquired if bpage is
 					found. NULL otherwise. If NULL
 					is passed then the hash_lock

=== modified file 'Percona-Server/storage/innobase/include/dict0dict.h'
--- Percona-Server/storage/innobase/include/dict0dict.h	2013-08-14 03:57:21 +0000
+++ Percona-Server/storage/innobase/include/dict0dict.h	2013-09-26 14:59:13 +0000
@@ -1367,7 +1367,7 @@
 Gets the read-write lock of the index tree.
 @return	read-write lock */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 dict_index_get_lock(
 /*================*/
 	dict_index_t*	index)	/*!< in: index */
@@ -1552,7 +1552,7 @@
 
 /* Dictionary system struct */
 struct dict_sys_t{
-	ib_mutex_t		mutex;		/*!< mutex protecting the data
+	ib_prio_mutex_t		mutex;		/*!< mutex protecting the data
 					dictionary; protects also the
 					disk-based dictionary system tables;
 					this mutex serializes CREATE TABLE

=== modified file 'Percona-Server/storage/innobase/include/dict0dict.ic'
--- Percona-Server/storage/innobase/include/dict0dict.ic	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/dict0dict.ic	2013-09-26 14:59:13 +0000
@@ -1175,7 +1175,7 @@
 Gets the read-write lock of the index tree.
 @return	read-write lock */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 dict_index_get_lock(
 /*================*/
 	dict_index_t*	index)	/*!< in: index */

=== modified file 'Percona-Server/storage/innobase/include/dict0mem.h'
--- Percona-Server/storage/innobase/include/dict0mem.h	2013-09-02 10:01:38 +0000
+++ Percona-Server/storage/innobase/include/dict0mem.h	2013-09-26 14:59:13 +0000
@@ -544,7 +544,7 @@
 initialized to 0, NULL or FALSE in dict_mem_index_create(). */
 struct dict_index_t{
 	index_id_t	id;	/*!< id of the index */
-	rw_lock_t*	search_latch; /*!< latch protecting the AHI partition
+	prio_rw_lock_t*	search_latch; /*!< latch protecting the AHI partition
 				      corresponding to this index */
 	hash_table_t*	search_table; /*!< hash table protected by
 				      search_latch */
@@ -632,7 +632,7 @@
 				/*!< approximate number of leaf pages in the
 				index tree */
 	/* @} */
-	rw_lock_t	lock;	/*!< read-write lock protecting the
+	prio_rw_lock_t	lock;	/*!< read-write lock protecting the
 				upper levels of the index tree */
 	trx_id_t	trx_id; /*!< id of the transaction that created this
 				index, or 0 if the index existed

=== modified file 'Percona-Server/storage/innobase/include/fil0fil.h'
--- Percona-Server/storage/innobase/include/fil0fil.h	2013-08-14 03:57:21 +0000
+++ Percona-Server/storage/innobase/include/fil0fil.h	2013-09-26 14:59:13 +0000
@@ -189,7 +189,7 @@
 Returns the latch of a file space.
 @return	latch protecting storage allocation */
 UNIV_INTERN
-rw_lock_t*
+prio_rw_lock_t*
 fil_space_get_latch(
 /*================*/
 	ulint	id,	/*!< in: space id */

=== modified file 'Percona-Server/storage/innobase/include/ha0ha.ic'
--- Percona-Server/storage/innobase/include/ha0ha.ic	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/ha0ha.ic	2013-09-26 14:59:13 +0000
@@ -123,7 +123,7 @@
 		ut_ad(mutex_own(hash_get_mutex(table, fold)));
 	} else if (table->type == HASH_TABLE_SYNC_RW_LOCK) {
 # ifdef UNIV_SYNC_DEBUG
-		rw_lock_t* lock = hash_get_lock(table, fold);
+		prio_rw_lock_t* lock = hash_get_lock(table, fold);
 		ut_ad(rw_lock_own(lock, RW_LOCK_EX));
 # endif
 	} else {
@@ -146,7 +146,7 @@
 		ut_ad(mutex_own(hash_get_mutex(table, fold)));
 	} else if (table->type == HASH_TABLE_SYNC_RW_LOCK) {
 # ifdef UNIV_SYNC_DEBUG
-		rw_lock_t* lock = hash_get_lock(table, fold);
+		prio_rw_lock_t* lock = hash_get_lock(table, fold);
 		ut_ad(rw_lock_own(lock, RW_LOCK_EX)
 		      || rw_lock_own(lock, RW_LOCK_SHARED));
 # endif

=== modified file 'Percona-Server/storage/innobase/include/hash0hash.h'
--- Percona-Server/storage/innobase/include/hash0hash.h	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/hash0hash.h	2013-09-26 14:59:13 +0000
@@ -382,7 +382,7 @@
 Gets the nth mutex in a hash table.
 @return	mutex */
 UNIV_INLINE
-ib_mutex_t*
+ib_prio_mutex_t*
 hash_get_nth_mutex(
 /*===============*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -391,7 +391,7 @@
 Gets the nth rw_lock in a hash table.
 @return	rw_lock */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 hash_get_nth_lock(
 /*==============*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -400,7 +400,7 @@
 Gets the mutex for a fold value in a hash table.
 @return	mutex */
 UNIV_INLINE
-ib_mutex_t*
+ib_prio_mutex_t*
 hash_get_mutex(
 /*===========*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -409,7 +409,7 @@
 Gets the rw_lock for a fold value in a hash table.
 @return	rw_lock */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 hash_get_lock(
 /*==========*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -450,8 +450,8 @@
 void
 hash_mutex_exit_all_but(
 /*====================*/
-	hash_table_t*	table,		/*!< in: hash table */
-	ib_mutex_t*	keep_mutex);	/*!< in: mutex to keep */
+	hash_table_t*		table,		/*!< in: hash table */
+	ib_prio_mutex_t*	keep_mutex);	/*!< in: mutex to keep */
 /************************************************************//**
 s-lock a lock for a fold value in a hash table. */
 UNIV_INTERN
@@ -506,7 +506,7 @@
 hash_unlock_x_all_but(
 /*==================*/
 	hash_table_t*	table,		/*!< in: hash table */
-	rw_lock_t*	keep_lock);	/*!< in: lock to keep */
+	prio_rw_lock_t*	keep_lock);	/*!< in: lock to keep */
 
 #else /* !UNIV_HOTBACKUP */
 # define hash_get_heap(table, fold)	((table)->heap)
@@ -547,10 +547,11 @@
 					rw_locks depending on the type.
 					Must be a power of 2 */
 	union {
-		ib_mutex_t*	mutexes;/* NULL, or an array of mutexes
+		ib_prio_mutex_t*	mutexes;
+					/* NULL, or an array of mutexes
 					used to protect segments of the
 					hash table */
-		rw_lock_t*	rw_locks;/* NULL, or an array of rw_lcoks
+		prio_rw_lock_t*	rw_locks;/* NULL, or an array of rw_lcoks
 					used to protect segments of the
 					hash table */
 	} sync_obj;

=== modified file 'Percona-Server/storage/innobase/include/hash0hash.ic'
--- Percona-Server/storage/innobase/include/hash0hash.ic	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/hash0hash.ic	2013-09-26 14:59:13 +0000
@@ -150,7 +150,7 @@
 Gets the nth mutex in a hash table.
 @return	mutex */
 UNIV_INLINE
-ib_mutex_t*
+ib_prio_mutex_t*
 hash_get_nth_mutex(
 /*===============*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -168,7 +168,7 @@
 Gets the mutex for a fold value in a hash table.
 @return	mutex */
 UNIV_INLINE
-ib_mutex_t*
+ib_prio_mutex_t*
 hash_get_mutex(
 /*===========*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -188,7 +188,7 @@
 Gets the nth rw_lock in a hash table.
 @return	rw_lock */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 hash_get_nth_lock(
 /*==============*/
 	hash_table_t*	table,	/*!< in: hash table */
@@ -206,7 +206,7 @@
 Gets the rw_lock for a fold value in a hash table.
 @return	rw_lock */
 UNIV_INLINE
-rw_lock_t*
+prio_rw_lock_t*
 hash_get_lock(
 /*==========*/
 	hash_table_t*	table,	/*!< in: hash table */

=== modified file 'Percona-Server/storage/innobase/include/log0log.h'
--- Percona-Server/storage/innobase/include/log0log.h	2013-06-25 13:13:06 +0000
+++ Percona-Server/storage/innobase/include/log0log.h	2013-09-26 14:59:13 +0000
@@ -796,7 +796,7 @@
 	ulint		buf_free;	/*!< first free offset within the log
 					buffer */
 #ifndef UNIV_HOTBACKUP
-	ib_mutex_t		mutex;		/*!< mutex protecting the log */
+	ib_prio_mutex_t		mutex;		/*!< mutex protecting the log */
 
 	ib_mutex_t		log_flush_order_mutex;/*!< mutex to serialize access to
 					the flush list when we are putting

=== modified file 'Percona-Server/storage/innobase/include/mtr0mtr.h'
--- Percona-Server/storage/innobase/include/mtr0mtr.h	2013-06-25 13:13:06 +0000
+++ Percona-Server/storage/innobase/include/mtr0mtr.h	2013-09-26 14:59:13 +0000
@@ -234,7 +234,7 @@
 /*=============================*/
 	mtr_t*		mtr,		/*!< in: mtr */
 	ulint		savepoint,	/*!< in: savepoint */
-	rw_lock_t*	lock);		/*!< in: latch to release */
+	prio_rw_lock_t*	lock);		/*!< in: latch to release */
 #else /* !UNIV_HOTBACKUP */
 # define mtr_release_s_latch_at_savepoint(mtr,savepoint,lock) ((void) 0)
 #endif /* !UNIV_HOTBACKUP */
@@ -281,7 +281,7 @@
 void
 mtr_s_lock_func(
 /*============*/
-	rw_lock_t*	lock,	/*!< in: rw-lock */
+	prio_rw_lock_t*	lock,	/*!< in: rw-lock */
 	const char*	file,	/*!< in: file name */
 	ulint		line,	/*!< in: line number */
 	mtr_t*		mtr);	/*!< in: mtr */
@@ -292,7 +292,7 @@
 void
 mtr_x_lock_func(
 /*============*/
-	rw_lock_t*	lock,	/*!< in: rw-lock */
+	prio_rw_lock_t*	lock,	/*!< in: rw-lock */
 	const char*	file,	/*!< in: file name */
 	ulint		line,	/*!< in: line number */
 	mtr_t*		mtr);	/*!< in: mtr */

=== modified file 'Percona-Server/storage/innobase/include/mtr0mtr.ic'
--- Percona-Server/storage/innobase/include/mtr0mtr.ic	2013-05-26 14:20:00 +0000
+++ Percona-Server/storage/innobase/include/mtr0mtr.ic	2013-09-26 14:59:13 +0000
@@ -130,7 +130,7 @@
 /*=============================*/
 	mtr_t*		mtr,		/*!< in: mtr */
 	ulint		savepoint,	/*!< in: savepoint */
-	rw_lock_t*	lock)		/*!< in: latch to release */
+	prio_rw_lock_t*	lock)		/*!< in: latch to release */
 {
 	mtr_memo_slot_t* slot;
 	dyn_array_t*	memo;
@@ -261,7 +261,7 @@
 void
 mtr_s_lock_func(
 /*============*/
-	rw_lock_t*	lock,	/*!< in: rw-lock */
+	prio_rw_lock_t*	lock,	/*!< in: rw-lock */
 	const char*	file,	/*!< in: file name */
 	ulint		line,	/*!< in: line number */
 	mtr_t*		mtr)	/*!< in: mtr */
@@ -280,7 +280,7 @@
 void
 mtr_x_lock_func(
 /*============*/
-	rw_lock_t*	lock,	/*!< in: rw-lock */
+	prio_rw_lock_t*	lock,	/*!< in: rw-lock */
 	const char*	file,	/*!< in: file name */
 	ulint		line,	/*!< in: line number */
 	mtr_t*		mtr)	/*!< in: mtr */

=== modified file 'Percona-Server/storage/innobase/include/srv0srv.h'
--- Percona-Server/storage/innobase/include/srv0srv.h	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/srv0srv.h	2013-09-26 14:59:13 +0000
@@ -479,6 +479,18 @@
 extern const char* srv_io_thread_op_info[];
 extern const char* srv_io_thread_function[];
 
+/* The relative priority of the purge coordinator and worker threads.  */
+extern my_bool srv_purge_thread_priority;
+
+/* The relative priority of the I/O threads.  */
+extern my_bool srv_io_thread_priority;
+
+/* The relative priority of the cleaner thread.  */
+extern my_bool srv_cleaner_thread_priority;
+
+/* The relative priority of the master thread.  */
+extern my_bool srv_master_thread_priority;
+
 /* the number of purge threads to use from the worker pool (currently 0 or 1) */
 extern ulong srv_n_purge_threads;
 

=== modified file 'Percona-Server/storage/innobase/include/sync0rw.h'
--- Percona-Server/storage/innobase/include/sync0rw.h	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/sync0rw.h	2013-09-26 14:59:13 +0000
@@ -96,6 +96,7 @@
 #define X_LOCK_DECR		0x00100000
 
 struct rw_lock_t;
+struct prio_rw_lock_t;
 #ifdef UNIV_SYNC_DEBUG
 struct rw_lock_debug_t;
 #endif /* UNIV_SYNC_DEBUG */
@@ -299,6 +300,24 @@
 #endif /* UNIV_DEBUG */
 	const char*	cmutex_name);	/*!< in: mutex name */
 /******************************************************************//**
+Creates, or rather, initializes a priority rw-lock object in a specified memory
+location (which must be appropriately aligned). The rw-lock is initialized
+to the non-locked state. Explicit freeing of the rw-lock with rw_lock_free
+is necessary only if the memory block containing it is freed. */
+UNIV_INTERN
+void
+rw_lock_create_func(
+/*================*/
+	prio_rw_lock_t*	lock,		/*!< in: pointer to memory */
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+	ulint		level,		/*!< in: level */
+# endif /* UNIV_SYNC_DEBUG */
+	const char*	cfile_name,	/*!< in: file name where created */
+	ulint		cline,		/*!< in: file line where created */
+#endif /* UNIV_DEBUG */
+	const char*	cmutex_name);	/*!< in: mutex name */
+/******************************************************************//**
 Calling this function is obligatory only if the memory buffer containing
 the rw-lock is freed. Removes an rw-lock object from the global list. The
 rw-lock is checked to be in the non-locked state. */
@@ -307,6 +326,15 @@
 rw_lock_free_func(
 /*==============*/
 	rw_lock_t*	lock);	/*!< in: rw-lock */
+/******************************************************************//**
+Calling this function is obligatory only if the memory buffer containing
+the priority rw-lock is freed. Removes an rw-lock object from the global list.
+The rw-lock is checked to be in the non-locked state. */
+UNIV_INTERN
+void
+rw_lock_free_func(
+/*==============*/
+	prio_rw_lock_t*	lock);	/*!< in: rw-lock */
 #ifdef UNIV_DEBUG
 /******************************************************************//**
 Checks that the rw-lock has been initialized and that there are no
@@ -317,6 +345,15 @@
 rw_lock_validate(
 /*=============*/
 	rw_lock_t*	lock);	/*!< in: rw-lock */
+/******************************************************************//**
+Checks that the priority rw-lock has been initialized and that there are no
+simultaneous shared and exclusive locks.
+@return	TRUE */
+UNIV_INTERN
+ibool
+rw_lock_validate(
+/*=============*/
+	prio_rw_lock_t*	lock);	/*!< in: rw-lock */
 #endif /* UNIV_DEBUG */
 /******************************************************************//**
 Low-level function which tries to lock an rw-lock in s-mode. Performs no
@@ -349,6 +386,22 @@
 	const char*	file_name,/*!< in: file name where lock requested */
 	ulint		line);	/*!< in: line where requested */
 /******************************************************************//**
+NOTE! Use the corresponding macro, not directly this function, except if
+you supply the file name and line number. Lock a priority rw-lock in shared
+mode for the current thread, using the relative thread priority.  If the
+rw-lock is locked in exclusive mode, or there is an exclusive lock request
+waiting, the function spins a preset time (controlled by SYNC_SPIN_ROUNDS),
+waiting for the lock, before suspending the thread. */
+UNIV_INLINE
+void
+rw_lock_s_lock_func(
+/*================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+/******************************************************************//**
 NOTE! Use the corresponding macro, not directly this function! Lock an
 rw-lock in exclusive mode for the current thread if the lock can be
 obtained immediately.
@@ -373,6 +426,17 @@
 	rw_lock_t*	lock);	/*!< in/out: rw-lock */
 
 /******************************************************************//**
+Releases a shared mode priority lock. */
+UNIV_INLINE
+void
+rw_lock_s_unlock_func(
+/*==================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the lock may have
+				been passed to another thread to unlock */
+#endif
+	prio_rw_lock_t*	lock);	/*!< in/out: rw-lock */
+/******************************************************************//**
 NOTE! Use the corresponding macro, not directly this function! Lock an
 rw-lock in exclusive mode for the current thread. If the rw-lock is locked
 in shared or exclusive mode, or there is an exclusive lock request waiting,
@@ -385,7 +449,30 @@
 void
 rw_lock_x_lock_func(
 /*================*/
-	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	rw_lock_t*      lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line,	/*!< in: line where requested */
+	bool		priority_lock = false,
+				/*!< in: whether the lock is a priority lock */
+	bool		high_priority = false);
+				/*!< in: whether we are acquiring a priority
+				lock with high priority */
+/******************************************************************//**
+NOTE! Use the corresponding macro, not directly this function! Lock a priority
+rw-lock in exclusive mode for the current thread. If the rw-lock is locked
+in shared or exclusive mode, or there is an exclusive lock request waiting,
+the function spins a preset time (controlled by SYNC_SPIN_ROUNDS), waiting
+for the lock, before suspending the thread. If the same thread has an x-lock
+on the rw-lock, locking succeed, with the following exception: if pass != 0,
+only a single x-lock may be taken on the lock. NOTE: If the same thread has
+an s-lock, locking does not succeed! */
+UNIV_INTERN
+void
+rw_lock_x_lock_func(
+/*================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
 	ulint		pass,	/*!< in: pass value; != 0, if the lock will
 				be passed to another thread to unlock */
 	const char*	file_name,/*!< in: file name where lock requested */
@@ -402,6 +489,17 @@
 #endif
 	rw_lock_t*	lock);	/*!< in/out: rw-lock */
 /******************************************************************//**
+Releases an exclusive mode priority lock. */
+UNIV_INLINE
+void
+rw_lock_x_unlock_func(
+/*==================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the lock may have
+				been passed to another thread to unlock */
+#endif
+	prio_rw_lock_t*	lock);	/*!< in/out: rw-lock */
+/******************************************************************//**
 This function is used in the insert buffer to move the ownership of an
 x-latch on a buffer frame to the current thread. The x-latch was set by
 the buffer read operation and it protected the buffer frame while the
@@ -424,6 +522,15 @@
 rw_lock_get_x_lock_count(
 /*=====================*/
 	const rw_lock_t*	lock);	/*!< in: rw-lock */
+/******************************************************************//**
+Returns the value of writer_count for the priority lock. Does not reserve the
+lock mutex, so the caller must be sure it is not changed during the call.
+@return	value of writer_count */
+UNIV_INLINE
+ulint
+rw_lock_get_x_lock_count(
+/*=====================*/
+	const prio_rw_lock_t*	lock);	/*!< in: rw-lock */
 /********************************************************************//**
 Check if there are threads waiting for the rw-lock.
 @return	1 if waiters, 0 otherwise */
@@ -432,6 +539,14 @@
 rw_lock_get_waiters(
 /*================*/
 	const rw_lock_t*	lock);	/*!< in: rw-lock */
+/********************************************************************//**
+Check if there are threads waiting for the priority rw-lock.
+@return	1 if waiters, 0 otherwise */
+UNIV_INLINE
+ulint
+rw_lock_get_waiters(
+/*================*/
+	const prio_rw_lock_t*	lock);	/*!< in: rw-lock */
 /******************************************************************//**
 Returns the write-status of the lock - this function made more sense
 with the old rw_lock implementation.
@@ -442,6 +557,15 @@
 /*===============*/
 	const rw_lock_t*	lock);	/*!< in: rw-lock */
 /******************************************************************//**
+Returns the write-status of the priority lock - this function made more sense
+with the old rw_lock implementation.
+@return	RW_LOCK_NOT_LOCKED, RW_LOCK_EX, RW_LOCK_WAIT_EX */
+UNIV_INLINE
+ulint
+rw_lock_get_writer(
+/*===============*/
+	const prio_rw_lock_t*	lock);	/*!< in: rw-lock */
+/******************************************************************//**
 Returns the number of readers.
 @return	number of readers */
 UNIV_INLINE
@@ -450,6 +574,14 @@
 /*=====================*/
 	const rw_lock_t*	lock);	/*!< in: rw-lock */
 /******************************************************************//**
+Returns the number of readers.
+@return	number of readers */
+UNIV_INLINE
+ulint
+rw_lock_get_reader_count(
+/*=====================*/
+	const prio_rw_lock_t*	lock);	/*!< in: rw-lock */
+/******************************************************************//**
 Decrements lock_word the specified amount if it is greater than 0.
 This is used by both s_lock and x_lock operations.
 @return	TRUE if decr occurs */
@@ -496,6 +628,17 @@
 	ulint		lock_type)	/*!< in: lock type: RW_LOCK_SHARED,
 					RW_LOCK_EX */
 	__attribute__((warn_unused_result));
+/******************************************************************//**
+Checks if the thread has locked the priority rw-lock in the specified mode,
+with the pass value == 0. */
+UNIV_INTERN
+ibool
+rw_lock_own(
+/*========*/
+	prio_rw_lock_t*	lock,		/*!< in: rw-lock */
+	ulint		lock_type)	/*!< in: lock type: RW_LOCK_SHARED,
+					RW_LOCK_EX */
+	__attribute__((warn_unused_result));
 #endif /* UNIV_SYNC_DEBUG */
 /******************************************************************//**
 Checks if somebody has locked the rw-lock in the specified mode. */
@@ -631,6 +774,27 @@
 #endif /* UNIV_DEBUG */
 };
 
+/** The structure implementing a priority rw lock.  */
+struct prio_rw_lock_t {
+	struct rw_lock_t	base_lock;	/* The regular rw latch
+						provides the lock word etc. for
+						the priority rw lock  */
+	volatile ulint		high_priority_s_waiters;
+						/* If 1, high priority S
+						waiters exist */
+	os_event_t		high_priority_s_event; /* High priority wait
+						array event for S waiters */
+	volatile ulint		high_priority_x_waiters;
+						/* If 1, high priority X
+						waiters exist */
+	os_event_t		high_priority_x_event;
+						/* High priority wait arraay
+						event for X waiters */
+	volatile ulint		high_priority_wait_ex_waiter;
+						/* If 1, a waiting next-writer
+						exists and is high-priority */
+};
+
 #ifdef UNIV_SYNC_DEBUG
 /** The structure for storing debug info of an rw-lock.  All access to this
 structure must be protected by rw_lock_debug_mutex_enter(). */
@@ -695,6 +859,26 @@
 	const char*	cmutex_name);	/*!< in: mutex name */
 
 /******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_create_func()
+NOTE! Please use the corresponding macro rw_lock_create(), not
+directly this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_create_func(
+/*====================*/
+	PSI_rwlock_key  key,		/*!< in: key registered with
+					performance schema */
+	prio_rw_lock_t*	lock,		/*!< in: rw lock */
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+	ulint		level,		/*!< in: level */
+# endif /* UNIV_SYNC_DEBUG */
+	const char*	cfile_name,	/*!< in: file name where created */
+	ulint		cline,		/*!< in: file line where created */
+#endif /* UNIV_DEBUG */
+	const char*	cmutex_name);	/*!< in: mutex name */
+
+/******************************************************************//**
 Performance schema instrumented wrap function for rw_lock_x_lock_func()
 NOTE! Please use the corresponding macro rw_lock_x_lock(), not
 directly this function! */
@@ -707,6 +891,21 @@
 				be passed to another thread to unlock */
 	const char*	file_name,/*!< in: file name where lock requested */
 	ulint		line);	/*!< in: line where requested */
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_x_lock_func()
+NOTE! Please use the corresponding macro rw_lock_x_lock(), not
+directly this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_lock_func(
+/*====================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+
 /******************************************************************//**
 Performance schema instrumented wrap function for
 rw_lock_x_lock_func_nowait()
@@ -719,75 +918,147 @@
 	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
 	const char*	file_name,/*!< in: file name where lock requested */
 	ulint		line);	/*!< in: line where requested */
-/******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_s_lock_func()
-NOTE! Please use the corresponding macro rw_lock_s_lock(), not directly
-this function! */
-UNIV_INLINE
-void
-pfs_rw_lock_s_lock_func(
-/*====================*/
-	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
-	ulint		pass,	/*!< in: pass value; != 0, if the lock will
-				be passed to another thread to unlock */
-	const char*	file_name,/*!< in: file name where lock requested */
-	ulint		line);	/*!< in: line where requested */
-/******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_s_lock_func()
-NOTE! Please use the corresponding macro rw_lock_s_lock(), not directly
-this function!
-@return TRUE if success */
-UNIV_INLINE
-ibool
-pfs_rw_lock_s_lock_low(
-/*===================*/
-	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
-	ulint		pass,	/*!< in: pass value; != 0, if the
-				lock will be passed to another
-				thread to unlock */
-	const char*	file_name, /*!< in: file name where lock requested */
-	ulint		line);	/*!< in: line where requested */
-/******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_x_lock_func()
-NOTE! Please use the corresponding macro rw_lock_x_lock(), not directly
-this function! */
-UNIV_INLINE
-void
-pfs_rw_lock_x_lock_func(
-/*====================*/
-	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
-	ulint		pass,	/*!< in: pass value; != 0, if the lock will
-				be passed to another thread to unlock */
-	const char*	file_name,/*!< in: file name where lock requested */
-	ulint		line);	/*!< in: line where requested */
-/******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_s_unlock_func()
-NOTE! Please use the corresponding macro rw_lock_s_unlock(), not directly
-this function! */
-UNIV_INLINE
-void
-pfs_rw_lock_s_unlock_func(
-/*======================*/
-#ifdef UNIV_SYNC_DEBUG
-	ulint		pass,	/*!< in: pass value; != 0, if the
-				lock may have been passed to another
-				thread to unlock */
-#endif
-	rw_lock_t*	lock);	/*!< in/out: rw-lock */
-/******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_s_unlock_func()
-NOTE! Please use the corresponding macro rw_lock_x_unlock(), not directly
-this function! */
-UNIV_INLINE
-void
-pfs_rw_lock_x_unlock_func(
-/*======================*/
-#ifdef UNIV_SYNC_DEBUG
-	ulint		pass,	/*!< in: pass value; != 0, if the
-				lock may have been passed to another
-				thread to unlock */
-#endif
-	rw_lock_t*	lock);	/*!< in/out: rw-lock */
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_lock_func()
+NOTE! Please use the corresponding macro rw_lock_s_lock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_lock_func(
+/*====================*/
+	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_lock_func()
+NOTE! Please use the corresponding macro rw_lock_s_lock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_lock_func(
+/*====================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_lock_func()
+NOTE! Please use the corresponding macro rw_lock_s_lock(), not directly
+this function!
+@return TRUE if success */
+UNIV_INLINE
+ibool
+pfs_rw_lock_s_lock_low(
+/*===================*/
+	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock will be passed to another
+				thread to unlock */
+	const char*	file_name, /*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_lock_func()
+NOTE! Please use the corresponding macro rw_lock_s_lock(), not directly
+this function!
+@return TRUE if success */
+UNIV_INLINE
+ibool
+pfs_rw_lock_s_lock_low(
+/*===================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock will be passed to another
+				thread to unlock */
+	const char*	file_name, /*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_x_lock_func()
+NOTE! Please use the corresponding macro rw_lock_x_lock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_lock_func(
+/*====================*/
+	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_x_lock_func()
+NOTE! Please use the corresponding macro rw_lock_x_lock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_lock_func(
+/*====================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line);	/*!< in: line where requested */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_unlock_func()
+NOTE! Please use the corresponding macro rw_lock_s_unlock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock may have been passed to another
+				thread to unlock */
+#endif
+	rw_lock_t*	lock);	/*!< in/out: rw-lock */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_unlock_func()
+NOTE! Please use the corresponding macro rw_lock_s_unlock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock may have been passed to another
+			        thread to unlock */
+#endif
+	prio_rw_lock_t*	lock);	/*!< in/out: rw-lock */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_unlock_func()
+NOTE! Please use the corresponding macro rw_lock_x_unlock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock may have been passed to another
+				thread to unlock */
+#endif
+	rw_lock_t*	lock);	/*!< in/out: rw-lock */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_unlock_func()
+NOTE! Please use the corresponding macro rw_lock_x_unlock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock may have been passed to another
+				thread to unlock */
+#endif
+	prio_rw_lock_t*	lock);	/*!< in/out: rw-lock */
 /******************************************************************//**
 Performance schema instrumented wrap function for rw_lock_free_func()
 NOTE! Please use the corresponding macro rw_lock_free(), not directly
@@ -797,6 +1068,15 @@
 pfs_rw_lock_free_func(
 /*==================*/
 	rw_lock_t*	lock);	/*!< in: rw-lock */
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_free_func()
+NOTE! Please use the corresponding macro rw_lock_free(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_free_func(
+/*==================*/
+	prio_rw_lock_t*	lock);	/*!< in: rw-lock */
 #endif  /* UNIV_PFS_RWLOCK */
 
 

=== modified file 'Percona-Server/storage/innobase/include/sync0rw.ic'
--- Percona-Server/storage/innobase/include/sync0rw.ic	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/sync0rw.ic	2013-09-26 14:59:13 +0000
@@ -31,17 +31,22 @@
 *******************************************************/
 
 /******************************************************************//**
-Lock an rw-lock in shared mode for the current thread. If the rw-lock is
-locked in exclusive mode, or there is an exclusive lock request waiting,
-the function spins a preset time (controlled by SYNC_SPIN_ROUNDS),
+Lock a regular or priority rw-lock in shared mode for the current thread. If
+the rw-lock is locked in exclusive mode, or there is an exclusive lock request
+waiting, the function spins a preset time (controlled by SYNC_SPIN_ROUNDS),
 waiting for the lock before suspending the thread. */
 UNIV_INTERN
 void
 rw_lock_s_lock_spin(
 /*================*/
-	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	void*		_lock,	/*!< in: pointer to rw-lock */
 	ulint		pass,	/*!< in: pass value; != 0, if the lock will
 				be passed to another thread to unlock */
+	bool		priority_lock,
+				/*!< in: whether the lock is a priority lock */
+	bool		high_priority,
+				/*!< in: whether we are acquiring a priority
+				lock with high priority */
 	const char*	file_name,/*!< in: file name where lock requested */
 	ulint		line);	/*!< in: line where requested */
 #ifdef UNIV_SYNC_DEBUG
@@ -80,6 +85,20 @@
 }
 
 /********************************************************************//**
+Check if there are threads waiting for the priority rw-lock.
+@return	1 if waiters, 0 otherwise */
+UNIV_INLINE
+ulint
+rw_lock_get_waiters(
+/*================*/
+	const prio_rw_lock_t*	lock)	/*!< in: rw-lock */
+{
+	return rw_lock_get_waiters(&lock->base_lock)
+		| lock->high_priority_s_waiters
+		| lock->high_priority_x_waiters;
+}
+
+/********************************************************************//**
 Sets lock->waiters to 1. It is not an error if lock->waiters is already
 1. On platforms where ATOMIC builtins are used this function enforces a
 memory barrier. */
@@ -137,6 +156,19 @@
 }
 
 /******************************************************************//**
+Returns the write-status of the priority lock - this function made more sense
+with the old rw_lock implementation.
+@return	RW_LOCK_NOT_LOCKED, RW_LOCK_EX, RW_LOCK_WAIT_EX */
+UNIV_INLINE
+ulint
+rw_lock_get_writer(
+/*===============*/
+	const prio_rw_lock_t*	lock)	/*!< in: rw-lock */
+{
+	return(rw_lock_get_writer(&lock->base_lock));
+}
+
+/******************************************************************//**
 Returns the number of readers.
 @return	number of readers */
 UNIV_INLINE
@@ -156,6 +188,18 @@
 	return(0);
 }
 
+/******************************************************************//**
+Returns the number of readers.
+@return	number of readers */
+UNIV_INLINE
+ulint
+rw_lock_get_reader_count(
+/*=====================*/
+	const prio_rw_lock_t*	lock)	/*!< in: rw-lock */
+{
+	return(rw_lock_get_reader_count(&lock->base_lock));
+}
+
 #ifndef INNODB_RW_LOCKS_USE_ATOMICS
 UNIV_INLINE
 ib_mutex_t*
@@ -185,6 +229,19 @@
 }
 
 /******************************************************************//**
+Returns the value of writer_count for the priority lock. Does not reserve the
+lock mutex, so the caller must be sure it is not changed during the call.
+@return	value of writer_count */
+UNIV_INLINE
+ulint
+rw_lock_get_x_lock_count(
+/*=====================*/
+	const prio_rw_lock_t*	lock)	/*!< in: rw-lock */
+{
+	return(rw_lock_get_x_lock_count(&lock->base_lock));
+}
+
+/******************************************************************//**
 Two different implementations for decrementing the lock_word of a rw_lock:
 one for systems supporting atomic operations, one for others. This does
 does not support recusive x-locks: they should be handled by the caller and
@@ -361,7 +418,75 @@
 	} else {
 		/* Did not succeed, try spin wait */
 
-		rw_lock_s_lock_spin(lock, pass, file_name, line);
+		rw_lock_s_lock_spin(lock, pass, false, false, file_name, line);
+
+		return;
+	}
+}
+
+/******************************************************************//**
+Return true if waiters of higher priority than the current thread
+exist.
+@true if waiterss of higher priority exist */
+UNIV_INLINE
+bool
+rw_lock_higher_prio_waiters_exist(
+/*==============================*/
+	bool	priority_lock,	/*!< in: whether the lock is a priority lock */
+	bool	high_priority,	/*!< in: whether we are acquiring a priority
+				lock with high priority */
+	void*	lock)		/*!< in: rw lock */
+{
+	if (high_priority || !priority_lock) {
+		ut_ad(!(!priority_lock && high_priority));
+		return(false);
+	}
+
+	ut_ad(priority_lock && !high_priority);
+
+	prio_rw_lock_t *prio_rw_lock = (prio_rw_lock_t *) lock;
+	return prio_rw_lock->high_priority_wait_ex_waiter > 0
+		|| prio_rw_lock->high_priority_s_waiters > 0
+		|| prio_rw_lock->high_priority_x_waiters > 0;
+}
+
+/******************************************************************//**
+NOTE! Use the corresponding macro, not directly this function, except if
+you supply the file name and line number. Lock a priority rw-lock in shared
+mode for the current thread, using the relative thread priority.  If the
+rw-lock is locked in exclusive mode, or there is an exclusive lock request
+waiting, the function spins a preset time (controlled by SYNC_SPIN_ROUNDS),
+waiting for the lock, before suspending the thread. */
+UNIV_INLINE
+void
+rw_lock_s_lock_func(
+/*================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line)	/*!< in: line where requested */
+{
+#ifdef UNIV_SYNC_DEBUG
+	ut_ad(!rw_lock_own(lock, RW_LOCK_SHARED)); /* see NOTE above */
+	ut_ad(!rw_lock_own(lock, RW_LOCK_EX));
+#endif /* UNIV_SYNC_DEBUG */
+
+	bool	high_priority = srv_current_thread_priority > 0;
+
+	/* Do not attempt to acquire a low-priority S latch if there are
+	high-priority waiters even if such attempt would be successful.  This
+	is to prevent a high priority X request from being starved by a
+	sequence of overlapping regular priority S requests.  */
+
+	if (!rw_lock_higher_prio_waiters_exist(true, high_priority, lock)
+	    && rw_lock_s_lock_low(&lock->base_lock, pass, file_name, line)) {
+
+		return; /* Success */
+	} else {
+		/* Did not succeed, try spin wait */
+		rw_lock_s_lock_spin(lock, pass, true, high_priority, file_name,
+				    line);
 
 		return;
 	}
@@ -380,8 +505,6 @@
 	const char*	file_name,/*!< in: file name where lock requested */
 	ulint		line)	/*!< in: line where requested */
 {
-	os_thread_id_t	curr_thread	= os_thread_get_curr_id();
-
 	ibool success;
 
 #ifdef INNODB_RW_LOCKS_USE_ATOMICS
@@ -401,7 +524,8 @@
 		rw_lock_set_writer_id_and_recursion_flag(lock, TRUE);
 
 	} else if (lock->recursive
-		   && os_thread_eq(lock->writer_thread, curr_thread)) {
+		   && os_thread_eq(lock->writer_thread,
+				   os_thread_get_curr_id())) {
 		/* Relock: this lock_word modification is safe since no other
 		threads can modify (lock, unlock, or reserve) lock_word while
 		there is an exclusive writer and this is the writer thread. */
@@ -469,13 +593,75 @@
 }
 
 /******************************************************************//**
-Releases an exclusive mode lock. */
+Releases a shared mode priority lock. */
 UNIV_INLINE
 void
-rw_lock_x_unlock_func(
+rw_lock_s_unlock_func(
 /*==================*/
 #ifdef UNIV_SYNC_DEBUG
 	ulint		pass,	/*!< in: pass value; != 0, if the lock may have
+				  been passed to another thread to unlock */
+#endif
+	prio_rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	ut_ad(lock->base_lock.lock_word > -X_LOCK_DECR);
+	ut_ad(lock->base_lock.lock_word != 0);
+	ut_ad(lock->base_lock.lock_word < X_LOCK_DECR);
+
+#ifdef UNIV_SYNC_DEBUG
+	rw_lock_remove_debug_info(&lock->base_lock, pass, RW_LOCK_SHARED);
+#endif
+
+	/* Increment lock_word to indicate 1 less reader */
+	if (rw_lock_lock_word_incr(&lock->base_lock, 1) == 0) {
+
+		/* A waiting next-writer exists, either high priority or
+		regular.  Wake up the first waiter in this order: 1) high
+		priority next-writer; 2) high priority X waiters; 3) high
+		priority S waiters; 4) regular priority next-waiter.  This
+		allows high priority requests to overtake an already-waiting
+		regular priority next-waiter.  */
+		if (lock->high_priority_wait_ex_waiter) {
+
+			lock->high_priority_wait_ex_waiter = 0;
+			/* Note that we do not have a separate high priority
+			next-waiter event.  There can be only one such waiter,
+			here we already know it's high priority, no
+			regular-priority wakeup may happen.  */
+			os_event_set(lock->base_lock.wait_ex_event);
+		} else if (lock->high_priority_x_waiters) {
+
+			lock->high_priority_x_waiters = 0;
+			os_event_set(lock->high_priority_x_event);
+		} else if (lock->high_priority_s_waiters) {
+
+			lock->high_priority_s_waiters = 0;
+			os_event_set(lock->high_priority_s_event);
+		} else {
+
+			os_event_set(lock->base_lock.wait_ex_event);
+		}
+		sync_array_object_signalled();
+	}
+
+	ut_ad(rw_lock_validate(lock));
+
+#ifdef UNIV_SYNC_PERF_STAT
+	rw_s_exit_count++;
+#endif
+}
+
+/******************************************************************//**
+Prepares an exclusive mode lock release: resets the recursion flag and removes
+the debug information if needed  and returns the required lock word increment
+value.
+@return lock word increment value to perform the unlock */
+UNIV_INLINE
+ulint
+rw_lock_x_prepare_unlock(
+/*=====================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the lock may have
 				been passed to another thread to unlock */
 #endif
 	rw_lock_t*	lock)	/*!< in/out: rw-lock */
@@ -507,10 +693,32 @@
 		x_lock_incr = 1;
 	}
 
+	return(x_lock_incr);
+}
+
+/******************************************************************//**
+Releases an exclusive mode lock. */
+UNIV_INLINE
+void
+rw_lock_x_unlock_func(
+/*==================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the lock may have
+				been passed to another thread to unlock */
+#endif
+	rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	ulint x_lock_incr = rw_lock_x_prepare_unlock(
+#ifdef UNIV_SYNC_DEBUG
+						     pass,
+#endif
+						     lock);
+
 	if (rw_lock_lock_word_incr(lock, x_lock_incr) == X_LOCK_DECR) {
 		/* Lock is now free. May have to signal read/write waiters.
 		We do not need to signal wait_ex waiters, since they cannot
 		exist when there is a writer. */
+
 		if (lock->waiters) {
 			rw_lock_reset_waiter_flag(lock);
 			os_event_set(lock->event);
@@ -525,6 +733,58 @@
 #endif
 }
 
+/******************************************************************//**
+Releases an exclusive mode priority lock. */
+UNIV_INLINE
+void
+rw_lock_x_unlock_func(
+/*==================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the lock may have
+				been passed to another thread to unlock */
+#endif
+	prio_rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	ulint x_lock_incr = rw_lock_x_prepare_unlock(
+#ifdef UNIV_SYNC_DEBUG
+						     pass,
+#endif
+						     &lock->base_lock);
+
+	if (rw_lock_lock_word_incr(&lock->base_lock, x_lock_incr)
+	    == X_LOCK_DECR) {
+
+		/* Priority lock is now free.  Signal any waiters in this
+	        order: 1) high priority X waiters; 2) high priority S waiters;
+	        3) regular priority waiters.
+		We do not need to signal wait_ex waiters, since they cannot
+		exist when there is a writer. */
+
+		if (lock->high_priority_x_waiters) {
+
+			lock->high_priority_x_waiters = 0;
+			os_event_set(lock->high_priority_x_event);
+			sync_array_object_signalled();
+		} else if (lock->high_priority_s_waiters) {
+
+			lock->high_priority_s_waiters = 0;
+			os_event_set(lock->high_priority_s_event);
+			sync_array_object_signalled();
+		} else if (lock->base_lock.waiters) {
+
+			rw_lock_reset_waiter_flag(&lock->base_lock);
+			os_event_set(lock->base_lock.event);
+			sync_array_object_signalled();
+		}
+	}
+
+	ut_ad(rw_lock_validate(lock));
+
+#ifdef UNIV_SYNC_PERF_STAT
+	rw_x_exit_count++;
+#endif
+}
+
 #ifdef UNIV_PFS_RWLOCK
 
 /******************************************************************//**
@@ -561,6 +821,42 @@
 # endif /* UNIV_DEBUG */
 			    cmutex_name);
 }
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_create_func().
+NOTE! Please use the corresponding macro rw_lock_create(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_create_func(
+/*====================*/
+	mysql_pfs_key_t	key,		/*!< in: key registered with
+					performance schema */
+	prio_rw_lock_t*	lock,		/*!< in: pointer to memory */
+# ifdef UNIV_DEBUG
+#  ifdef UNIV_SYNC_DEBUG
+	ulint		level,		/*!< in: level */
+#  endif /* UNIV_SYNC_DEBUG */
+	const char*	cfile_name,	/*!< in: file name where created */
+	ulint		cline,		/*!< in: file line where created */
+# endif /* UNIV_DEBUG */
+	const char*	cmutex_name)	/*!< in: mutex name */
+{
+	/* Initialize the rwlock for performance schema */
+	lock->base_lock.pfs_psi = PSI_RWLOCK_CALL(init_rwlock)(key, lock);
+
+	/* The actual function to initialize an rwlock */
+	rw_lock_create_func(lock,
+# ifdef UNIV_DEBUG
+#  ifdef UNIV_SYNC_DEBUG
+			    level,
+#  endif /* UNIV_SYNC_DEBUG */
+			    cfile_name,
+			    cline,
+# endif /* UNIV_DEBUG */
+			    cmutex_name);
+}
+
 /******************************************************************//**
 Performance schema instrumented wrap function for rw_lock_x_lock_func()
 NOTE! Please use the corresponding macro rw_lock_x_lock(), not directly
@@ -594,6 +890,42 @@
 		rw_lock_x_lock_func(lock, pass, file_name, line);
 	}
 }
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_x_lock_func()
+NOTE! Please use the corresponding macro rw_lock_x_lock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_lock_func(
+/*====================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line)	/*!< in: line where requested */
+{
+	if (lock->base_lock.pfs_psi != NULL)
+	{
+		PSI_rwlock_locker*	locker;
+		PSI_rwlock_locker_state	state;
+
+		/* Record the entry of rw x lock request in performance schema */
+		locker = PSI_RWLOCK_CALL(start_rwlock_wrwait)(
+			&state, lock->base_lock.pfs_psi, PSI_RWLOCK_WRITELOCK,
+			file_name, line);
+
+		rw_lock_x_lock_func(lock, pass, file_name, line);
+
+		if (locker != NULL)
+			PSI_RWLOCK_CALL(end_rwlock_wrwait)(locker, 0);
+	}
+	else
+	{
+		rw_lock_x_lock_func(lock, pass, file_name, line);
+	}
+}
+
 /******************************************************************//**
 Performance schema instrumented wrap function for
 rw_lock_x_lock_func_nowait()
@@ -650,6 +982,26 @@
 
 	rw_lock_free_func(lock);
 }
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_free_func()
+NOTE! Please use the corresponding macro rw_lock_free(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_free_func(
+/*==================*/
+	prio_rw_lock_t*	lock)	/*!< in: pointer to rw-lock */
+{
+	if (lock->base_lock.pfs_psi != NULL)
+	{
+		PSI_RWLOCK_CALL(destroy_rwlock)(lock->base_lock.pfs_psi);
+		lock->base_lock.pfs_psi = NULL;
+	}
+
+	rw_lock_free_func(lock);
+}
+
 /******************************************************************//**
 Performance schema instrumented wrap function for rw_lock_s_lock_func()
 NOTE! Please use the corresponding macro rw_lock_s_lock(), not
@@ -687,6 +1039,46 @@
 
 	return;
 }
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_lock_func()
+NOTE! Please use the corresponding macro rw_lock_s_lock(), not
+directly this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_lock_func(
+/*====================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock will be passed to another
+				thread to unlock */
+	const char*	file_name,/*!< in: file name where lock
+				requested */
+	ulint		line)	/*!< in: line where requested */
+{
+	if (lock->base_lock.pfs_psi != NULL)
+	{
+		PSI_rwlock_locker*	locker;
+		PSI_rwlock_locker_state	state;
+
+		/* Instrumented to inform we are aquiring a shared rwlock */
+		locker = PSI_RWLOCK_CALL(start_rwlock_rdwait)(
+			&state, lock->base_lock.pfs_psi, PSI_RWLOCK_READLOCK,
+			file_name, line);
+
+		rw_lock_s_lock_func(lock, pass, file_name, line);
+
+		if (locker != NULL)
+			PSI_RWLOCK_CALL(end_rwlock_rdwait)(locker, 0);
+	}
+	else
+	{
+		rw_lock_s_lock_func(lock, pass, file_name, line);
+	}
+
+	return;
+}
+
 /******************************************************************//**
 Performance schema instrumented wrap function for rw_lock_s_lock_func()
 NOTE! Please use the corresponding macro rw_lock_s_lock(), not
@@ -728,55 +1120,129 @@
 }
 
 /******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_x_unlock_func()
-NOTE! Please use the corresponding macro rw_lock_x_unlock(), not directly
-this function! */
-UNIV_INLINE
-void
-pfs_rw_lock_x_unlock_func(
-/*======================*/
-#ifdef UNIV_SYNC_DEBUG
-	ulint		pass,	/*!< in: pass value; != 0, if the
-				lock may have been passed to another
-				thread to unlock */
-#endif
-	rw_lock_t*	lock)	/*!< in/out: rw-lock */
-{
-	/* Inform performance schema we are unlocking the lock */
-	if (lock->pfs_psi != NULL)
-		PSI_RWLOCK_CALL(unlock_rwlock)(lock->pfs_psi);
-
-	rw_lock_x_unlock_func(
-#ifdef UNIV_SYNC_DEBUG
-		pass,
-#endif
-		lock);
-}
-
-/******************************************************************//**
-Performance schema instrumented wrap function for rw_lock_s_unlock_func()
-NOTE! Please use the corresponding macro pfs_rw_lock_s_unlock(), not
-directly this function! */
-UNIV_INLINE
-void
-pfs_rw_lock_s_unlock_func(
-/*======================*/
-#ifdef UNIV_SYNC_DEBUG
-	ulint		pass,	/*!< in: pass value; != 0, if the
-				lock may have been passed to another
-				thread to unlock */
-#endif
-	rw_lock_t*	lock)	/*!< in/out: rw-lock */
-{
-	/* Inform performance schema we are unlocking the lock */
-	if (lock->pfs_psi != NULL)
-		PSI_RWLOCK_CALL(unlock_rwlock)(lock->pfs_psi);
-
-	rw_lock_s_unlock_func(
-#ifdef UNIV_SYNC_DEBUG
-		pass,
-#endif
-		lock);
-
-}
+Performance schema instrumented wrap function for rw_lock_s_lock_func()
+NOTE! Please use the corresponding macro rw_lock_s_lock(), not
+directly this function!
+@return	TRUE if success */
+UNIV_INLINE
+ibool
+pfs_rw_lock_s_lock_low(
+/*===================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				  lock will be passed to another
+				  thread to unlock */
+	const char*	file_name, /*!< in: file name where lock requested */
+	ulint		line)	/*!< in: line where requested */
+{
+	return(pfs_rw_lock_s_lock_low(&lock->base_lock, pass,
+				      file_name, line));
+}
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_x_unlock_func()
+NOTE! Please use the corresponding macro rw_lock_x_unlock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock may have been passed to another
+				thread to unlock */
+#endif
+	rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	/* Inform performance schema we are unlocking the lock */
+	if (lock->pfs_psi != NULL)
+		PSI_RWLOCK_CALL(unlock_rwlock)(lock->pfs_psi);
+
+	rw_lock_x_unlock_func(
+#ifdef UNIV_SYNC_DEBUG
+		pass,
+#endif
+		lock);
+}
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_x_unlock_func()
+NOTE! Please use the corresponding macro rw_lock_x_unlock(), not directly
+this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_x_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+		ulint		pass,	/*!< in: pass value; != 0, if the
+					lock may have been passed to another
+					thread to unlock */
+#endif
+		prio_rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	/* Inform performance schema we are unlocking the lock */
+	if (lock->base_lock.pfs_psi != NULL)
+		PSI_RWLOCK_CALL(unlock_rwlock)(lock->base_lock.pfs_psi);
+
+	rw_lock_x_unlock_func(
+#ifdef UNIV_SYNC_DEBUG
+		pass,
+#endif
+		lock);
+}
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_unlock_func()
+NOTE! Please use the corresponding macro pfs_rw_lock_s_unlock(), not
+directly this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				lock may have been passed to another
+				thread to unlock */
+#endif
+	rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	/* Inform performance schema we are unlocking the lock */
+	if (lock->pfs_psi != NULL)
+		PSI_RWLOCK_CALL(unlock_rwlock)(lock->pfs_psi);
+
+	rw_lock_s_unlock_func(
+#ifdef UNIV_SYNC_DEBUG
+		pass,
+#endif
+		lock);
+
+}
+
+/******************************************************************//**
+Performance schema instrumented wrap function for rw_lock_s_unlock_func()
+NOTE! Please use the corresponding macro pfs_rw_lock_s_unlock(), not
+directly this function! */
+UNIV_INLINE
+void
+pfs_rw_lock_s_unlock_func(
+/*======================*/
+#ifdef UNIV_SYNC_DEBUG
+	ulint		pass,	/*!< in: pass value; != 0, if the
+				  lock may have been passed to another
+				  thread to unlock */
+#endif
+	prio_rw_lock_t*	lock)	/*!< in/out: rw-lock */
+{
+	/* Inform performance schema we are unlocking the lock */
+	if (lock->base_lock.pfs_psi != NULL)
+		PSI_RWLOCK_CALL(unlock_rwlock)(lock->base_lock.pfs_psi);
+
+	rw_lock_s_unlock_func(
+#ifdef UNIV_SYNC_DEBUG
+		pass,
+#endif
+		lock);
+
+}
+
 #endif /* UNIV_PFS_RWLOCK */

=== modified file 'Percona-Server/storage/innobase/include/sync0sync.h'
--- Percona-Server/storage/innobase/include/sync0sync.h	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/include/sync0sync.h	2013-09-26 14:59:13 +0000
@@ -160,6 +160,8 @@
 
 mutex_create
 mutex_enter
+mutex_enter_first
+mutex_enter_last
 mutex_exit
 mutex_enter_nowait
 mutex_free
@@ -195,6 +197,12 @@
 # define mutex_enter_nowait(M)					\
 	pfs_mutex_enter_nowait_func((M), __FILE__, __LINE__)
 
+# define mutex_enter_first(M)					\
+	pfs_mutex_enter_func((M), __FILE__, __LINE__, HIGH_PRIO)
+
+# define mutex_enter_last(M)					\
+	pfs_mutex_enter_func((M), __FILE__, __LINE__, LOW_PRIO)
+
 # define mutex_exit(M)	pfs_mutex_exit_func(M)
 
 # define mutex_free(M)	pfs_mutex_free_func(M)
@@ -221,6 +229,12 @@
 # define mutex_enter_nowait(M)	\
 	mutex_enter_nowait_func((M), __FILE__, __LINE__)
 
+# define mutex_enter_first(M)	\
+	mutex_enter_func((M), __FILE__, __LINE__, HIGH_PRIO)
+
+# define mutex_enter_last(M)	\
+	mutex_enter_func((M), __FILE__, __LINE__, LOW_PRIO)
+
 # define mutex_exit(M)	mutex_exit_func(M)
 
 # define mutex_free(M)	mutex_free_func(M)
@@ -247,6 +261,26 @@
 	const char*	cmutex_name);	/*!< in: mutex name */
 
 /******************************************************************//**
+Creates, or rather, initializes a priority mutex object in a specified memory
+location (which must be appropriately aligned). The mutex is initialized
+in the reset state. Explicit freeing of the mutex with mutex_free is
+necessary only if the memory block containing it is freed. */
+UNIV_INTERN
+void
+mutex_create_func(
+/*==============*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to memory */
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+	ulint			level,		/*!< in: level */
+# endif /* UNIV_SYNC_DEBUG */
+	const char*		cfile_name,	/*!< in: file name where
+						created */
+	ulint			cline,		/*!< in: file line where
+						created */
+#endif /* UNIV_DEBUG */
+	const char*		cmutex_name);	/*!< in: mutex name */
+/******************************************************************//**
 NOTE! Use the corresponding macro mutex_free(), not directly this function!
 Calling this function is obligatory only if the memory buffer containing
 the mutex is freed. Removes a mutex object from the mutex list. The mutex
@@ -256,6 +290,16 @@
 mutex_free_func(
 /*============*/
 	ib_mutex_t*	mutex);	/*!< in: mutex */
+/******************************************************************//**
+NOTE! Use the corresponding macro mutex_free(), not directly this function!
+Calling this function is obligatory only if the memory buffer containing
+the mutex is freed. Removes a priority mutex object from the mutex list. The
+mutex is checked to be in the reset state. */
+UNIV_INTERN
+void
+mutex_free_func(
+/*============*/
+	ib_prio_mutex_t*	mutex);	/*!< in: mutex */
 /**************************************************************//**
 NOTE! The following macro should be used in mutex locking, not the
 corresponding function. */
@@ -275,6 +319,26 @@
 	ib_mutex_t*	mutex,		/*!< in: pointer to mutex */
 	const char*	file_name,	/*!< in: file name where locked */
 	ulint		line);		/*!< in: line where locked */
+/******************************************************************//**
+NOTE! Use the corresponding macro in the header file, not this function
+directly. Locks a priority mutex for the current thread. If the mutex is
+reserved the function spins a preset time (controlled by SYNC_SPIN_ROUNDS)
+waiting for the mutex before suspending the thread. If the thread is suspended,
+the priority argument value determines the relative order for its wake up.  Any
+HIGH_PRIO waiters will be woken up before any LOW_PRIO waiters.  In case of
+DEFAULT_PRIO, the relative priority will be set according to
+srv_current_thread_priority.  */
+UNIV_INLINE
+void
+mutex_enter_func(
+/*=============*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where
+						locked */
+	ulint			line,		/*!< in: line where locked */
+	enum ib_sync_priority	priority = DEFAULT_PRIO);
+						/*!<in: mutex acquisition
+						priority */
 /********************************************************************//**
 NOTE! Use the corresponding macro in the header file, not this function
 directly. Tries to lock the mutex for the current thread. If the lock is not
@@ -288,6 +352,20 @@
 	const char*	file_name,	/*!< in: file name where mutex
 					requested */
 	ulint		line);		/*!< in: line where requested */
+/********************************************************************//**
+NOTE! Use the corresponding macro in the header file, not this function
+directly. Tries to lock the mutex for the current thread. If the lock is not
+acquired immediately, returns with return value 1.
+@return	0 if succeed, 1 if not */
+UNIV_INTERN
+ulint
+mutex_enter_nowait_func(
+/*====================*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where mutex
+						requested */
+	ulint			line);		/*!< in: line where
+						requested */
 /******************************************************************//**
 NOTE! Use the corresponding macro mutex_exit(), not directly this function!
 Unlocks a mutex owned by the current thread. */
@@ -296,6 +374,14 @@
 mutex_exit_func(
 /*============*/
 	ib_mutex_t*	mutex);	/*!< in: pointer to mutex */
+/******************************************************************//**
+NOTE! Use the corresponding macro mutex_exit(), not directly this function!
+Unlocks a priority mutex owned by the current thread. */
+UNIV_INLINE
+void
+mutex_exit_func(
+/*============*/
+	ib_prio_mutex_t*	mutex);	/*!< in: pointer to mutex */
 
 
 #ifdef UNIV_PFS_MUTEX
@@ -320,6 +406,29 @@
 # endif /* UNIV_DEBUG */
 	const char*	cmutex_name);
 /******************************************************************//**
+NOTE! Please use the corresponding macro mutex_create(), not directly
+this function!
+A wrapper function for mutex_create_func(), registers the mutex
+with peformance schema if "UNIV_PFS_MUTEX" is defined when
+creating the performance mutex */
+UNIV_INLINE
+void
+pfs_mutex_create_func(
+/*==================*/
+	PSI_mutex_key		key,		/*!< in: Performance Schema
+						key */
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to memory */
+# ifdef UNIV_DEBUG
+#  ifdef UNIV_SYNC_DEBUG
+	ulint			level,		/*!< in: level */
+#  endif /* UNIV_SYNC_DEBUG */
+	const char*		cfile_name,	/*!< in: file name where
+						created */
+	ulint			cline,		/*!< in: file line where
+						created */
+# endif /* UNIV_DEBUG */
+	const char*		cmutex_name);
+/******************************************************************//**
 NOTE! Please use the corresponding macro mutex_enter(), not directly
 this function!
 This is a performance schema instrumented wrapper function for
@@ -331,6 +440,22 @@
 	ib_mutex_t*	mutex,		/*!< in: pointer to mutex */
 	const char*	file_name,	/*!< in: file name where locked */
 	ulint		line);		/*!< in: line where locked */
+/******************************************************************//**
+NOTE! Please use the corresponding macro mutex_enter(), not directly
+this function!
+This is a performance schema instrumented wrapper function for
+mutex_enter_func(). */
+UNIV_INLINE
+void
+pfs_mutex_enter_func(
+/*=================*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where
+						locked */
+	ulint			line,		/*!< in: line where locked */
+	enum ib_sync_priority	priority = DEFAULT_PRIO);
+						/*!<in: mutex acquisition
+						priority */
 /********************************************************************//**
 NOTE! Please use the corresponding macro mutex_enter_nowait(), not directly
 this function!
@@ -345,6 +470,21 @@
 	const char*	file_name,	/*!< in: file name where mutex
 					requested */
 	ulint		line);		/*!< in: line where requested */
+/********************************************************************//**
+NOTE! Please use the corresponding macro mutex_enter_nowait(), not directly
+this function!
+This is a performance schema instrumented wrapper function for
+mutex_enter_nowait_func.
+@return	0 if succeed, 1 if not */
+UNIV_INLINE
+ulint
+pfs_mutex_enter_nowait_func(
+/*========================*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where mutex
+						requested */
+	ulint			line);		/*!< in: line where
+						requested */
 /******************************************************************//**
 NOTE! Please use the corresponding macro mutex_exit(), not directly
 this function!
@@ -355,6 +495,16 @@
 pfs_mutex_exit_func(
 /*================*/
 	ib_mutex_t*	mutex);	/*!< in: pointer to mutex */
+/******************************************************************//**
+NOTE! Please use the corresponding macro mutex_exit(), not directly
+this function!
+A wrap function of mutex_exit_func() with peformance schema instrumentation.
+Unlocks a priority mutex owned by the current thread. */
+UNIV_INLINE
+void
+pfs_mutex_exit_func(
+/*================*/
+	ib_prio_mutex_t*	mutex);	/*!< in: pointer to mutex */
 
 /******************************************************************//**
 NOTE! Please use the corresponding macro mutex_free(), not directly
@@ -366,6 +516,16 @@
 pfs_mutex_free_func(
 /*================*/
 	ib_mutex_t*	mutex);	/*!< in: mutex */
+/******************************************************************//**
+NOTE! Please use the corresponding macro mutex_free(), not directly
+this function!
+Wrapper function for mutex_free_func(). Also destroys the performance
+schema probes when freeing the priority mutex */
+UNIV_INLINE
+void
+pfs_mutex_free_func(
+/*================*/
+	ib_prio_mutex_t*	mutex);	/*!< in: mutex */
 
 #endif /* UNIV_PFS_MUTEX */
 
@@ -414,6 +574,16 @@
 /*======*/
 	const ib_mutex_t*	mutex)	/*!< in: mutex */
 	__attribute__((warn_unused_result));
+/******************************************************************//**
+Checks that the current thread owns the priority mutex. Works only
+in the debug version.
+@return	TRUE if owns */
+UNIV_INTERN
+ibool
+mutex_own(
+/*======*/
+	const ib_prio_mutex_t*	mutex)	/*!< in: priority mutex */
+	__attribute__((warn_unused_result));
 #endif /* UNIV_DEBUG */
 #ifdef UNIV_SYNC_DEBUG
 /******************************************************************//**
@@ -748,6 +918,9 @@
 #define RW_LOCK_SHARED		352
 #define RW_LOCK_WAIT_EX		353
 #define SYNC_MUTEX		354
+#define SYNC_PRIO_MUTEX		355
+#define PRIO_RW_LOCK_EX		356
+#define PRIO_RW_LOCK_SHARED	357
 
 /* NOTE! The structure appears here only for the compiler to know its size.
 Do not use its fields directly! The structure used in the spin lock
@@ -798,6 +971,20 @@
 #endif
 };
 
+/** XtraDB priority mutex */
+struct ib_prio_mutex_t {
+	ib_mutex_t	base_mutex;	/* The regular mutex provides the lock
+					word etc. for the priority mutex  */
+	os_event_t	high_priority_event; /* High priority wait array
+					event */
+	volatile ulint	high_priority_waiters; /* Set to 1 if there are (or
+					may be) threads that asked for this
+					mutex to be acquired with high priority
+					in the global wait array for this mutex
+					to be released.  Otherwise, this is
+					0.  */
+};
+
 /** Constant determining how long spin wait is continued before suspending
 the thread. A value 600 rounds on a 1995 100 MHz Pentium seems to correspond
 to 20 microseconds. */

=== modified file 'Percona-Server/storage/innobase/include/sync0sync.ic'
--- Percona-Server/storage/innobase/include/sync0sync.ic	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/sync0sync.ic	2013-09-26 14:59:13 +0000
@@ -39,14 +39,17 @@
 	ib_mutex_t*	mutex,	/*!< in: mutex */
 	ulint		n);	/*!< in: value to set */
 /******************************************************************//**
-Reserves a mutex for the current thread. If the mutex is reserved, the
-function spins a preset time (controlled by SYNC_SPIN_ROUNDS) waiting
-for the mutex before suspending the thread. */
+Reserves a mutex or a priority mutex for the current thread. If the mutex is
+reserved, the function spins a preset time (controlled by SYNC_SPIN_ROUNDS)
+waiting for the mutex before suspending the thread. */
 UNIV_INTERN
 void
 mutex_spin_wait(
 /*============*/
-	ib_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	void*		_mutex,		/*!< in: pointer to mutex */
+	bool		high_priority,	/*!< in: whether the mutex is a
+					priority mutex with high priority
+					specified */
 	const char*	file_name,	/*!< in: file name where mutex
 					requested */
 	ulint		line);		/*!< in: line where requested */
@@ -192,6 +195,55 @@
 }
 
 /******************************************************************//**
+NOTE! Use the corresponding macro mutex_exit(), not directly this function!
+Unlocks a priority mutex owned by the current thread. */
+UNIV_INLINE
+void
+mutex_exit_func(
+/*============*/
+	ib_prio_mutex_t*	mutex)	/*!< in: pointer to mutex */
+{
+	ut_ad(mutex_own(mutex));
+
+	ut_d(mutex->base_mutex.thread_id = (os_thread_id_t) ULINT_UNDEFINED);
+
+#ifdef UNIV_SYNC_DEBUG
+	sync_thread_reset_level(&mutex->base_mutex);
+#endif
+	mutex_reset_lock_word(&mutex->base_mutex);
+
+	/* A problem: we assume that mutex_reset_lock word
+	is a memory barrier, that is when we read the waiters
+	field next, the read must be serialized in memory
+	after the reset. A speculative processor might
+	perform the read first, which could leave a waiting
+	thread hanging indefinitely.
+
+	Our current solution call every second
+	sync_arr_wake_threads_if_sema_free()
+	to wake up possible hanging threads if
+	they are missed in mutex_signal_object. */
+
+	/* Wake up any high priority waiters first.  */
+	if (mutex->high_priority_waiters != 0) {
+
+		mutex->high_priority_waiters = 0;
+		os_event_set(mutex->high_priority_event);
+		sync_array_object_signalled();
+
+	} else if (mutex_get_waiters(&mutex->base_mutex) != 0) {
+
+		mutex_signal_object(&mutex->base_mutex);
+	}
+
+#ifdef UNIV_SYNC_PERF_STAT
+	mutex_exit_count++;
+#endif
+
+}
+
+
+/******************************************************************//**
 Locks a mutex for the current thread. If the mutex is reserved, the function
 spins a preset time (controlled by SYNC_SPIN_ROUNDS), waiting for the mutex
 before suspending the thread. */
@@ -217,8 +269,54 @@
 		return;	/* Succeeded! */
 	}
 
-	mutex_spin_wait(mutex, file_name, line);
-}
+	mutex_spin_wait(mutex, false, file_name, line);
+}
+
+/******************************************************************//**
+NOTE! Use the corresponding macro in the header file, not this function
+directly. Locks a priority mutex for the current thread. If the mutex is
+reserved the function spins a preset time (controlled by SYNC_SPIN_ROUNDS)
+waiting for the mutex before suspending the thread. If the thread is suspended,
+the priority argument value determines the relative order for its wake up.  Any
+HIGH_PRIO waiters will be woken up before any LOW_PRIO waiters.  In case of
+DEFAULT_PRIO, the relative priority will be set according to
+srv_current_thread_priority.  */
+UNIV_INLINE
+void
+mutex_enter_func(
+/*=============*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where
+						locked */
+	ulint			line,		/*!< in: line where locked */
+	enum ib_sync_priority	priority)
+						/*!<in: mutex acquisition
+						priority */
+{
+	bool	high_priority;
+
+	ut_ad(mutex_validate(&mutex->base_mutex));
+	ut_ad(!mutex_own(mutex));
+
+	/* Note that we do not peek at the value of lock_word before trying
+	the atomic test_and_set; we could peek, and possibly save time. */
+
+	if (!ib_mutex_test_and_set(&mutex->base_mutex)) {
+		ut_d(mutex->base_mutex.thread_id = os_thread_get_curr_id());
+#ifdef UNIV_SYNC_DEBUG
+		mutex_set_debug_info(&mutex->base_mutex, file_name, line);
+#endif
+		return;	/* Succeeded! */
+	}
+
+	if (UNIV_LIKELY(priority == DEFAULT_PRIO)) {
+		high_priority = srv_current_thread_priority;
+	} else {
+		high_priority = (priority == HIGH_PRIO);
+	}
+	mutex_spin_wait(mutex, high_priority, file_name, line);
+}
+
 
 #ifdef UNIV_PFS_MUTEX
 /******************************************************************//**
@@ -252,6 +350,40 @@
 	}
 }
 
+/******************************************************************//**
+NOTE! Please use the corresponding macro mutex_enter(), not directly
+this function!
+This is a performance schema instrumented wrapper function for
+mutex_enter_func(). */
+UNIV_INLINE
+void
+pfs_mutex_enter_func(
+/*=================*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where
+						locked */
+	ulint			line,		/*!< in: line where locked */
+	enum ib_sync_priority	priority)	/*!<in: mutex acquisition
+						priority */
+{
+	if (mutex->base_mutex.pfs_psi != NULL) {
+		PSI_mutex_locker*	locker;
+		PSI_mutex_locker_state	state;
+
+		locker = PSI_MUTEX_CALL(start_mutex_wait)(
+			&state, mutex->base_mutex.pfs_psi,
+			PSI_MUTEX_LOCK, file_name, line);
+
+		mutex_enter_func(mutex, file_name, line, priority);
+
+		if (locker != NULL) {
+			PSI_MUTEX_CALL(end_mutex_wait)(locker, 0);
+		}
+	} else {
+		mutex_enter_func(mutex, file_name, line, priority);
+	}
+}
+
 /********************************************************************//**
 NOTE! Please use the corresponding macro mutex_enter_nowait(), not directly
 this function!
@@ -289,6 +421,26 @@
 	return(ret);
 }
 
+/********************************************************************//**
+NOTE! Please use the corresponding macro mutex_enter_nowait(), not directly
+this function!
+This is a performance schema instrumented wrapper function for
+mutex_enter_nowait_func.
+@return	0 if succeed, 1 if not */
+UNIV_INLINE
+ulint
+pfs_mutex_enter_nowait_func(
+/*========================*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	const char*		file_name,	/*!< in: file name where mutex
+						  requested */
+	ulint			line)		/*!< in: line where
+						  requested */
+{
+	return pfs_mutex_enter_nowait_func(&mutex->base_mutex, file_name,
+					   line);
+}
+
 /******************************************************************//**
 NOTE! Please use the corresponding macro mutex_exit(), not directly
 this function!
@@ -308,6 +460,25 @@
 }
 
 /******************************************************************//**
+NOTE! Please use the corresponding macro mutex_exit(), not directly
+this function!
+A wrap function of mutex_exit_func() with peformance schema instrumentation.
+Unlocks a priority mutex owned by the current thread. */
+UNIV_INLINE
+void
+pfs_mutex_exit_func(
+/*================*/
+	ib_prio_mutex_t*	mutex)	/*!< in: pointer to mutex */
+{
+	if (mutex->base_mutex.pfs_psi != NULL) {
+		PSI_MUTEX_CALL(unlock_mutex)(mutex->base_mutex.pfs_psi);
+	}
+
+	mutex_exit_func(mutex);
+}
+
+
+/******************************************************************//**
 NOTE! Please use the corresponding macro mutex_create(), not directly
 this function!
 A wrapper function for mutex_create_func(), registers the mutex
@@ -342,6 +513,44 @@
 }
 
 /******************************************************************//**
+NOTE! Please use the corresponding macro mutex_create(), not directly
+this function!
+A wrapper function for mutex_create_func(), registers the mutex
+with peformance schema if "UNIV_PFS_MUTEX" is defined when
+creating the performance mutex */
+UNIV_INLINE
+void
+pfs_mutex_create_func(
+/*==================*/
+	PSI_mutex_key		key,		/*!< in: Performance Schema
+						key */
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to memory */
+# ifdef UNIV_DEBUG
+#  ifdef UNIV_SYNC_DEBUG
+	ulint			level,		/*!< in: level */
+#  endif /* UNIV_SYNC_DEBUG */
+	const char*		cfile_name,	/*!< in: file name where
+						created */
+	ulint			cline,		/*!< in: file line where
+						  created */
+# endif /* UNIV_DEBUG */
+	const char*		cmutex_name)
+{
+	mutex->base_mutex.pfs_psi = PSI_MUTEX_CALL(init_mutex)(key, mutex);
+
+	mutex_create_func(mutex,
+# ifdef UNIV_DEBUG
+#  ifdef UNIV_SYNC_DEBUG
+			  level,
+#  endif /* UNIV_SYNC_DEBUG */
+			  cfile_name,
+			  cline,
+# endif /* UNIV_DEBUG */
+			  cmutex_name);
+}
+
+
+/******************************************************************//**
 NOTE! Please use the corresponding macro mutex_free(), not directly
 this function!
 Wrapper function for mutex_free_func(). Also destroys the performance
@@ -360,6 +569,26 @@
 	mutex_free_func(mutex);
 }
 
+/******************************************************************//**
+NOTE! Please use the corresponding macro mutex_free(), not directly
+this function!
+Wrapper function for mutex_free_func(). Also destroys the performance
+schema probes when freeing the priority mutex */
+UNIV_INLINE
+void
+pfs_mutex_free_func(
+/*================*/
+	ib_prio_mutex_t*	mutex)	/*!< in: mutex */
+{
+	if (mutex->base_mutex.pfs_psi != NULL) {
+		PSI_MUTEX_CALL(destroy_mutex)(mutex->base_mutex.pfs_psi);
+		mutex->base_mutex.pfs_psi = NULL;
+	}
+
+	mutex_free_func(mutex);
+}
+
+
 #endif /* UNIV_PFS_MUTEX */
 
 #ifndef HAVE_ATOMIC_BUILTINS

=== modified file 'Percona-Server/storage/innobase/include/sync0types.h'
--- Percona-Server/storage/innobase/include/sync0types.h	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/sync0types.h	2013-09-26 14:59:13 +0000
@@ -28,4 +28,17 @@
 
 struct ib_mutex_t;
 
+/* The relative priority of the current thread.  If 0, low priority; if 1, high
+priority.  */
+extern UNIV_THREAD_LOCAL ulint srv_current_thread_priority;
+
+struct ib_prio_mutex_t;
+
+/** Priority mutex and rwlatch acquisition priorities */
+enum ib_sync_priority {
+	DEFAULT_PRIO,
+	LOW_PRIO,
+	HIGH_PRIO
+};
+
 #endif

=== modified file 'Percona-Server/storage/innobase/include/trx0purge.h'
--- Percona-Server/storage/innobase/include/trx0purge.h	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/include/trx0purge.h	2013-09-26 14:59:13 +0000
@@ -138,7 +138,7 @@
 					purge query: this trx is not in the
 					trx list of the trx system and it
 					never ends */
-	rw_lock_t	latch;		/*!< The latch protecting the purge
+	prio_rw_lock_t	latch;		/*!< The latch protecting the purge
 					view. A purge operation must acquire an
 					x-latch here for the instant at which
 					it changes the purge view: an undo

=== modified file 'Percona-Server/storage/innobase/include/trx0trx.h'
--- Percona-Server/storage/innobase/include/trx0trx.h	2013-09-02 10:01:38 +0000
+++ Percona-Server/storage/innobase/include/trx0trx.h	2013-09-26 14:59:13 +0000
@@ -1138,7 +1138,7 @@
 
 Bear in mind (3) and (4) when using the hash index.
 */
-extern rw_lock_t*	btr_search_latch_arr;
+extern prio_rw_lock_t*	btr_search_latch_arr;
 
 #ifndef UNIV_NONINL
 #include "trx0trx.ic"

=== modified file 'Percona-Server/storage/innobase/include/univ.i'
--- Percona-Server/storage/innobase/include/univ.i	2013-08-29 17:05:40 +0000
+++ Percona-Server/storage/innobase/include/univ.i	2013-09-26 14:59:13 +0000
@@ -277,6 +277,13 @@
 # define UNIV_COLD /* empty */
 #endif
 
+#ifdef UNIV_LINUX
+# define UNIV_THREAD_LOCAL __thread
+#else
+/* FIXME: the TLS variables are silently broken on other platforms for now */
+# define UNIV_THREAD_LOCAL
+#endif
+
 #ifndef UNIV_MUST_NOT_INLINE
 /* Definition for inline version */
 

=== modified file 'Percona-Server/storage/innobase/mtr/mtr0mtr.cc'
--- Percona-Server/storage/innobase/mtr/mtr0mtr.cc	2013-06-25 13:13:06 +0000
+++ Percona-Server/storage/innobase/mtr/mtr0mtr.cc	2013-09-26 14:59:13 +0000
@@ -82,10 +82,10 @@
 		buf_page_release((buf_block_t*) object, slot->type);
 		break;
 	case MTR_MEMO_S_LOCK:
-		rw_lock_s_unlock((rw_lock_t*) object);
+		rw_lock_s_unlock((prio_rw_lock_t*) object);
 		break;
 	case MTR_MEMO_X_LOCK:
-		rw_lock_x_unlock((rw_lock_t*) object);
+		rw_lock_x_unlock((prio_rw_lock_t*) object);
 		break;
 #ifdef UNIV_DEBUG
 	default:

=== modified file 'Percona-Server/storage/innobase/srv/srv0srv.cc'
--- Percona-Server/storage/innobase/srv/srv0srv.cc	2013-09-24 13:11:08 +0000
+++ Percona-Server/storage/innobase/srv/srv0srv.cc	2013-09-26 14:59:13 +0000
@@ -319,6 +319,22 @@
 /* Number of iterations over which adaptive flushing is averaged. */
 UNIV_INTERN ulong	srv_flushing_avg_loops		= 30;
 
+/* The relative priority of the current thread.  If 0, low priority; if 1, high
+priority.  */
+UNIV_INTERN UNIV_THREAD_LOCAL ulint srv_current_thread_priority = 0;
+
+/* The relative priority of the purge coordinator and worker threads.  */
+UNIV_INTERN my_bool	srv_purge_thread_priority	= FALSE;
+
+/* The relative priority of the I/O threads.  */
+UNIV_INTERN my_bool	srv_io_thread_priority		= FALSE;
+
+/* The relative priority of the cleaner thread.  */
+UNIV_INTERN my_bool	srv_cleaner_thread_priority	= FALSE;
+
+/* The relative priority of the master thread.  */
+UNIV_INTERN my_bool	srv_master_thread_priority	= FALSE;
+
 /* The number of purge threads to use.*/
 UNIV_INTERN ulong	srv_n_purge_threads = 1;
 
@@ -2863,6 +2879,8 @@
 
 		MONITOR_INC(MONITOR_MASTER_THREAD_SLEEP);
 
+		srv_current_thread_priority = srv_master_thread_priority;
+
 		if (srv_check_activity(old_activity_count)) {
 			old_activity_count = srv_get_activity_count();
 			srv_master_do_active_tasks();
@@ -3003,6 +3021,8 @@
 
 		os_event_wait(slot->event);
 
+		srv_current_thread_priority = srv_purge_thread_priority;
+
 		if (srv_task_execute()) {
 
 			/* If there are tasks in the queue, wakeup
@@ -3279,6 +3299,8 @@
 
 		n_total_purged = 0;
 
+		srv_current_thread_priority = srv_purge_thread_priority;
+
 		rseg_history_len = srv_do_purge(
 			srv_n_purge_threads, &n_total_purged);
 

=== modified file 'Percona-Server/storage/innobase/srv/srv0start.cc'
--- Percona-Server/storage/innobase/srv/srv0start.cc	2013-08-14 03:57:21 +0000
+++ Percona-Server/storage/innobase/srv/srv0start.cc	2013-09-26 14:59:13 +0000
@@ -476,6 +476,7 @@
 #endif /* UNIV_PFS_THREAD */
 
 	while (srv_shutdown_state != SRV_SHUTDOWN_EXIT_THREADS) {
+		srv_current_thread_priority = srv_io_thread_priority;
 		fil_aio_wait(segment);
 	}
 

=== modified file 'Percona-Server/storage/innobase/sync/sync0arr.cc'
--- Percona-Server/storage/innobase/sync/sync0arr.cc	2013-08-14 03:57:21 +0000
+++ Percona-Server/storage/innobase/sync/sync0arr.cc	2013-09-26 14:59:13 +0000
@@ -83,10 +83,11 @@
 	void*		wait_object;	/*!< pointer to the object the
 					thread is waiting for; if NULL
 					the cell is free for use */
-	ib_mutex_t*	old_wait_mutex;	/*!< the latest wait mutex in cell */
-	rw_lock_t*	old_wait_rw_lock;
-					/*!< the latest wait rw-lock
-					in cell */
+	void*		old_wait_mutex;	/*!< the latest regular or priority
+					wait mutex in cell */
+	void*		old_wait_rw_lock;
+					/*!< the latest regular or priority
+					wait rw-lock in cell */
 	ulint		request_type;	/*!< lock type requested on the
 					object */
 	const char*	file;		/*!< in debug version file where
@@ -295,9 +296,19 @@
 
 	if (type == SYNC_MUTEX) {
 		return(((ib_mutex_t*) cell->wait_object)->event);
+	} else if (type == SYNC_PRIO_MUTEX) {
+		return(((ib_prio_mutex_t*) cell->wait_object)
+		       ->high_priority_event);
 	} else if (type == RW_LOCK_WAIT_EX) {
 		return(((rw_lock_t*) cell->wait_object)->wait_ex_event);
+	} else if (type == PRIO_RW_LOCK_SHARED) {
+		return(((prio_rw_lock_t *) cell->wait_object)
+		       ->high_priority_s_event);
+	} else if (type == PRIO_RW_LOCK_EX) {
+		return(((prio_rw_lock_t *) cell->wait_object)
+		       ->high_priority_x_event);
 	} else { /* RW_LOCK_SHARED and RW_LOCK_EX wait on the same event */
+		ut_ad(type == RW_LOCK_SHARED || type == RW_LOCK_EX);
 		return(((rw_lock_t*) cell->wait_object)->event);
 	}
 }
@@ -336,12 +347,10 @@
 			cell->waiting = FALSE;
 			cell->wait_object = object;
 
-			if (type == SYNC_MUTEX) {
-				cell->old_wait_mutex =
-					static_cast<ib_mutex_t*>(object);
+			if (type == SYNC_MUTEX || type == SYNC_PRIO_MUTEX) {
+				cell->old_wait_mutex = object;
 			} else {
-				cell->old_wait_rw_lock =
-					static_cast<rw_lock_t*>(object);
+				cell->old_wait_rw_lock = object;
 			}
 
 			cell->request_type = type;
@@ -436,7 +445,9 @@
 	sync_cell_t*	cell)	/*!< in: sync cell */
 {
 	ib_mutex_t*	mutex;
+	ib_prio_mutex_t*	prio_mutex;
 	rw_lock_t*	rwlock;
+	prio_rw_lock_t*	prio_rwlock	= NULL;
 	ulint		type;
 	ulint		writer;
 
@@ -449,10 +460,19 @@
 		innobase_basename(cell->file), (ulong) cell->line,
 		difftime(time(NULL), cell->reservation_time));
 
-	if (type == SYNC_MUTEX) {
+	if (type == SYNC_MUTEX || type == SYNC_PRIO_MUTEX) {
 		/* We use old_wait_mutex in case the cell has already
 		been freed meanwhile */
-		mutex = cell->old_wait_mutex;
+		if (type == SYNC_MUTEX) {
+
+			mutex = static_cast<ib_mutex_t*>(cell->old_wait_mutex);
+		} else {
+
+			prio_mutex = static_cast<ib_prio_mutex_t*>
+				(cell->old_wait_mutex);
+			mutex = &prio_mutex->base_mutex;
+		}
+
 
 		fprintf(file,
 			"Mutex at %p '%s', lock var %lu\n"
@@ -467,15 +487,38 @@
 #endif /* UNIV_SYNC_DEBUG */
 			(ulong) mutex->waiters);
 
+		if (type == SYNC_PRIO_MUTEX) {
+
+			fprintf(file,
+				"high-priority waiters flag %lu\n",
+				(ulong) prio_mutex->high_priority_waiters);
+		}
+
 	} else if (type == RW_LOCK_EX
 		   || type == RW_LOCK_WAIT_EX
-		   || type == RW_LOCK_SHARED) {
+		   || type == RW_LOCK_SHARED
+		   || type == PRIO_RW_LOCK_SHARED
+		   || type == PRIO_RW_LOCK_EX) {
 
-		fputs(type == RW_LOCK_EX ? "X-lock on"
+		fputs((type == RW_LOCK_EX || type == PRIO_RW_LOCK_EX)
+		      ? "X-lock on"
 		      : type == RW_LOCK_WAIT_EX ? "X-lock (wait_ex) on"
 		      : "S-lock on", file);
 
-		rwlock = cell->old_wait_rw_lock;
+		/* Currently we are unable to tell high priority
+	        RW_LOCK_WAIT_EX waiter from a regular priority one.  Assume
+	        it's a regular one.  */
+		if (type == RW_LOCK_EX || type == RW_LOCK_WAIT_EX
+		    || type == RW_LOCK_SHARED) {
+
+			rwlock = static_cast<rw_lock_t *>
+				(cell->old_wait_rw_lock);
+		} else {
+
+			prio_rwlock = static_cast<prio_rw_lock_t *>
+				(cell->old_wait_rw_lock);
+			rwlock = &prio_rwlock->base_lock;
+		}
 
 		fprintf(file,
 			" RW-latch at %p '%s'\n",
@@ -503,6 +546,15 @@
 			(ulong) rwlock->last_s_line,
 			rwlock->last_x_file_name,
 			(ulong) rwlock->last_x_line);
+		if (prio_rwlock) {
+			fprintf(stderr, "high priority S waiters flag %lu, "
+				"high priority X waiters flag %lu, "
+				"wait-exclusive waiter is "
+				"high priority if exists: %lu\n",
+				prio_rwlock->high_priority_s_waiters,
+				prio_rwlock->high_priority_x_waiters,
+				prio_rwlock->high_priority_wait_ex_waiter);
+		}
 	} else {
 		ut_error;
 	}
@@ -615,9 +667,15 @@
 		return(FALSE); /* No deadlock here */
 	}
 
-	if (cell->request_type == SYNC_MUTEX) {
+	if (cell->request_type == SYNC_MUTEX
+	    || cell->request_type == SYNC_PRIO_MUTEX) {
 
-		mutex = static_cast<ib_mutex_t*>(cell->wait_object);
+		if (cell->request_type == SYNC_MUTEX) {
+			mutex = static_cast<ib_mutex_t*>(cell->wait_object);
+		} else {
+			mutex = &(static_cast<ib_prio_mutex_t*>(
+					  cell->wait_object))->base_mutex;
+		}
 
 		if (mutex_get_lock_word(mutex) != 0) {
 
@@ -647,6 +705,7 @@
 		return(FALSE); /* No deadlock */
 
 	} else if (cell->request_type == RW_LOCK_EX
+		   || cell->request_type == PRIO_RW_LOCK_EX
 		   || cell->request_type == RW_LOCK_WAIT_EX) {
 
 		lock = static_cast<rw_lock_t*>(cell->wait_object);
@@ -685,7 +744,8 @@
 
 		return(FALSE);
 
-	} else if (cell->request_type == RW_LOCK_SHARED) {
+	} else if (cell->request_type == RW_LOCK_SHARED
+		   || cell->request_type == PRIO_RW_LOCK_SHARED) {
 
 		lock = static_cast<rw_lock_t*>(cell->wait_object);
 
@@ -734,16 +794,23 @@
 	ib_mutex_t*	mutex;
 	rw_lock_t*	lock;
 
-	if (cell->request_type == SYNC_MUTEX) {
+	if (cell->request_type == SYNC_MUTEX
+	    || cell->request_type == SYNC_PRIO_MUTEX) {
 
-		mutex = static_cast<ib_mutex_t*>(cell->wait_object);
+		if (cell->request_type == SYNC_MUTEX) {
+			mutex = static_cast<ib_mutex_t*>(cell->wait_object);
+		} else {
+			mutex = &(static_cast<ib_prio_mutex_t*>(
+					  cell->wait_object))->base_mutex;
+		}
 
 		if (mutex_get_lock_word(mutex) == 0) {
 
 			return(TRUE);
 		}
 
-	} else if (cell->request_type == RW_LOCK_EX) {
+	} else if (cell->request_type == RW_LOCK_EX
+		   || cell->request_type == PRIO_RW_LOCK_EX) {
 
 		lock = static_cast<rw_lock_t*>(cell->wait_object);
 
@@ -762,7 +829,8 @@
 
 			return(TRUE);
 		}
-	} else if (cell->request_type == RW_LOCK_SHARED) {
+	} else if (cell->request_type == RW_LOCK_SHARED
+		   || cell->request_type == PRIO_RW_LOCK_SHARED) {
 		lock = static_cast<rw_lock_t*>(cell->wait_object);
 
                 /* lock_word > 0 means no writer or reserved writer */
@@ -770,6 +838,9 @@
 
 			return(TRUE);
 		}
+	} else {
+
+		ut_error;
 	}
 
 	return(FALSE);

=== modified file 'Percona-Server/storage/innobase/sync/sync0rw.cc'
--- Percona-Server/storage/innobase/sync/sync0rw.cc	2013-08-06 15:16:34 +0000
+++ Percona-Server/storage/innobase/sync/sync0rw.cc	2013-09-26 14:59:13 +0000
@@ -277,6 +277,41 @@
 }
 
 /******************************************************************//**
+Creates, or rather, initializes a priority rw-lock object in a specified memory
+location (which must be appropriately aligned). The rw-lock is initialized
+to the non-locked state. Explicit freeing of the rw-lock with rw_lock_free
+is necessary only if the memory block containing it is freed. */
+UNIV_INTERN
+void
+rw_lock_create_func(
+/*================*/
+	prio_rw_lock_t*	lock,		/*!< in: pointer to memory */
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+	ulint		level,		/*!< in: level */
+# endif /* UNIV_SYNC_DEBUG */
+	const char*	cfile_name,	/*!< in: file name where created */
+	ulint		cline,		/*!< in: file line where created */
+#endif /* UNIV_DEBUG */
+	const char*	cmutex_name)	/*!< in: mutex name */
+{
+	rw_lock_create_func(&lock->base_lock,
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+			    level,
+# endif
+			    cfile_name,
+			    cline,
+#endif
+			    cmutex_name);
+	lock->high_priority_s_waiters = 0;
+	lock->high_priority_s_event = os_event_create();
+	lock->high_priority_x_waiters = 0;
+	lock->high_priority_x_event = os_event_create();
+	lock->high_priority_wait_ex_waiter = 0;
+}
+
+/******************************************************************//**
 Calling this function is obligatory only if the memory buffer containing
 the rw-lock is freed. Removes an rw-lock object from the global list. The
 rw-lock is checked to be in the non-locked state. */
@@ -321,6 +356,21 @@
 #endif /* !INNODB_RW_LOCKS_USE_ATOMICS */
 }
 
+/******************************************************************//**
+Calling this function is obligatory only if the memory buffer containing
+the priority rw-lock is freed. Removes an rw-lock object from the global list.
+The rw-lock is checked to be in the non-locked state. */
+UNIV_INTERN
+void
+rw_lock_free_func(
+/*==============*/
+	prio_rw_lock_t*	lock)	/*!< in: rw-lock */
+{
+	os_event_free(lock->high_priority_s_event);
+	os_event_free(lock->high_priority_x_event);
+	rw_lock_free_func(&lock->base_lock);
+}
+
 #ifdef UNIV_DEBUG
 /******************************************************************//**
 Checks that the rw-lock has been initialized and that there are no
@@ -347,20 +397,41 @@
 
 	return(TRUE);
 }
+
+/******************************************************************//**
+Checks that the priority rw-lock has been initialized and that there are no
+simultaneous shared and exclusive locks.
+@return	TRUE */
+UNIV_INTERN
+ibool
+rw_lock_validate(
+/*=============*/
+	prio_rw_lock_t*	lock)	/*!< in: rw-lock */
+{
+	ut_ad(lock->high_priority_s_waiters < 2);
+	ut_ad(lock->high_priority_x_waiters < 2);
+	return(rw_lock_validate(&lock->base_lock));
+}
+
 #endif /* UNIV_DEBUG */
 
 /******************************************************************//**
-Lock an rw-lock in shared mode for the current thread. If the rw-lock is
-locked in exclusive mode, or there is an exclusive lock request waiting,
-the function spins a preset time (controlled by SYNC_SPIN_ROUNDS), waiting
-for the lock, before suspending the thread. */
+Lock a regular or priority rw-lock in shared mode for the current thread. If
+the rw-lock is locked in exclusive mode, or there is an exclusive lock request
+waiting, the function spins a preset time (controlled by SYNC_SPIN_ROUNDS),
+waiting for the lock, before suspending the thread. */
 UNIV_INTERN
 void
 rw_lock_s_lock_spin(
 /*================*/
-	rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	void*		_lock,	/*!< in: pointer to rw-lock */
 	ulint		pass,	/*!< in: pass value; != 0, if the lock
 				will be passed to another thread to unlock */
+	bool		priority_lock,
+				/*!< in: whether the lock is a priority lock */
+	bool		high_priority,
+				/*!< in: whether we are acquiring a priority
+				lock with high priority */
 	const char*	file_name, /*!< in: file name where lock requested */
 	ulint		line)	/*!< in: line where requested */
 {
@@ -368,6 +439,7 @@
 	ulint		i = 0;	/* spin round count */
 	sync_array_t*	sync_arr;
 	size_t		counter_index;
+	rw_lock_t*	lock = (rw_lock_t *) _lock;
 
 	/* We reuse the thread id to index into the counter, cache
 	it here for efficiency. */
@@ -379,35 +451,47 @@
 	rw_lock_stats.rw_s_spin_wait_count.add(counter_index, 1);
 lock_loop:
 
-	/* Spin waiting for the writer field to become free */
-	while (i < SYNC_SPIN_ROUNDS && lock->lock_word <= 0) {
-		if (srv_spin_wait_delay) {
-			ut_delay(ut_rnd_interval(0, srv_spin_wait_delay));
-		}
-
-		i++;
-	}
-
-	if (i == SYNC_SPIN_ROUNDS) {
+	if (!rw_lock_higher_prio_waiters_exist(priority_lock, high_priority,
+					       lock)) {
+
+		/* Spin waiting for the writer field to become free */
+		while (i < SYNC_SPIN_ROUNDS && lock->lock_word <= 0) {
+			if (srv_spin_wait_delay) {
+				ut_delay(ut_rnd_interval(0,
+							 srv_spin_wait_delay));
+			}
+
+			i++;
+		}
+
+		if (i == SYNC_SPIN_ROUNDS) {
+			os_thread_yield();
+		}
+
+		if (srv_print_latch_waits) {
+			fprintf(stderr,
+				"Thread " ULINTPF " spin wait rw-s-lock at %p"
+				" '%s' rnds " ULINTPF "\n",
+				os_thread_pf(os_thread_get_curr_id()),
+				(void*) lock, lock->lock_name, i);
+		}
+	} else {
+
+		/* In case of higher priority waiters already present, perform
+		only this part of the spinning code path.  */
 		os_thread_yield();
 	}
 
-	if (srv_print_latch_waits) {
-		fprintf(stderr,
-			"Thread " ULINTPF " spin wait rw-s-lock at %p"
-			" '%s' rnds " ULINTPF "\n",
-			os_thread_pf(os_thread_get_curr_id()),
-			(void*) lock, lock->lock_name, i);
-	}
-
 	/* We try once again to obtain the lock */
-	if (TRUE == rw_lock_s_lock_low(lock, pass, file_name, line)) {
+	if (!rw_lock_higher_prio_waiters_exist(priority_lock, high_priority,
+					       lock)
+	    && (TRUE == rw_lock_s_lock_low(lock, pass, file_name, line))) {
 		rw_lock_stats.rw_s_spin_round_count.add(counter_index, i);
 
 		return; /* Success */
 	} else {
 
-		if (i < SYNC_SPIN_ROUNDS) {
+		if (i > 0 && i < SYNC_SPIN_ROUNDS) {
 			goto lock_loop;
 		}
 
@@ -416,14 +500,24 @@
 		sync_arr = sync_array_get();
 
 		sync_array_reserve_cell(
-			sync_arr, lock, RW_LOCK_SHARED,
+			sync_arr, lock,
+			high_priority ? PRIO_RW_LOCK_SHARED : RW_LOCK_SHARED,
 			file_name, line, &index);
 
 		/* Set waiters before checking lock_word to ensure wake-up
 		signal is sent. This may lead to some unnecessary signals. */
-		rw_lock_set_waiter_flag(lock);
+		if (high_priority) {
+			prio_rw_lock_t*	prio_rw_lock
+				= (prio_rw_lock_t *) _lock;
+			prio_rw_lock->high_priority_s_waiters = 1;
+		} else {
+			rw_lock_set_waiter_flag(lock);
+		}
 
-		if (TRUE == rw_lock_s_lock_low(lock, pass, file_name, line)) {
+		if (!rw_lock_higher_prio_waiters_exist(priority_lock,
+						       high_priority, lock)
+		    && (TRUE == rw_lock_s_lock_low(lock, pass,
+						   file_name, line))) {
 			sync_array_free_cell(sync_arr, index);
 			return; /* Success */
 		}
@@ -618,7 +712,12 @@
 	ulint		pass,	/*!< in: pass value; != 0, if the lock will
 				be passed to another thread to unlock */
 	const char*	file_name,/*!< in: file name where lock requested */
-	ulint		line)	/*!< in: line where requested */
+	ulint		line,	/*!< in: line where requested */
+	bool		priority_lock,
+				/*!< in: whether the lock is a priority lock */
+	bool		high_priority)
+				/*!< in: whether we are acquiring a priority
+				lock with high priority */
 {
 	ulint		i;	/*!< spin round count */
 	ulint		index;	/*!< index of the reserved wait cell */
@@ -640,12 +739,15 @@
 
 lock_loop:
 
-	if (rw_lock_x_lock_low(lock, pass, file_name, line)) {
+	if (!rw_lock_higher_prio_waiters_exist(priority_lock, high_priority,
+					       lock)
+	    && rw_lock_x_lock_low(lock, pass, file_name, line)) {
 		rw_lock_stats.rw_x_spin_round_count.add(counter_index, i);
 
 		return;	/* Locking succeeded */
 
-	} else {
+	} else if (!rw_lock_higher_prio_waiters_exist(priority_lock,
+						      high_priority, lock)) {
 
 		if (!spinning) {
 			spinning = TRUE;
@@ -669,26 +771,42 @@
 		} else {
 			goto lock_loop;
 		}
+	} else {
+
+		/* In case we skipped spinning because of higher-priority
+		waiters already waiting, perform only this bit of the spinning
+		code path.  */
+		os_thread_yield();
 	}
 
-	rw_lock_stats.rw_x_spin_round_count.add(counter_index, i);
-
-	if (srv_print_latch_waits) {
-		fprintf(stderr,
-			"Thread " ULINTPF " spin wait rw-x-lock at %p"
-			" '%s' rnds " ULINTPF "\n",
-			os_thread_pf(os_thread_get_curr_id()), (void*) lock,
-			lock->lock_name, i);
+	if (spinning) {
+
+		rw_lock_stats.rw_x_spin_round_count.add(counter_index, i);
+
+		if (srv_print_latch_waits) {
+			fprintf(stderr,
+				"Thread " ULINTPF " spin wait rw-x-lock at %p"
+				" '%s' rnds " ULINTPF "\n",
+				os_thread_pf(os_thread_get_curr_id()),
+				(void*) lock,lock->lock_name, i);
+		}
 	}
 
 	sync_arr = sync_array_get();
 
 	sync_array_reserve_cell(
-		sync_arr, lock, RW_LOCK_EX, file_name, line, &index);
+		sync_arr, lock,
+		high_priority ? PRIO_RW_LOCK_EX : RW_LOCK_EX,
+		file_name, line, &index);
 
 	/* Waiters must be set before checking lock_word, to ensure signal
 	is sent. This could lead to a few unnecessary wake-up signals. */
-	rw_lock_set_waiter_flag(lock);
+	if (high_priority) {
+		prio_rw_lock_t*	prio_lock = (prio_rw_lock_t *)lock;
+		prio_lock->high_priority_x_waiters = 1;
+	} else {
+		rw_lock_set_waiter_flag(lock);
+	}
 
 	if (rw_lock_x_lock_low(lock, pass, file_name, line)) {
 		sync_array_free_cell(sync_arr, index);
@@ -713,6 +831,29 @@
 	goto lock_loop;
 }
 
+/******************************************************************//**
+NOTE! Use the corresponding macro, not directly this function! Lock a priority
+rw-lock in exclusive mode for the current thread. If the rw-lock is locked
+in shared or exclusive mode, or there is an exclusive lock request waiting,
+the function spins a preset time (controlled by SYNC_SPIN_ROUNDS), waiting
+for the lock, before suspending the thread. If the same thread has an x-lock
+on the rw-lock, locking succeed, with the following exception: if pass != 0,
+only a single x-lock may be taken on the lock. NOTE: If the same thread has
+an s-lock, locking does not succeed! */
+UNIV_INTERN
+void
+rw_lock_x_lock_func(
+/*================*/
+	prio_rw_lock_t*	lock,	/*!< in: pointer to rw-lock */
+	ulint		pass,	/*!< in: pass value; != 0, if the lock will
+				  be passed to another thread to unlock */
+	const char*	file_name,/*!< in: file name where lock requested */
+	ulint		line)	/*!< in: line where requested */
+{
+	rw_lock_x_lock_func(&lock->base_lock, pass, file_name, line, true,
+			    srv_current_thread_priority > 0);
+}
+
 #ifdef UNIV_SYNC_DEBUG
 /******************************************************************//**
 Acquires the debug mutex. We cannot use the mutex defined in sync0sync,
@@ -881,6 +1022,21 @@
 
 	return(FALSE);
 }
+
+/******************************************************************//**
+Checks if the thread has locked the priority rw-lock in the specified mode,
+with the pass value == 0. */
+UNIV_INTERN
+ibool
+rw_lock_own(
+/*========*/
+	prio_rw_lock_t*	lock,		/*!< in: rw-lock */
+	ulint		lock_type)	/*!< in: lock type: RW_LOCK_SHARED,
+					RW_LOCK_EX */
+{
+	return(rw_lock_own(&lock->base_lock, lock_type));
+}
+
 #endif /* UNIV_SYNC_DEBUG */
 
 /******************************************************************//**

=== modified file 'Percona-Server/storage/innobase/sync/sync0sync.cc'
--- Percona-Server/storage/innobase/sync/sync0sync.cc	2013-09-20 05:27:28 +0000
+++ Percona-Server/storage/innobase/sync/sync0sync.cc	2013-09-26 14:59:13 +0000
@@ -321,6 +321,40 @@
 }
 
 /******************************************************************//**
+Creates, or rather, initializes a priority mutex object in a specified memory
+location (which must be appropriately aligned). The mutex is initialized
+in the reset state. Explicit freeing of the mutex with mutex_free is
+necessary only if the memory block containing it is freed. */
+UNIV_INTERN
+void
+mutex_create_func(
+/*==============*/
+	ib_prio_mutex_t*	mutex,		/*!< in: pointer to memory */
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+	ulint			level,		/*!< in: level */
+# endif /* UNIV_SYNC_DEBUG */
+	const char*		cfile_name,	/*!< in: file name where
+						created */
+	ulint			cline,		/*!< in: file line where
+						created */
+#endif /* UNIV_DEBUG */
+	const char*		cmutex_name)	/*!< in: mutex name */
+{
+	mutex_create_func(&mutex->base_mutex,
+#ifdef UNIV_DEBUG
+# ifdef UNIV_SYNC_DEBUG
+			  level,
+#endif /* UNIV_SYNC_DEBUG */
+			  cfile_name,
+			  cline,
+#endif /* UNIV_DEBUG */
+			  cmutex_name);
+	mutex->high_priority_waiters = 0;
+	mutex->high_priority_event = os_event_create();
+}
+
+/******************************************************************//**
 NOTE! Use the corresponding macro mutex_free(), not directly this function!
 Calling this function is obligatory only if the memory buffer containing
 the mutex is freed. Removes a mutex object from the mutex list. The mutex
@@ -380,6 +414,22 @@
 	return;
 }
 
+/******************************************************************//**
+NOTE! Use the corresponding macro mutex_free(), not directly this function!
+Calling this function is obligatory only if the memory buffer containing
+the mutex is freed. Removes a priority mutex object from the mutex list. The
+mutex is checked to be in the reset state. */
+UNIV_INTERN
+void
+mutex_free_func(
+/*============*/
+	ib_prio_mutex_t*	mutex)	/*!< in: mutex */
+{
+	ut_a(mutex->high_priority_waiters == 0);
+	os_event_free(mutex->high_priority_event);
+	mutex_free_func(&mutex->base_mutex);
+}
+
 /********************************************************************//**
 NOTE! Use the corresponding macro in the header file, not this function
 directly. Tries to lock the mutex for the current thread. If the lock is not
@@ -442,6 +492,20 @@
 	return(mutex_get_lock_word(mutex) == 1
 	       && os_thread_eq(mutex->thread_id, os_thread_get_curr_id()));
 }
+
+/******************************************************************//**
+Checks that the current thread owns the priority mutex. Works only
+in the debug version.
+@return	TRUE if owns */
+UNIV_INTERN
+ibool
+mutex_own(
+/*======*/
+	const ib_prio_mutex_t*	mutex)	/*!< in: priority mutex */
+{
+	return mutex_own(&mutex->base_mutex);
+}
+
 #endif /* UNIV_DEBUG */
 
 /******************************************************************//**
@@ -464,14 +528,17 @@
 }
 
 /******************************************************************//**
-Reserves a mutex for the current thread. If the mutex is reserved, the
-function spins a preset time (controlled by SYNC_SPIN_ROUNDS), waiting
-for the mutex before suspending the thread. */
+Reserves a mutex or a priority mutex for the current thread. If the mutex is
+reserved, the function spins a preset time (controlled by SYNC_SPIN_ROUNDS),
+waiting for the mutex before suspending the thread. */
 UNIV_INTERN
 void
 mutex_spin_wait(
 /*============*/
-	ib_mutex_t*	mutex,		/*!< in: pointer to mutex */
+	void*		_mutex,		/*!< in: pointer to mutex */
+	bool		high_priority,	/*!< in: whether the mutex is a
+					priority mutex with high priority
+					specified */
 	const char*	file_name,	/*!< in: file name where mutex
 					requested */
 	ulint		line)		/*!< in: line where requested */
@@ -480,6 +547,10 @@
 	ulint		index;		/* index of the reserved wait cell */
 	sync_array_t*	sync_arr;
 	size_t		counter_index;
+	/* The typecast below is performed for some of the priority mutexes
+	too, when !high_priority.  This exploits the fact that regular mutex is
+	a prefix of the priority mutex in memory.  */
+	ib_mutex_t*	mutex = (ib_mutex_t *) _mutex;
 
 	counter_index = (size_t) os_thread_get_curr_id();
 
@@ -543,7 +614,8 @@
 	sync_arr = sync_array_get();
 
 	sync_array_reserve_cell(
-		sync_arr, mutex, SYNC_MUTEX, file_name, line, &index);
+		sync_arr, mutex, high_priority ? SYNC_PRIO_MUTEX : SYNC_MUTEX,
+		file_name, line, &index);
 
 	/* The memory order of the array reservation and the change in the
 	waiters field is important: when we suspend a thread, we first
@@ -551,7 +623,11 @@
 	released in mutex_exit, the waiters field is first set to zero and
 	then the event is set to the signaled state. */
 
-	mutex_set_waiters(mutex, 1);
+	if (high_priority) {
+		((ib_prio_mutex_t *)_mutex)->high_priority_waiters = 1;
+	} else {
+		mutex_set_waiters(mutex, 1);
+	}
 
 	/* Try to reserve still a few times */
 	for (i = 0; i < 4; i++) {