Merge lp:~gl-az/percona-server/ST-31919-5.1 into lp:percona-server/5.1

[George Ormond Lorch III]

Fix for upstream 68354 and lp 1182572:
  Server crashes on update/join FEDERATED + local table when only 1 local row

This issue seems to be the result of a poor or changed assumption in the order
in which handler calls are made. In this specific scenario (a join with a local
table that has exactly one matching record) there is an assumption in the code
that a call to position will always come after a 'rnd_init/index_init' and
'rnd_next/index_read' but before 'rnd_end/index_end' is called.

The failing call order is:
  ha_federated::index_init
  ha_federated::index_read
  ha_federated::index_end
  ha_federated::position
  ha_federated::rnd_init
  ha_federated::extra
  ha_federated::rnd_pos

So here you can see position is getting called _outside_ of the scope of an
_init.._end table and/or index scan. This does indicate a change/problem in
the server but that is outside the scope of this fix.

As results are fetched, they are added to an interal list of result sets called
'results'. The last result set/current position is also held in the
'stored_result' value. If 'index_end' or 'rnd_end' is called before 'position'
has been called, the 'stored_result' is zeroed out and that result is removed
from the 'stored_results' list. When 'position' is called, a NULL result is
returned, which is then passed back through 'rnd_pos', which then dereferences
the NULL value and tries to manipulate some of the result internals, crashing.

So seems that the intention of the whole 'stored_result' was to have a place
to track all results allocated for the purposes of keeping them valid for the
'position' and 'rnd_pos' calls.

My approch to fixing this is to restructure this somewhan and instead of
tracking all results allocated in the 'results' list and popping them out when
they are discovered to not be needed, I went the other way. The results only
get addes to a storage list when it has been proven that they are in fact
needed later. So now for each new result set, it is temporarily stored in a
new value called 'last_result'. Whenever a new last_result needs created, it is
set by calling a new function 'set_last_result'. This function will check to
see if 'position' has been called on that result and if so, add it to the
(renamed) list 'stored_results' and clears the 'position_called' flag, else it
will free the 'last_result'. It then sets the new 'last_result' value.

Because of the out of scan scope call of position, we can no longer clear the
'stored_results' lists on '_end'. It will be cleared now when 'reset' is
called.

I have created a test case (suite/federated/federated_bug_68354.test) and
verified that it fails without the fix. Everything tests out under the federated
suite and the issue appears fixed.

The only downside is that a lengthy query could use a lot of memory while
hanging onto old 'position'ed results but I can't see how that can be safely
avoided in this particular handler.

[George Ormond Lorch III (gl-az)]

http://jenkins.percona.com/view/PS%205.1/job/percona-server-5.1-param/556/

[Alexey Kopytov (akopytov)]

George,

I agree with the general idea of the fix. I would try to keep the
patch minimal and avoid refactoring to simplify maintenance, but
that's more a personal preference. The only significant comment that
I have is that we seem to introduce a memory leak:

- before the patch, 'stored_result' is guaranteed to also be in the
  'results' array. Which ensures that ha_federated::reset() calls
  mysql_free_result() for all stored results. Now last_result is
  guaranteed to NOT be in the 'stored_results' array. Don't we need
  an explicit mysql_free_result() for it on reset?

Minor stuff:

- the test case needs a header referencing bug number(s)
- no need to include include/not_embedded.inc in the test case, as
  it is already included from suite/federated/include/federated.inc
- unnecessary whitespace changes on lines 411 (the server code
  formatting rules require 2 blank lines between functions), 439 and 447
- wrong comment formatting in ha_federated.h (again, server code
  formatting is different from InnoDB)

review: Needs Fixing

[Alexey Kopytov (akopytov)]

Approved, though the changes to ha_federated::rnd_end() and the blank line change on line 420 are not needed.

review: Approve

[Laurynas Biveinis (laurynas-biveinis)]

George -

Feel free to send this diff (and its 5.5/5.6 counterparts) to the upstream bug (use Contributions tab) now that we have OCA signed.

[George Ormond Lorch III (gl-az)]

Fixed rnd_end and odd blank line (wasn't a blank line, but a line with a single space in the original and I had removed the space) and repushed to same brranch.

[George Ormond Lorch III (gl-az)]

On 6/19/2013 4:21 AM, Laurynas Biveinis wrote:
> George -
>
> Feel free to send this diff (and its 5.5/5.6 counterparts) to the upstream bug (use Contributions tab) now that we have OCA signed.
Good idea...done.

--
George O. Lorch III
Software Engineer, Percona
[phone] x542 US/Arizona (GMT -7)
skype: george.ormond.lorch.iii

[Laurynas Biveinis (laurynas-biveinis)]

Since I merged a previous version of this MP (yay!), will create a new MP to revert

@@ -2591,14 +2598,14 @@
 int ha_federated::rnd_end()
 {
   DBUG_ENTER("ha_federated::rnd_end");
- DBUG_RETURN(index_end());
+ active_index= MAX_KEY;
+ DBUG_RETURN(0);
 }

and

-
+
 int ha_federated::external_lock(THD *thd, int lock_type)

=== added file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.result'
--- Percona-Server/mysql-test/suite/federated/federated_bug_68354.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/federated/federated_bug_68354.result	2013-06-25 17:20:39 +0000
@@ -0,0 +1,80 @@
+CREATE DATABASE federated;
+CREATE DATABASE federated;
+SET @OLD_MASTER_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+SET @OLD_SLAVE_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+DROP TABLE IF EXISTS federated.t1;
+Warnings:
+Note	1051	Unknown table 't1'
+CREATE TABLE federated.t1 (
+`id` int(20) NOT NULL,
+`dummy` int(20) DEFAULT NULL,
+PRIMARY KEY (`id`))
+ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+INSERT INTO federated.t1 (`id`) VALUES
+(1001), (1002), (1003), (1004), (1005),
+(1006), (1007), (1008), (1009), (1010);
+CREATE TABLE federated.t1 (
+`id` int(20) NOT NULL,
+`dummy` int(20) DEFAULT NULL,
+PRIMARY KEY (`id`))
+ENGINE="FEDERATED"
+  DEFAULT CHARSET=latin1
+CONNECTION='mysql://root@127.0.0.1:SLAVE_PORT/federated/t1';
+CREATE TABLE federated.t2 (
+`id` int(20) NOT NULL,
+`dummy` int(20) NOT NULL,
+PRIMARY KEY (`id`))
+ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+INSERT INTO federated.t2 (`id`, `dummy`) VALUES (1005, 5);
+SELECT * FROM federated.t1;
+id	dummy
+1001	NULL
+1002	NULL
+1003	NULL
+1004	NULL
+1005	NULL
+1006	NULL
+1007	NULL
+1008	NULL
+1009	NULL
+1010	NULL
+UPDATE federated.t1 ft1 INNER JOIN federated.t2 ft2 ON ft1.id = ft2.id SET ft1.dummy = ft2.dummy WHERE ft1.dummy IS NULL;
+SELECT * FROM federated.t1;
+id	dummy
+1001	NULL
+1002	NULL
+1003	NULL
+1004	NULL
+1005	5
+1006	NULL
+1007	NULL
+1008	NULL
+1009	NULL
+1010	NULL
+DELETE FROM federated.t1 WHERE id = 1005;
+SELECT * FROM federated.t1 WHERE id = 1005;
+id	dummy
+SELECT * FROM federated.t1;
+id	dummy
+1001	NULL
+1002	NULL
+1003	NULL
+1004	NULL
+1006	NULL
+1007	NULL
+1008	NULL
+1009	NULL
+1010	NULL
+DROP TABLE federated.t2;
+DROP TABLE federated.t1;
+DROP TABLE federated.t1;
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_MASTER_CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_SLAVE_CONCURRENT_INSERT;
+DROP TABLE IF EXISTS federated.t1;
+DROP DATABASE federated;
+DROP TABLE IF EXISTS federated.t1;
+DROP DATABASE federated;

=== renamed file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.result' => 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.result.moved'
=== added file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.test'
--- Percona-Server/mysql-test/suite/federated/federated_bug_68354.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/federated/federated_bug_68354.test	2013-06-25 17:20:39 +0000
@@ -0,0 +1,79 @@
+# Bug 1182572 68354: Server crashes on update/join FEDERATED + local table when 
+# only 1 local row.
+#
+# When updating a federated table with UPDATE ... JOIN, the server consistently
+# crashes with Signal 11 when only 1 row exists in the local table involved in
+# the join and that 1 row can be joined with a row in the federated table.
+#
+
+--source suite/federated/include/federated.inc
+
+connection default;
+
+# Disable concurrent inserts to avoid test failures when reading
+# data from concurrent connections (insert might return before
+# the data is actually in the table).
+SET @OLD_MASTER_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+
+connection slave;
+SET @OLD_SLAVE_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+DROP TABLE IF EXISTS federated.t1;
+CREATE TABLE federated.t1 (
+    `id` int(20) NOT NULL,
+    `dummy` int(20) DEFAULT NULL,
+    PRIMARY KEY (`id`))
+  ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+
+INSERT INTO federated.t1 (`id`) VALUES
+    (1001), (1002), (1003), (1004), (1005),
+    (1006), (1007), (1008), (1009), (1010);
+
+connection master;
+--replace_result $SLAVE_MYPORT SLAVE_PORT
+eval CREATE TABLE federated.t1 (
+    `id` int(20) NOT NULL,
+    `dummy` int(20) DEFAULT NULL,
+    PRIMARY KEY (`id`))
+  ENGINE="FEDERATED"
+  DEFAULT CHARSET=latin1
+  CONNECTION='mysql://root@127.0.0.1:$SLAVE_MYPORT/federated/t1';
+
+CREATE TABLE federated.t2 (
+    `id` int(20) NOT NULL,
+    `dummy` int(20) NOT NULL,
+    PRIMARY KEY (`id`))
+  ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+
+INSERT INTO federated.t2 (`id`, `dummy`) VALUES (1005, 5);
+
+SELECT * FROM federated.t1;
+
+# master would crash here before bug fix
+UPDATE federated.t1 ft1 INNER JOIN federated.t2 ft2 ON ft1.id = ft2.id SET ft1.dummy = ft2.dummy WHERE ft1.dummy IS NULL;
+
+# sanity test to make sure correct rows are being returned and server doesn't
+# also crash during DELETE
+SELECT * FROM federated.t1;
+DELETE FROM federated.t1 WHERE id = 1005;
+SELECT * FROM federated.t1 WHERE id = 1005;
+
+SELECT * FROM federated.t1;
+
+DROP TABLE federated.t2;
+DROP TABLE federated.t1;
+
+connection slave;
+DROP TABLE federated.t1;
+
+connection default;
+
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_MASTER_CONCURRENT_INSERT;
+connection slave;
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_SLAVE_CONCURRENT_INSERT;
+
+connection default;
+--source suite/federated/include/federated_cleanup.inc

=== renamed file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.test' => 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.test.moved'
=== modified file 'Percona-Server/storage/federated/ha_federated.cc'
--- Percona-Server/storage/federated/ha_federated.cc	2013-06-18 18:39:22 +0000
+++ Percona-Server/storage/federated/ha_federated.cc	2013-06-25 17:20:39 +0000
@@ -2329,6 +2329,7 @@
 
   DBUG_ENTER("ha_federated::index_read");
 
+<<<<<<< TREE
   if ((retval= index_read_idx_with_result_set(buf, active_index, key,
                                              key_len, find_flag,
                                              &mysql_result)))
@@ -2336,6 +2337,15 @@
 
   set_last_result(mysql_result);
   DBUG_RETURN(0);
+=======
+  if ((retval= index_read_idx_with_result_set(buf, active_index, key,
+                                              key_len, find_flag,
+                                              &mysql_result)))
+    DBUG_RETURN(retval);
+
+  set_last_result(mysql_result);
+  DBUG_RETURN(0);
+>>>>>>> MERGE-SOURCE
 }
 
 
@@ -2505,10 +2515,17 @@
     retval= HA_ERR_END_OF_FILE;
     goto error;
   }
+<<<<<<< TREE
 
   set_last_result(mysql_result);
   
   DBUG_RETURN(read_next(table->record[0], mysql_result));
+=======
+
+  set_last_result(mysql_result);
+
+  DBUG_RETURN(read_next(table->record[0], mysql_result));
+>>>>>>> MERGE-SOURCE
 
 error:
   table->status= STATUS_NOT_FOUND;
@@ -2714,8 +2731,12 @@
   /* Store data cursor position. */
   memcpy_fixed(ref + sizeof(MYSQL_RES *), &current_position,
                sizeof(MYSQL_ROW_OFFSET));
+<<<<<<< TREE
 
   position_called= true;
+=======
+  position_called= true;
+>>>>>>> MERGE-SOURCE
   DBUG_VOID_RETURN;
 }