Merge lp:~gl-az/percona-server/ST-31919-5.5 into lp:percona-server/5.5

[George Ormond Lorch III]

Resubmitting with correct GCA procedure followed whcih will cause issues now when merging to trunk but _should_ leave the trunk in a clean state.

Jenkins --suite=federated: http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/775/

Fix for upstream 68354 and lp 1182572:
  Server crashes on update/join FEDERATED + local table when only 1 local row

This issue seems to be the result of a poor or changed assumption in the order
in which handler calls are made. In this specific scenario (a join with a local
table that has exactly one matching record) there is an assumption in the code
that a call to position will always come after a 'rnd_init/index_init' and
'rnd_next/index_read' but before 'rnd_end/index_end' is called.

The failing call order is:
  ha_federated::index_init
  ha_federated::index_read
  ha_federated::index_end
  ha_federated::position
  ha_federated::rnd_init
  ha_federated::extra
  ha_federated::rnd_pos

So here you can see position is getting called _outside_ of the scope of an
_init.._end table and/or index scan. This does indicate a change/problem in
the server but that is outside the scope of this fix.

As results are fetched, they are added to an interal list of result sets called
'results'. The last result set/current position is also held in the
'stored_result' value. If 'index_end' or 'rnd_end' is called before 'position'
has been called, the 'stored_result' is zeroed out and that result is removed
from the 'stored_results' list. When 'position' is called, a NULL result is
returned, which is then passed back through 'rnd_pos', which then dereferences
the NULL value and tries to manipulate some of the result internals, crashing.

So seems that the intention of the whole 'stored_result' was to have a place
to track all results allocated for the purposes of keeping them valid for the
'position' and 'rnd_pos' calls.

My approch to fixing this is to restructure this somewhan and instead of
tracking all results allocated in the 'results' list and popping them out when
they are discovered to not be needed, I went the other way. The results only
get addes to a storage list when it has been proven that they are in fact
needed later. So now for each new result set, it is temporarily stored in a
new value called 'last_result'. Whenever a new last_result needs created, it is
set by calling a new function 'set_last_result'. This function will check to
see if 'position' has been called on that result and if so, add it to the
(renamed) list 'stored_results' and clears the 'position_called' flag, else it
will free the 'last_result'. It then sets the new 'last_result' value.

Because of the out of scan scope call of position, we can no longer clear the
'stored_results' lists on '_end'. It will be cleared now when 'reset' is
called.

I have created a test case (suite/federated/federated_bug_68354.test) and
verified that it fails without the fix. Everything tests out under the federated
suite and the issue appears fixed.

The only downside is that a lengthy query could use a lot of memory while
hanging onto old 'position'ed results but I can't see how that can be safely
avoided in this particular handler.

[George Ormond Lorch III (gl-az)]

http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/767/

[Alexey Kopytov (akopytov)]

Same comments as in the 5.1 MP plus:

   - copyright change in ha_federated.cc
   - 5.6 MP?

review: Needs Fixing

[Alexey Kopytov (akopytov)]

Found the 5.6 MP, it was just not linked to the bug.

[Alexey Kopytov (akopytov)]

Approved with the same comments as in the 5.1 MP + the copyright change in ha_federated.cc is still there. That would be nice to fix before merging.

review: Approve

[George Ormond Lorch III (gl-az)]

Fixed same as 5.1, good catch on the copyright, must have pulled that in as part of my merge up and missed it in the diff. Pushed to same branch.

[George Ormond Lorch III (gl-az)]

Same as 5.5, fixed and pushed to same branch.

[George Ormond Lorch III (gl-az)]

Oops, wrong comment/branch/browser tab

[Alexey Kopytov (akopytov)]

review: Approve

=== added file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.result'
--- Percona-Server/mysql-test/suite/federated/federated_bug_68354.result	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/federated/federated_bug_68354.result	2013-06-25 17:38:33 +0000
@@ -0,0 +1,80 @@
+CREATE DATABASE federated;
+CREATE DATABASE federated;
+SET @OLD_MASTER_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+SET @OLD_SLAVE_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+DROP TABLE IF EXISTS federated.t1;
+Warnings:
+Note	1051	Unknown table 't1'
+CREATE TABLE federated.t1 (
+`id` int(20) NOT NULL,
+`dummy` int(20) DEFAULT NULL,
+PRIMARY KEY (`id`))
+ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+INSERT INTO federated.t1 (`id`) VALUES
+(1001), (1002), (1003), (1004), (1005),
+(1006), (1007), (1008), (1009), (1010);
+CREATE TABLE federated.t1 (
+`id` int(20) NOT NULL,
+`dummy` int(20) DEFAULT NULL,
+PRIMARY KEY (`id`))
+ENGINE="FEDERATED"
+  DEFAULT CHARSET=latin1
+CONNECTION='mysql://root@127.0.0.1:SLAVE_PORT/federated/t1';
+CREATE TABLE federated.t2 (
+`id` int(20) NOT NULL,
+`dummy` int(20) NOT NULL,
+PRIMARY KEY (`id`))
+ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+INSERT INTO federated.t2 (`id`, `dummy`) VALUES (1005, 5);
+SELECT * FROM federated.t1;
+id	dummy
+1001	NULL
+1002	NULL
+1003	NULL
+1004	NULL
+1005	NULL
+1006	NULL
+1007	NULL
+1008	NULL
+1009	NULL
+1010	NULL
+UPDATE federated.t1 ft1 INNER JOIN federated.t2 ft2 ON ft1.id = ft2.id SET ft1.dummy = ft2.dummy WHERE ft1.dummy IS NULL;
+SELECT * FROM federated.t1;
+id	dummy
+1001	NULL
+1002	NULL
+1003	NULL
+1004	NULL
+1005	5
+1006	NULL
+1007	NULL
+1008	NULL
+1009	NULL
+1010	NULL
+DELETE FROM federated.t1 WHERE id = 1005;
+SELECT * FROM federated.t1 WHERE id = 1005;
+id	dummy
+SELECT * FROM federated.t1;
+id	dummy
+1001	NULL
+1002	NULL
+1003	NULL
+1004	NULL
+1006	NULL
+1007	NULL
+1008	NULL
+1009	NULL
+1010	NULL
+DROP TABLE federated.t2;
+DROP TABLE federated.t1;
+DROP TABLE federated.t1;
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_MASTER_CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_SLAVE_CONCURRENT_INSERT;
+DROP TABLE IF EXISTS federated.t1;
+DROP DATABASE federated;
+DROP TABLE IF EXISTS federated.t1;
+DROP DATABASE federated;

=== renamed file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.result' => 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.result.moved'
=== added file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.test'
--- Percona-Server/mysql-test/suite/federated/federated_bug_68354.test	1970-01-01 00:00:00 +0000
+++ Percona-Server/mysql-test/suite/federated/federated_bug_68354.test	2013-06-25 17:38:33 +0000
@@ -0,0 +1,79 @@
+# Bug 1182572 68354: Server crashes on update/join FEDERATED + local table when 
+# only 1 local row.
+#
+# When updating a federated table with UPDATE ... JOIN, the server consistently
+# crashes with Signal 11 when only 1 row exists in the local table involved in
+# the join and that 1 row can be joined with a row in the federated table.
+#
+
+--source suite/federated/include/federated.inc
+
+connection default;
+
+# Disable concurrent inserts to avoid test failures when reading
+# data from concurrent connections (insert might return before
+# the data is actually in the table).
+SET @OLD_MASTER_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+
+connection slave;
+SET @OLD_SLAVE_CONCURRENT_INSERT= @@GLOBAL.CONCURRENT_INSERT;
+SET @@GLOBAL.CONCURRENT_INSERT= 0;
+DROP TABLE IF EXISTS federated.t1;
+CREATE TABLE federated.t1 (
+    `id` int(20) NOT NULL,
+    `dummy` int(20) DEFAULT NULL,
+    PRIMARY KEY (`id`))
+  ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+
+INSERT INTO federated.t1 (`id`) VALUES
+    (1001), (1002), (1003), (1004), (1005),
+    (1006), (1007), (1008), (1009), (1010);
+
+connection master;
+--replace_result $SLAVE_MYPORT SLAVE_PORT
+eval CREATE TABLE federated.t1 (
+    `id` int(20) NOT NULL,
+    `dummy` int(20) DEFAULT NULL,
+    PRIMARY KEY (`id`))
+  ENGINE="FEDERATED"
+  DEFAULT CHARSET=latin1
+  CONNECTION='mysql://root@127.0.0.1:$SLAVE_MYPORT/federated/t1';
+
+CREATE TABLE federated.t2 (
+    `id` int(20) NOT NULL,
+    `dummy` int(20) NOT NULL,
+    PRIMARY KEY (`id`))
+  ENGINE="MyISAM"
+  DEFAULT CHARSET=latin1;
+
+INSERT INTO federated.t2 (`id`, `dummy`) VALUES (1005, 5);
+
+SELECT * FROM federated.t1;
+
+# master would crash here before bug fix
+UPDATE federated.t1 ft1 INNER JOIN federated.t2 ft2 ON ft1.id = ft2.id SET ft1.dummy = ft2.dummy WHERE ft1.dummy IS NULL;
+
+# sanity test to make sure correct rows are being returned and server doesn't
+# also crash during DELETE
+SELECT * FROM federated.t1;
+DELETE FROM federated.t1 WHERE id = 1005;
+SELECT * FROM federated.t1 WHERE id = 1005;
+
+SELECT * FROM federated.t1;
+
+DROP TABLE federated.t2;
+DROP TABLE federated.t1;
+
+connection slave;
+DROP TABLE federated.t1;
+
+connection default;
+
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_MASTER_CONCURRENT_INSERT;
+connection slave;
+SET @@GLOBAL.CONCURRENT_INSERT= @OLD_SLAVE_CONCURRENT_INSERT;
+
+connection default;
+--source suite/federated/include/federated_cleanup.inc

=== renamed file 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.test' => 'Percona-Server/mysql-test/suite/federated/federated_bug_68354.test.moved'
=== modified file 'Percona-Server/storage/federated/ha_federated.cc'
--- Percona-Server/storage/federated/ha_federated.cc	2013-06-18 18:39:30 +0000
+++ Percona-Server/storage/federated/ha_federated.cc	2013-06-25 17:38:33 +0000
@@ -2780,8 +2780,12 @@
   /* Store data cursor position. */
   memcpy(ref + sizeof(MYSQL_RES *), &current_position,
                sizeof(MYSQL_ROW_OFFSET));
+<<<<<<< TREE
 
   position_called= true;
+=======
+  position_called= true;
+>>>>>>> MERGE-SOURCE
   DBUG_VOID_RETURN;
 }