Merge lp:~vlad-lesin/percona-server/5.5-mysqlbinlog-replacedb into lp:percona-server/5.5

Proposed by Vlad Lesin
Status: Merged
Approved by: Laurynas Biveinis
Approved revision: no longer in the source branch.
Merged at revision: 629
Proposed branch: lp:~vlad-lesin/percona-server/5.5-mysqlbinlog-replacedb
Merge into: lp:percona-server/5.5
Diff against target: 1027 lines (+858/-4) (has conflicts)
15 files modified
client/CMakeLists.txt (+6/-0)
client/client_priv.h (+1/-0)
client/mysqlbinlog.cc (+164/-3)
mysql-test/extra/rpl_tests/grep_pattern.inc (+22/-0)
mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result (+33/-0)
mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result (+29/-0)
mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result (+29/-0)
mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result (+152/-0)
mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test (+95/-0)
mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test (+79/-0)
mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test (+61/-0)
mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt (+1/-0)
mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test (+80/-0)
sql/log_event.cc (+102/-0)
sql/log_event.h (+4/-1)
Text conflict in client/CMakeLists.txt
To merge this branch: bzr merge lp:~vlad-lesin/percona-server/5.5-mysqlbinlog-replacedb
Reviewer Review Type Date Requested Status
Laurynas Biveinis (community) Approve
Stewart Smith (community) Needs Fixing
George Ormond Lorch III (community) g2 Approve
Review via email: mp+196578@code.launchpad.net

Description of the change

Port --rewrite-db mysqlbinlog option from 5.1.

--rewrite-db is a string option which allows to replace one db-name to another
in mysqlbinlog output. The format of the string value is the following:
"from1->to1,from2->to2,from3->to3". Where "to[123]" are the names to which
the correspondent "from[123]" names must be replaced.

The original implementation does not replace db names in statements for
statement-based binlog, it just replaces db name of binlog event and issues
"USE" statement with replaced db name if necessary. I think this restriction
concerned with unwillingness to implement complex logic for parsing db names
from statements for such simple feature.

But for row-based binlog it replaces db names in statements correctly because
each table name is mapped to some number in binlog and it is quite easy to
extract such table ids from binlog event and replace one name by another.

http://jenkins.percona.com/job/percona-server-5.5-param/894/

To post a comment you must log in.
Revision history for this message
George Ormond Lorch III (gl-az) :
review: Approve (g2)
Revision history for this message
Stewart Smith (stewart) wrote :

same comment as 5.6:

It states that you can do "a->b,c->d" but I don't see the handling for the comma in the mysqlbinlog.cc command line parsing. Am I missing something? We should also test this.

review: Needs Fixing
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

    - This needs two associated blueprints, one for 5.5, and one for
      5.6

    - Oracle has implemented this functionality in 5.7.1. From the
      user perspective the difference is option syntax, ours:
      --rewrite-db='a->b,c->d'
      theirs:
      --rewrite-db='a->b' --rewrite-db='c->d'
      To ensure smooth migration once we switch to their
      implementation in 5.7, please test both syntaxes in the
      testcase.

    - What should be the interaction between --rewrite-db and
      --database options, i.e. which one should come first? My first
      thought is rewrite-db before database, i.e. first we map, and
      then we filter. This is what the current patch appears to do
      for SBR, but it's reversed for RBR (judging from
      shall_skip_database() and rewrite_db() call order). This would
      also need a testcase.

    - Rewrite rules are not applied to CREATE_FILE_EVENT. This
      wouldn't be an issue in itself, but these events are filtered
      against --database, thus, depending on how the previous item is
      resolved, these events might need rewrite-db application too,
      so that they can be filtered properly. This would also need a
      testcase.

    - LOAD DATA events (EXECUTE_LOAD_QUERY_EVENT) need test coverage
      too. Probably this can be handled together with the previous
      item.

    - Because RBR event DB rewrite requires shifting of
      Table_map_og_event contents around if the DB name length
      changes, please test this case for both shorter and longer
      names.

    - Since Oracle has implemented this, there are some Oracle
      testcases too. Consider importing binlog_mysqlbinlog_rewrite_db
      binlog_rewrite_db_noleak binlog_rewrite_suppress_use from MySQL
      5.7. If you choose to, make sure they get the correct bzr file
      ids (i.e. bzr add --file-ids-from=path to 5.7 tree)

review: Needs Fixing
Revision history for this message
Stewart Smith (stewart) wrote :

Laurynas Biveinis <email address hidden> writes:
> - Oracle has implemented this functionality in 5.7.1. From the
> user perspective the difference is option syntax, ours:
> --rewrite-db='a->b,c->d'
> theirs:
> --rewrite-db='a->b' --rewrite-db='c->d'
> To ensure smooth migration once we switch to their
> implementation in 5.7, please test both syntaxes in the
> testcase.

We should probably throw a warning if the user uses the syntax that will
break in 5.7, as really we should just take the Oracle impl in 5.7.

--
Stewart Smith

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> Laurynas Biveinis <email address hidden> writes:
> > - Oracle has implemented this functionality in 5.7.1. From the
> > user perspective the difference is option syntax, ours:
> > --rewrite-db='a->b,c->d'
> > theirs:
> > --rewrite-db='a->b' --rewrite-db='c->d'
> > To ensure smooth migration once we switch to their
> > implementation in 5.7, please test both syntaxes in the
> > testcase.
>
> We should probably throw a warning if the user uses the syntax that will
> break in 5.7, as really we should just take the Oracle impl in 5.7.
>
>
> --
> Stewart Smith

We can also (if the customer does not object) take the 5.7 syntax in 5.5 already for a bit simpler code/testcases.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

On 12/20/2013 12:58 PM, Laurynas Biveinis wrote:
> - This needs two associated blueprints, one for 5.5, and one for
> 5.6
Done.

> - Oracle has implemented this functionality in 5.7.1. From the
> user perspective the difference is option syntax, ours:
> --rewrite-db='a->b,c->d'
> theirs:
> --rewrite-db='a->b' --rewrite-db='c->d'
> To ensure smooth migration once we switch to their
> implementation in 5.7, please test both syntaxes in the
> testcase.
Yes, currently both formats are allowed, but if the old
format(comma-separated rules) is used the warning is issued.

> - What should be the interaction between --rewrite-db and
> --database options, i.e. which one should come first? My first
> thought is rewrite-db before database, i.e. first we map, and
> then we filter. This is what the current patch appears to do
> for SBR, but it's reversed for RBR (judging from
> shall_skip_database() and rewrite_db() call order). This would
> also need a testcase.
Yes, you are right, the order of rewrite-db and database filtering was
reversed for RBR. Fixed and the corresponding test is added.

> - Rewrite rules are not applied to CREATE_FILE_EVENT. This
> wouldn't be an issue in itself, but these events are filtered
> against --database, thus, depending on how the previous item is
> resolved, these events might need rewrite-db application too,
> so that they can be filtered properly. This would also need a
> testcase.
I added the code which replaces db name in the event processing, but I
have no idea how to test it as there is the following comment in
log_event.h:

"The events which really update data are Query_log_event,
Execute_load_query_log_event and old Load_log_event and
Execute_load_log_event events (Execute_load_query is used together with
Begin_load_query and Append_block events to replicate LOAD DATA INFILE.
Create_file/Append_block/Execute_load (which includes Load_log_event)
were used to replicate LOAD DATA before the 5.0.3)."

So CREATE_FILE_EVENT were used before 5.0.3.

> - LOAD DATA events (EXECUTE_LOAD_QUERY_EVENT) need test coverage
> too. Probably this can be handled together with the previous
> item.
The corresponding test case is made.

> - Because RBR event DB rewrite requires shifting of
> Table_map_og_event contents around if the DB name length
> changes, please test this case for both shorter and longer
> names.
Done.
> - Since Oracle has implemented this, there are some Oracle
> testcases too. Consider importing binlog_mysqlbinlog_rewrite_db
> binlog_rewrite_db_noleak binlog_rewrite_suppress_use from MySQL
> 5.7. If you choose to, make sure they get the correct bzr file
> ids (i.e. bzr add --file-ids-from=path to 5.7 tree)
Done.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote :
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> > - Rewrite rules are not applied to CREATE_FILE_EVENT. This
> > wouldn't be an issue in itself, but these events are filtered
> > against --database, thus, depending on how the previous item is
> > resolved, these events might need rewrite-db application too,
> > so that they can be filtered properly. This would also need a
> > testcase.
> I added the code which replaces db name in the event processing, but I
> have no idea how to test it as there is the following comment in
> log_event.h:
>
> "The events which really update data are Query_log_event,
> Execute_load_query_log_event and old Load_log_event and
> Execute_load_log_event events (Execute_load_query is used together with
> Begin_load_query and Append_block events to replicate LOAD DATA INFILE.
> Create_file/Append_block/Execute_load (which includes Load_log_event)
> were used to replicate LOAD DATA before the 5.0.3)."
>
> So CREATE_FILE_EVENT were used before 5.0.3.

Ah sorry I missed this, this is a non-issue then.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

    - The warning for "a->b,c->d" syntax seems to be repeated for
      each comma, so "a->b,c->d,e->f" would print the warning twice,
      which is unnecessary.

    - Does running our testcases on MySQL 5.7 give the expected
      results?

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> - The warning for "a->b,c->d" syntax seems to be repeated for
> each comma, so "a->b,c->d,e->f" would print the warning twice,
> which is unnecessary.
Fixed.

> - Does running our testcases on MySQL 5.7 give the expected
> results?
No because 5.7 does not support our syntax.

http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/930/

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> > - The warning for "a->b,c->d" syntax seems to be repeated for
> > each comma, so "a->b,c->d,e->f" would print the warning twice,
> > which is unnecessary.
> Fixed.

What was the fix?

The relevant code reads

      if (*ptr == ',')
      {
 warning("The comma-separated list of rewritings syntax is obsolete and is discarded in 5.7\n");
        *(ptr++)= 0;
      }

In order to warn only once for e.g. --rewrite-db="a->b,c->d,e->f" --rewrite-db="g->h,i->j,k->l", it would need some sort of global flag already_warned_for_comma_syntax.

Or am I missing something here?

If you need to repush, then also please fix indentation at diff lines 216, 220, 227, 231, and curly brace not having own line at 217.

> > - Does running our testcases on MySQL 5.7 give the expected
> > results?
> No because 5.7 does not support our syntax.

Right, but accounted for syntax changes? Does our and their order of applying --database= --rewrite-db= agree?

review: Needs Information
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> > > - The warning for "a->b,c->d" syntax seems to be repeated for
> > > each comma, so "a->b,c->d,e->f" would print the warning twice,
> > > which is unnecessary.
> > Fixed.
>
> What was the fix?
...
> Or am I missing something here?

Yes, you are right, I was sure I fixed it. But somehow I did not. I believe it is fixed now.

> > > - Does running our testcases on MySQL 5.7 give the expected
> > > results?
> > No because 5.7 does not support our syntax.
>
> Right, but accounted for syntax changes? Does our and their order of applying
> --database= --rewrite-db= agree?

Yes. For 5.7 dump_local_log_entries() and dump_remote_log_entries() invoke Log_event::read_log_event() where db name is rewritten and then invoke process_event() where db name is checked by shall_skip_database() for filtering out. So db name is rewritten and then filtered like in PS.

As well I have rebased the code to the newest 5.5. Here id jenkins testing: http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/942

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> > > > - The warning for "a->b,c->d" syntax seems to be repeated for
> > > > each comma, so "a->b,c->d,e->f" would print the warning twice,
> > > > which is unnecessary.
> > > Fixed.
> >
> > What was the fix?
> ...
> > Or am I missing something here?
>
> Yes, you are right, I was sure I fixed it. But somehow I did not. I believe it
> is fixed now.

The flag is local to OPT_REWRITE_DB, so --rewrite-db="a->b,c->d,e->f" --rewrite-db="g->h,i->j,k->l", would still print the warning twice, wouldn't it?

> As well I have rebased the code to the newest 5.5.

Uhm, why? This is not a GCA branch anymore then?

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> The flag is local to OPT_REWRITE_DB, so --rewrite-db="a->b,c->d,e->f"
> --rewrite-db="g->h,i->j,k->l", would still print the warning twice, wouldn't
> it?
Yep. Fixed.

> > As well I have rebased the code to the newest 5.5.
> Uhm, why? This is not a GCA branch anymore then?
To make merging to trunk a little bit easier. The structure of trunk source tree was changed and there were a couple of conflicts due to some code change which was resolved manually.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

As discussed on IRC, this will need a GCA rebase.

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> As discussed on IRC, this will need a GCA rebase.
Rebased.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'client/CMakeLists.txt'
--- client/CMakeLists.txt 2013-07-23 04:41:43 +0000
+++ client/CMakeLists.txt 2014-02-15 00:33:28 +0000
@@ -57,8 +57,14 @@
57MYSQL_ADD_EXECUTABLE(mysql_plugin mysql_plugin.c)57MYSQL_ADD_EXECUTABLE(mysql_plugin mysql_plugin.c)
58TARGET_LINK_LIBRARIES(mysql_plugin perconaserverclient)58TARGET_LINK_LIBRARIES(mysql_plugin perconaserverclient)
5959
60<<<<<<< TREE
60MYSQL_ADD_EXECUTABLE(mysqlbinlog mysqlbinlog.cc sql_string.cc)61MYSQL_ADD_EXECUTABLE(mysqlbinlog mysqlbinlog.cc sql_string.cc)
61TARGET_LINK_LIBRARIES(mysqlbinlog perconaserverclient)62TARGET_LINK_LIBRARIES(mysqlbinlog perconaserverclient)
63=======
64MYSQL_ADD_EXECUTABLE(mysqlbinlog mysqlbinlog.cc sql_string.cc
65 ../sql/rpl_filter.cc ../sql/sql_list.cc)
66TARGET_LINK_LIBRARIES(mysqlbinlog mysqlclient)
67>>>>>>> MERGE-SOURCE
6268
63MYSQL_ADD_EXECUTABLE(mysqladmin mysqladmin.cc)69MYSQL_ADD_EXECUTABLE(mysqladmin mysqladmin.cc)
64TARGET_LINK_LIBRARIES(mysqladmin perconaserverclient)70TARGET_LINK_LIBRARIES(mysqladmin perconaserverclient)
6571
=== modified file 'client/client_priv.h'
--- client/client_priv.h 2013-05-15 05:05:18 +0000
+++ client/client_priv.h 2014-02-15 00:33:28 +0000
@@ -94,6 +94,7 @@
94 OPT_DEFAULT_PLUGIN,94 OPT_DEFAULT_PLUGIN,
95 OPT_ENABLE_CLEARTEXT_PLUGIN,95 OPT_ENABLE_CLEARTEXT_PLUGIN,
96 OPT_INNODB_OPTIMIZE_KEYS,96 OPT_INNODB_OPTIMIZE_KEYS,
97 OPT_REWRITE_DB,
97 OPT_MAX_CLIENT_OPTION98 OPT_MAX_CLIENT_OPTION
98};99};
99100
100101
=== modified file 'client/mysqlbinlog.cc'
--- client/mysqlbinlog.cc 2013-10-31 18:24:55 +0000
+++ client/mysqlbinlog.cc 2014-02-15 00:33:28 +0000
@@ -39,6 +39,18 @@
39#include "my_dir.h"39#include "my_dir.h"
40#include <welcome_copyright_notice.h> // ORACLE_WELCOME_COPYRIGHT_NOTICE40#include <welcome_copyright_notice.h> // ORACLE_WELCOME_COPYRIGHT_NOTICE
4141
42/* Needed for Rpl_filter */
43CHARSET_INFO* system_charset_info= &my_charset_utf8_general_ci;
44
45#include "rpl_filter.h"
46
47/*
48 True if obsolette syntax warning has been already shown
49 during parsing --rewrite-db command line option
50*/
51bool rewrite_db_obs_syn_warn= false;
52Rpl_filter *binlog_filter= NULL;
53
42#define BIN_LOG_HEADER_SIZE 454#define BIN_LOG_HEADER_SIZE 4
43#define PROBE_HEADER_LEN (EVENT_LEN_OFFSET+4)55#define PROBE_HEADER_LEN (EVENT_LEN_OFFSET+4)
4456
@@ -631,6 +643,35 @@
631 strcmp(log_dbname, database);643 strcmp(log_dbname, database);
632}644}
633645
646/**
647 Rewrites db name in T instance if binlog_filter contains
648 name for replacement(see --rewrite-db option).
649
650 T::db must be weak pointer and can point to the memory owned by
651 "binlog_filter" after this function execution, that is why "ev" must be
652 destroyed before "binlog_filter".
653
654 @param ev Event to process
655*/
656template <typename T>
657static void rewrite_db(T &ev)
658{
659 size_t len_to= 0;
660 const char* db_to;
661
662 DBUG_ASSERT(binlog_filter);
663
664 if (!ev.db)
665 return;
666
667 db_to= binlog_filter->get_rewrite_db(ev.db, &len_to);
668
669 if (!len_to)
670 return;
671
672 ev.db= db_to;
673 ev.db_len= len_to;
674}
634675
635/**676/**
636 Prints the given event in base64 format.677 Prints the given event in base64 format.
@@ -752,8 +793,16 @@
752793
753 switch (ev_type) {794 switch (ev_type) {
754 case QUERY_EVENT:795 case QUERY_EVENT:
755 if (!((Query_log_event*)ev)->is_trans_keyword() &&796 {
756 shall_skip_database(((Query_log_event*)ev)->db))797 Query_log_event *qlev = static_cast<Query_log_event *>(ev);
798 /*
799 ev is deleted at the end of this function(before binlog_filter deletion)
800 so it is safe to set ev->db to some memory owned by binlog_filter here.
801 */
802 if (binlog_filter)
803 rewrite_db(*qlev);
804 if (!qlev->is_trans_keyword() &&
805 shall_skip_database(qlev->db))
757 goto end;806 goto end;
758 if (opt_base64_output_mode == BASE64_OUTPUT_ALWAYS)807 if (opt_base64_output_mode == BASE64_OUTPUT_ALWAYS)
759 {808 {
@@ -767,11 +816,17 @@
767 if (head->error == -1)816 if (head->error == -1)
768 goto err;817 goto err;
769 break;818 break;
770819 }
771 case CREATE_FILE_EVENT:820 case CREATE_FILE_EVENT:
772 {821 {
773 Create_file_log_event* ce= (Create_file_log_event*)ev;822 Create_file_log_event* ce= (Create_file_log_event*)ev;
774 /*823 /*
824 ev is deleted at the end of this function(before binlog_filter deletion)
825 so it is safe to set ev->db to some memory owned by binlog_filter here.
826 */
827 if (binlog_filter)
828 rewrite_db(*ce);
829 /*
775 We test if this event has to be ignored. If yes, we don't save830 We test if this event has to be ignored. If yes, we don't save
776 this event; this will have the good side-effect of ignoring all831 this event; this will have the good side-effect of ignoring all
777 related Append_block and Exec_load.832 related Append_block and Exec_load.
@@ -903,6 +958,12 @@
903 case EXECUTE_LOAD_QUERY_EVENT:958 case EXECUTE_LOAD_QUERY_EVENT:
904 {959 {
905 Execute_load_query_log_event *exlq= (Execute_load_query_log_event*)ev;960 Execute_load_query_log_event *exlq= (Execute_load_query_log_event*)ev;
961 /*
962 ev is deleted at the end of this function(before binlog_filter deletion)
963 so it is safe to set ev->db to some memory owned by binlog_filter here.
964 */
965 if (binlog_filter)
966 rewrite_db(*exlq);
906 char *fname= load_processor.grab_fname(exlq->file_id);967 char *fname= load_processor.grab_fname(exlq->file_id);
907968
908 if (!shall_skip_database(exlq->db))969 if (!shall_skip_database(exlq->db))
@@ -930,6 +991,19 @@
930 case TABLE_MAP_EVENT:991 case TABLE_MAP_EVENT:
931 {992 {
932 Table_map_log_event *map= ((Table_map_log_event *)ev);993 Table_map_log_event *map= ((Table_map_log_event *)ev);
994 // Rewrite db name here (see --rewrite-db option)
995 if (binlog_filter)
996 {
997 size_t len_to= 0;
998 const char* db_to= binlog_filter->get_rewrite_db(map->get_db_name(),
999 &len_to);
1000 if (len_to && map->rewrite_db(db_to, len_to, glob_description_event))
1001 {
1002 error("Could not rewrite database name");
1003 goto err;
1004 }
1005 }
1006
933 if (shall_skip_database(map->get_db_name()))1007 if (shall_skip_database(map->get_db_name()))
934 {1008 {
935 print_event_info->m_table_map_ignored.set_table(map->get_table_id(), map);1009 print_event_info->m_table_map_ignored.set_table(map->get_table_id(), map);
@@ -1225,6 +1299,10 @@
1225 "Used to reserve file descriptors for use by this program.",1299 "Used to reserve file descriptors for use by this program.",
1226 &open_files_limit, &open_files_limit, 0, GET_ULONG,1300 &open_files_limit, &open_files_limit, 0, GET_ULONG,
1227 REQUIRED_ARG, MY_NFILE, 8, OS_FILE_LIMIT, 0, 1, 0},1301 REQUIRED_ARG, MY_NFILE, 8, OS_FILE_LIMIT, 0, 1, 0},
1302 {"rewrite-db", OPT_REWRITE_DB,
1303 "Updates to a database with a different name than the original. "
1304 "Example: rewrite-db='from->to'.",
1305 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
1228 {0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}1306 {0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}
1229};1307};
12301308
@@ -1303,6 +1381,7 @@
1303 my_free(dirname_for_local_load);1381 my_free(dirname_for_local_load);
13041382
1305 delete glob_description_event;1383 delete glob_description_event;
1384 delete binlog_filter;
1306 if (mysql)1385 if (mysql)
1307 mysql_close(mysql);1386 mysql_close(mysql);
1308}1387}
@@ -1406,6 +1485,88 @@
1406 (find_type_or_exit(argument, &base64_output_mode_typelib, opt->name)-1);1485 (find_type_or_exit(argument, &base64_output_mode_typelib, opt->name)-1);
1407 }1486 }
1408 break;1487 break;
1488 case OPT_REWRITE_DB: // db_from->db_to
1489 {
1490 /* See also handling of OPT_REPLICATE_REWRITE_DB in sql/mysqld.cc */
1491 char* ptr= argument;
1492 do {
1493 char* key= ptr; // db-from
1494 char* val; // db-to
1495
1496 // Where key begins
1497 while (*key && my_isspace(&my_charset_latin1, *key))
1498 key++;
1499
1500 // Where val begins
1501 if (!(ptr= strstr(key, "->")))
1502 {
1503 sql_print_error("Bad syntax in rewrite-db: missing '->'!\n");
1504 return 1;
1505 }
1506 val= ptr + 2;
1507 while (*val && my_isspace(&my_charset_latin1, *val))
1508 val++;
1509
1510 // Write \0 and skip blanks at the end of key
1511 *ptr-- = 0;
1512 while (my_isspace(&my_charset_latin1, *ptr) && ptr > argument)
1513 *ptr-- = 0;
1514
1515 if (!*key)
1516 {
1517 sql_print_error("Bad syntax in rewrite-db: empty db-from!\n");
1518 return 1;
1519 }
1520
1521 // Skip blanks at the end of val
1522 ptr= val;
1523 while (*ptr && !my_isspace(&my_charset_latin1, *ptr) && *ptr != ',')
1524 ++ptr;
1525
1526 if (my_isspace(&my_charset_latin1, *ptr))
1527 {
1528 *(ptr++)= 0;
1529 while (*ptr && *ptr != ',')
1530 {
1531 if (!my_isspace(&my_charset_latin1, *ptr)) {
1532 sql_print_error("Bad syntax in rewrite-db: db-to must contain db "
1533 "name without spaces!\n");
1534 return 1;
1535 }
1536 ++ptr;
1537 }
1538 }
1539
1540 if (*ptr == ',')
1541 {
1542 if (!rewrite_db_obs_syn_warn)
1543 {
1544 warning("The comma-separated list of rewritings syntax is obsolete and "
1545 "discarded in 5.7\n");
1546 rewrite_db_obs_syn_warn= true;
1547 }
1548 *(ptr++)= 0;
1549 }
1550 else
1551 *ptr= 0;
1552
1553 if (!*val)
1554 {
1555 sql_print_error("Bad syntax in rewrite-db: empty db-to!\n");
1556 return 1;
1557 }
1558
1559 if (!binlog_filter &&
1560 !(binlog_filter= new Rpl_filter))
1561 {
1562 sql_print_error("Failed to create Rpl_filter\n");
1563 return 1;
1564 }
1565
1566 binlog_filter->add_db_rewrite(key, val);
1567 } while (*ptr);
1568 break;
1569 }
1409 case 'v':1570 case 'v':
1410 if (argument == disabled_my_option)1571 if (argument == disabled_my_option)
1411 verbose= 0;1572 verbose= 0;
14121573
=== added file 'mysql-test/extra/rpl_tests/grep_pattern.inc'
--- mysql-test/extra/rpl_tests/grep_pattern.inc 1970-01-01 00:00:00 +0000
+++ mysql-test/extra/rpl_tests/grep_pattern.inc 2014-02-15 00:33:28 +0000
@@ -0,0 +1,22 @@
1# Please set GREP_FILE and GREP_PATTERN environment variables
2# to work this file properly.
3--perl
4 use strict;
5 my $file= $ENV{'GREP_FILE'} or die "grep file not set";
6 my $pattern= $ENV{'GREP_PATTERN'} or die "pattern is not set";
7 open(FILE, "$file") or die("Unable to open $file: $!\n");
8 my $count = 0;
9 print "Matching lines are:\n";
10 while (<FILE>) {
11 my $line = $_;
12 if ($line =~ /$pattern/) {
13 print "$line\n";
14 $count++;
15 }
16 }
17 if ($count == 0) {
18 print "None\n";
19 }
20 print "Occurrences of the $pattern in the input file : $count\n";
21 close(FILE);
22EOF
023
=== added file 'mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result'
--- mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result 2014-02-15 00:33:28 +0000
@@ -0,0 +1,33 @@
1RESET MASTER;
2CREATE DATABASE db1;
3USE db1;
4CREATE TABLE t1 (i INT);
5INSERT INTO t1 VALUES(1);
6INSERT INTO t1 VALUES(2);
7UPDATE t1 SET i= i+1;
8DELETE FROM t1 WHERE i=2;
9[Syntax error in the use of the new option: The from database name is missing]
10[Syntax error in the use of the new option: The '->' is missing]
11[Syntax error in the use of the new option: The to database name is missing]
12[VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 1]
13[Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4,db5->db6"]
14[log_grep.inc] file: mysqltest.log pattern: The comma-separated list of rewritings syntax is obsolete and discarded
15[log_grep.inc] lines: 1
16[VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 2]
17[Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4" --rewrite-db="db5->db6,db6->db7"]
18[log_grep.inc] file: mysqltest.log pattern: The comma-separated list of rewritings syntax is obsolete and discarded
19[log_grep.inc] lines: 2
20[VALID SYNTAX,The from->to database names are correctly mentioned]
21#Dropping the database db1 and creating the table in the new database db2.
22CREATE DATABASE db2;
23DROP DATABASE db1;
24RESET MASTER;
25SELECT * FROM db2.t1;
26ERROR 42S02: Table 'db2.t1' doesn't exist
27DROP DATABASE db1;
28RESET MASTER;
29[The event of table db1.t1 has been successfully applied to db2.t1]
30include/assert.inc [Assert that table db2.t1 has one row after applying the sql file.]
31CLEANUP
32DROP DATABASE db1;
33DROP DATABASE db2;
034
=== added file 'mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result'
--- mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result 2014-02-15 00:33:28 +0000
@@ -0,0 +1,29 @@
1RESET MASTER;
2CREATE DATABASE db1;
3USE db1;
4CREATE TABLE t1 (i INT);
5INSERT INTO db1.t1 VALUES(1);
6INSERT INTO db1.t1 VALUES(2);
7UPDATE t1 SET i= i+1;
8DELETE FROM t1 WHERE i=2;
9CREATE DATABASE db2;
10CREATE TABLE db2.t1 (i INT);
11INSERT INTO db2.t1 VALUES(3);
12INSERT INTO db2.t1 VALUES(4);
13INSERT INTO db2.t1 VALUES(5);
14UPDATE db2.t1 SET i= i+1;
15DELETE FROM db2.t1 WHERE i=4;
16call mtr.add_suppression("Slave SQL: Error executing row event:*");
17Dropping the database db1 creating the new database db3.
18DROP DATABASE db1;
19CREATE DATABASE db3;
20DROP DATABASE db2;
21RESET MASTER;
22[The sql file will be applied on the current database]
23[The content of table db3.t1 and db2.t1 will be different confirming no leak]
24include/assert.inc [The content of the table t1 in database db3 and db2 is different]
25include/assert.inc [Table t1 in db3 have different row count than t1 in db2]
26CLEANUP
27DROP DATABASE db1;
28DROP DATABASE db2;
29DROP DATABASE db3;
030
=== added file 'mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result'
--- mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result 2014-02-15 00:33:28 +0000
@@ -0,0 +1,29 @@
1RESET MASTER;
2CREATE DATABASE db1;
3USE db1;
4CREATE TABLE t1 (i INT);
5INSERT INTO t1 VALUES(1);
6INSERT INTO t1 VALUES(2);
7UPDATE t1 SET i= i+1;
8DELETE FROM t1 WHERE i=2;
9[The use <db_name> is not suppressed in the general use of mysqlbinlog]
10Matching lines are:
11use `db1`/*!*/;
12
13Occurrences of the use `db1` in the input file : 1
14[The use <db_name> is suppressed on using rewrite-db option of mysqlbinlog]
15Matching lines are:
16None
17Occurrences of the use `db1` in the input file : 0
18CREATE DATABASE db2;
19DROP DATABASE db1;
20RESET MASTER;
21SELECT * FROM db2.t1;
22ERROR 42S02: Table 'db2.t1' doesn't exist
23DROP DATABASE db1;
24RESET MASTER;
25[The event of table db1.t1 has been successfully applied to db2.t1]
26include/assert.inc [Assert that table db2.t1 has no rows after applying the sql file.]
27[CLEANUP]
28DROP DATABASE db1;
29DROP DATABASE db2;
030
=== added file 'mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result'
--- mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result 2014-02-15 00:33:28 +0000
@@ -0,0 +1,152 @@
1#
2# Apply log with ''
3#
4# use a
5f
60
70
8# use bbbbbbbbbbbbbbbbbbbb
9f
101
111
12# use cccccccccccccccc
13f
142
152
16# use d
17f
18#
19# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="d"'
20#
21# use a
22f
23# use bbbbbbbbbbbbbbbbbbbb
24f
25# use cccccccccccccccc
26f
27# use d
28f
292
302
31#
32# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="cccccccccccccccc"'
33#
34# use a
35f
36# use bbbbbbbbbbbbbbbbbbbb
37f
38# use cccccccccccccccc
39f
40# use d
41f
42#
43# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="bbbbbbbbbbbbbbbbbbbb"'
44#
45# use a
46f
47# use bbbbbbbbbbbbbbbbbbbb
48f
490
500
511
521
53# use cccccccccccccccc
54f
55# use d
56f
57#
58# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="a"'
59#
60# use a
61f
62# use bbbbbbbbbbbbbbbbbbbb
63f
64# use cccccccccccccccc
65f
66# use d
67f
68#
69# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d"'
70#
71# use a
72f
73# use bbbbbbbbbbbbbbbbbbbb
74f
750
760
771
781
79# use cccccccccccccccc
80f
81# use d
82f
832
842
85#
86# Apply log with '--rewrite-db=" a -> bbbbbbbbbbbbbbbbbbbb , cccccccccccccccc -> d "'
87#
88# use a
89f
90# use bbbbbbbbbbbbbbbbbbbb
91f
920
930
941
951
96# use cccccccccccccccc
97f
98# use d
99f
1002
1012
102#
103# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb,cccccccccccccccc->d"'
104#
105# use a
106f
107# use bbbbbbbbbbbbbbbbbbbb
108f
1090
1100
1111
1121
113# use cccccccccccccccc
114f
115# use d
116f
1172
1182
119#
120# Apply log with '--rewrite-db="a->d"'
121#
122# use a
123f
124# use bbbbbbbbbbbbbbbbbbbb
125f
1261
1271
128# use cccccccccccccccc
129f
1302
1312
132# use d
133f
1340
1350
136#
137# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb"'
138#
139# use a
140f
141# use bbbbbbbbbbbbbbbbbbbb
142f
1430
1440
1451
1461
147# use cccccccccccccccc
148f
1492
1502
151# use d
152f
0153
=== added file 'mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test'
--- mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test 2014-02-15 00:33:28 +0000
@@ -0,0 +1,95 @@
1# WL#6404 - Add rewrite-db option to mysqlbinlog on RBR
2#
3# The test checks the new rewrite-db option for the mysqlbinlog.
4#
5# The test checks the following aspects of the new option:
6
7# 1. The valid syntax for the use of the new option and the errors in
8# case the usage is not correct.
9# 2. Apply the new binlog file ( with database rewritten ) to a new database
10# and check if it works.
11#
12--source include/have_binlog_format_row.inc
13
14RESET MASTER;
15CREATE DATABASE db1;
16USE db1;
17CREATE TABLE t1 (i INT);
18
19# Get some INSERT, UPDATE and DELETE ROW events.
20INSERT INTO t1 VALUES(1);
21INSERT INTO t1 VALUES(2);
22UPDATE t1 SET i= i+1;
23DELETE FROM t1 WHERE i=2;
24
25--let $MYSQLD_DATADIR= `select @@datadir`
26--exec $MYSQL_BINLOG --force-if-open $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event.sql
27
28# Using the new option to apply the row event on some other database (from db1 -> db2 in the current case)
29
30--echo [Syntax error in the use of the new option: The from database name is missing]
31--error 1
32--exec $MYSQL_BINLOG --force-if-open --rewrite-db="->db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
33
34--echo [Syntax error in the use of the new option: The '->' is missing]
35--error 1
36--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1 db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
37
38--echo [Syntax error in the use of the new option: The to database name is missing]
39--error 1
40--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
41
42--echo [VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 1]
43--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2,db3->db4,db5->db6" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
44
45--echo [Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4,db5->db6"]
46--let log_file=mysqltest.log
47--let log_file_full_path=$MYSQLTEST_VARDIR/log/current_test
48--let grep_pattern=The comma-separated list of rewritings syntax is obsolete and discarded
49--source include/log_grep.inc
50
51--echo [VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 2]
52--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2,db3->db4" --rewrite-db="db5->db6,db6->db7" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
53
54--echo [Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4" --rewrite-db="db5->db6,db6->db7"]
55--let log_file=mysqltest.log
56--let log_file_full_path=$MYSQLTEST_VARDIR/log/current_test
57--let grep_pattern=The comma-separated list of rewritings syntax is obsolete and discarded
58--source include/log_grep.inc
59
60--echo [VALID SYNTAX,The from->to database names are correctly mentioned]
61--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
62
63--echo #Dropping the database db1 and creating the table in the new database db2.
64
65CREATE DATABASE db2;
66DROP DATABASE db1;
67
68# The SQL file will be applied but nothing is applied on Database db2 since the row event was
69# generated for database db1 and table t1.
70
71# With gtid-mode=on we need purge gtid_executed, if not transactions
72# replayed through mysqlbinlog will be skipped.
73RESET MASTER;
74--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event.sql
75--error ER_NO_SUCH_TABLE
76SELECT * FROM db2.t1;
77
78# The SQL file should be applied since the row event was extracted using the new mysqlbinlog option.
79
80DROP DATABASE db1;
81# With gtid-mode=on we need purge gtid_executed, if not transactions
82# replayed through mysqlbinlog will be skipped.
83RESET MASTER;
84--echo [The event of table db1.t1 has been successfully applied to db2.t1]
85--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
86--let $assert_text= Assert that table db2.t1 has one row after applying the sql file.
87--let $assert_cond= `SELECT COUNT(*)=1 from db2.t1`
88--source include/assert.inc
89
90--echo CLEANUP
91
92--remove_file $MYSQLTEST_VARDIR/tmp/row_event.sql
93--remove_file $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
94DROP DATABASE db1;
95DROP DATABASE db2;
096
=== added file 'mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test'
--- mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test 2014-02-15 00:33:28 +0000
@@ -0,0 +1,79 @@
1# WL#6404 - Add rewrite-db option to mysqlbinlog on RBR
2#
3# The test checks that there is no leak of the global variables used
4# to decide whether rewrite-db option is ON or OFF.
5#
6# The test checks this by creating two tables in two different databases.
7# insert rows in both of them.
8#
9# Read the binlog file using the new option --rewrite-db="db1->db3" and
10# writing in the .sql file.
11#
12# On applying the .sql file on database db3, we get an error since not
13# all events are converted for database db3. Only DB1 is converted to db3.
14# The db2 event cause the error.
15#
16
17--source include/have_binlog_format_row.inc
18RESET MASTER;
19CREATE DATABASE db1;
20USE db1;
21CREATE TABLE t1 (i INT);
22
23# Get some INSERT, UPDATE and DELETE ROW events.
24INSERT INTO db1.t1 VALUES(1);
25INSERT INTO db1.t1 VALUES(2);
26UPDATE t1 SET i= i+1;
27DELETE FROM t1 WHERE i=2;
28
29CREATE DATABASE db2;
30CREATE TABLE db2.t1 (i INT);
31
32# Get some INSERT, UPDATE and DELETE ROW events.
33INSERT INTO db2.t1 VALUES(3);
34INSERT INTO db2.t1 VALUES(4);
35INSERT INTO db2.t1 VALUES(5);
36UPDATE db2.t1 SET i= i+1;
37DELETE FROM db2.t1 WHERE i=4;
38
39--let $MYSQLD_DATADIR= `select @@datadir`
40
41# Using the new option to apply the row event on some other database (from db1 -> db3 in the current case)
42
43--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db3" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/rewrite_noleak.sql
44call mtr.add_suppression("Slave SQL: Error executing row event:*");
45
46--echo Dropping the database db1 creating the new database db3.
47DROP DATABASE db1;
48CREATE DATABASE db3;
49
50# The SQL file will be applied but only db1 events will be replayed on db3.
51# Database db2 will not be applied on db3 confirming there is no leak.
52DROP DATABASE db2;
53
54# With gtid-mode=on we need purge gtid_executed, if not transactions
55# replayed through mysqlbinlog will be skipped.
56RESET MASTER;
57--echo [The sql file will be applied on the current database]
58--exec $MYSQL --database=db3 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/rewrite_noleak.sql
59
60--echo [The content of table db3.t1 and db2.t1 will be different confirming no leak]
61
62--let $db3_max= `SELECT MAX(i) FROM db3.t1`
63--let $db2_max= `SELECT MAX(i) FROM db2.t1`
64--let $assert_text= The content of the table t1 in database db3 and db2 is different
65--let $assert_cond= $db3_max <> $db2_max
66--source include/assert.inc
67
68--let $db3_count= `SELECT COUNT(*) FROM db3.t1`
69--let $db2_count= `SELECT COUNT(*) FROM db2.t1`
70--let $assert_text= Table t1 in db3 have different row count than t1 in db2
71--let $assert_cond= $db3_count=1 AND $db2_count<>1
72--source include/assert.inc
73
74--echo CLEANUP
75
76#--remove_file $MYSQLTEST_VARDIR/tmp/rewrite_noleak.sql
77DROP DATABASE db1;
78DROP DATABASE db2;
79DROP DATABASE db3;
080
=== added file 'mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test'
--- mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test 2014-02-15 00:33:28 +0000
@@ -0,0 +1,61 @@
1# WL#6404 - Add rewrite-db option to mysqlbinlog on RBR
2#
3# The test aims to check that the use of rewrite-db option of
4# mysqlbinlog suppresses the USE DATABASE command logged in
5# binlog file.
6#
7
8--source include/have_binlog_format_row.inc
9RESET MASTER;
10CREATE DATABASE db1;
11USE db1;
12CREATE TABLE t1 (i INT);
13
14# Get some INSERT, UPDATE and DELETE ROW events.
15INSERT INTO t1 VALUES(1);
16INSERT INTO t1 VALUES(2);
17UPDATE t1 SET i= i+1;
18DELETE FROM t1 WHERE i=2;
19
20--let $MYSQLD_DATADIR= `select @@datadir`
21
22# Checking for the suppression of the USE DATABASE command on using the new option.
23# Reading binlog file without the rewrite-db option.
24--echo [The use <db_name> is not suppressed in the general use of mysqlbinlog]
25--exec $MYSQL_BINLOG --force-if-open $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event.sql
26--let GREP_FILE=$MYSQLTEST_VARDIR/tmp/row_event.sql
27--let GREP_PATTERN=use `db1`
28--source extra/rpl_tests/grep_pattern.inc
29
30# Reading binlog file with the rewrite-db option.
31--echo [The use <db_name> is suppressed on using rewrite-db option of mysqlbinlog]
32--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
33--let GREP_FILE=$MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
34--let GREP_PATTERN=use `db1`
35--source extra/rpl_tests/grep_pattern.inc
36
37CREATE DATABASE db2;
38DROP DATABASE db1;
39# With gtid-mode=on we need purge gtid_executed, if not transactions
40# replayed through mysqlbinlog will be skipped.
41RESET MASTER;
42--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event.sql
43--error ER_NO_SUCH_TABLE
44SELECT * FROM db2.t1;
45
46DROP DATABASE db1;
47# With gtid-mode=on we need purge gtid_executed, if not transactions
48# replayed through mysqlbinlog will be skipped.
49RESET MASTER;
50--echo [The event of table db1.t1 has been successfully applied to db2.t1]
51--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
52--let $assert_text= Assert that table db2.t1 has no rows after applying the sql file.
53--let $assert_cond= `SELECT COUNT(*)=1 from db2.t1`
54--source include/assert.inc
55
56--echo [CLEANUP]
57--remove_file $MYSQLTEST_VARDIR/tmp/row_event.sql
58--remove_file $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
59
60DROP DATABASE db1;
61DROP DATABASE db2;
062
=== added file 'mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt'
--- mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt 2014-02-15 00:33:28 +0000
@@ -0,0 +1,1 @@
1--local-infile=1
02
=== added file 'mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test'
--- mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test 1970-01-01 00:00:00 +0000
+++ mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test 2014-02-15 00:33:28 +0000
@@ -0,0 +1,80 @@
1--source include/have_log_bin.inc
2#
3# WL #36: http://askmonty.org/worklog/Server-Sprint/?tid=36
4# This is basic test for --rewrite-db option.
5# mysqlbinlog --rewrite-db="from->to" should rewrite database name
6#
7
8--disable_query_log
9
10# Because RBR event DB rewrite requires shifting of
11# Table_map_log_event contents around if the DB name length
12# changes, this case is tested for both shorter and longer
13# names.
14
15--let DB1=a
16--let DB2=bbbbbbbbbbbbbbbbbbbb
17--let DB3=cccccccccccccccc
18--let DB4=d
19
20eval CREATE DATABASE $DB1; eval USE $DB1; CREATE TABLE t1(f INT);
21eval CREATE DATABASE $DB2; eval USE $DB2; CREATE TABLE t1(f INT);
22eval CREATE DATABASE $DB3; eval USE $DB3; CREATE TABLE t1(f INT);
23eval CREATE DATABASE $DB4; eval USE $DB4; CREATE TABLE t1(f INT);
24
25RESET MASTER;
26--let MYSQLD_DATA_DIR=`select @@datadir`
27--let LOAD_DATA_FILE=$MYSQLD_DATA_DIR/table.values
28--let BIN_LOG_FILE_NAME=query_get_value(show master status, File, 1)
29--let BIN_LOG_START_POSITION=query_get_value(show master status, Position, 1)
30eval USE $DB1; INSERT INTO t1 VALUES(0); INSERT INTO t1 VALUES(0);
31eval USE $DB2; INSERT INTO t1 VALUES(1); INSERT INTO t1 VALUES(1);
32write_file $LOAD_DATA_FILE;
332
342
35EOF
36eval USE $DB3;
37--eval LOAD DATA INFILE "$LOAD_DATA_FILE" INTO TABLE t1
38--let BIN_LOG_STOP_POSITION=query_get_value(show master status, Position, 1)
39FLUSH LOGS;
40
41--let BIN_LOG_FULL_PATH=`SELECT CONCAT("$MYSQLD_DATA_DIR", "$BIN_LOG_FILE_NAME")`
42
43--let i=10
44
45while ($i)
46{
47# Test for different combinations of syntax and db name filtering and rewriting
48--let REWRITE=`SELECT CASE $i WHEN 10 THEN '' WHEN 9 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB4"' WHEN 8 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB3"' WHEN 7 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB2"' WHEN 6 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB1"' WHEN 5 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4"' WHEN 4 THEN '--rewrite-db=" $DB1 -> $DB2 , $DB3 -> $DB4 "' WHEN 3 THEN '--rewrite-db="$DB1->$DB2,$DB3->$DB4"' WHEN 2 THEN '--rewrite-db="$DB1->$DB4"' WHEN 1 THEN '--rewrite-db="$DB1->$DB2"' END`
49
50eval USE $DB1; DELETE FROM t1;
51eval USE $DB2; DELETE FROM t1;
52eval USE $DB3; DELETE FROM t1;
53eval USE $DB4; DELETE FROM t1;
54
55--echo #
56--echo # Apply log with '$REWRITE'
57--echo #
58
59--exec $MYSQL_BINLOG $REWRITE --start-position=$BIN_LOG_START_POSITION --stop-position=$BIN_LOG_STOP_POSITION $BIN_LOG_FULL_PATH | $MYSQL --local-infile=1
60
61--echo # use $DB1
62eval USE $DB1; SELECT * FROM t1 ORDER BY f;
63--echo # use $DB2
64eval USE $DB2; SELECT * FROM t1 ORDER BY f;
65--echo # use $DB3
66eval USE $DB3; SELECT * FROM t1 ORDER BY f;
67--echo # use $DB4
68eval USE $DB4; SELECT * FROM t1 ORDER BY f;
69
70dec $i;
71}
72
73eval USE $DB1; DROP TABLE t1; eval DROP DATABASE $DB1;
74eval USE $DB2; DROP TABLE t1; eval DROP DATABASE $DB2;
75eval USE $DB3; DROP TABLE t1; eval DROP DATABASE $DB3;
76eval USE $DB4; DROP TABLE t1; eval DROP DATABASE $DB4;
77
78--remove_file $LOAD_DATA_FILE
79
80--enable_query_log
081
=== modified file 'sql/log_event.cc'
--- sql/log_event.cc 2014-02-03 04:39:37 +0000
+++ sql/log_event.cc 2014-02-15 00:33:28 +0000
@@ -8714,6 +8714,108 @@
8714 my_free(m_memory);8714 my_free(m_memory);
8715}8715}
87168716
8717#ifdef MYSQL_CLIENT
8718
8719/*
8720 Rewrite database name for the event to name specified by new_db
8721 SYNOPSIS
8722 new_db Database name to change to
8723 new_len Length
8724 desc Event describing binlog that we're writing to.
8725
8726 DESCRIPTION
8727 Reset db name. This function assumes that temp_buf member contains event
8728 representation taken from a binary log. It resets m_dbnam and m_dblen and
8729 rewrites temp_buf with new db name.
8730
8731 RETURN
8732 0 - Success
8733 other - Error
8734*/
8735
8736int Table_map_log_event::rewrite_db(const char* new_db, size_t new_len,
8737 const Format_description_log_event* desc)
8738{
8739 DBUG_ENTER("Table_map_log_event::rewrite_db");
8740 DBUG_ASSERT(temp_buf);
8741
8742 uint header_len= min(desc->common_header_len,
8743 LOG_EVENT_MINIMAL_HEADER_LEN) + TABLE_MAP_HEADER_LEN;
8744 int len_diff;
8745
8746 if (!(len_diff= new_len - m_dblen))
8747 {
8748 memcpy((void*) (temp_buf + header_len + 1), new_db, m_dblen + 1);
8749 memcpy((void*) m_dbnam, new_db, m_dblen + 1);
8750 DBUG_RETURN(0);
8751 }
8752
8753 // Create new temp_buf
8754 ulong event_cur_len= uint4korr(temp_buf + EVENT_LEN_OFFSET);
8755 ulong event_new_len= event_cur_len + len_diff;
8756 char* new_temp_buf= (char*) my_malloc(event_new_len, MYF(MY_WME));
8757
8758 if (!new_temp_buf)
8759 {
8760 sql_print_error("Table_map_log_event::rewrite_db: "
8761 "failed to allocate new temp_buf (%d bytes required)",
8762 event_new_len);
8763 DBUG_RETURN(-1);
8764 }
8765
8766 // Rewrite temp_buf
8767 char* ptr= new_temp_buf;
8768 ulong cnt= 0;
8769
8770 // Copy header and change event length
8771 memcpy(ptr, temp_buf, header_len);
8772 int4store(ptr + EVENT_LEN_OFFSET, event_new_len);
8773 ptr += header_len;
8774 cnt += header_len;
8775
8776 // Write new db name length and new name
8777 *ptr++ = new_len;
8778 memcpy(ptr, new_db, new_len + 1);
8779 ptr += new_len + 1;
8780 cnt += m_dblen + 2;
8781
8782 // Copy rest part
8783 memcpy(ptr, temp_buf + cnt, event_cur_len - cnt);
8784
8785 // Reregister temp buf
8786 free_temp_buf();
8787 register_temp_buf(new_temp_buf);
8788
8789 // Reset m_dbnam and m_dblen members
8790 m_dblen= new_len;
8791
8792 // m_dbnam resides in m_memory together with m_tblnam and m_coltype
8793 uchar* memory= m_memory;
8794 char const* tblnam= m_tblnam;
8795 uchar* coltype= m_coltype;
8796
8797 m_memory= (uchar*) my_multi_malloc(MYF(MY_WME),
8798 &m_dbnam, (uint) m_dblen + 1,
8799 &m_tblnam, (uint) m_tbllen + 1,
8800 &m_coltype, (uint) m_colcnt,
8801 NullS);
8802
8803 if (!m_memory)
8804 {
8805 sql_print_error("Table_map_log_event::rewrite_db: "
8806 "failed to allocate new m_memory (%d + %d + %d bytes required)",
8807 m_dblen + 1, m_tbllen + 1, m_colcnt);
8808 DBUG_RETURN(-1);
8809 }
8810
8811 memcpy((void*)m_dbnam, new_db, m_dblen + 1);
8812 memcpy((void*)m_tblnam, tblnam, m_tbllen + 1);
8813 memcpy(m_coltype, coltype, m_colcnt);
8814
8815 my_free(memory);
8816 DBUG_RETURN(0);
8817}
8818#endif /* MYSQL_CLIENT */
8717/*8819/*
8718 Return value is an error code, one of:8820 Return value is an error code, one of:
87198821
87208822
=== modified file 'sql/log_event.h'
--- sql/log_event.h 2014-02-03 04:39:37 +0000
+++ sql/log_event.h 2014-02-15 00:33:28 +0000
@@ -3504,7 +3504,10 @@
3504 ulong get_table_id() const { return m_table_id; }3504 ulong get_table_id() const { return m_table_id; }
3505 const char *get_table_name() const { return m_tblnam; }3505 const char *get_table_name() const { return m_tblnam; }
3506 const char *get_db_name() const { return m_dbnam; }3506 const char *get_db_name() const { return m_dbnam; }
35073507#ifdef MYSQL_CLIENT
3508 int rewrite_db(const char* new_name, size_t new_name_len,
3509 const Format_description_log_event*);
3510#endif
3508 virtual Log_event_type get_type_code() { return TABLE_MAP_EVENT; }3511 virtual Log_event_type get_type_code() { return TABLE_MAP_EVENT; }
3509 virtual bool is_valid() const { return m_memory != NULL; /* we check malloc */ }3512 virtual bool is_valid() const { return m_memory != NULL; /* we check malloc */ }
35103513

Subscribers

People subscribed via source and target branches