Merge lp:~vlad-lesin/percona-server/5.5-mysqlbinlog-replacedb into lp:percona-server/5.5

Proposed by Vlad Lesin
Status: Merged
Approved by: Laurynas Biveinis
Approved revision: no longer in the source branch.
Merged at revision: 629
Proposed branch: lp:~vlad-lesin/percona-server/5.5-mysqlbinlog-replacedb
Merge into: lp:percona-server/5.5
Diff against target: 1027 lines (+858/-4) (has conflicts)
15 files modified
client/CMakeLists.txt (+6/-0)
client/client_priv.h (+1/-0)
client/mysqlbinlog.cc (+164/-3)
mysql-test/extra/rpl_tests/grep_pattern.inc (+22/-0)
mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result (+33/-0)
mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result (+29/-0)
mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result (+29/-0)
mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result (+152/-0)
mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test (+95/-0)
mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test (+79/-0)
mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test (+61/-0)
mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt (+1/-0)
mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test (+80/-0)
sql/log_event.cc (+102/-0)
sql/log_event.h (+4/-1)
Text conflict in client/CMakeLists.txt
To merge this branch: bzr merge lp:~vlad-lesin/percona-server/5.5-mysqlbinlog-replacedb
Reviewer Review Type Date Requested Status
Laurynas Biveinis (community) Approve
Stewart Smith (community) Needs Fixing
George Ormond Lorch III (community) g2 Approve
Review via email: mp+196578@code.launchpad.net

Description of the change

Port --rewrite-db mysqlbinlog option from 5.1.

--rewrite-db is a string option which allows to replace one db-name to another
in mysqlbinlog output. The format of the string value is the following:
"from1->to1,from2->to2,from3->to3". Where "to[123]" are the names to which
the correspondent "from[123]" names must be replaced.

The original implementation does not replace db names in statements for
statement-based binlog, it just replaces db name of binlog event and issues
"USE" statement with replaced db name if necessary. I think this restriction
concerned with unwillingness to implement complex logic for parsing db names
from statements for such simple feature.

But for row-based binlog it replaces db names in statements correctly because
each table name is mapped to some number in binlog and it is quite easy to
extract such table ids from binlog event and replace one name by another.

http://jenkins.percona.com/job/percona-server-5.5-param/894/

To post a comment you must log in.
Revision history for this message
George Ormond Lorch III (gl-az) :
review: Approve (g2)
Revision history for this message
Stewart Smith (stewart) wrote :

same comment as 5.6:

It states that you can do "a->b,c->d" but I don't see the handling for the comma in the mysqlbinlog.cc command line parsing. Am I missing something? We should also test this.

review: Needs Fixing
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

    - This needs two associated blueprints, one for 5.5, and one for
      5.6

    - Oracle has implemented this functionality in 5.7.1. From the
      user perspective the difference is option syntax, ours:
      --rewrite-db='a->b,c->d'
      theirs:
      --rewrite-db='a->b' --rewrite-db='c->d'
      To ensure smooth migration once we switch to their
      implementation in 5.7, please test both syntaxes in the
      testcase.

    - What should be the interaction between --rewrite-db and
      --database options, i.e. which one should come first? My first
      thought is rewrite-db before database, i.e. first we map, and
      then we filter. This is what the current patch appears to do
      for SBR, but it's reversed for RBR (judging from
      shall_skip_database() and rewrite_db() call order). This would
      also need a testcase.

    - Rewrite rules are not applied to CREATE_FILE_EVENT. This
      wouldn't be an issue in itself, but these events are filtered
      against --database, thus, depending on how the previous item is
      resolved, these events might need rewrite-db application too,
      so that they can be filtered properly. This would also need a
      testcase.

    - LOAD DATA events (EXECUTE_LOAD_QUERY_EVENT) need test coverage
      too. Probably this can be handled together with the previous
      item.

    - Because RBR event DB rewrite requires shifting of
      Table_map_og_event contents around if the DB name length
      changes, please test this case for both shorter and longer
      names.

    - Since Oracle has implemented this, there are some Oracle
      testcases too. Consider importing binlog_mysqlbinlog_rewrite_db
      binlog_rewrite_db_noleak binlog_rewrite_suppress_use from MySQL
      5.7. If you choose to, make sure they get the correct bzr file
      ids (i.e. bzr add --file-ids-from=path to 5.7 tree)

review: Needs Fixing
Revision history for this message
Stewart Smith (stewart) wrote :

Laurynas Biveinis <email address hidden> writes:
> - Oracle has implemented this functionality in 5.7.1. From the
> user perspective the difference is option syntax, ours:
> --rewrite-db='a->b,c->d'
> theirs:
> --rewrite-db='a->b' --rewrite-db='c->d'
> To ensure smooth migration once we switch to their
> implementation in 5.7, please test both syntaxes in the
> testcase.

We should probably throw a warning if the user uses the syntax that will
break in 5.7, as really we should just take the Oracle impl in 5.7.

--
Stewart Smith

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> Laurynas Biveinis <email address hidden> writes:
> > - Oracle has implemented this functionality in 5.7.1. From the
> > user perspective the difference is option syntax, ours:
> > --rewrite-db='a->b,c->d'
> > theirs:
> > --rewrite-db='a->b' --rewrite-db='c->d'
> > To ensure smooth migration once we switch to their
> > implementation in 5.7, please test both syntaxes in the
> > testcase.
>
> We should probably throw a warning if the user uses the syntax that will
> break in 5.7, as really we should just take the Oracle impl in 5.7.
>
>
> --
> Stewart Smith

We can also (if the customer does not object) take the 5.7 syntax in 5.5 already for a bit simpler code/testcases.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

On 12/20/2013 12:58 PM, Laurynas Biveinis wrote:
> - This needs two associated blueprints, one for 5.5, and one for
> 5.6
Done.

> - Oracle has implemented this functionality in 5.7.1. From the
> user perspective the difference is option syntax, ours:
> --rewrite-db='a->b,c->d'
> theirs:
> --rewrite-db='a->b' --rewrite-db='c->d'
> To ensure smooth migration once we switch to their
> implementation in 5.7, please test both syntaxes in the
> testcase.
Yes, currently both formats are allowed, but if the old
format(comma-separated rules) is used the warning is issued.

> - What should be the interaction between --rewrite-db and
> --database options, i.e. which one should come first? My first
> thought is rewrite-db before database, i.e. first we map, and
> then we filter. This is what the current patch appears to do
> for SBR, but it's reversed for RBR (judging from
> shall_skip_database() and rewrite_db() call order). This would
> also need a testcase.
Yes, you are right, the order of rewrite-db and database filtering was
reversed for RBR. Fixed and the corresponding test is added.

> - Rewrite rules are not applied to CREATE_FILE_EVENT. This
> wouldn't be an issue in itself, but these events are filtered
> against --database, thus, depending on how the previous item is
> resolved, these events might need rewrite-db application too,
> so that they can be filtered properly. This would also need a
> testcase.
I added the code which replaces db name in the event processing, but I
have no idea how to test it as there is the following comment in
log_event.h:

"The events which really update data are Query_log_event,
Execute_load_query_log_event and old Load_log_event and
Execute_load_log_event events (Execute_load_query is used together with
Begin_load_query and Append_block events to replicate LOAD DATA INFILE.
Create_file/Append_block/Execute_load (which includes Load_log_event)
were used to replicate LOAD DATA before the 5.0.3)."

So CREATE_FILE_EVENT were used before 5.0.3.

> - LOAD DATA events (EXECUTE_LOAD_QUERY_EVENT) need test coverage
> too. Probably this can be handled together with the previous
> item.
The corresponding test case is made.

> - Because RBR event DB rewrite requires shifting of
> Table_map_og_event contents around if the DB name length
> changes, please test this case for both shorter and longer
> names.
Done.
> - Since Oracle has implemented this, there are some Oracle
> testcases too. Consider importing binlog_mysqlbinlog_rewrite_db
> binlog_rewrite_db_noleak binlog_rewrite_suppress_use from MySQL
> 5.7. If you choose to, make sure they get the correct bzr file
> ids (i.e. bzr add --file-ids-from=path to 5.7 tree)
Done.

Revision history for this message
Vlad Lesin (vlad-lesin) wrote :
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> > - Rewrite rules are not applied to CREATE_FILE_EVENT. This
> > wouldn't be an issue in itself, but these events are filtered
> > against --database, thus, depending on how the previous item is
> > resolved, these events might need rewrite-db application too,
> > so that they can be filtered properly. This would also need a
> > testcase.
> I added the code which replaces db name in the event processing, but I
> have no idea how to test it as there is the following comment in
> log_event.h:
>
> "The events which really update data are Query_log_event,
> Execute_load_query_log_event and old Load_log_event and
> Execute_load_log_event events (Execute_load_query is used together with
> Begin_load_query and Append_block events to replicate LOAD DATA INFILE.
> Create_file/Append_block/Execute_load (which includes Load_log_event)
> were used to replicate LOAD DATA before the 5.0.3)."
>
> So CREATE_FILE_EVENT were used before 5.0.3.

Ah sorry I missed this, this is a non-issue then.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

    - The warning for "a->b,c->d" syntax seems to be repeated for
      each comma, so "a->b,c->d,e->f" would print the warning twice,
      which is unnecessary.

    - Does running our testcases on MySQL 5.7 give the expected
      results?

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> - The warning for "a->b,c->d" syntax seems to be repeated for
> each comma, so "a->b,c->d,e->f" would print the warning twice,
> which is unnecessary.
Fixed.

> - Does running our testcases on MySQL 5.7 give the expected
> results?
No because 5.7 does not support our syntax.

http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/930/

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> > - The warning for "a->b,c->d" syntax seems to be repeated for
> > each comma, so "a->b,c->d,e->f" would print the warning twice,
> > which is unnecessary.
> Fixed.

What was the fix?

The relevant code reads

      if (*ptr == ',')
      {
 warning("The comma-separated list of rewritings syntax is obsolete and is discarded in 5.7\n");
        *(ptr++)= 0;
      }

In order to warn only once for e.g. --rewrite-db="a->b,c->d,e->f" --rewrite-db="g->h,i->j,k->l", it would need some sort of global flag already_warned_for_comma_syntax.

Or am I missing something here?

If you need to repush, then also please fix indentation at diff lines 216, 220, 227, 231, and curly brace not having own line at 217.

> > - Does running our testcases on MySQL 5.7 give the expected
> > results?
> No because 5.7 does not support our syntax.

Right, but accounted for syntax changes? Does our and their order of applying --database= --rewrite-db= agree?

review: Needs Information
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> > > - The warning for "a->b,c->d" syntax seems to be repeated for
> > > each comma, so "a->b,c->d,e->f" would print the warning twice,
> > > which is unnecessary.
> > Fixed.
>
> What was the fix?
...
> Or am I missing something here?

Yes, you are right, I was sure I fixed it. But somehow I did not. I believe it is fixed now.

> > > - Does running our testcases on MySQL 5.7 give the expected
> > > results?
> > No because 5.7 does not support our syntax.
>
> Right, but accounted for syntax changes? Does our and their order of applying
> --database= --rewrite-db= agree?

Yes. For 5.7 dump_local_log_entries() and dump_remote_log_entries() invoke Log_event::read_log_event() where db name is rewritten and then invoke process_event() where db name is checked by shall_skip_database() for filtering out. So db name is rewritten and then filtered like in PS.

As well I have rebased the code to the newest 5.5. Here id jenkins testing: http://jenkins.percona.com/view/PS%205.5/job/percona-server-5.5-param/942

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

> > > > - The warning for "a->b,c->d" syntax seems to be repeated for
> > > > each comma, so "a->b,c->d,e->f" would print the warning twice,
> > > > which is unnecessary.
> > > Fixed.
> >
> > What was the fix?
> ...
> > Or am I missing something here?
>
> Yes, you are right, I was sure I fixed it. But somehow I did not. I believe it
> is fixed now.

The flag is local to OPT_REWRITE_DB, so --rewrite-db="a->b,c->d,e->f" --rewrite-db="g->h,i->j,k->l", would still print the warning twice, wouldn't it?

> As well I have rebased the code to the newest 5.5.

Uhm, why? This is not a GCA branch anymore then?

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> The flag is local to OPT_REWRITE_DB, so --rewrite-db="a->b,c->d,e->f"
> --rewrite-db="g->h,i->j,k->l", would still print the warning twice, wouldn't
> it?
Yep. Fixed.

> > As well I have rebased the code to the newest 5.5.
> Uhm, why? This is not a GCA branch anymore then?
To make merging to trunk a little bit easier. The structure of trunk source tree was changed and there were a couple of conflicts due to some code change which was resolved manually.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

As discussed on IRC, this will need a GCA rebase.

review: Needs Fixing
Revision history for this message
Vlad Lesin (vlad-lesin) wrote :

> As discussed on IRC, this will need a GCA rebase.
Rebased.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'client/CMakeLists.txt'
2--- client/CMakeLists.txt 2013-07-23 04:41:43 +0000
3+++ client/CMakeLists.txt 2014-02-15 00:33:28 +0000
4@@ -57,8 +57,14 @@
5 MYSQL_ADD_EXECUTABLE(mysql_plugin mysql_plugin.c)
6 TARGET_LINK_LIBRARIES(mysql_plugin perconaserverclient)
7
8+<<<<<<< TREE
9 MYSQL_ADD_EXECUTABLE(mysqlbinlog mysqlbinlog.cc sql_string.cc)
10 TARGET_LINK_LIBRARIES(mysqlbinlog perconaserverclient)
11+=======
12+MYSQL_ADD_EXECUTABLE(mysqlbinlog mysqlbinlog.cc sql_string.cc
13+ ../sql/rpl_filter.cc ../sql/sql_list.cc)
14+TARGET_LINK_LIBRARIES(mysqlbinlog mysqlclient)
15+>>>>>>> MERGE-SOURCE
16
17 MYSQL_ADD_EXECUTABLE(mysqladmin mysqladmin.cc)
18 TARGET_LINK_LIBRARIES(mysqladmin perconaserverclient)
19
20=== modified file 'client/client_priv.h'
21--- client/client_priv.h 2013-05-15 05:05:18 +0000
22+++ client/client_priv.h 2014-02-15 00:33:28 +0000
23@@ -94,6 +94,7 @@
24 OPT_DEFAULT_PLUGIN,
25 OPT_ENABLE_CLEARTEXT_PLUGIN,
26 OPT_INNODB_OPTIMIZE_KEYS,
27+ OPT_REWRITE_DB,
28 OPT_MAX_CLIENT_OPTION
29 };
30
31
32=== modified file 'client/mysqlbinlog.cc'
33--- client/mysqlbinlog.cc 2013-10-31 18:24:55 +0000
34+++ client/mysqlbinlog.cc 2014-02-15 00:33:28 +0000
35@@ -39,6 +39,18 @@
36 #include "my_dir.h"
37 #include <welcome_copyright_notice.h> // ORACLE_WELCOME_COPYRIGHT_NOTICE
38
39+/* Needed for Rpl_filter */
40+CHARSET_INFO* system_charset_info= &my_charset_utf8_general_ci;
41+
42+#include "rpl_filter.h"
43+
44+/*
45+ True if obsolette syntax warning has been already shown
46+ during parsing --rewrite-db command line option
47+*/
48+bool rewrite_db_obs_syn_warn= false;
49+Rpl_filter *binlog_filter= NULL;
50+
51 #define BIN_LOG_HEADER_SIZE 4
52 #define PROBE_HEADER_LEN (EVENT_LEN_OFFSET+4)
53
54@@ -631,6 +643,35 @@
55 strcmp(log_dbname, database);
56 }
57
58+/**
59+ Rewrites db name in T instance if binlog_filter contains
60+ name for replacement(see --rewrite-db option).
61+
62+ T::db must be weak pointer and can point to the memory owned by
63+ "binlog_filter" after this function execution, that is why "ev" must be
64+ destroyed before "binlog_filter".
65+
66+ @param ev Event to process
67+*/
68+template <typename T>
69+static void rewrite_db(T &ev)
70+{
71+ size_t len_to= 0;
72+ const char* db_to;
73+
74+ DBUG_ASSERT(binlog_filter);
75+
76+ if (!ev.db)
77+ return;
78+
79+ db_to= binlog_filter->get_rewrite_db(ev.db, &len_to);
80+
81+ if (!len_to)
82+ return;
83+
84+ ev.db= db_to;
85+ ev.db_len= len_to;
86+}
87
88 /**
89 Prints the given event in base64 format.
90@@ -752,8 +793,16 @@
91
92 switch (ev_type) {
93 case QUERY_EVENT:
94- if (!((Query_log_event*)ev)->is_trans_keyword() &&
95- shall_skip_database(((Query_log_event*)ev)->db))
96+ {
97+ Query_log_event *qlev = static_cast<Query_log_event *>(ev);
98+ /*
99+ ev is deleted at the end of this function(before binlog_filter deletion)
100+ so it is safe to set ev->db to some memory owned by binlog_filter here.
101+ */
102+ if (binlog_filter)
103+ rewrite_db(*qlev);
104+ if (!qlev->is_trans_keyword() &&
105+ shall_skip_database(qlev->db))
106 goto end;
107 if (opt_base64_output_mode == BASE64_OUTPUT_ALWAYS)
108 {
109@@ -767,11 +816,17 @@
110 if (head->error == -1)
111 goto err;
112 break;
113-
114+ }
115 case CREATE_FILE_EVENT:
116 {
117 Create_file_log_event* ce= (Create_file_log_event*)ev;
118 /*
119+ ev is deleted at the end of this function(before binlog_filter deletion)
120+ so it is safe to set ev->db to some memory owned by binlog_filter here.
121+ */
122+ if (binlog_filter)
123+ rewrite_db(*ce);
124+ /*
125 We test if this event has to be ignored. If yes, we don't save
126 this event; this will have the good side-effect of ignoring all
127 related Append_block and Exec_load.
128@@ -903,6 +958,12 @@
129 case EXECUTE_LOAD_QUERY_EVENT:
130 {
131 Execute_load_query_log_event *exlq= (Execute_load_query_log_event*)ev;
132+ /*
133+ ev is deleted at the end of this function(before binlog_filter deletion)
134+ so it is safe to set ev->db to some memory owned by binlog_filter here.
135+ */
136+ if (binlog_filter)
137+ rewrite_db(*exlq);
138 char *fname= load_processor.grab_fname(exlq->file_id);
139
140 if (!shall_skip_database(exlq->db))
141@@ -930,6 +991,19 @@
142 case TABLE_MAP_EVENT:
143 {
144 Table_map_log_event *map= ((Table_map_log_event *)ev);
145+ // Rewrite db name here (see --rewrite-db option)
146+ if (binlog_filter)
147+ {
148+ size_t len_to= 0;
149+ const char* db_to= binlog_filter->get_rewrite_db(map->get_db_name(),
150+ &len_to);
151+ if (len_to && map->rewrite_db(db_to, len_to, glob_description_event))
152+ {
153+ error("Could not rewrite database name");
154+ goto err;
155+ }
156+ }
157+
158 if (shall_skip_database(map->get_db_name()))
159 {
160 print_event_info->m_table_map_ignored.set_table(map->get_table_id(), map);
161@@ -1225,6 +1299,10 @@
162 "Used to reserve file descriptors for use by this program.",
163 &open_files_limit, &open_files_limit, 0, GET_ULONG,
164 REQUIRED_ARG, MY_NFILE, 8, OS_FILE_LIMIT, 0, 1, 0},
165+ {"rewrite-db", OPT_REWRITE_DB,
166+ "Updates to a database with a different name than the original. "
167+ "Example: rewrite-db='from->to'.",
168+ 0, 0, 0, GET_STR, REQUIRED_ARG, 0, 0, 0, 0, 0, 0},
169 {0, 0, 0, 0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0}
170 };
171
172@@ -1303,6 +1381,7 @@
173 my_free(dirname_for_local_load);
174
175 delete glob_description_event;
176+ delete binlog_filter;
177 if (mysql)
178 mysql_close(mysql);
179 }
180@@ -1406,6 +1485,88 @@
181 (find_type_or_exit(argument, &base64_output_mode_typelib, opt->name)-1);
182 }
183 break;
184+ case OPT_REWRITE_DB: // db_from->db_to
185+ {
186+ /* See also handling of OPT_REPLICATE_REWRITE_DB in sql/mysqld.cc */
187+ char* ptr= argument;
188+ do {
189+ char* key= ptr; // db-from
190+ char* val; // db-to
191+
192+ // Where key begins
193+ while (*key && my_isspace(&my_charset_latin1, *key))
194+ key++;
195+
196+ // Where val begins
197+ if (!(ptr= strstr(key, "->")))
198+ {
199+ sql_print_error("Bad syntax in rewrite-db: missing '->'!\n");
200+ return 1;
201+ }
202+ val= ptr + 2;
203+ while (*val && my_isspace(&my_charset_latin1, *val))
204+ val++;
205+
206+ // Write \0 and skip blanks at the end of key
207+ *ptr-- = 0;
208+ while (my_isspace(&my_charset_latin1, *ptr) && ptr > argument)
209+ *ptr-- = 0;
210+
211+ if (!*key)
212+ {
213+ sql_print_error("Bad syntax in rewrite-db: empty db-from!\n");
214+ return 1;
215+ }
216+
217+ // Skip blanks at the end of val
218+ ptr= val;
219+ while (*ptr && !my_isspace(&my_charset_latin1, *ptr) && *ptr != ',')
220+ ++ptr;
221+
222+ if (my_isspace(&my_charset_latin1, *ptr))
223+ {
224+ *(ptr++)= 0;
225+ while (*ptr && *ptr != ',')
226+ {
227+ if (!my_isspace(&my_charset_latin1, *ptr)) {
228+ sql_print_error("Bad syntax in rewrite-db: db-to must contain db "
229+ "name without spaces!\n");
230+ return 1;
231+ }
232+ ++ptr;
233+ }
234+ }
235+
236+ if (*ptr == ',')
237+ {
238+ if (!rewrite_db_obs_syn_warn)
239+ {
240+ warning("The comma-separated list of rewritings syntax is obsolete and "
241+ "discarded in 5.7\n");
242+ rewrite_db_obs_syn_warn= true;
243+ }
244+ *(ptr++)= 0;
245+ }
246+ else
247+ *ptr= 0;
248+
249+ if (!*val)
250+ {
251+ sql_print_error("Bad syntax in rewrite-db: empty db-to!\n");
252+ return 1;
253+ }
254+
255+ if (!binlog_filter &&
256+ !(binlog_filter= new Rpl_filter))
257+ {
258+ sql_print_error("Failed to create Rpl_filter\n");
259+ return 1;
260+ }
261+
262+ binlog_filter->add_db_rewrite(key, val);
263+ } while (*ptr);
264+ break;
265+ }
266 case 'v':
267 if (argument == disabled_my_option)
268 verbose= 0;
269
270=== added file 'mysql-test/extra/rpl_tests/grep_pattern.inc'
271--- mysql-test/extra/rpl_tests/grep_pattern.inc 1970-01-01 00:00:00 +0000
272+++ mysql-test/extra/rpl_tests/grep_pattern.inc 2014-02-15 00:33:28 +0000
273@@ -0,0 +1,22 @@
274+# Please set GREP_FILE and GREP_PATTERN environment variables
275+# to work this file properly.
276+--perl
277+ use strict;
278+ my $file= $ENV{'GREP_FILE'} or die "grep file not set";
279+ my $pattern= $ENV{'GREP_PATTERN'} or die "pattern is not set";
280+ open(FILE, "$file") or die("Unable to open $file: $!\n");
281+ my $count = 0;
282+ print "Matching lines are:\n";
283+ while (<FILE>) {
284+ my $line = $_;
285+ if ($line =~ /$pattern/) {
286+ print "$line\n";
287+ $count++;
288+ }
289+ }
290+ if ($count == 0) {
291+ print "None\n";
292+ }
293+ print "Occurrences of the $pattern in the input file : $count\n";
294+ close(FILE);
295+EOF
296
297=== added file 'mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result'
298--- mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result 1970-01-01 00:00:00 +0000
299+++ mysql-test/suite/binlog/r/binlog_mysqlbinlog_rewrite_db.result 2014-02-15 00:33:28 +0000
300@@ -0,0 +1,33 @@
301+RESET MASTER;
302+CREATE DATABASE db1;
303+USE db1;
304+CREATE TABLE t1 (i INT);
305+INSERT INTO t1 VALUES(1);
306+INSERT INTO t1 VALUES(2);
307+UPDATE t1 SET i= i+1;
308+DELETE FROM t1 WHERE i=2;
309+[Syntax error in the use of the new option: The from database name is missing]
310+[Syntax error in the use of the new option: The '->' is missing]
311+[Syntax error in the use of the new option: The to database name is missing]
312+[VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 1]
313+[Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4,db5->db6"]
314+[log_grep.inc] file: mysqltest.log pattern: The comma-separated list of rewritings syntax is obsolete and discarded
315+[log_grep.inc] lines: 1
316+[VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 2]
317+[Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4" --rewrite-db="db5->db6,db6->db7"]
318+[log_grep.inc] file: mysqltest.log pattern: The comma-separated list of rewritings syntax is obsolete and discarded
319+[log_grep.inc] lines: 2
320+[VALID SYNTAX,The from->to database names are correctly mentioned]
321+#Dropping the database db1 and creating the table in the new database db2.
322+CREATE DATABASE db2;
323+DROP DATABASE db1;
324+RESET MASTER;
325+SELECT * FROM db2.t1;
326+ERROR 42S02: Table 'db2.t1' doesn't exist
327+DROP DATABASE db1;
328+RESET MASTER;
329+[The event of table db1.t1 has been successfully applied to db2.t1]
330+include/assert.inc [Assert that table db2.t1 has one row after applying the sql file.]
331+CLEANUP
332+DROP DATABASE db1;
333+DROP DATABASE db2;
334
335=== added file 'mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result'
336--- mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result 1970-01-01 00:00:00 +0000
337+++ mysql-test/suite/binlog/r/binlog_rewrite_db_noleak.result 2014-02-15 00:33:28 +0000
338@@ -0,0 +1,29 @@
339+RESET MASTER;
340+CREATE DATABASE db1;
341+USE db1;
342+CREATE TABLE t1 (i INT);
343+INSERT INTO db1.t1 VALUES(1);
344+INSERT INTO db1.t1 VALUES(2);
345+UPDATE t1 SET i= i+1;
346+DELETE FROM t1 WHERE i=2;
347+CREATE DATABASE db2;
348+CREATE TABLE db2.t1 (i INT);
349+INSERT INTO db2.t1 VALUES(3);
350+INSERT INTO db2.t1 VALUES(4);
351+INSERT INTO db2.t1 VALUES(5);
352+UPDATE db2.t1 SET i= i+1;
353+DELETE FROM db2.t1 WHERE i=4;
354+call mtr.add_suppression("Slave SQL: Error executing row event:*");
355+Dropping the database db1 creating the new database db3.
356+DROP DATABASE db1;
357+CREATE DATABASE db3;
358+DROP DATABASE db2;
359+RESET MASTER;
360+[The sql file will be applied on the current database]
361+[The content of table db3.t1 and db2.t1 will be different confirming no leak]
362+include/assert.inc [The content of the table t1 in database db3 and db2 is different]
363+include/assert.inc [Table t1 in db3 have different row count than t1 in db2]
364+CLEANUP
365+DROP DATABASE db1;
366+DROP DATABASE db2;
367+DROP DATABASE db3;
368
369=== added file 'mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result'
370--- mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result 1970-01-01 00:00:00 +0000
371+++ mysql-test/suite/binlog/r/binlog_rewrite_suppress_use.result 2014-02-15 00:33:28 +0000
372@@ -0,0 +1,29 @@
373+RESET MASTER;
374+CREATE DATABASE db1;
375+USE db1;
376+CREATE TABLE t1 (i INT);
377+INSERT INTO t1 VALUES(1);
378+INSERT INTO t1 VALUES(2);
379+UPDATE t1 SET i= i+1;
380+DELETE FROM t1 WHERE i=2;
381+[The use <db_name> is not suppressed in the general use of mysqlbinlog]
382+Matching lines are:
383+use `db1`/*!*/;
384+
385+Occurrences of the use `db1` in the input file : 1
386+[The use <db_name> is suppressed on using rewrite-db option of mysqlbinlog]
387+Matching lines are:
388+None
389+Occurrences of the use `db1` in the input file : 0
390+CREATE DATABASE db2;
391+DROP DATABASE db1;
392+RESET MASTER;
393+SELECT * FROM db2.t1;
394+ERROR 42S02: Table 'db2.t1' doesn't exist
395+DROP DATABASE db1;
396+RESET MASTER;
397+[The event of table db1.t1 has been successfully applied to db2.t1]
398+include/assert.inc [Assert that table db2.t1 has no rows after applying the sql file.]
399+[CLEANUP]
400+DROP DATABASE db1;
401+DROP DATABASE db2;
402
403=== added file 'mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result'
404--- mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result 1970-01-01 00:00:00 +0000
405+++ mysql-test/suite/binlog/r/percona_mysqlbinlog_rewritedb.result 2014-02-15 00:33:28 +0000
406@@ -0,0 +1,152 @@
407+#
408+# Apply log with ''
409+#
410+# use a
411+f
412+0
413+0
414+# use bbbbbbbbbbbbbbbbbbbb
415+f
416+1
417+1
418+# use cccccccccccccccc
419+f
420+2
421+2
422+# use d
423+f
424+#
425+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="d"'
426+#
427+# use a
428+f
429+# use bbbbbbbbbbbbbbbbbbbb
430+f
431+# use cccccccccccccccc
432+f
433+# use d
434+f
435+2
436+2
437+#
438+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="cccccccccccccccc"'
439+#
440+# use a
441+f
442+# use bbbbbbbbbbbbbbbbbbbb
443+f
444+# use cccccccccccccccc
445+f
446+# use d
447+f
448+#
449+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="bbbbbbbbbbbbbbbbbbbb"'
450+#
451+# use a
452+f
453+# use bbbbbbbbbbbbbbbbbbbb
454+f
455+0
456+0
457+1
458+1
459+# use cccccccccccccccc
460+f
461+# use d
462+f
463+#
464+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d" --database="a"'
465+#
466+# use a
467+f
468+# use bbbbbbbbbbbbbbbbbbbb
469+f
470+# use cccccccccccccccc
471+f
472+# use d
473+f
474+#
475+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb" --rewrite-db="cccccccccccccccc->d"'
476+#
477+# use a
478+f
479+# use bbbbbbbbbbbbbbbbbbbb
480+f
481+0
482+0
483+1
484+1
485+# use cccccccccccccccc
486+f
487+# use d
488+f
489+2
490+2
491+#
492+# Apply log with '--rewrite-db=" a -> bbbbbbbbbbbbbbbbbbbb , cccccccccccccccc -> d "'
493+#
494+# use a
495+f
496+# use bbbbbbbbbbbbbbbbbbbb
497+f
498+0
499+0
500+1
501+1
502+# use cccccccccccccccc
503+f
504+# use d
505+f
506+2
507+2
508+#
509+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb,cccccccccccccccc->d"'
510+#
511+# use a
512+f
513+# use bbbbbbbbbbbbbbbbbbbb
514+f
515+0
516+0
517+1
518+1
519+# use cccccccccccccccc
520+f
521+# use d
522+f
523+2
524+2
525+#
526+# Apply log with '--rewrite-db="a->d"'
527+#
528+# use a
529+f
530+# use bbbbbbbbbbbbbbbbbbbb
531+f
532+1
533+1
534+# use cccccccccccccccc
535+f
536+2
537+2
538+# use d
539+f
540+0
541+0
542+#
543+# Apply log with '--rewrite-db="a->bbbbbbbbbbbbbbbbbbbb"'
544+#
545+# use a
546+f
547+# use bbbbbbbbbbbbbbbbbbbb
548+f
549+0
550+0
551+1
552+1
553+# use cccccccccccccccc
554+f
555+2
556+2
557+# use d
558+f
559
560=== added file 'mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test'
561--- mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test 1970-01-01 00:00:00 +0000
562+++ mysql-test/suite/binlog/t/binlog_mysqlbinlog_rewrite_db.test 2014-02-15 00:33:28 +0000
563@@ -0,0 +1,95 @@
564+# WL#6404 - Add rewrite-db option to mysqlbinlog on RBR
565+#
566+# The test checks the new rewrite-db option for the mysqlbinlog.
567+#
568+# The test checks the following aspects of the new option:
569+
570+# 1. The valid syntax for the use of the new option and the errors in
571+# case the usage is not correct.
572+# 2. Apply the new binlog file ( with database rewritten ) to a new database
573+# and check if it works.
574+#
575+--source include/have_binlog_format_row.inc
576+
577+RESET MASTER;
578+CREATE DATABASE db1;
579+USE db1;
580+CREATE TABLE t1 (i INT);
581+
582+# Get some INSERT, UPDATE and DELETE ROW events.
583+INSERT INTO t1 VALUES(1);
584+INSERT INTO t1 VALUES(2);
585+UPDATE t1 SET i= i+1;
586+DELETE FROM t1 WHERE i=2;
587+
588+--let $MYSQLD_DATADIR= `select @@datadir`
589+--exec $MYSQL_BINLOG --force-if-open $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event.sql
590+
591+# Using the new option to apply the row event on some other database (from db1 -> db2 in the current case)
592+
593+--echo [Syntax error in the use of the new option: The from database name is missing]
594+--error 1
595+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="->db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
596+
597+--echo [Syntax error in the use of the new option: The '->' is missing]
598+--error 1
599+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1 db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
600+
601+--echo [Syntax error in the use of the new option: The to database name is missing]
602+--error 1
603+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
604+
605+--echo [VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 1]
606+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2,db3->db4,db5->db6" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
607+
608+--echo [Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4,db5->db6"]
609+--let log_file=mysqltest.log
610+--let log_file_full_path=$MYSQLTEST_VARDIR/log/current_test
611+--let grep_pattern=The comma-separated list of rewritings syntax is obsolete and discarded
612+--source include/log_grep.inc
613+
614+--echo [VALID SYNTAX,The from->to database names are correctly mentioned, but there is obsolete syntax warning 2]
615+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2,db3->db4" --rewrite-db="db5->db6,db6->db7" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
616+
617+--echo [Test that warning about obsolete syntax is shown only once for --rewrite-db="db1->db2,db3->db4" --rewrite-db="db5->db6,db6->db7"]
618+--let log_file=mysqltest.log
619+--let log_file_full_path=$MYSQLTEST_VARDIR/log/current_test
620+--let grep_pattern=The comma-separated list of rewritings syntax is obsolete and discarded
621+--source include/log_grep.inc
622+
623+--echo [VALID SYNTAX,The from->to database names are correctly mentioned]
624+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
625+
626+--echo #Dropping the database db1 and creating the table in the new database db2.
627+
628+CREATE DATABASE db2;
629+DROP DATABASE db1;
630+
631+# The SQL file will be applied but nothing is applied on Database db2 since the row event was
632+# generated for database db1 and table t1.
633+
634+# With gtid-mode=on we need purge gtid_executed, if not transactions
635+# replayed through mysqlbinlog will be skipped.
636+RESET MASTER;
637+--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event.sql
638+--error ER_NO_SUCH_TABLE
639+SELECT * FROM db2.t1;
640+
641+# The SQL file should be applied since the row event was extracted using the new mysqlbinlog option.
642+
643+DROP DATABASE db1;
644+# With gtid-mode=on we need purge gtid_executed, if not transactions
645+# replayed through mysqlbinlog will be skipped.
646+RESET MASTER;
647+--echo [The event of table db1.t1 has been successfully applied to db2.t1]
648+--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
649+--let $assert_text= Assert that table db2.t1 has one row after applying the sql file.
650+--let $assert_cond= `SELECT COUNT(*)=1 from db2.t1`
651+--source include/assert.inc
652+
653+--echo CLEANUP
654+
655+--remove_file $MYSQLTEST_VARDIR/tmp/row_event.sql
656+--remove_file $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
657+DROP DATABASE db1;
658+DROP DATABASE db2;
659
660=== added file 'mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test'
661--- mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test 1970-01-01 00:00:00 +0000
662+++ mysql-test/suite/binlog/t/binlog_rewrite_db_noleak.test 2014-02-15 00:33:28 +0000
663@@ -0,0 +1,79 @@
664+# WL#6404 - Add rewrite-db option to mysqlbinlog on RBR
665+#
666+# The test checks that there is no leak of the global variables used
667+# to decide whether rewrite-db option is ON or OFF.
668+#
669+# The test checks this by creating two tables in two different databases.
670+# insert rows in both of them.
671+#
672+# Read the binlog file using the new option --rewrite-db="db1->db3" and
673+# writing in the .sql file.
674+#
675+# On applying the .sql file on database db3, we get an error since not
676+# all events are converted for database db3. Only DB1 is converted to db3.
677+# The db2 event cause the error.
678+#
679+
680+--source include/have_binlog_format_row.inc
681+RESET MASTER;
682+CREATE DATABASE db1;
683+USE db1;
684+CREATE TABLE t1 (i INT);
685+
686+# Get some INSERT, UPDATE and DELETE ROW events.
687+INSERT INTO db1.t1 VALUES(1);
688+INSERT INTO db1.t1 VALUES(2);
689+UPDATE t1 SET i= i+1;
690+DELETE FROM t1 WHERE i=2;
691+
692+CREATE DATABASE db2;
693+CREATE TABLE db2.t1 (i INT);
694+
695+# Get some INSERT, UPDATE and DELETE ROW events.
696+INSERT INTO db2.t1 VALUES(3);
697+INSERT INTO db2.t1 VALUES(4);
698+INSERT INTO db2.t1 VALUES(5);
699+UPDATE db2.t1 SET i= i+1;
700+DELETE FROM db2.t1 WHERE i=4;
701+
702+--let $MYSQLD_DATADIR= `select @@datadir`
703+
704+# Using the new option to apply the row event on some other database (from db1 -> db3 in the current case)
705+
706+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db3" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/rewrite_noleak.sql
707+call mtr.add_suppression("Slave SQL: Error executing row event:*");
708+
709+--echo Dropping the database db1 creating the new database db3.
710+DROP DATABASE db1;
711+CREATE DATABASE db3;
712+
713+# The SQL file will be applied but only db1 events will be replayed on db3.
714+# Database db2 will not be applied on db3 confirming there is no leak.
715+DROP DATABASE db2;
716+
717+# With gtid-mode=on we need purge gtid_executed, if not transactions
718+# replayed through mysqlbinlog will be skipped.
719+RESET MASTER;
720+--echo [The sql file will be applied on the current database]
721+--exec $MYSQL --database=db3 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/rewrite_noleak.sql
722+
723+--echo [The content of table db3.t1 and db2.t1 will be different confirming no leak]
724+
725+--let $db3_max= `SELECT MAX(i) FROM db3.t1`
726+--let $db2_max= `SELECT MAX(i) FROM db2.t1`
727+--let $assert_text= The content of the table t1 in database db3 and db2 is different
728+--let $assert_cond= $db3_max <> $db2_max
729+--source include/assert.inc
730+
731+--let $db3_count= `SELECT COUNT(*) FROM db3.t1`
732+--let $db2_count= `SELECT COUNT(*) FROM db2.t1`
733+--let $assert_text= Table t1 in db3 have different row count than t1 in db2
734+--let $assert_cond= $db3_count=1 AND $db2_count<>1
735+--source include/assert.inc
736+
737+--echo CLEANUP
738+
739+#--remove_file $MYSQLTEST_VARDIR/tmp/rewrite_noleak.sql
740+DROP DATABASE db1;
741+DROP DATABASE db2;
742+DROP DATABASE db3;
743
744=== added file 'mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test'
745--- mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test 1970-01-01 00:00:00 +0000
746+++ mysql-test/suite/binlog/t/binlog_rewrite_suppress_use.test 2014-02-15 00:33:28 +0000
747@@ -0,0 +1,61 @@
748+# WL#6404 - Add rewrite-db option to mysqlbinlog on RBR
749+#
750+# The test aims to check that the use of rewrite-db option of
751+# mysqlbinlog suppresses the USE DATABASE command logged in
752+# binlog file.
753+#
754+
755+--source include/have_binlog_format_row.inc
756+RESET MASTER;
757+CREATE DATABASE db1;
758+USE db1;
759+CREATE TABLE t1 (i INT);
760+
761+# Get some INSERT, UPDATE and DELETE ROW events.
762+INSERT INTO t1 VALUES(1);
763+INSERT INTO t1 VALUES(2);
764+UPDATE t1 SET i= i+1;
765+DELETE FROM t1 WHERE i=2;
766+
767+--let $MYSQLD_DATADIR= `select @@datadir`
768+
769+# Checking for the suppression of the USE DATABASE command on using the new option.
770+# Reading binlog file without the rewrite-db option.
771+--echo [The use <db_name> is not suppressed in the general use of mysqlbinlog]
772+--exec $MYSQL_BINLOG --force-if-open $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event.sql
773+--let GREP_FILE=$MYSQLTEST_VARDIR/tmp/row_event.sql
774+--let GREP_PATTERN=use `db1`
775+--source extra/rpl_tests/grep_pattern.inc
776+
777+# Reading binlog file with the rewrite-db option.
778+--echo [The use <db_name> is suppressed on using rewrite-db option of mysqlbinlog]
779+--exec $MYSQL_BINLOG --force-if-open --rewrite-db="db1->db2" $MYSQLD_DATADIR/master-bin.000001 > $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
780+--let GREP_FILE=$MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
781+--let GREP_PATTERN=use `db1`
782+--source extra/rpl_tests/grep_pattern.inc
783+
784+CREATE DATABASE db2;
785+DROP DATABASE db1;
786+# With gtid-mode=on we need purge gtid_executed, if not transactions
787+# replayed through mysqlbinlog will be skipped.
788+RESET MASTER;
789+--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event.sql
790+--error ER_NO_SUCH_TABLE
791+SELECT * FROM db2.t1;
792+
793+DROP DATABASE db1;
794+# With gtid-mode=on we need purge gtid_executed, if not transactions
795+# replayed through mysqlbinlog will be skipped.
796+RESET MASTER;
797+--echo [The event of table db1.t1 has been successfully applied to db2.t1]
798+--exec $MYSQL --database=db2 --local-infile=1 < $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
799+--let $assert_text= Assert that table db2.t1 has no rows after applying the sql file.
800+--let $assert_cond= `SELECT COUNT(*)=1 from db2.t1`
801+--source include/assert.inc
802+
803+--echo [CLEANUP]
804+--remove_file $MYSQLTEST_VARDIR/tmp/row_event.sql
805+--remove_file $MYSQLTEST_VARDIR/tmp/row_event_rewrite.sql
806+
807+DROP DATABASE db1;
808+DROP DATABASE db2;
809
810=== added file 'mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt'
811--- mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt 1970-01-01 00:00:00 +0000
812+++ mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb-master.opt 2014-02-15 00:33:28 +0000
813@@ -0,0 +1,1 @@
814+--local-infile=1
815
816=== added file 'mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test'
817--- mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test 1970-01-01 00:00:00 +0000
818+++ mysql-test/suite/binlog/t/percona_mysqlbinlog_rewritedb.test 2014-02-15 00:33:28 +0000
819@@ -0,0 +1,80 @@
820+--source include/have_log_bin.inc
821+#
822+# WL #36: http://askmonty.org/worklog/Server-Sprint/?tid=36
823+# This is basic test for --rewrite-db option.
824+# mysqlbinlog --rewrite-db="from->to" should rewrite database name
825+#
826+
827+--disable_query_log
828+
829+# Because RBR event DB rewrite requires shifting of
830+# Table_map_log_event contents around if the DB name length
831+# changes, this case is tested for both shorter and longer
832+# names.
833+
834+--let DB1=a
835+--let DB2=bbbbbbbbbbbbbbbbbbbb
836+--let DB3=cccccccccccccccc
837+--let DB4=d
838+
839+eval CREATE DATABASE $DB1; eval USE $DB1; CREATE TABLE t1(f INT);
840+eval CREATE DATABASE $DB2; eval USE $DB2; CREATE TABLE t1(f INT);
841+eval CREATE DATABASE $DB3; eval USE $DB3; CREATE TABLE t1(f INT);
842+eval CREATE DATABASE $DB4; eval USE $DB4; CREATE TABLE t1(f INT);
843+
844+RESET MASTER;
845+--let MYSQLD_DATA_DIR=`select @@datadir`
846+--let LOAD_DATA_FILE=$MYSQLD_DATA_DIR/table.values
847+--let BIN_LOG_FILE_NAME=query_get_value(show master status, File, 1)
848+--let BIN_LOG_START_POSITION=query_get_value(show master status, Position, 1)
849+eval USE $DB1; INSERT INTO t1 VALUES(0); INSERT INTO t1 VALUES(0);
850+eval USE $DB2; INSERT INTO t1 VALUES(1); INSERT INTO t1 VALUES(1);
851+write_file $LOAD_DATA_FILE;
852+2
853+2
854+EOF
855+eval USE $DB3;
856+--eval LOAD DATA INFILE "$LOAD_DATA_FILE" INTO TABLE t1
857+--let BIN_LOG_STOP_POSITION=query_get_value(show master status, Position, 1)
858+FLUSH LOGS;
859+
860+--let BIN_LOG_FULL_PATH=`SELECT CONCAT("$MYSQLD_DATA_DIR", "$BIN_LOG_FILE_NAME")`
861+
862+--let i=10
863+
864+while ($i)
865+{
866+# Test for different combinations of syntax and db name filtering and rewriting
867+--let REWRITE=`SELECT CASE $i WHEN 10 THEN '' WHEN 9 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB4"' WHEN 8 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB3"' WHEN 7 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB2"' WHEN 6 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4" --database="$DB1"' WHEN 5 THEN '--rewrite-db="$DB1->$DB2" --rewrite-db="$DB3->$DB4"' WHEN 4 THEN '--rewrite-db=" $DB1 -> $DB2 , $DB3 -> $DB4 "' WHEN 3 THEN '--rewrite-db="$DB1->$DB2,$DB3->$DB4"' WHEN 2 THEN '--rewrite-db="$DB1->$DB4"' WHEN 1 THEN '--rewrite-db="$DB1->$DB2"' END`
868+
869+eval USE $DB1; DELETE FROM t1;
870+eval USE $DB2; DELETE FROM t1;
871+eval USE $DB3; DELETE FROM t1;
872+eval USE $DB4; DELETE FROM t1;
873+
874+--echo #
875+--echo # Apply log with '$REWRITE'
876+--echo #
877+
878+--exec $MYSQL_BINLOG $REWRITE --start-position=$BIN_LOG_START_POSITION --stop-position=$BIN_LOG_STOP_POSITION $BIN_LOG_FULL_PATH | $MYSQL --local-infile=1
879+
880+--echo # use $DB1
881+eval USE $DB1; SELECT * FROM t1 ORDER BY f;
882+--echo # use $DB2
883+eval USE $DB2; SELECT * FROM t1 ORDER BY f;
884+--echo # use $DB3
885+eval USE $DB3; SELECT * FROM t1 ORDER BY f;
886+--echo # use $DB4
887+eval USE $DB4; SELECT * FROM t1 ORDER BY f;
888+
889+dec $i;
890+}
891+
892+eval USE $DB1; DROP TABLE t1; eval DROP DATABASE $DB1;
893+eval USE $DB2; DROP TABLE t1; eval DROP DATABASE $DB2;
894+eval USE $DB3; DROP TABLE t1; eval DROP DATABASE $DB3;
895+eval USE $DB4; DROP TABLE t1; eval DROP DATABASE $DB4;
896+
897+--remove_file $LOAD_DATA_FILE
898+
899+--enable_query_log
900
901=== modified file 'sql/log_event.cc'
902--- sql/log_event.cc 2014-02-03 04:39:37 +0000
903+++ sql/log_event.cc 2014-02-15 00:33:28 +0000
904@@ -8714,6 +8714,108 @@
905 my_free(m_memory);
906 }
907
908+#ifdef MYSQL_CLIENT
909+
910+/*
911+ Rewrite database name for the event to name specified by new_db
912+ SYNOPSIS
913+ new_db Database name to change to
914+ new_len Length
915+ desc Event describing binlog that we're writing to.
916+
917+ DESCRIPTION
918+ Reset db name. This function assumes that temp_buf member contains event
919+ representation taken from a binary log. It resets m_dbnam and m_dblen and
920+ rewrites temp_buf with new db name.
921+
922+ RETURN
923+ 0 - Success
924+ other - Error
925+*/
926+
927+int Table_map_log_event::rewrite_db(const char* new_db, size_t new_len,
928+ const Format_description_log_event* desc)
929+{
930+ DBUG_ENTER("Table_map_log_event::rewrite_db");
931+ DBUG_ASSERT(temp_buf);
932+
933+ uint header_len= min(desc->common_header_len,
934+ LOG_EVENT_MINIMAL_HEADER_LEN) + TABLE_MAP_HEADER_LEN;
935+ int len_diff;
936+
937+ if (!(len_diff= new_len - m_dblen))
938+ {
939+ memcpy((void*) (temp_buf + header_len + 1), new_db, m_dblen + 1);
940+ memcpy((void*) m_dbnam, new_db, m_dblen + 1);
941+ DBUG_RETURN(0);
942+ }
943+
944+ // Create new temp_buf
945+ ulong event_cur_len= uint4korr(temp_buf + EVENT_LEN_OFFSET);
946+ ulong event_new_len= event_cur_len + len_diff;
947+ char* new_temp_buf= (char*) my_malloc(event_new_len, MYF(MY_WME));
948+
949+ if (!new_temp_buf)
950+ {
951+ sql_print_error("Table_map_log_event::rewrite_db: "
952+ "failed to allocate new temp_buf (%d bytes required)",
953+ event_new_len);
954+ DBUG_RETURN(-1);
955+ }
956+
957+ // Rewrite temp_buf
958+ char* ptr= new_temp_buf;
959+ ulong cnt= 0;
960+
961+ // Copy header and change event length
962+ memcpy(ptr, temp_buf, header_len);
963+ int4store(ptr + EVENT_LEN_OFFSET, event_new_len);
964+ ptr += header_len;
965+ cnt += header_len;
966+
967+ // Write new db name length and new name
968+ *ptr++ = new_len;
969+ memcpy(ptr, new_db, new_len + 1);
970+ ptr += new_len + 1;
971+ cnt += m_dblen + 2;
972+
973+ // Copy rest part
974+ memcpy(ptr, temp_buf + cnt, event_cur_len - cnt);
975+
976+ // Reregister temp buf
977+ free_temp_buf();
978+ register_temp_buf(new_temp_buf);
979+
980+ // Reset m_dbnam and m_dblen members
981+ m_dblen= new_len;
982+
983+ // m_dbnam resides in m_memory together with m_tblnam and m_coltype
984+ uchar* memory= m_memory;
985+ char const* tblnam= m_tblnam;
986+ uchar* coltype= m_coltype;
987+
988+ m_memory= (uchar*) my_multi_malloc(MYF(MY_WME),
989+ &m_dbnam, (uint) m_dblen + 1,
990+ &m_tblnam, (uint) m_tbllen + 1,
991+ &m_coltype, (uint) m_colcnt,
992+ NullS);
993+
994+ if (!m_memory)
995+ {
996+ sql_print_error("Table_map_log_event::rewrite_db: "
997+ "failed to allocate new m_memory (%d + %d + %d bytes required)",
998+ m_dblen + 1, m_tbllen + 1, m_colcnt);
999+ DBUG_RETURN(-1);
1000+ }
1001+
1002+ memcpy((void*)m_dbnam, new_db, m_dblen + 1);
1003+ memcpy((void*)m_tblnam, tblnam, m_tbllen + 1);
1004+ memcpy(m_coltype, coltype, m_colcnt);
1005+
1006+ my_free(memory);
1007+ DBUG_RETURN(0);
1008+}
1009+#endif /* MYSQL_CLIENT */
1010 /*
1011 Return value is an error code, one of:
1012
1013
1014=== modified file 'sql/log_event.h'
1015--- sql/log_event.h 2014-02-03 04:39:37 +0000
1016+++ sql/log_event.h 2014-02-15 00:33:28 +0000
1017@@ -3504,7 +3504,10 @@
1018 ulong get_table_id() const { return m_table_id; }
1019 const char *get_table_name() const { return m_tblnam; }
1020 const char *get_db_name() const { return m_dbnam; }
1021-
1022+#ifdef MYSQL_CLIENT
1023+ int rewrite_db(const char* new_name, size_t new_name_len,
1024+ const Format_description_log_event*);
1025+#endif
1026 virtual Log_event_type get_type_code() { return TABLE_MAP_EVENT; }
1027 virtual bool is_valid() const { return m_memory != NULL; /* we check malloc */ }
1028

Subscribers

People subscribed via source and target branches