Merge lp:~patrick-crews/drizzle/drizzle_dump_restore_test into lp:~drizzle-trunk/drizzle/development

Proposed by Patrick Crews on 2010-05-19
Status: Merged
Merge reported by: Patrick Crews
Merged at revision: not available
Proposed branch: lp:~patrick-crews/drizzle/drizzle_dump_restore_test
Merge into: lp:~drizzle-trunk/drizzle/development
Diff against target: 331 lines (+254/-24)
5 files modified
tests/include/diff_tables.inc (+22/-24)
tests/include/drizzledump.inc (+45/-0)
tests/r/drizzledump_restore.result (+83/-0)
tests/t/drizzledump_restore.test (+91/-0)
tests/test-run.pl (+13/-0)
To merge this branch: bzr merge lp:~patrick-crews/drizzle/drizzle_dump_restore_test
Reviewer Review Type Date Requested Status
Monty Taylor 2010-05-19 Needs Fixing on 2010-05-20
Review via email: mp+25648@code.launchpad.net

This proposal has been superseded by a proposal from 2010-05-20.

Description of the change

I ported a simple test of dump/restore that I had added to MySQL some time ago. It is drizzledump_restore

This test could use some expanded scenarios, but it does verify that we can dump and restore a table.

Altered test-run.pl so that we can call the drizzle client from within tests ($DRIZZLE), which was needed for the restore portion of the test.

Included a partial port of MySQL Bug#51057 - Weak code in diff_tables.inc can lead to 100% CPU consumption. If one of the tables to be compared did not exist, the test would timeout and waste time and resources.

Ran into some issues with diff_tables and ORDINAL_POSITION, but I suspect this was not needed and the new test is the only one that actually uses this include file.

To post a comment you must log in.
Monty Taylor (mordred) wrote :

Looks great!

Two things...

First, it looks like you forgot to add the new test files...

Second, just go ahead and delete the code you commented out in diff_tables. If we need it, we can also pull it from a previous version.

review: Needs Fixing
Patrick Crews (patrick-crews) wrote :

Sorry about the omission of the new files. My bzr skills are have gathered some dust. Things are fixed and polished - removed the unneeded code from diff_tables.inc and added the new files

Stewart Smith (stewart) wrote :

On Thu, 20 May 2010 03:40:54 -0000, Patrick Crews <email address hidden> wrote:
> Sorry about the omission of the new files. My bzr skills are have
> gathered some dust. Things are fixed and polished - removed the
> unneeded code from diff_tables.inc and added the new files

'bzr status' can tell you what's around. useful to run before
committing, it'll tell you unknown files (taht you've probably forgotten
to add)

--
Stewart Smith

1548. By Patrick Crews on 2010-05-25

Fixed .result issue where mysqldump was incorrectly replaced with drizzledump in MySQL bug names

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'tests/include/diff_tables.inc'
2--- tests/include/diff_tables.inc 2008-06-25 05:29:13 +0000
3+++ tests/include/diff_tables.inc 2010-05-20 03:32:27 +0000
4@@ -65,7 +65,7 @@
5 while ($_diff_i) {
6
7 # Parse out any leading "master:" or "slave:" from the table
8- # specification and connect the appropriate server.
9+ # specification and connect to the appropriate server.
10 let $_diff_conn_master=`SELECT SUBSTR('$_diff_table', 1, 7) = 'master:'`;
11 if ($_diff_conn_master) {
12 let $_diff_table=`SELECT SUBSTR('$_diff_table', 8)`;
13@@ -78,32 +78,30 @@
14 }
15
16 # Sanity-check the input.
17- let $_diff_error= `SELECT '$_diff_table' NOT LIKE '_%._%'`;
18- if ($_diff_error) {
19- --echo !!!ERROR IN TEST: \$diff_table_$_diff_i='$_diff_table' is not in the form database.table
20- exit;
21- }
22-
23- # We need the output files to be sorted (so that diff_files does not
24- # think the files are different just because they are differently
25- # ordered). To this end, we first generate a query that sorts the
26- # table by all columns. Since ORDER BY accept column indices, we
27- # just generate a comma-separated list of all numbers from 1 to the
28- # number of columns in the table.
29- let $_diff_column_index=`SELECT MAX(ordinal_position)
30- FROM information_schema.columns
31- WHERE CONCAT(table_schema, '.', table_name) =
32- '$_diff_table'`;
33- let $_diff_column_list=$_diff_column_index;
34- dec $_diff_column_index;
35- while ($_diff_column_index) {
36- let $_diff_column_list=$_diff_column_index, $_diff_column_list;
37- dec $_diff_column_index;
38- }
39+ if (`SELECT '$_diff_table' NOT LIKE '_%._%'`) {
40+ --echo !!!ERROR IN TEST: \$diff_table_$_diff_i='$_diff_table'
41+ --echo is not in the form database.table
42+ --die
43+ }
44+
45+ # Check if the table exists
46+ if (`SELECT COUNT(*) = 0 FROM data_dictionary.tables
47+ WHERE CONCAT(table_schema, '.', table_name) = '$_diff_table'`) {
48+ --echo !!!ERROR IN TEST: The table '$_diff_table' does not exist
49+ --die
50+ }
51+
52+
53+ #let $_diff_column_list=$_diff_column_index;
54+ #dec $_diff_column_index;
55+ #while ($_diff_column_index) {
56+ # let $_diff_column_list=$_diff_column_index, $_diff_column_list;
57+ # dec $_diff_column_index;
58+ #}
59
60 # Now that we have the comma-separated list of columns, we can write
61 # the table to a file.
62- eval SELECT * FROM $_diff_table ORDER BY $_diff_column_list
63+ eval SELECT * FROM $_diff_table ORDER BY 1
64 INTO OUTFILE '$MYSQLTEST_VARDIR/tmp/diff_table_$_diff_i';
65
66 # Do the same for $diff_table_1.
67
68=== added file 'tests/include/drizzledump.inc'
69--- tests/include/drizzledump.inc 1970-01-01 00:00:00 +0000
70+++ tests/include/drizzledump.inc 2010-05-20 03:32:27 +0000
71@@ -0,0 +1,45 @@
72+################################################################################
73+# drizzledump.inc
74+#
75+# SUMMARY: include file to facilitate testing the quality of drizzledump output
76+#
77+# INPUTS: Two variables:
78+# $table_name - the name of the table that was dumped
79+# $drizzledumpfile - the name of the file that captured drizzledump output
80+#
81+# OUTPUTS: minor echo data:
82+# We 'echo' some stage information to the .result file:
83+# 'altering original table', 'restoring from dumpfile', 'comparing'
84+#
85+# OTHER FILES: We use include/diff_tables.inc to compare the original, renamed
86+# table with the 'restored' one.
87+#
88+# DESCRIPTION: This file works by being fed the name of the original table
89+# and a drizzledump output file. The original table is then renamed
90+# to <table_name>_orig, the drizzledump file is used to recreate the
91+# table, then diff_tables.inc is called to compare them.
92+#
93+# LIMITATIONS: Does *NOT* work with xml output!
94+#
95+################################################################################
96+
97+--echo # Begin testing drizzledump output + restore
98+--echo # Create 'original table name - <table>_orig
99+# NOTE: We use SET then let as query_get_value has issues with the extra commas
100+# used in the CONCAT statement.
101+eval SET @orig_table_name = CONCAT('$table_name', '_orig');
102+let $orig_table_name = query_get_value(SELECT @orig_table_name,@orig_table_name,1);
103+--echo # Rename original table
104+eval ALTER TABLE $table_name RENAME to $orig_table_name;
105+--echo # Recreate table from drizzledump output
106+--exec $DRIZZLE test < $drizzledumpfile
107+--echo # Compare original and recreated tables
108+--echo # Recreated table: $table_name
109+--echo # Original table: $orig_table_name
110+let $diff_table_1 = $table_name;
111+let $diff_table_2 = $orig_table_name;
112+--source include/diff_tables.inc
113+--echo # Cleanup
114+--remove_file $drizzledumpfile
115+eval DROP TABLE $table_name, $orig_table_name;
116+
117
118=== added file 'tests/r/drizzledump_restore.result'
119--- tests/r/drizzledump_restore.result 1970-01-01 00:00:00 +0000
120+++ tests/r/drizzledump_restore.result 2010-05-20 03:32:27 +0000
121@@ -0,0 +1,83 @@
122+# Pre-test cleanup
123+DROP TABLE IF EXISTS t1;
124+# Begin tests
125+#
126+# Bug#2005 Long decimal comparison bug.
127+#
128+CREATE TABLE t1 (a DECIMAL(64, 20));
129+INSERT INTO t1 VALUES ("1234567890123456789012345678901234567890"),
130+("0987654321098765432109876543210987654321");
131+# Begin testing drizzledump output + restore
132+# Create 'original table name - <table>_orig
133+SET @orig_table_name = CONCAT('test.t1', '_orig');
134+# Rename original table
135+ALTER TABLE test.t1 RENAME to test.t1_orig;
136+# Recreate table from drizzledump output
137+# Compare original and recreated tables
138+# Recreated table: test.t1
139+# Original table: test.t1_orig
140+Comparing tables test.t1 and test.t1_orig
141+# Cleanup
142+DROP TABLE test.t1, test.t1_orig;
143+#
144+# Bug#3361 drizzledump quotes DECIMAL values inconsistently
145+#
146+CREATE TABLE t1 (a DECIMAL(10,5), b FLOAT);
147+INSERT INTO t1 VALUES (1.2345, 2.3456);
148+INSERT INTO t1 VALUES ('1.2345', 2.3456);
149+INSERT INTO t1 VALUES ("1.2345", 2.3456);
150+INSERT INTO t1 VALUES (1.2345, 2.3456);
151+INSERT INTO t1 VALUES ('1.2345', 2.3456);
152+INSERT INTO t1 VALUES ("1.2345", 2.3456);
153+# Begin testing drizzledump output + restore
154+# Create 'original table name - <table>_orig
155+SET @orig_table_name = CONCAT('test.t1', '_orig');
156+# Rename original table
157+ALTER TABLE test.t1 RENAME to test.t1_orig;
158+# Recreate table from drizzledump output
159+# Compare original and recreated tables
160+# Recreated table: test.t1
161+# Original table: test.t1_orig
162+Comparing tables test.t1 and test.t1_orig
163+# Cleanup
164+DROP TABLE test.t1, test.t1_orig;
165+#
166+# WL#2319 Exclude Tables from dump
167+#
168+CREATE TABLE t1 (a INT);
169+CREATE TABLE t2 (a INT);
170+INSERT INTO t1 VALUES (1),(2),(3);
171+INSERT INTO t2 VALUES (4),(5),(6);
172+# Begin testing drizzledump output + restore
173+# Create 'original table name - <table>_orig
174+SET @orig_table_name = CONCAT('test.t2', '_orig');
175+# Rename original table
176+ALTER TABLE test.t2 RENAME to test.t2_orig;
177+# Recreate table from drizzledump output
178+# Compare original and recreated tables
179+# Recreated table: test.t2
180+# Original table: test.t2_orig
181+Comparing tables test.t2 and test.t2_orig
182+# Cleanup
183+DROP TABLE test.t2, test.t2_orig;
184+DROP TABLE t1;
185+#
186+# Bug#8830 drizzledump --skip-extended-insert causes --hex-blob to dump wrong values
187+#
188+CREATE TABLE t1 (`b` blob);
189+INSERT INTO `t1` VALUES (0x602010000280100005E71A);
190+# Begin testing drizzledump output + restore
191+# Create 'original table name - <table>_orig
192+SET @orig_table_name = CONCAT('test.t1', '_orig');
193+# Rename original table
194+ALTER TABLE test.t1 RENAME to test.t1_orig;
195+# Recreate table from drizzledump output
196+# Compare original and recreated tables
197+# Recreated table: test.t1
198+# Original table: test.t1_orig
199+Comparing tables test.t1 and test.t1_orig
200+# Cleanup
201+DROP TABLE test.t1, test.t1_orig;
202+# End tests
203+# Cleanup
204+# remove drizzledumpfile
205
206=== added file 'tests/t/drizzledump_restore.test'
207--- tests/t/drizzledump_restore.test 1970-01-01 00:00:00 +0000
208+++ tests/t/drizzledump_restore.test 2010-05-20 03:32:27 +0000
209@@ -0,0 +1,91 @@
210+###############################################################################
211+# drizzledump_restore.test
212+#
213+# Purpose: Tests if drizzledump output can be used to successfully restore
214+# tables and data.
215+# We CREATE a table, drizzledump it to a file, ALTER the original
216+# table's name, recreate the table from the drizzledump file, then
217+# utilize include/diff_tables to compare the original and recreated
218+# tables.
219+#
220+# We use several examples from drizzledump.test here and include
221+# the relevant bug numbers and headers from that test.
222+#
223+# NOTE: This test is not currently complete and offers only basic
224+# cases of drizzledump output being restored.
225+# Also, does NOT work with -X (xml) output!
226+#
227+# Author: pcrews
228+# Created: 2009-05-21
229+# Last Change:
230+# Change date:
231+###############################################################################
232+
233+#--source include/have_log_bin.inc
234+
235+
236+# Define drizzledumpfile here. It is used to capture drizzledump output
237+# in order to test the output's ability to restore an exact copy of the table
238+let $drizzledumpfile = $MYSQLTEST_VARDIR/tmp/drizzledumpfile.sql;
239+
240+--echo # Pre-test cleanup
241+--disable_warnings
242+DROP TABLE IF EXISTS t1;
243+--enable_warnings
244+
245+--echo # Begin tests
246+--echo #
247+--echo # Bug#2005 Long decimal comparison bug.
248+--echo #
249+CREATE TABLE t1 (a DECIMAL(64, 20));
250+INSERT INTO t1 VALUES ("1234567890123456789012345678901234567890"),
251+("0987654321098765432109876543210987654321");
252+--exec $DRIZZLE_DUMP --compact test t1 > $drizzledumpfile
253+let $table_name = test.t1;
254+--source include/drizzledump.inc
255+
256+--echo #
257+--echo # Bug#3361 mysqldump quotes DECIMAL values inconsistently
258+--echo #
259+CREATE TABLE t1 (a DECIMAL(10,5), b FLOAT);
260+# Check at first how mysql work with quoted decimal
261+INSERT INTO t1 VALUES (1.2345, 2.3456);
262+INSERT INTO t1 VALUES ('1.2345', 2.3456);
263+INSERT INTO t1 VALUES ("1.2345", 2.3456);
264+INSERT INTO t1 VALUES (1.2345, 2.3456);
265+INSERT INTO t1 VALUES ('1.2345', 2.3456);
266+INSERT INTO t1 VALUES ("1.2345", 2.3456);
267+
268+# check how drizzledump make quoting
269+--exec $DRIZZLE_DUMP --compact test t1 > $drizzledumpfile
270+let $table_name = test.t1;
271+--source include/drizzledump.inc
272+
273+
274+--echo #
275+--echo # WL#2319 Exclude Tables from dump
276+--echo #
277+CREATE TABLE t1 (a INT);
278+CREATE TABLE t2 (a INT);
279+INSERT INTO t1 VALUES (1),(2),(3);
280+INSERT INTO t2 VALUES (4),(5),(6);
281+--exec $DRIZZLE_DUMP --skip-comments --ignore-table=test.t1 test > $drizzledumpfile
282+let $table_name = test.t2;
283+--source include/drizzledump.inc
284+DROP TABLE t1;
285+
286+--echo #
287+--echo # Bug#8830 mysqldump --skip-extended-insert causes --hex-blob to dump wrong values
288+--echo #
289+CREATE TABLE t1 (`b` blob);
290+INSERT INTO `t1` VALUES (0x602010000280100005E71A);
291+--exec $DRIZZLE_DUMP --skip-extended-insert --hex-blob test --skip-comments t1 > $drizzledumpfile
292+let $table_name = test.t1;
293+--source include/drizzledump.inc
294+
295+--echo # End tests
296+
297+--echo # Cleanup
298+--echo # remove drizzledumpfile
299+--error 0,1
300+--remove_file $drizzledumpfile
301
302=== modified file 'tests/test-run.pl'
303--- tests/test-run.pl 2010-04-20 07:21:29 +0000
304+++ tests/test-run.pl 2010-05-20 03:32:27 +0000
305@@ -1322,6 +1322,13 @@
306 "--port=$mysqld->{'port'} ";
307 }
308
309+sub generate_cmdline_drizzle ($) {
310+ my($mysqld) = @_;
311+ return
312+ mtr_native_path($exe_drizzle) .
313+ " -uroot --port=$mysqld->{'port'} ";
314+}
315+
316
317 ##############################################################################
318 #
319@@ -1492,6 +1499,12 @@
320 $ENV{'DRIZZLE_DUMP_SECONDARY'}= $cmdline_mysqldump_secondary;
321
322 # ----------------------------------------------------
323+ # Setup env so we can execute drizzle client
324+ # ----------------------------------------------------
325+ my $cmdline_drizzle = generate_cmdline_drizzle($master->[0]);
326+ $ENV{'DRIZZLE'}= $cmdline_drizzle;
327+
328+ # ----------------------------------------------------
329 # Setup env so childs can execute mysqlslap
330 # ----------------------------------------------------
331 if ( $exe_drizzleslap )