Merge lp:~percona-toolkit-dev/percona-toolkit/release-2.2.10 into lp:percona-toolkit/2.2
- release-2.2.10
- Merge into 2.2
Proposed by
Tomislav Plavcic
Status: | Merged |
---|---|
Merged at revision: | 609 |
Proposed branch: | lp:~percona-toolkit-dev/percona-toolkit/release-2.2.10 |
Merge into: | lp:percona-toolkit/2.2 |
Diff against target: |
1855 lines (+521/-273) 57 files modified
Changelog (+9/-1) Makefile.PL (+1/-1) bin/pt-align (+1/-1) bin/pt-archiver (+2/-2) bin/pt-config-diff (+2/-2) bin/pt-deadlock-logger (+2/-2) bin/pt-diskstats (+2/-2) bin/pt-duplicate-key-checker (+2/-2) bin/pt-fifo-split (+1/-1) bin/pt-find (+2/-2) bin/pt-fingerprint (+1/-1) bin/pt-fk-error-logger (+2/-2) bin/pt-heartbeat (+2/-2) bin/pt-index-usage (+25/-11) bin/pt-ioprofile (+1/-1) bin/pt-kill (+2/-2) bin/pt-mext (+3/-2) bin/pt-mysql-summary (+3/-2) bin/pt-online-schema-change (+45/-37) bin/pt-pmp (+1/-1) bin/pt-query-digest (+14/-8) bin/pt-show-grants (+1/-1) bin/pt-sift (+1/-1) bin/pt-slave-delay (+2/-2) bin/pt-slave-find (+1/-1) bin/pt-slave-restart (+2/-2) bin/pt-stalk (+1/-1) bin/pt-summary (+3/-2) bin/pt-table-checksum (+43/-19) bin/pt-table-sync (+2/-2) bin/pt-table-usage (+42/-17) bin/pt-upgrade (+25/-11) bin/pt-variable-advisor (+2/-2) bin/pt-visual-explain (+1/-1) config/deb/changelog (+11/-0) config/sphinx-build/conf.py (+1/-1) docs/percona-toolkit.pod (+1/-1) docs/release_notes.rst (+29/-0) lib/MySQLStatusWaiter.pm (+3/-2) lib/Percona/Toolkit.pm (+1/-1) lib/RowChecksum.pm (+22/-13) lib/SlowLogParser.pm (+14/-6) lib/bash/parse_options.sh (+2/-1) t/lib/MySQLStatusWaiter.t (+18/-14) t/lib/SlowLogParser.t (+64/-0) t/lib/samples/slowlogs/slow060.txt (+10/-0) t/lib/samples/slowlogs/slow061.txt (+10/-0) t/pt-online-schema-change/bug-1315130.t (+0/-78) t/pt-online-schema-change/bugs.t (+64/-0) t/pt-online-schema-change/samples/bug-1340728.sql (+10/-0) t/pt-online-schema-change/samples/bug-1340728_cleanup.sql (+2/-0) t/pt-query-digest/since_until.t (+3/-3) t/pt-table-checksum/basics.t (+3/-2) t/pt-table-checksum/samples/default-results-5.5.txt (+1/-1) t/pt-table-checksum/samples/default-results-5.6.txt (+1/-1) t/pt-table-checksum/samples/static-chunk-size-results-5.5.txt (+1/-1) t/pt-table-checksum/samples/static-chunk-size-results-5.6.txt (+1/-1) |
To merge this branch: | bzr merge lp:~percona-toolkit-dev/percona-toolkit/release-2.2.10 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Daniel Nichter | Pending | ||
Review via email: mp+230115@code.launchpad.net |
Commit message
Description of the change
Release branch for Percona Toolkit version 2.2.10
To post a comment you must log in.
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file 'Changelog' |
2 | --- Changelog 2014-07-08 12:06:12 +0000 |
3 | +++ Changelog 2014-08-08 14:10:17 +0000 |
4 | @@ -1,8 +1,16 @@ |
5 | Changelog for Percona Toolkit |
6 | |
7 | +v2.2.10 released 2014-08-06 |
8 | + |
9 | + * Fixed bug 1287253: pt-table-checksum deadlock |
10 | + * Fixed bug 1299387: 5.6 slow query log Thead_id becomes Id |
11 | + * Fixed bug 1311654: pt-table-checksum + PXC inconsistent results upon --resume |
12 | + * Fixed bug 1340728: pt-online-schema-change doesn't work with HASH indexes |
13 | + * Fixed bug 1253872: pt-table-checksum max load 20% rounds down |
14 | + * Fixed bug 1340364: some shell tools output error when queried for --version |
15 | + |
16 | v2.2.9 released 2014-07-08 |
17 | |
18 | - |
19 | * Fixed bug 1258135: pt-deadlock-logger introduces a noise to MySQL |
20 | * Fixed bug 1329422: pt-online-schema-change foreign-keys-method=none breaks constraints |
21 | * Fixed bug 1315130: pt-online-schema-change not properly detecting foreign keys |
22 | |
23 | === modified file 'Makefile.PL' |
24 | --- Makefile.PL 2014-07-08 12:06:12 +0000 |
25 | +++ Makefile.PL 2014-08-08 14:10:17 +0000 |
26 | @@ -2,7 +2,7 @@ |
27 | |
28 | WriteMakefile( |
29 | NAME => 'percona-toolkit', |
30 | - VERSION => '2.2.9', |
31 | + VERSION => '2.2.10', |
32 | EXE_FILES => [ <bin/*> ], |
33 | MAN1PODS => { |
34 | 'docs/percona-toolkit.pod' => 'blib/man1/percona-toolkit.1p', |
35 | |
36 | === modified file 'bin/pt-align' |
37 | --- bin/pt-align 2014-07-04 17:16:08 +0000 |
38 | +++ bin/pt-align 2014-08-08 14:10:17 +0000 |
39 | @@ -1323,6 +1323,6 @@ |
40 | |
41 | =head1 VERSION |
42 | |
43 | -pt-align 2.2.9 |
44 | +pt-align 2.2.10 |
45 | |
46 | =cut |
47 | |
48 | === modified file 'bin/pt-archiver' |
49 | --- bin/pt-archiver 2014-07-04 17:16:08 +0000 |
50 | +++ bin/pt-archiver 2014-08-08 14:10:17 +0000 |
51 | @@ -43,7 +43,7 @@ |
52 | { |
53 | package Percona::Toolkit; |
54 | |
55 | -our $VERSION = '2.2.9'; |
56 | +our $VERSION = '2.2.10'; |
57 | |
58 | use strict; |
59 | use warnings FATAL => 'all'; |
60 | @@ -7892,6 +7892,6 @@ |
61 | |
62 | =head1 VERSION |
63 | |
64 | -pt-archiver 2.2.9 |
65 | +pt-archiver 2.2.10 |
66 | |
67 | =cut |
68 | |
69 | === modified file 'bin/pt-config-diff' |
70 | --- bin/pt-config-diff 2014-07-04 17:16:08 +0000 |
71 | +++ bin/pt-config-diff 2014-08-08 14:10:17 +0000 |
72 | @@ -43,7 +43,7 @@ |
73 | { |
74 | package Percona::Toolkit; |
75 | |
76 | -our $VERSION = '2.2.9'; |
77 | +our $VERSION = '2.2.10'; |
78 | |
79 | use strict; |
80 | use warnings FATAL => 'all'; |
81 | @@ -5733,6 +5733,6 @@ |
82 | |
83 | =head1 VERSION |
84 | |
85 | -pt-config-diff 2.2.9 |
86 | +pt-config-diff 2.2.10 |
87 | |
88 | =cut |
89 | |
90 | === modified file 'bin/pt-deadlock-logger' |
91 | --- bin/pt-deadlock-logger 2014-07-04 17:16:08 +0000 |
92 | +++ bin/pt-deadlock-logger 2014-08-08 14:10:17 +0000 |
93 | @@ -42,7 +42,7 @@ |
94 | { |
95 | package Percona::Toolkit; |
96 | |
97 | -our $VERSION = '2.2.9'; |
98 | +our $VERSION = '2.2.10'; |
99 | |
100 | use strict; |
101 | use warnings FATAL => 'all'; |
102 | @@ -5523,6 +5523,6 @@ |
103 | |
104 | =head1 VERSION |
105 | |
106 | -pt-deadlock-logger 2.2.9 |
107 | +pt-deadlock-logger 2.2.10 |
108 | |
109 | =cut |
110 | |
111 | === modified file 'bin/pt-diskstats' |
112 | --- bin/pt-diskstats 2014-07-04 17:16:08 +0000 |
113 | +++ bin/pt-diskstats 2014-08-08 14:10:17 +0000 |
114 | @@ -38,7 +38,7 @@ |
115 | { |
116 | package Percona::Toolkit; |
117 | |
118 | -our $VERSION = '2.2.9'; |
119 | +our $VERSION = '2.2.10'; |
120 | |
121 | use strict; |
122 | use warnings FATAL => 'all'; |
123 | @@ -5579,6 +5579,6 @@ |
124 | |
125 | =head1 VERSION |
126 | |
127 | -pt-diskstats 2.2.9 |
128 | +pt-diskstats 2.2.10 |
129 | |
130 | =cut |
131 | |
132 | === modified file 'bin/pt-duplicate-key-checker' |
133 | --- bin/pt-duplicate-key-checker 2014-07-04 17:16:08 +0000 |
134 | +++ bin/pt-duplicate-key-checker 2014-08-08 14:10:17 +0000 |
135 | @@ -39,7 +39,7 @@ |
136 | { |
137 | package Percona::Toolkit; |
138 | |
139 | -our $VERSION = '2.2.9'; |
140 | +our $VERSION = '2.2.10'; |
141 | |
142 | use strict; |
143 | use warnings FATAL => 'all'; |
144 | @@ -5600,6 +5600,6 @@ |
145 | |
146 | =head1 VERSION |
147 | |
148 | -pt-duplicate-key-checker 2.2.9 |
149 | +pt-duplicate-key-checker 2.2.10 |
150 | |
151 | =cut |
152 | |
153 | === modified file 'bin/pt-fifo-split' |
154 | --- bin/pt-fifo-split 2014-07-04 17:16:08 +0000 |
155 | +++ bin/pt-fifo-split 2014-08-08 14:10:17 +0000 |
156 | @@ -1612,6 +1612,6 @@ |
157 | |
158 | =head1 VERSION |
159 | |
160 | -pt-fifo-split 2.2.9 |
161 | +pt-fifo-split 2.2.10 |
162 | |
163 | =cut |
164 | |
165 | === modified file 'bin/pt-find' |
166 | --- bin/pt-find 2014-07-04 17:16:08 +0000 |
167 | +++ bin/pt-find 2014-08-08 14:10:17 +0000 |
168 | @@ -35,7 +35,7 @@ |
169 | { |
170 | package Percona::Toolkit; |
171 | |
172 | -our $VERSION = '2.2.9'; |
173 | +our $VERSION = '2.2.10'; |
174 | |
175 | use strict; |
176 | use warnings FATAL => 'all'; |
177 | @@ -4984,6 +4984,6 @@ |
178 | |
179 | =head1 VERSION |
180 | |
181 | -pt-find 2.2.9 |
182 | +pt-find 2.2.10 |
183 | |
184 | =cut |
185 | |
186 | === modified file 'bin/pt-fingerprint' |
187 | --- bin/pt-fingerprint 2014-07-04 17:16:08 +0000 |
188 | +++ bin/pt-fingerprint 2014-08-08 14:10:17 +0000 |
189 | @@ -2203,6 +2203,6 @@ |
190 | |
191 | =head1 VERSION |
192 | |
193 | -pt-fingerprint 2.2.9 |
194 | +pt-fingerprint 2.2.10 |
195 | |
196 | =cut |
197 | |
198 | === modified file 'bin/pt-fk-error-logger' |
199 | --- bin/pt-fk-error-logger 2014-07-04 17:16:08 +0000 |
200 | +++ bin/pt-fk-error-logger 2014-08-08 14:10:17 +0000 |
201 | @@ -37,7 +37,7 @@ |
202 | { |
203 | package Percona::Toolkit; |
204 | |
205 | -our $VERSION = '2.2.9'; |
206 | +our $VERSION = '2.2.10'; |
207 | |
208 | use strict; |
209 | use warnings FATAL => 'all'; |
210 | @@ -4509,6 +4509,6 @@ |
211 | |
212 | =head1 VERSION |
213 | |
214 | -pt-fk-error-logger 2.2.9 |
215 | +pt-fk-error-logger 2.2.10 |
216 | |
217 | =cut |
218 | |
219 | === modified file 'bin/pt-heartbeat' |
220 | --- bin/pt-heartbeat 2014-07-04 17:16:08 +0000 |
221 | +++ bin/pt-heartbeat 2014-08-08 14:10:17 +0000 |
222 | @@ -38,7 +38,7 @@ |
223 | { |
224 | package Percona::Toolkit; |
225 | |
226 | -our $VERSION = '2.2.9'; |
227 | +our $VERSION = '2.2.10'; |
228 | |
229 | use strict; |
230 | use warnings FATAL => 'all'; |
231 | @@ -6192,6 +6192,6 @@ |
232 | |
233 | =head1 VERSION |
234 | |
235 | -pt-heartbeat 2.2.9 |
236 | +pt-heartbeat 2.2.10 |
237 | |
238 | =cut |
239 | |
240 | === modified file 'bin/pt-index-usage' |
241 | --- bin/pt-index-usage 2014-07-04 17:16:08 +0000 |
242 | +++ bin/pt-index-usage 2014-08-08 14:10:17 +0000 |
243 | @@ -45,7 +45,7 @@ |
244 | { |
245 | package Percona::Toolkit; |
246 | |
247 | -our $VERSION = '2.2.9'; |
248 | +our $VERSION = '2.2.10'; |
249 | |
250 | use strict; |
251 | use warnings FATAL => 'all'; |
252 | @@ -2772,12 +2772,13 @@ |
253 | my ( $class ) = @_; |
254 | my $self = { |
255 | pending => [], |
256 | + last_event_offset => undef, |
257 | }; |
258 | return bless $self, $class; |
259 | } |
260 | |
261 | my $slow_log_ts_line = qr/^# Time: ([0-9: ]{15})/; |
262 | -my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]/; |
263 | +my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]\s*(?:Id:\s*(\d+))?/; |
264 | my $slow_log_hd_line = qr{ |
265 | ^(?: |
266 | T[cC][pP]\s[pP]ort:\s+\d+ # case differs on windows/unix |
267 | @@ -2808,6 +2809,7 @@ |
268 | or defined($stmt = $next_event->()) |
269 | ) { |
270 | my @properties = ('cmd', 'Query', 'pos_in_log', $pos_in_log); |
271 | + $self->{last_event_offset} = $pos_in_log; |
272 | $pos_in_log = $tell->(); |
273 | |
274 | if ( $stmt =~ s/$slow_log_hd_line//go ){ # Throw away header lines in log |
275 | @@ -2840,19 +2842,25 @@ |
276 | push @properties, 'ts', $time; |
277 | ++$got_ts; |
278 | if ( !$got_uh |
279 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
280 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
281 | ) { |
282 | PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
283 | push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
284 | - ++$got_uh; |
285 | + if ( $thread_id ) { |
286 | + push @properties, 'Thread_id', $thread_id; |
287 | + } |
288 | + ++$got_uh; |
289 | } |
290 | } |
291 | |
292 | elsif ( !$got_uh |
293 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
294 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
295 | ) { |
296 | - PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
297 | - push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
298 | + PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
299 | + push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
300 | + if ( $thread_id ) { |
301 | + push @properties, 'Thread_id', $thread_id; |
302 | + } |
303 | ++$got_uh; |
304 | } |
305 | |
306 | @@ -2933,9 +2941,15 @@ |
307 | |
308 | PTDEBUG && _d('Properties of event:', Dumper(\@properties)); |
309 | my $event = { @properties }; |
310 | - if ( $args{stats} ) { |
311 | - $args{stats}->{events_read}++; |
312 | - $args{stats}->{events_parsed}++; |
313 | + if ( !$event->{arg} ) { |
314 | + PTDEBUG && _d('Partial event, no arg'); |
315 | + } |
316 | + else { |
317 | + $self->{last_event_offset} = undef; |
318 | + if ( $args{stats} ) { |
319 | + $args{stats}->{events_read}++; |
320 | + $args{stats}->{events_parsed}++; |
321 | + } |
322 | } |
323 | return $event; |
324 | } # EVENT |
325 | @@ -7532,6 +7546,6 @@ |
326 | |
327 | =head1 VERSION |
328 | |
329 | -pt-index-usage 2.2.9 |
330 | +pt-index-usage 2.2.10 |
331 | |
332 | =cut |
333 | |
334 | === modified file 'bin/pt-ioprofile' |
335 | --- bin/pt-ioprofile 2014-07-04 17:16:08 +0000 |
336 | +++ bin/pt-ioprofile 2014-08-08 14:10:17 +0000 |
337 | @@ -1119,7 +1119,7 @@ |
338 | |
339 | =head1 VERSION |
340 | |
341 | -pt-ioprofile 2.2.9 |
342 | +pt-ioprofile 2.2.10 |
343 | |
344 | =cut |
345 | |
346 | |
347 | === modified file 'bin/pt-kill' |
348 | --- bin/pt-kill 2014-07-04 17:16:08 +0000 |
349 | +++ bin/pt-kill 2014-08-08 14:10:17 +0000 |
350 | @@ -47,7 +47,7 @@ |
351 | { |
352 | package Percona::Toolkit; |
353 | |
354 | -our $VERSION = '2.2.9'; |
355 | +our $VERSION = '2.2.10'; |
356 | |
357 | use strict; |
358 | use warnings FATAL => 'all'; |
359 | @@ -8183,6 +8183,6 @@ |
360 | |
361 | =head1 VERSION |
362 | |
363 | -pt-kill 2.2.9 |
364 | +pt-kill 2.2.10 |
365 | |
366 | =cut |
367 | |
368 | === modified file 'bin/pt-mext' |
369 | --- bin/pt-mext 2014-07-04 17:16:08 +0000 |
370 | +++ bin/pt-mext 2014-08-08 14:10:17 +0000 |
371 | @@ -127,9 +127,10 @@ |
372 | |
373 | usage_or_errors() { |
374 | local file="$1" |
375 | + local version="" |
376 | |
377 | if [ "$OPT_VERSION" ]; then |
378 | - local version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
379 | + version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
380 | echo "$version" |
381 | return 1 |
382 | fi |
383 | @@ -791,7 +792,7 @@ |
384 | |
385 | =head1 VERSION |
386 | |
387 | -pt-mext 2.2.9 |
388 | +pt-mext 2.2.10 |
389 | |
390 | =cut |
391 | |
392 | |
393 | === modified file 'bin/pt-mysql-summary' |
394 | --- bin/pt-mysql-summary 2014-07-04 17:16:08 +0000 |
395 | +++ bin/pt-mysql-summary 2014-08-08 14:10:17 +0000 |
396 | @@ -88,9 +88,10 @@ |
397 | |
398 | usage_or_errors() { |
399 | local file="$1" |
400 | + local version="" |
401 | |
402 | if [ "$OPT_VERSION" ]; then |
403 | - local version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
404 | + version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
405 | echo "$version" |
406 | return 1 |
407 | fi |
408 | @@ -3089,7 +3090,7 @@ |
409 | |
410 | =head1 VERSION |
411 | |
412 | -pt-mysql-summary 2.2.9 |
413 | +pt-mysql-summary 2.2.10 |
414 | |
415 | =cut |
416 | |
417 | |
418 | === modified file 'bin/pt-online-schema-change' |
419 | --- bin/pt-online-schema-change 2014-07-04 17:16:08 +0000 |
420 | +++ bin/pt-online-schema-change 2014-08-08 14:10:17 +0000 |
421 | @@ -54,7 +54,7 @@ |
422 | { |
423 | package Percona::Toolkit; |
424 | |
425 | -our $VERSION = '2.2.9'; |
426 | +our $VERSION = '2.2.10'; |
427 | |
428 | use strict; |
429 | use warnings FATAL => 'all'; |
430 | @@ -4785,6 +4785,7 @@ |
431 | |
432 | use strict; |
433 | use warnings FATAL => 'all'; |
434 | +use POSIX qw( ceil ); |
435 | use English qw(-no_match_vars); |
436 | use constant PTDEBUG => $ENV{PTDEBUG} || 0; |
437 | |
438 | @@ -4950,8 +4951,8 @@ |
439 | } |
440 | else { |
441 | PTDEBUG && _d('Initial', $var, 'value:', $init_val); |
442 | - $val = int(($init_val * $threshold_factor) + $init_val); |
443 | - $vars->{$var} = $val; |
444 | + $val = ($init_val * $threshold_factor) + $init_val; |
445 | + $vars->{$var} = int(ceil($val)); |
446 | } |
447 | PTDEBUG && _d('Wait if', $var, '>=', $val); |
448 | } |
449 | @@ -8908,28 +8909,31 @@ |
450 | # boundary sql. This check applies to the next nibble. So if |
451 | # the current nibble number is 5, then nibble 5 is already done |
452 | # and we're checking nibble number 6. |
453 | - my $expl = explain_statement( |
454 | - tbl => $tbl, |
455 | - sth => $sth->{explain_upper_boundary}, |
456 | - vals => [ @{$boundary->{lower}}, $nibble_iter->limit() ], |
457 | - ); |
458 | - if (lc($expl->{key} || '') ne lc($nibble_iter->nibble_index() || '')) { |
459 | - my $msg |
460 | - = "Aborting copying table $tbl->{name} at chunk " |
461 | - . ($nibble_iter->nibble_number() + 1) |
462 | - . " because it is not safe to ascend. Chunking should " |
463 | - . "use the " |
464 | - . ($nibble_iter->nibble_index() || '?') |
465 | - . " index, but MySQL EXPLAIN reports that " |
466 | - . ($expl->{key} ? "the $expl->{key}" : "no") |
467 | - . " index will be used for " |
468 | - . $sth->{upper_boundary}->{Statement} |
469 | - . " with values " |
470 | - . join(", ", map { defined $_ ? $_ : "NULL" } |
471 | - (@{$boundary->{lower}}, $nibble_iter->limit())) |
472 | - . "\n"; |
473 | - die ts($msg); |
474 | - } |
475 | + # Skip if --nocheck-plan See: https://bugs.launchpad.net/percona-toolkit/+bug/1340728 |
476 | + if ( $o->get('check-plan') ) { |
477 | + my $expl = explain_statement( |
478 | + tbl => $tbl, |
479 | + sth => $sth->{explain_upper_boundary}, |
480 | + vals => [ @{$boundary->{lower}}, $nibble_iter->limit() ], |
481 | + ); |
482 | + if ( lc($expl->{key} || '') ne lc($nibble_iter->nibble_index() || '') ) { |
483 | + my $msg |
484 | + = "Aborting copying table $tbl->{name} at chunk " |
485 | + . ($nibble_iter->nibble_number() + 1) |
486 | + . " because it is not safe to ascend. Chunking should " |
487 | + . "use the " |
488 | + . ($nibble_iter->nibble_index() || '?') |
489 | + . " index, but MySQL EXPLAIN reports that " |
490 | + . ($expl->{key} ? "the $expl->{key}" : "no") |
491 | + . " index will be used for " |
492 | + . $sth->{upper_boundary}->{Statement} |
493 | + . " with values " |
494 | + . join(", ", map { defined $_ ? $_ : "NULL" } |
495 | + (@{$boundary->{lower}}, $nibble_iter->limit())) |
496 | + . "\n"; |
497 | + die ts($msg); |
498 | + } |
499 | + } |
500 | |
501 | # Once nibbling begins for a table, control does not return to this |
502 | # tool until nibbling is done because, as noted above, all work is |
503 | @@ -9531,15 +9535,17 @@ |
504 | vals => [ @{$boundary->{lower}}, @{$boundary->{upper}} ], |
505 | ); |
506 | |
507 | - # Ensure that MySQL is using the chunk index if the table is being chunked. |
508 | - if ( !$nibble_iter->one_nibble() |
509 | - && lc($expl->{key} || '') ne lc($nibble_iter->nibble_index() || '') ) |
510 | - { |
511 | - die ts("Error copying rows at chunk " . $nibble_iter->nibble_number() |
512 | - . " of $tbl->{db}.$tbl->{tbl} because MySQL chose " |
513 | - . ($expl->{key} ? "the $expl->{key}" : "no") . " index " |
514 | - . " instead of the " . $nibble_iter->nibble_index() . "index.\n"); |
515 | - } |
516 | + # Ensure that MySQL is using the chunk index if the table is being chunked. |
517 | + # Skip if --nocheck-plan See: https://bugs.launchpad.net/percona-toolkit/+bug/1340728 |
518 | + if ( !$nibble_iter->one_nibble() |
519 | + && lc($expl->{key} || '') ne lc($nibble_iter->nibble_index() || '') |
520 | + && $o->get('check-plan') ) |
521 | + { |
522 | + die ts("Error copying rows at chunk " . $nibble_iter->nibble_number() |
523 | + . " of $tbl->{db}.$tbl->{tbl} because MySQL chose " |
524 | + . ($expl->{key} ? "the $expl->{key}" : "no") . " index " |
525 | + . " instead of the " . $nibble_iter->nibble_index() . "index.\n"); |
526 | + } |
527 | |
528 | # Ensure that the chunk isn't too large if there's a --chunk-size-limit. |
529 | # If single-chunking the table, this has already been checked, so it |
530 | @@ -9564,11 +9570,13 @@ |
531 | } |
532 | } |
533 | |
534 | - # Ensure that MySQL is still using the entire index. |
535 | + # Ensure that MySQL is still using the entire index. |
536 | # https://bugs.launchpad.net/percona-toolkit/+bug/1010232 |
537 | + # Skip if --nocheck-plan See: https://bugs.launchpad.net/percona-toolkit/+bug/1340728 |
538 | if ( !$nibble_iter->one_nibble() |
539 | && $tbl->{key_len} |
540 | - && ($expl->{key_len} || 0) < $tbl->{key_len} ) |
541 | + && ($expl->{key_len} || 0) < $tbl->{key_len} |
542 | + && $o->get('check-plan') ) |
543 | { |
544 | die ts("Error copying rows at chunk " . $nibble_iter->nibble_number() |
545 | . " of $tbl->{db}.$tbl->{tbl} because MySQL used " |
546 | @@ -11555,6 +11563,6 @@ |
547 | |
548 | =head1 VERSION |
549 | |
550 | -pt-online-schema-change 2.2.9 |
551 | +pt-online-schema-change 2.2.10 |
552 | |
553 | =cut |
554 | |
555 | === modified file 'bin/pt-pmp' |
556 | --- bin/pt-pmp 2014-07-04 17:16:08 +0000 |
557 | +++ bin/pt-pmp 2014-08-08 14:10:17 +0000 |
558 | @@ -889,7 +889,7 @@ |
559 | |
560 | =head1 VERSION |
561 | |
562 | -pt-pmp 2.2.9 |
563 | +pt-pmp 2.2.10 |
564 | |
565 | =cut |
566 | |
567 | |
568 | === modified file 'bin/pt-query-digest' |
569 | --- bin/pt-query-digest 2014-07-04 17:16:08 +0000 |
570 | +++ bin/pt-query-digest 2014-08-08 14:10:17 +0000 |
571 | @@ -64,7 +64,7 @@ |
572 | { |
573 | package Percona::Toolkit; |
574 | |
575 | -our $VERSION = '2.2.9'; |
576 | +our $VERSION = '2.2.10'; |
577 | |
578 | use strict; |
579 | use warnings FATAL => 'all'; |
580 | @@ -4972,7 +4972,7 @@ |
581 | } |
582 | |
583 | my $slow_log_ts_line = qr/^# Time: ([0-9: ]{15})/; |
584 | -my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]/; |
585 | +my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]\s*(?:Id:\s*(\d+))?/; |
586 | my $slow_log_hd_line = qr{ |
587 | ^(?: |
588 | T[cC][pP]\s[pP]ort:\s+\d+ # case differs on windows/unix |
589 | @@ -5036,19 +5036,25 @@ |
590 | push @properties, 'ts', $time; |
591 | ++$got_ts; |
592 | if ( !$got_uh |
593 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
594 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
595 | ) { |
596 | PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
597 | push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
598 | - ++$got_uh; |
599 | + if ( $thread_id ) { |
600 | + push @properties, 'Thread_id', $thread_id; |
601 | + } |
602 | + ++$got_uh; |
603 | } |
604 | } |
605 | |
606 | elsif ( !$got_uh |
607 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
608 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
609 | ) { |
610 | - PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
611 | - push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
612 | + PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
613 | + push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
614 | + if ( $thread_id ) { |
615 | + push @properties, 'Thread_id', $thread_id; |
616 | + } |
617 | ++$got_uh; |
618 | } |
619 | |
620 | @@ -16587,6 +16593,6 @@ |
621 | |
622 | =head1 VERSION |
623 | |
624 | -pt-query-digest 2.2.9 |
625 | +pt-query-digest 2.2.10 |
626 | |
627 | =cut |
628 | |
629 | === modified file 'bin/pt-show-grants' |
630 | --- bin/pt-show-grants 2014-07-04 17:16:08 +0000 |
631 | +++ bin/pt-show-grants 2014-08-08 14:10:17 +0000 |
632 | @@ -2406,6 +2406,6 @@ |
633 | |
634 | =head1 VERSION |
635 | |
636 | -pt-show-grants 2.2.9 |
637 | +pt-show-grants 2.2.10 |
638 | |
639 | =cut |
640 | |
641 | === modified file 'bin/pt-sift' |
642 | --- bin/pt-sift 2014-07-04 17:16:08 +0000 |
643 | +++ bin/pt-sift 2014-08-08 14:10:17 +0000 |
644 | @@ -1237,7 +1237,7 @@ |
645 | |
646 | =head1 VERSION |
647 | |
648 | -pt-sift 2.2.9 |
649 | +pt-sift 2.2.10 |
650 | |
651 | =cut |
652 | |
653 | |
654 | === modified file 'bin/pt-slave-delay' |
655 | --- bin/pt-slave-delay 2014-07-04 17:16:08 +0000 |
656 | +++ bin/pt-slave-delay 2014-08-08 14:10:17 +0000 |
657 | @@ -40,7 +40,7 @@ |
658 | { |
659 | package Percona::Toolkit; |
660 | |
661 | -our $VERSION = '2.2.9'; |
662 | +our $VERSION = '2.2.10'; |
663 | |
664 | use strict; |
665 | use warnings FATAL => 'all'; |
666 | @@ -4869,6 +4869,6 @@ |
667 | |
668 | =head1 VERSION |
669 | |
670 | -pt-slave-delay 2.2.9 |
671 | +pt-slave-delay 2.2.10 |
672 | |
673 | =cut |
674 | |
675 | === modified file 'bin/pt-slave-find' |
676 | --- bin/pt-slave-find 2014-07-04 17:16:08 +0000 |
677 | +++ bin/pt-slave-find 2014-08-08 14:10:17 +0000 |
678 | @@ -4334,6 +4334,6 @@ |
679 | |
680 | =head1 VERSION |
681 | |
682 | -pt-slave-find 2.2.9 |
683 | +pt-slave-find 2.2.10 |
684 | |
685 | =cut |
686 | |
687 | === modified file 'bin/pt-slave-restart' |
688 | --- bin/pt-slave-restart 2014-07-04 17:16:08 +0000 |
689 | +++ bin/pt-slave-restart 2014-08-08 14:10:17 +0000 |
690 | @@ -41,7 +41,7 @@ |
691 | { |
692 | package Percona::Toolkit; |
693 | |
694 | -our $VERSION = '2.2.9'; |
695 | +our $VERSION = '2.2.10'; |
696 | |
697 | use strict; |
698 | use warnings FATAL => 'all'; |
699 | @@ -5937,6 +5937,6 @@ |
700 | |
701 | =head1 VERSION |
702 | |
703 | -pt-slave-restart 2.2.9 |
704 | +pt-slave-restart 2.2.10 |
705 | |
706 | =cut |
707 | |
708 | === modified file 'bin/pt-stalk' |
709 | --- bin/pt-stalk 2014-07-04 17:16:08 +0000 |
710 | +++ bin/pt-stalk 2014-08-08 14:10:17 +0000 |
711 | @@ -2211,7 +2211,7 @@ |
712 | |
713 | =head1 VERSION |
714 | |
715 | -pt-stalk 2.2.9 |
716 | +pt-stalk 2.2.10 |
717 | |
718 | =cut |
719 | |
720 | |
721 | === modified file 'bin/pt-summary' |
722 | --- bin/pt-summary 2014-07-04 17:16:08 +0000 |
723 | +++ bin/pt-summary 2014-08-08 14:10:17 +0000 |
724 | @@ -95,9 +95,10 @@ |
725 | |
726 | usage_or_errors() { |
727 | local file="$1" |
728 | + local version="" |
729 | |
730 | if [ "$OPT_VERSION" ]; then |
731 | - local version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
732 | + version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
733 | echo "$version" |
734 | return 1 |
735 | fi |
736 | @@ -2690,7 +2691,7 @@ |
737 | |
738 | =head1 VERSION |
739 | |
740 | -pt-summary 2.2.9 |
741 | +pt-summary 2.2.10 |
742 | |
743 | =cut |
744 | |
745 | |
746 | === modified file 'bin/pt-table-checksum' |
747 | --- bin/pt-table-checksum 2014-07-04 17:16:08 +0000 |
748 | +++ bin/pt-table-checksum 2014-08-08 14:10:17 +0000 |
749 | @@ -57,7 +57,7 @@ |
750 | { |
751 | package Percona::Toolkit; |
752 | |
753 | -our $VERSION = '2.2.9'; |
754 | +our $VERSION = '2.2.10'; |
755 | |
756 | use strict; |
757 | use warnings FATAL => 'all'; |
758 | @@ -5977,19 +5977,28 @@ |
759 | } |
760 | my ($dbh, $repl_table) = @args{@required_args}; |
761 | |
762 | - my $sql |
763 | - = "SELECT CONCAT(db, '.', tbl) AS `table`, " |
764 | - . "chunk, chunk_index, lower_boundary, upper_boundary, " |
765 | - . "COALESCE(this_cnt-master_cnt, 0) AS cnt_diff, " |
766 | - . "COALESCE(" |
767 | - . "this_crc <> master_crc OR ISNULL(master_crc) <> ISNULL(this_crc), 0" |
768 | - . ") AS crc_diff, this_cnt, master_cnt, this_crc, master_crc " |
769 | - . "FROM $repl_table " |
770 | - . "WHERE (master_cnt <> this_cnt OR master_crc <> this_crc " |
771 | - . "OR ISNULL(master_crc) <> ISNULL(this_crc))" |
772 | - . ($args{where} ? " AND ($args{where})" : ""); |
773 | - PTDEBUG && _d($sql); |
774 | - my $diffs = $dbh->selectall_arrayref($sql, { Slice => {} }); |
775 | + |
776 | + my $tries = $self->{'OptionParser'}->get('replicate-check-retries') || 1; |
777 | + my $diffs; |
778 | + while ($tries--) { |
779 | + my $sql |
780 | + = "SELECT CONCAT(db, '.', tbl) AS `table`, " |
781 | + . "chunk, chunk_index, lower_boundary, upper_boundary, " |
782 | + . "COALESCE(this_cnt-master_cnt, 0) AS cnt_diff, " |
783 | + . "COALESCE(" |
784 | + . "this_crc <> master_crc OR ISNULL(master_crc) <> ISNULL(this_crc), 0" |
785 | + . ") AS crc_diff, this_cnt, master_cnt, this_crc, master_crc " |
786 | + . "FROM $repl_table " |
787 | + . "WHERE (master_cnt <> this_cnt OR master_crc <> this_crc " |
788 | + . "OR ISNULL(master_crc) <> ISNULL(this_crc)) " |
789 | + . ($args{where} ? " AND ($args{where})" : ""); |
790 | + PTDEBUG && _d($sql); |
791 | + $diffs = $dbh->selectall_arrayref($sql, { Slice => {} }); |
792 | + if (!@$diffs || !$tries) { # if no differences are found OR we are out of tries left... |
793 | + last; # get out now |
794 | + } |
795 | + sleep 1; |
796 | + } |
797 | return $diffs; |
798 | } |
799 | |
800 | @@ -8347,6 +8356,7 @@ |
801 | |
802 | use strict; |
803 | use warnings FATAL => 'all'; |
804 | +use POSIX qw( ceil ); |
805 | use English qw(-no_match_vars); |
806 | use constant PTDEBUG => $ENV{PTDEBUG} || 0; |
807 | |
808 | @@ -8512,8 +8522,8 @@ |
809 | } |
810 | else { |
811 | PTDEBUG && _d('Initial', $var, 'value:', $init_val); |
812 | - $val = int(($init_val * $threshold_factor) + $init_val); |
813 | - $vars->{$var} = $val; |
814 | + $val = ($init_val * $threshold_factor) + $init_val; |
815 | + $vars->{$var} = int(ceil($val)); |
816 | } |
817 | PTDEBUG && _d('Wait if', $var, '>=', $val); |
818 | } |
819 | @@ -8964,6 +8974,7 @@ |
820 | $print_header = 1; |
821 | $exit_status = 0; |
822 | |
823 | + |
824 | # ######################################################################## |
825 | # Get configuration information. |
826 | # ######################################################################## |
827 | @@ -10072,6 +10083,7 @@ |
828 | # Should be done automatically, but I like to be explicit. |
829 | $fetch_sth->finish(); |
830 | $update_sth->finish(); |
831 | + $delete_sth->finish(); |
832 | |
833 | # Update rate, chunk size, and progress if the nibble actually |
834 | # selected some rows. |
835 | @@ -10382,6 +10394,7 @@ |
836 | return $msg ? "$ts $msg" : $ts; |
837 | } |
838 | |
839 | + |
840 | sub nibble_is_safe { |
841 | my (%args) = @_; |
842 | my @required_args = qw(Cxn tbl NibbleIterator OptionParser); |
843 | @@ -10561,6 +10574,7 @@ |
844 | |
845 | if ( $error =~ m/Lock wait timeout exceeded/ |
846 | || $error =~ m/Query execution was interrupted/ |
847 | + || $error =~ m/Deadlock found/ |
848 | ) { |
849 | # These errors/warnings can be retried, so don't print |
850 | # a warning yet; do that in final_fail. |
851 | @@ -10584,8 +10598,9 @@ |
852 | my (%args) = @_; |
853 | my $error = $args{error}; |
854 | |
855 | - if ( $error =~ /Lock wait timeout exceeded/ |
856 | - || $error =~ /Query execution was interrupted/ |
857 | + if ( $error =~ m/Lock wait timeout exceeded/ |
858 | + || $error =~ m/Query execution was interrupted/ |
859 | + || $error =~ m/Deadlock found/ |
860 | ) { |
861 | # These errors/warnings are not fatal but only cause this |
862 | # nibble to be skipped. |
863 | @@ -12287,6 +12302,15 @@ |
864 | if you run pt-table-checksum quietly in a cron job, for example, and later want |
865 | a report on the results of the cron job, perhaps to implement a Nagios check. |
866 | |
867 | +=item --replicate-check-retries |
868 | + |
869 | +type: int; default: 1 |
870 | + |
871 | +Retry checksum comparison this many times when a difference is encountered. |
872 | +Only when a difference persists after this number of checks is it considered valid. |
873 | +Using this option with a value of 2 or more alleviates spurious differences that |
874 | +arise when using the --resume option. |
875 | + |
876 | =item --replicate-database |
877 | |
878 | type: string |
879 | @@ -12674,6 +12698,6 @@ |
880 | |
881 | =head1 VERSION |
882 | |
883 | -pt-table-checksum 2.2.9 |
884 | +pt-table-checksum 2.2.10 |
885 | |
886 | =cut |
887 | |
888 | === modified file 'bin/pt-table-sync' |
889 | --- bin/pt-table-sync 2014-07-04 17:16:08 +0000 |
890 | +++ bin/pt-table-sync 2014-08-08 14:10:17 +0000 |
891 | @@ -55,7 +55,7 @@ |
892 | { |
893 | package Percona::Toolkit; |
894 | |
895 | -our $VERSION = '2.2.9'; |
896 | +our $VERSION = '2.2.10'; |
897 | |
898 | use strict; |
899 | use warnings FATAL => 'all'; |
900 | @@ -12768,6 +12768,6 @@ |
901 | |
902 | =head1 VERSION |
903 | |
904 | -pt-table-sync 2.2.9 |
905 | +pt-table-sync 2.2.10 |
906 | |
907 | =cut |
908 | |
909 | === modified file 'bin/pt-table-usage' |
910 | --- bin/pt-table-usage 2014-07-04 17:16:08 +0000 |
911 | +++ bin/pt-table-usage 2014-08-08 14:10:17 +0000 |
912 | @@ -1556,12 +1556,13 @@ |
913 | my ( $class ) = @_; |
914 | my $self = { |
915 | pending => [], |
916 | + last_event_offset => undef, |
917 | }; |
918 | return bless $self, $class; |
919 | } |
920 | |
921 | my $slow_log_ts_line = qr/^# Time: ([0-9: ]{15})/; |
922 | -my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]/; |
923 | +my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]\s*(?:Id:\s*(\d+))?/; |
924 | my $slow_log_hd_line = qr{ |
925 | ^(?: |
926 | T[cC][pP]\s[pP]ort:\s+\d+ # case differs on windows/unix |
927 | @@ -1592,6 +1593,7 @@ |
928 | or defined($stmt = $next_event->()) |
929 | ) { |
930 | my @properties = ('cmd', 'Query', 'pos_in_log', $pos_in_log); |
931 | + $self->{last_event_offset} = $pos_in_log; |
932 | $pos_in_log = $tell->(); |
933 | |
934 | if ( $stmt =~ s/$slow_log_hd_line//go ){ # Throw away header lines in log |
935 | @@ -1624,19 +1626,25 @@ |
936 | push @properties, 'ts', $time; |
937 | ++$got_ts; |
938 | if ( !$got_uh |
939 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
940 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
941 | ) { |
942 | PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
943 | push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
944 | - ++$got_uh; |
945 | + if ( $thread_id ) { |
946 | + push @properties, 'Thread_id', $thread_id; |
947 | + } |
948 | + ++$got_uh; |
949 | } |
950 | } |
951 | |
952 | elsif ( !$got_uh |
953 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
954 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
955 | ) { |
956 | - PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
957 | - push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
958 | + PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
959 | + push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
960 | + if ( $thread_id ) { |
961 | + push @properties, 'Thread_id', $thread_id; |
962 | + } |
963 | ++$got_uh; |
964 | } |
965 | |
966 | @@ -1676,14 +1684,25 @@ |
967 | |
968 | if ( !$found_arg && $pos == $len ) { |
969 | PTDEBUG && _d("Did not find arg, looking for special cases"); |
970 | - local $INPUT_RECORD_SEPARATOR = ";\n"; |
971 | + local $INPUT_RECORD_SEPARATOR = ";\n"; # get next line |
972 | if ( defined(my $l = $next_event->()) ) { |
973 | - chomp $l; |
974 | - $l =~ s/^\s+//; |
975 | - PTDEBUG && _d("Found admin statement", $l); |
976 | - push @properties, 'cmd', 'Admin', 'arg', $l; |
977 | - push @properties, 'bytes', length($properties[-1]); |
978 | - $found_arg++; |
979 | + if ( $l =~ /^\s*[A-Z][a-z_]+: / ) { |
980 | + PTDEBUG && _d("Found NULL query before", $l); |
981 | + local $INPUT_RECORD_SEPARATOR = ";\n#"; |
982 | + my $rest_of_event = $next_event->(); |
983 | + push @{$self->{pending}}, $l . $rest_of_event; |
984 | + push @properties, 'cmd', 'Query', 'arg', '/* No query */'; |
985 | + push @properties, 'bytes', 0; |
986 | + $found_arg++; |
987 | + } |
988 | + else { |
989 | + chomp $l; |
990 | + $l =~ s/^\s+//; |
991 | + PTDEBUG && _d("Found admin statement", $l); |
992 | + push @properties, 'cmd', 'Admin', 'arg', $l; |
993 | + push @properties, 'bytes', length($properties[-1]); |
994 | + $found_arg++; |
995 | + } |
996 | } |
997 | else { |
998 | PTDEBUG && _d("I can't figure out what to do with this line"); |
999 | @@ -1706,9 +1725,15 @@ |
1000 | |
1001 | PTDEBUG && _d('Properties of event:', Dumper(\@properties)); |
1002 | my $event = { @properties }; |
1003 | - if ( $args{stats} ) { |
1004 | - $args{stats}->{events_read}++; |
1005 | - $args{stats}->{events_parsed}++; |
1006 | + if ( !$event->{arg} ) { |
1007 | + PTDEBUG && _d('Partial event, no arg'); |
1008 | + } |
1009 | + else { |
1010 | + $self->{last_event_offset} = undef; |
1011 | + if ( $args{stats} ) { |
1012 | + $args{stats}->{events_read}++; |
1013 | + $args{stats}->{events_parsed}++; |
1014 | + } |
1015 | } |
1016 | return $event; |
1017 | } # EVENT |
1018 | @@ -7536,6 +7561,6 @@ |
1019 | |
1020 | =head1 VERSION |
1021 | |
1022 | -pt-table-usage 2.2.9 |
1023 | +pt-table-usage 2.2.10 |
1024 | |
1025 | =cut |
1026 | |
1027 | === modified file 'bin/pt-upgrade' |
1028 | --- bin/pt-upgrade 2014-07-04 17:16:08 +0000 |
1029 | +++ bin/pt-upgrade 2014-08-08 14:10:17 +0000 |
1030 | @@ -61,7 +61,7 @@ |
1031 | { |
1032 | package Percona::Toolkit; |
1033 | |
1034 | -our $VERSION = '2.2.9'; |
1035 | +our $VERSION = '2.2.10'; |
1036 | |
1037 | use strict; |
1038 | use warnings FATAL => 'all'; |
1039 | @@ -6537,12 +6537,13 @@ |
1040 | my ( $class ) = @_; |
1041 | my $self = { |
1042 | pending => [], |
1043 | + last_event_offset => undef, |
1044 | }; |
1045 | return bless $self, $class; |
1046 | } |
1047 | |
1048 | my $slow_log_ts_line = qr/^# Time: ([0-9: ]{15})/; |
1049 | -my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]/; |
1050 | +my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]\s*(?:Id:\s*(\d+))?/; |
1051 | my $slow_log_hd_line = qr{ |
1052 | ^(?: |
1053 | T[cC][pP]\s[pP]ort:\s+\d+ # case differs on windows/unix |
1054 | @@ -6573,6 +6574,7 @@ |
1055 | or defined($stmt = $next_event->()) |
1056 | ) { |
1057 | my @properties = ('cmd', 'Query', 'pos_in_log', $pos_in_log); |
1058 | + $self->{last_event_offset} = $pos_in_log; |
1059 | $pos_in_log = $tell->(); |
1060 | |
1061 | if ( $stmt =~ s/$slow_log_hd_line//go ){ # Throw away header lines in log |
1062 | @@ -6605,19 +6607,25 @@ |
1063 | push @properties, 'ts', $time; |
1064 | ++$got_ts; |
1065 | if ( !$got_uh |
1066 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
1067 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
1068 | ) { |
1069 | PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
1070 | push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
1071 | - ++$got_uh; |
1072 | + if ( $thread_id ) { |
1073 | + push @properties, 'Thread_id', $thread_id; |
1074 | + } |
1075 | + ++$got_uh; |
1076 | } |
1077 | } |
1078 | |
1079 | elsif ( !$got_uh |
1080 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
1081 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
1082 | ) { |
1083 | - PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
1084 | - push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
1085 | + PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
1086 | + push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
1087 | + if ( $thread_id ) { |
1088 | + push @properties, 'Thread_id', $thread_id; |
1089 | + } |
1090 | ++$got_uh; |
1091 | } |
1092 | |
1093 | @@ -6698,9 +6706,15 @@ |
1094 | |
1095 | PTDEBUG && _d('Properties of event:', Dumper(\@properties)); |
1096 | my $event = { @properties }; |
1097 | - if ( $args{stats} ) { |
1098 | - $args{stats}->{events_read}++; |
1099 | - $args{stats}->{events_parsed}++; |
1100 | + if ( !$event->{arg} ) { |
1101 | + PTDEBUG && _d('Partial event, no arg'); |
1102 | + } |
1103 | + else { |
1104 | + $self->{last_event_offset} = undef; |
1105 | + if ( $args{stats} ) { |
1106 | + $args{stats}->{events_read}++; |
1107 | + $args{stats}->{events_parsed}++; |
1108 | + } |
1109 | } |
1110 | return $event; |
1111 | } # EVENT |
1112 | @@ -11224,6 +11238,6 @@ |
1113 | |
1114 | =head1 VERSION |
1115 | |
1116 | -pt-upgrade 2.2.9 |
1117 | +pt-upgrade 2.2.10 |
1118 | |
1119 | =cut |
1120 | |
1121 | === modified file 'bin/pt-variable-advisor' |
1122 | --- bin/pt-variable-advisor 2014-07-04 17:16:08 +0000 |
1123 | +++ bin/pt-variable-advisor 2014-08-08 14:10:17 +0000 |
1124 | @@ -44,7 +44,7 @@ |
1125 | { |
1126 | package Percona::Toolkit; |
1127 | |
1128 | -our $VERSION = '2.2.9'; |
1129 | +our $VERSION = '2.2.10'; |
1130 | |
1131 | use strict; |
1132 | use warnings FATAL => 'all'; |
1133 | @@ -6138,6 +6138,6 @@ |
1134 | |
1135 | =head1 VERSION |
1136 | |
1137 | -pt-variable-advisor 2.2.9 |
1138 | +pt-variable-advisor 2.2.10 |
1139 | |
1140 | =cut |
1141 | |
1142 | === modified file 'bin/pt-visual-explain' |
1143 | --- bin/pt-visual-explain 2014-07-04 17:16:08 +0000 |
1144 | +++ bin/pt-visual-explain 2014-08-08 14:10:17 +0000 |
1145 | @@ -3243,6 +3243,6 @@ |
1146 | |
1147 | =head1 VERSION |
1148 | |
1149 | -pt-visual-explain 2.2.9 |
1150 | +pt-visual-explain 2.2.10 |
1151 | |
1152 | =cut |
1153 | |
1154 | === modified file 'config/deb/changelog' |
1155 | --- config/deb/changelog 2014-07-08 12:06:12 +0000 |
1156 | +++ config/deb/changelog 2014-08-08 14:10:17 +0000 |
1157 | @@ -1,3 +1,14 @@ |
1158 | +percona-toolkit (2.2.10) unstable; urgency=low |
1159 | + |
1160 | + * Fixed bug 1287253: pt-table-checksum deadlock |
1161 | + * Fixed bug 1299387: 5.6 slow query log Thead_id becomes Id |
1162 | + * Fixed bug 1311654: pt-table-checksum + PXC inconsistent results upon --resume |
1163 | + * Fixed bug 1340728: pt-online-schema-change doesn't work with HASH indexes |
1164 | + * Fixed bug 1253872: pt-table-checksum max load 20% rounds down |
1165 | + * Fixed bug 1340364: some shell tools output error when queried for --version |
1166 | + |
1167 | + -- Percona Toolkit Developers <toolkit-dev@percona.com> Wed, 06 Aug 2014 19:53:53 +0000 |
1168 | + |
1169 | percona-toolkit (2.2.9) unstable; urgency=low |
1170 | |
1171 | * Fixed bug 1258135: pt-deadlock-logger introduces a noise to MySQL |
1172 | |
1173 | === modified file 'config/sphinx-build/conf.py' |
1174 | --- config/sphinx-build/conf.py 2014-07-08 12:06:12 +0000 |
1175 | +++ config/sphinx-build/conf.py 2014-08-08 14:10:17 +0000 |
1176 | @@ -50,7 +50,7 @@ |
1177 | # The short X.Y version. |
1178 | version = '2.2' |
1179 | # The full version, including alpha/beta/rc tags. |
1180 | -release = '2.2.9' |
1181 | +release = '2.2.10' |
1182 | |
1183 | # The language for content autogenerated by Sphinx. Refer to documentation |
1184 | # for a list of supported languages. |
1185 | |
1186 | === modified file 'docs/percona-toolkit.pod' |
1187 | --- docs/percona-toolkit.pod 2014-07-08 12:06:12 +0000 |
1188 | +++ docs/percona-toolkit.pod 2014-08-08 14:10:17 +0000 |
1189 | @@ -557,6 +557,6 @@ |
1190 | |
1191 | =head1 VERSION |
1192 | |
1193 | -Percona Toolkit v2.2.9 released 2014-07-08 |
1194 | +Percona Toolkit v2.2.10 released 2014-08-06 |
1195 | |
1196 | =cut |
1197 | |
1198 | === modified file 'docs/release_notes.rst' |
1199 | --- docs/release_notes.rst 2014-07-08 12:06:12 +0000 |
1200 | +++ docs/release_notes.rst 2014-08-08 14:10:17 +0000 |
1201 | @@ -1,6 +1,35 @@ |
1202 | Release Notes |
1203 | ************* |
1204 | |
1205 | +v2.2.10 released 2014-08-06 |
1206 | +=========================== |
1207 | + |
1208 | +Percona Toolkit 2.2.10 has been released. This release contains six bug fixes. |
1209 | + |
1210 | +Bugs Fixed: |
1211 | + |
1212 | +* Fixed bug 1287253: pt-table-checksum would exit with error if it would encounter deadlock when doing checksum. This was fixed by retrying the command in case of deadlock error. |
1213 | + |
1214 | +* Fixed bug 1311654: When used with Percona XtraDB Cluster, pt-table-checksum could show incorrect result if --resume option was used. This was fixed by adding a new ``--replicate-check-retries`` command line parameter. If you are having resume problems you can now set ``--replicate-check-retries`` N , where N is the number of times to retry a discrepant checksum (default = 1 , no retries). Setting a value of ``3`` is enough to completely eliminate spurious differences. |
1215 | + |
1216 | +* Fixed bug 1299387: pt-query-digest didn't work correctly do to a changed logging format when field ``Thread_id`` has been renamed to ``Id``. Fixed by implementing support for the new format. |
1217 | + |
1218 | +* Fixed bug 1340728: in some cases, where the index was of type "hash" , pt-online-schema-change would refuse to run because MySQL reported it would not use an index for the select. This check should have been able to be skipped using --nocheck-plan option, but it wasn't. ``--nocheck-plan`` now ignores the chosen index correctly. |
1219 | + |
1220 | +* Fixed bug 1253872: When running pt-table-checksum or pt-online-schema on a server that is unused, setting the 20% max load would fail due to tools rounding the value down. This has been fixed by rounding the value up. |
1221 | + |
1222 | +* Fixed bug 1340364: Due to incompatibility of dash and bash syntax some shell tools were showing error when queried for version. |
1223 | + |
1224 | +Changelog |
1225 | +--------- |
1226 | + |
1227 | +* Fixed bug 1287253: pt-table-checksum deadlock |
1228 | +* Fixed bug 1299387: 5.6 slow query log Thead_id becomes Id |
1229 | +* Fixed bug 1311654: pt-table-checksum + PXC inconsistent results upon --resume |
1230 | +* Fixed bug 1340728: pt-online-schema-change doesn't work with HASH indexes |
1231 | +* Fixed bug 1253872: pt-table-checksum max load 20% rounds down |
1232 | +* Fixed bug 1340364: some shell tools output error when queried for --version |
1233 | + |
1234 | v2.2.9 released 2014-07-08 |
1235 | ========================== |
1236 | |
1237 | |
1238 | === modified file 'lib/MySQLStatusWaiter.pm' |
1239 | --- lib/MySQLStatusWaiter.pm 2013-01-03 00:19:16 +0000 |
1240 | +++ lib/MySQLStatusWaiter.pm 2014-08-08 14:10:17 +0000 |
1241 | @@ -24,6 +24,7 @@ |
1242 | |
1243 | use strict; |
1244 | use warnings FATAL => 'all'; |
1245 | +use POSIX qw( ceil ); |
1246 | use English qw(-no_match_vars); |
1247 | use constant PTDEBUG => $ENV{PTDEBUG} || 0; |
1248 | |
1249 | @@ -221,8 +222,8 @@ |
1250 | } |
1251 | else { |
1252 | PTDEBUG && _d('Initial', $var, 'value:', $init_val); |
1253 | - $val = int(($init_val * $threshold_factor) + $init_val); |
1254 | - $vars->{$var} = $val; |
1255 | + $val = ($init_val * $threshold_factor) + $init_val; |
1256 | + $vars->{$var} = int(ceil($val)); |
1257 | } |
1258 | PTDEBUG && _d('Wait if', $var, '>=', $val); |
1259 | } |
1260 | |
1261 | === modified file 'lib/Percona/Toolkit.pm' |
1262 | --- lib/Percona/Toolkit.pm 2014-07-04 16:13:38 +0000 |
1263 | +++ lib/Percona/Toolkit.pm 2014-08-08 14:10:17 +0000 |
1264 | @@ -18,7 +18,7 @@ |
1265 | # ########################################################################### |
1266 | package Percona::Toolkit; |
1267 | |
1268 | -our $VERSION = '2.2.9'; |
1269 | +our $VERSION = '2.2.10'; |
1270 | |
1271 | use strict; |
1272 | use warnings FATAL => 'all'; |
1273 | |
1274 | === modified file 'lib/RowChecksum.pm' |
1275 | --- lib/RowChecksum.pm 2013-01-03 00:19:16 +0000 |
1276 | +++ lib/RowChecksum.pm 2014-08-08 14:10:17 +0000 |
1277 | @@ -459,19 +459,28 @@ |
1278 | } |
1279 | my ($dbh, $repl_table) = @args{@required_args}; |
1280 | |
1281 | - my $sql |
1282 | - = "SELECT CONCAT(db, '.', tbl) AS `table`, " |
1283 | - . "chunk, chunk_index, lower_boundary, upper_boundary, " |
1284 | - . "COALESCE(this_cnt-master_cnt, 0) AS cnt_diff, " |
1285 | - . "COALESCE(" |
1286 | - . "this_crc <> master_crc OR ISNULL(master_crc) <> ISNULL(this_crc), 0" |
1287 | - . ") AS crc_diff, this_cnt, master_cnt, this_crc, master_crc " |
1288 | - . "FROM $repl_table " |
1289 | - . "WHERE (master_cnt <> this_cnt OR master_crc <> this_crc " |
1290 | - . "OR ISNULL(master_crc) <> ISNULL(this_crc))" |
1291 | - . ($args{where} ? " AND ($args{where})" : ""); |
1292 | - PTDEBUG && _d($sql); |
1293 | - my $diffs = $dbh->selectall_arrayref($sql, { Slice => {} }); |
1294 | + |
1295 | + my $tries = $self->{'OptionParser'}->get('replicate-check-retries') || 1; |
1296 | + my $diffs; |
1297 | + while ($tries--) { |
1298 | + my $sql |
1299 | + = "SELECT CONCAT(db, '.', tbl) AS `table`, " |
1300 | + . "chunk, chunk_index, lower_boundary, upper_boundary, " |
1301 | + . "COALESCE(this_cnt-master_cnt, 0) AS cnt_diff, " |
1302 | + . "COALESCE(" |
1303 | + . "this_crc <> master_crc OR ISNULL(master_crc) <> ISNULL(this_crc), 0" |
1304 | + . ") AS crc_diff, this_cnt, master_cnt, this_crc, master_crc " |
1305 | + . "FROM $repl_table " |
1306 | + . "WHERE (master_cnt <> this_cnt OR master_crc <> this_crc " |
1307 | + . "OR ISNULL(master_crc) <> ISNULL(this_crc)) " |
1308 | + . ($args{where} ? " AND ($args{where})" : ""); |
1309 | + PTDEBUG && _d($sql); |
1310 | + $diffs = $dbh->selectall_arrayref($sql, { Slice => {} }); |
1311 | + if (!@$diffs || !$tries) { # if no differences are found OR we are out of tries left... |
1312 | + last; # get out now |
1313 | + } |
1314 | + sleep 1; |
1315 | + } |
1316 | return $diffs; |
1317 | } |
1318 | |
1319 | |
1320 | === modified file 'lib/SlowLogParser.pm' |
1321 | --- lib/SlowLogParser.pm 2013-05-08 03:19:37 +0000 |
1322 | +++ lib/SlowLogParser.pm 2014-08-08 14:10:17 +0000 |
1323 | @@ -42,7 +42,7 @@ |
1324 | } |
1325 | |
1326 | my $slow_log_ts_line = qr/^# Time: ([0-9: ]{15})/; |
1327 | -my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]/; |
1328 | +my $slow_log_uh_line = qr/# User\@Host: ([^\[]+|\[[^[]+\]).*?@ (\S*) \[(.*)\]\s*(?:Id:\s*(\d+))?/; |
1329 | # These can appear in the log file when it's opened -- for example, when someone |
1330 | # runs FLUSH LOGS or the server starts. |
1331 | # /usr/sbin/mysqld, Version: 5.0.67-0ubuntu6-log ((Ubuntu)). started with: |
1332 | @@ -159,21 +159,29 @@ |
1333 | ++$got_ts; |
1334 | # The User@Host might be concatenated onto the end of the Time. |
1335 | if ( !$got_uh |
1336 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
1337 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
1338 | ) { |
1339 | PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
1340 | push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
1341 | - ++$got_uh; |
1342 | + # 5.6 has the thread id on the User@Host line |
1343 | + if ( $thread_id ) { |
1344 | + push @properties, 'Thread_id', $thread_id; |
1345 | + } |
1346 | + ++$got_uh; |
1347 | } |
1348 | } |
1349 | |
1350 | # Maybe it's the user/host line of a slow query log |
1351 | # # User@Host: root[root] @ localhost [] |
1352 | elsif ( !$got_uh |
1353 | - && ( my ( $user, $host, $ip ) = $line =~ m/$slow_log_uh_line/o ) |
1354 | + && ( my ( $user, $host, $ip, $thread_id ) = $line =~ m/$slow_log_uh_line/o ) |
1355 | ) { |
1356 | - PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
1357 | - push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
1358 | + PTDEBUG && _d("Got user, host, ip", $user, $host, $ip); |
1359 | + push @properties, 'user', $user, 'host', $host, 'ip', $ip; |
1360 | + # 5.6 has the thread id on the User@Host line |
1361 | + if ( $thread_id ) { |
1362 | + push @properties, 'Thread_id', $thread_id; |
1363 | + } |
1364 | ++$got_uh; |
1365 | } |
1366 | |
1367 | |
1368 | === modified file 'lib/bash/parse_options.sh' |
1369 | --- lib/bash/parse_options.sh 2013-08-09 21:21:55 +0000 |
1370 | +++ lib/bash/parse_options.sh 2014-08-08 14:10:17 +0000 |
1371 | @@ -76,9 +76,10 @@ |
1372 | |
1373 | usage_or_errors() { |
1374 | local file="$1" |
1375 | + local version="" |
1376 | |
1377 | if [ "$OPT_VERSION" ]; then |
1378 | - local version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
1379 | + version=$(grep '^pt-[^ ]\+ [0-9]' "$file") |
1380 | echo "$version" |
1381 | return 1 |
1382 | fi |
1383 | |
1384 | === modified file 't/lib/MySQLStatusWaiter.t' |
1385 | --- t/lib/MySQLStatusWaiter.t 2012-05-25 21:27:23 +0000 |
1386 | +++ t/lib/MySQLStatusWaiter.t 2014-08-08 14:10:17 +0000 |
1387 | @@ -8,6 +8,7 @@ |
1388 | |
1389 | use strict; |
1390 | use warnings FATAL => 'all'; |
1391 | +use POSIX qw( ceil floor ); |
1392 | use English qw(-no_match_vars); |
1393 | use Test::More tests => 17; |
1394 | |
1395 | @@ -19,6 +20,7 @@ |
1396 | my $slept = 0; |
1397 | my @vals = (); |
1398 | |
1399 | + |
1400 | sub oktorun { |
1401 | return $oktorun; |
1402 | } |
1403 | @@ -76,33 +78,35 @@ |
1404 | "Catch non-existent variable" |
1405 | ); |
1406 | |
1407 | -# ############################################################################ |
1408 | -# Use initial vals + 20%. |
1409 | -# ############################################################################ |
1410 | + |
1411 | +# ############################################################################ |
1412 | +# Initial vals + 20% |
1413 | +# ############################################################################ |
1414 | + |
1415 | @vals = ( |
1416 | # initial check for existence |
1417 | - { Threads_connected => 10, }, |
1418 | - { Threads_running => 5, }, |
1419 | + { Threads_connected => 9, }, |
1420 | + { Threads_running => 4, }, |
1421 | |
1422 | # first check, no wait |
1423 | { Threads_connected => 1, }, |
1424 | { Threads_running => 1, }, |
1425 | |
1426 | # second check, wait |
1427 | - { Threads_connected => 12, }, # too high |
1428 | - { Threads_running => 6, }, # too high |
1429 | + { Threads_connected => 11, }, # too high |
1430 | + { Threads_running => 5, }, # too high |
1431 | |
1432 | # third check, wait |
1433 | - { Threads_connected => 12, }, # too high |
1434 | - { Threads_running => 5, }, |
1435 | + { Threads_connected => 11, }, # too high |
1436 | + { Threads_running => 4, }, |
1437 | |
1438 | # fourth check, wait |
1439 | { Threads_connected => 10, }, |
1440 | - { Threads_running => 6, }, # too high |
1441 | + { Threads_running => 5, }, # too high |
1442 | |
1443 | # fifth check, no wait |
1444 | { Threads_connected => 10, }, |
1445 | - { Threads_running => 5, }, |
1446 | + { Threads_running => 4, }, |
1447 | ); |
1448 | |
1449 | $oktorun = 1; |
1450 | @@ -117,10 +121,10 @@ |
1451 | is_deeply( |
1452 | $sw->max_values(), |
1453 | { |
1454 | - Threads_connected => int(10 + (10 * 0.20)), |
1455 | - Threads_running => int(5 + (5 * 0.20)), |
1456 | + Threads_connected => ceil(9 + (9 * 0.20)), |
1457 | + Threads_running => ceil(4 + (4 * 0.20)), |
1458 | }, |
1459 | - "Initial values +20%" |
1460 | + "Threshold = ceil(InitialValue * 1.2)" |
1461 | ); |
1462 | |
1463 | # first check |
1464 | |
1465 | === modified file 't/lib/SlowLogParser.t' |
1466 | --- t/lib/SlowLogParser.t 2012-12-11 01:07:36 +0000 |
1467 | +++ t/lib/SlowLogParser.t 2014-08-08 14:10:17 +0000 |
1468 | @@ -1349,6 +1349,70 @@ |
1469 | ], |
1470 | ); |
1471 | |
1472 | + |
1473 | +# ############################################################################# |
1474 | +# pt-query-digest fails to parse Thread_id on 5.6 |
1475 | +# https://bugs.launchpad.net/percona-toolkit/+bug/1299387 |
1476 | +# ############################################################################# |
1477 | + |
1478 | +# first test with a Percona 5.5 slow log (includes an explicit Thread_id line) |
1479 | +# (MySQL 5.5 doesn't include thread id) |
1480 | +test_log_parser( |
1481 | + parser => $p, |
1482 | + file => "$sample/slow060.txt", |
1483 | + result => [ |
1484 | + { |
1485 | + Thread_id => '1', # here's our item |
1486 | + Bytes_sent => '64', |
1487 | + Killed => '0', |
1488 | + Last_errno => '0', |
1489 | + Lock_time => '0.000000', |
1490 | + Query_time => '10.000373', |
1491 | + Rows_affected => '0', |
1492 | + Rows_examined => '0', |
1493 | + Rows_read => '0', |
1494 | + Rows_sent => '1', |
1495 | + arg => 'select sleep(10)', |
1496 | + bytes => 16, |
1497 | + cmd => 'Query', |
1498 | + host => 'localhost', |
1499 | + ip => '127.0.0.1', |
1500 | + pos_in_log => 0, |
1501 | + timestamp => '1405358212', |
1502 | + ts => '140714 14:16:52', |
1503 | + user => 'root' |
1504 | + }, |
1505 | + ], |
1506 | +); |
1507 | + |
1508 | +# now test with a Percona 5.6 slow log. Thread_id is now on the user@host line, and is called 'Id'. |
1509 | +# (this is in line with MySQL 5.6) |
1510 | +test_log_parser( |
1511 | + parser => $p, |
1512 | + file => "$sample/slow061.txt", |
1513 | + result => [ |
1514 | + { |
1515 | + Thread_id => 1, # here's our item |
1516 | + Bytes_sent => '64', |
1517 | + Lock_time => '0.000000', |
1518 | + Query_time => '11.013723', |
1519 | + Rows_affected => '0', |
1520 | + Rows_examined => '0', |
1521 | + Rows_sent => '1', |
1522 | + arg => 'select sleep(11)', |
1523 | + bytes => 16, |
1524 | + cmd => 'Query', |
1525 | + host => 'localhost', |
1526 | + ip => '127.0.0.1', |
1527 | + pos_in_log => 0, |
1528 | + timestamp => '1405360304', |
1529 | + ts => '140714 14:51:44', |
1530 | + user => 'root', |
1531 | + }, |
1532 | + ], |
1533 | +); |
1534 | + |
1535 | + |
1536 | # ############################################################################# |
1537 | # Done. |
1538 | # ############################################################################# |
1539 | |
1540 | === added file 't/lib/samples/slowlogs/slow060.txt' |
1541 | --- t/lib/samples/slowlogs/slow060.txt 1970-01-01 00:00:00 +0000 |
1542 | +++ t/lib/samples/slowlogs/slow060.txt 2014-08-08 14:10:17 +0000 |
1543 | @@ -0,0 +1,10 @@ |
1544 | +bin/mysqld, Version: 5.5.38-35.2-log (Percona Server (GPL), Release 35.2, Revision 674). started with: |
1545 | +Tcp port: 3306 Unix socket: (null) |
1546 | +Time Id Command Argument |
1547 | +# Time: 140714 14:16:52 |
1548 | +# User@Host: root[root] @ localhost [127.0.0.1] |
1549 | +# Thread_id: 1 Schema: Last_errno: 0 Killed: 0 |
1550 | +# Query_time: 10.000373 Lock_time: 0.000000 Rows_sent: 1 Rows_examined: 0 Rows_affected: 0 Rows_read: 0 |
1551 | +# Bytes_sent: 64 |
1552 | +SET timestamp=1405358212; |
1553 | +select sleep(10); |
1554 | |
1555 | === added file 't/lib/samples/slowlogs/slow061.txt' |
1556 | --- t/lib/samples/slowlogs/slow061.txt 1970-01-01 00:00:00 +0000 |
1557 | +++ t/lib/samples/slowlogs/slow061.txt 2014-08-08 14:10:17 +0000 |
1558 | @@ -0,0 +1,10 @@ |
1559 | +bin/mysqld, Version: 5.6.19-67.0-log (Percona Server (GPL), Release 67.0, Revision 618). started with: |
1560 | +Tcp port: 3306 Unix socket: (null) |
1561 | +Time Id Command Argument |
1562 | +# Time: 140714 14:51:44 |
1563 | +# User@Host: root[root] @ localhost [127.0.0.1] Id: 1 |
1564 | +# Schema: Last_errno: 0 Killed: 0 |
1565 | +# Query_time: 11.013723 Lock_time: 0.000000 Rows_sent: 1 Rows_examined: 0 Rows_affected: 0 |
1566 | +# Bytes_sent: 64 |
1567 | +SET timestamp=1405360304; |
1568 | +select sleep(11); |
1569 | |
1570 | === removed file 't/pt-online-schema-change/bug-1315130.t' |
1571 | --- t/pt-online-schema-change/bug-1315130.t 2014-06-20 21:31:30 +0000 |
1572 | +++ t/pt-online-schema-change/bug-1315130.t 1970-01-01 00:00:00 +0000 |
1573 | @@ -1,78 +0,0 @@ |
1574 | -#!/usr/bin/env perl |
1575 | - |
1576 | -BEGIN { |
1577 | - die "The PERCONA_TOOLKIT_BRANCH environment variable is not set.\n" |
1578 | - unless $ENV{PERCONA_TOOLKIT_BRANCH} && -d $ENV{PERCONA_TOOLKIT_BRANCH}; |
1579 | - unshift @INC, "$ENV{PERCONA_TOOLKIT_BRANCH}/lib"; |
1580 | -}; |
1581 | - |
1582 | -use strict; |
1583 | -use warnings FATAL => 'all'; |
1584 | -use English qw(-no_match_vars); |
1585 | -use Test::More; |
1586 | -use Time::HiRes qw(sleep); |
1587 | - |
1588 | -$ENV{PTTEST_FAKE_TS} = 1; |
1589 | -$ENV{PERCONA_TOOLKIT_TEST_USE_DSN_NAMES} = 1; |
1590 | - |
1591 | -use PerconaTest; |
1592 | -use Sandbox; |
1593 | -require "$trunk/bin/pt-online-schema-change"; |
1594 | -require VersionParser; |
1595 | - |
1596 | -use Data::Dumper; |
1597 | -$Data::Dumper::Indent = 1; |
1598 | -$Data::Dumper::Sortkeys = 1; |
1599 | -$Data::Dumper::Quotekeys = 0; |
1600 | - |
1601 | -my $dp = new DSNParser(opts=>$dsn_opts); |
1602 | -my $sb = new Sandbox(basedir => '/tmp', DSNParser => $dp); |
1603 | -my $master_dbh = $sb->get_dbh_for('master'); |
1604 | -my $slave_dbh = $sb->get_dbh_for('slave1'); |
1605 | - |
1606 | -if ( !$master_dbh ) { |
1607 | - plan skip_all => 'Cannot connect to sandbox master'; |
1608 | -} |
1609 | -elsif ( !$slave_dbh ) { |
1610 | - plan skip_all => 'Cannot connect to sandbox slave'; |
1611 | -} |
1612 | - |
1613 | -my $q = new Quoter(); |
1614 | -my $tp = new TableParser(Quoter => $q); |
1615 | -my @args = qw(--set-vars innodb_lock_wait_timeout=3); |
1616 | -my $output = ""; |
1617 | -my $dsn = "h=127.1,P=12345,u=msandbox,p=msandbox"; |
1618 | -my $exit = 0; |
1619 | -my $sample = "t/pt-online-schema-change/samples"; |
1620 | -my $rows; |
1621 | - |
1622 | - |
1623 | -# ############################################################################# |
1624 | -# Issue 1315130 |
1625 | -# Failed to detect child tables in other schema, and falsely identified |
1626 | -# child tables in own schema |
1627 | -# ############################################################################# |
1628 | - |
1629 | -$sb->load_file('master', "$sample/bug-1315130_cleanup.sql"); |
1630 | -$sb->load_file('master', "$sample/bug-1315130.sql"); |
1631 | -($output, $exit) = full_output( |
1632 | - sub { pt_online_schema_change::main(@args, "$dsn,D=bug_1315130_a,t=parent_table", |
1633 | - '--dry-run', |
1634 | - '--alter', "add column c varchar(16)", |
1635 | - '--alter-foreign-keys-method', 'auto'), |
1636 | - }, |
1637 | -); |
1638 | -print STDERR "[$output]\n"; |
1639 | - like( |
1640 | - $output, |
1641 | - qr/Child tables:\s*`bug_1315130_a`\.`child_table_in_same_schema` \(approx\. 1 rows\)\s*`bug_1315130_b`\.`child_table_in_second_schema` \(approx\. 1 rows\)[^`]*?Will/s, |
1642 | - "Correctly identify child tables from other schemas and ignores tables from same schema referencig same named parent in other schema.", |
1643 | - ); |
1644 | -# clear databases with their foreign keys |
1645 | -$sb->load_file('master', "$sample/bug-1315130_cleanup.sql"); |
1646 | - |
1647 | -# Done. |
1648 | -# ############################################################################# |
1649 | -$sb->wipe_clean($master_dbh); |
1650 | -ok($sb->ok(), "Sandbox servers") or BAIL_OUT(__FILE__ . " broke the sandbox"); |
1651 | -done_testing; |
1652 | |
1653 | === modified file 't/pt-online-schema-change/bugs.t' |
1654 | --- t/pt-online-schema-change/bugs.t 2013-06-26 20:28:27 +0000 |
1655 | +++ t/pt-online-schema-change/bugs.t 2014-08-08 14:10:17 +0000 |
1656 | @@ -124,7 +124,11 @@ |
1657 | { |
1658 | my $o = new OptionParser(file => "$trunk/bin/pt-table-checksum"); |
1659 | $o->get_specs(); |
1660 | + |
1661 | + $o->set('check-plan', 1); # check-plan is true by default |
1662 | + |
1663 | no warnings; |
1664 | + |
1665 | local *pt_online_schema_change::explain_statement = sub { |
1666 | return { key => 'some_key' } |
1667 | }; |
1668 | @@ -369,6 +373,66 @@ |
1669 | "Bug 1188264: warning about expected MySQL error 1265" |
1670 | ); |
1671 | |
1672 | + |
1673 | +# ############################################################################# |
1674 | +# Issue 1315130 |
1675 | +# Failed to detect child tables in other schema, and falsely identified |
1676 | +# child tables in own schema |
1677 | +# ############################################################################# |
1678 | + |
1679 | +$sb->load_file('master', "$sample/bug-1315130_cleanup.sql"); |
1680 | +$sb->load_file('master', "$sample/bug-1315130.sql"); |
1681 | + |
1682 | +$output = output( |
1683 | + sub { pt_online_schema_change::main(@args, "$master_dsn,D=bug_1315130_a,t=parent_table", |
1684 | + '--dry-run', |
1685 | + '--alter', "add column c varchar(16)", |
1686 | + '--alter-foreign-keys-method', 'auto'), |
1687 | + }, |
1688 | +); |
1689 | + |
1690 | + like( |
1691 | + $output, |
1692 | + qr/Child tables:\s*`bug_1315130_a`\.`child_table_in_same_schema` \(approx\. 1 rows\)\s*`bug_1315130_b`\.`child_table_in_second_schema` \(approx\. 1 rows\)[^`]*?Will/s, |
1693 | + "Correctly identify child tables from other schemas and ignores tables from same schema referencig same named parent in other schema.", |
1694 | + ); |
1695 | +# clear databases with their foreign keys |
1696 | +$sb->load_file('master', "$sample/bug-1315130_cleanup.sql"); |
1697 | + |
1698 | + |
1699 | +# ############################################################################# |
1700 | +# Issue 1340728 |
1701 | +# fails when no index is returned in EXPLAIN, even though --nocheck-plan is set |
1702 | +# (happens on HASH indexes) |
1703 | +# ############################################################################# |
1704 | + |
1705 | +$sb->load_file('master', "$sample/bug-1340728_cleanup.sql"); |
1706 | +$sb->load_file('master', "$sample/bug-1340728.sql"); |
1707 | + |
1708 | +# insert a few thousand rows (else test isn't valid) |
1709 | +my $rows = 5000; |
1710 | +for (my $i = 0; $i < $rows; $i++) { |
1711 | + $master_dbh->do("INSERT INTO bug_1340728.test VALUES (NULL, 'xx')"); |
1712 | +} |
1713 | + |
1714 | + |
1715 | +$output = output( |
1716 | + sub { pt_online_schema_change::main(@args, "$master_dsn,D=bug_1340728,t=test", |
1717 | + '--execute', |
1718 | + '--alter', "ADD COLUMN c INT", |
1719 | + '--nocheck-plan', |
1720 | + ), |
1721 | + }, |
1722 | +); |
1723 | + |
1724 | + like( |
1725 | + $output, |
1726 | + qr/Successfully altered/s, |
1727 | + "--nocheck-plan ignores plans without index", |
1728 | + ); |
1729 | +# clear databases |
1730 | +$sb->load_file('master', "$sample/bug-1340728_cleanup.sql"); |
1731 | + |
1732 | # ############################################################################# |
1733 | # Done. |
1734 | # ############################################################################# |
1735 | |
1736 | === added file 't/pt-online-schema-change/samples/bug-1340728.sql' |
1737 | --- t/pt-online-schema-change/samples/bug-1340728.sql 1970-01-01 00:00:00 +0000 |
1738 | +++ t/pt-online-schema-change/samples/bug-1340728.sql 2014-08-08 14:10:17 +0000 |
1739 | @@ -0,0 +1,10 @@ |
1740 | +drop database if exists bug_1340728; |
1741 | +create database bug_1340728; |
1742 | +use bug_1340728; |
1743 | + |
1744 | +CREATE TABLE `test` ( |
1745 | + `id` int(10) unsigned NOT NULL AUTO_INCREMENT, |
1746 | + `name` char(2) NOT NULL, |
1747 | + PRIMARY KEY (`id`) USING HASH |
1748 | +) ENGINE=MEMORY DEFAULT CHARSET=latin1; |
1749 | + |
1750 | |
1751 | === added file 't/pt-online-schema-change/samples/bug-1340728_cleanup.sql' |
1752 | --- t/pt-online-schema-change/samples/bug-1340728_cleanup.sql 1970-01-01 00:00:00 +0000 |
1753 | +++ t/pt-online-schema-change/samples/bug-1340728_cleanup.sql 2014-08-08 14:10:17 +0000 |
1754 | @@ -0,0 +1,2 @@ |
1755 | +drop database if exists bug_1340728; |
1756 | + |
1757 | |
1758 | === modified file 't/pt-query-digest/since_until.t' |
1759 | --- t/pt-query-digest/since_until.t 2013-03-05 16:59:33 +0000 |
1760 | +++ t/pt-query-digest/since_until.t 2014-08-08 14:10:17 +0000 |
1761 | @@ -52,16 +52,16 @@ |
1762 | '--since 090727' |
1763 | ); |
1764 | |
1765 | -# This test will fail come July 2014. |
1766 | +# This test will fail come July 2015. |
1767 | ok( |
1768 | no_diff( |
1769 | sub { pt_query_digest::main(@args, |
1770 | - "$sample_in/slow033.txt", qw(--since 1825d), |
1771 | + "$sample_in/slow033.txt", qw(--since 2190d), |
1772 | )}, |
1773 | "t/pt-query-digest/samples/slow033-since-Nd.txt", |
1774 | stderr => 1, |
1775 | ), |
1776 | - '--since 1825d (5 years ago)' |
1777 | + '--since 2190d (6 years ago)' |
1778 | ); |
1779 | |
1780 | # --until |
1781 | |
1782 | === modified file 't/pt-table-checksum/basics.t' |
1783 | --- t/pt-table-checksum/basics.t 2014-07-07 00:43:50 +0000 |
1784 | +++ t/pt-table-checksum/basics.t 2014-08-08 14:10:17 +0000 |
1785 | @@ -80,8 +80,8 @@ |
1786 | # other than to ensure that there's at least one for each table. |
1787 | $row = $master_dbh->selectrow_arrayref("select count(*) from percona.checksums"); |
1788 | ok( |
1789 | - $row->[0] > 30 && $row->[0] < 50, |
1790 | - 'Between 30 and 50 chunks' |
1791 | + $row->[0] > 30 && $row->[0] < 60, |
1792 | + 'Between 30 and 60 chunks' |
1793 | ) or diag($row->[0]); |
1794 | |
1795 | # ############################################################################ |
1796 | @@ -97,6 +97,7 @@ |
1797 | "Static chunk size (--chunk-time 0)" |
1798 | ); |
1799 | |
1800 | + |
1801 | $row = $master_dbh->selectrow_arrayref("select count(*) from percona.checksums"); |
1802 | ok( |
1803 | $row->[0] >= 85 && $row->[0] <= 90, |
1804 | |
1805 | === modified file 't/pt-table-checksum/samples/default-results-5.5.txt' |
1806 | --- t/pt-table-checksum/samples/default-results-5.5.txt 2013-02-19 01:51:14 +0000 |
1807 | +++ t/pt-table-checksum/samples/default-results-5.5.txt 2014-08-08 14:10:17 +0000 |
1808 | @@ -21,7 +21,7 @@ |
1809 | 0 0 0 0 mysql.time_zone_transition |
1810 | 0 0 0 0 mysql.time_zone_transition_type |
1811 | 0 0 2 0 mysql.user |
1812 | -0 0 38 0 percona_test.checksums |
1813 | +0 0 22 0 percona_test.checksums |
1814 | 0 0 1 0 percona_test.load_data |
1815 | 0 0 1 0 percona_test.sentinel |
1816 | 0 0 200 0 sakila.actor |
1817 | |
1818 | === modified file 't/pt-table-checksum/samples/default-results-5.6.txt' |
1819 | --- t/pt-table-checksum/samples/default-results-5.6.txt 2013-02-19 01:51:44 +0000 |
1820 | +++ t/pt-table-checksum/samples/default-results-5.6.txt 2014-08-08 14:10:17 +0000 |
1821 | @@ -20,7 +20,7 @@ |
1822 | 0 0 0 0 mysql.time_zone_transition |
1823 | 0 0 0 0 mysql.time_zone_transition_type |
1824 | 0 0 2 0 mysql.user |
1825 | -0 0 37 0 percona_test.checksums |
1826 | +0 0 22 0 percona_test.checksums |
1827 | 0 0 1 0 percona_test.load_data |
1828 | 0 0 1 0 percona_test.sentinel |
1829 | 0 0 200 0 sakila.actor |
1830 | |
1831 | === modified file 't/pt-table-checksum/samples/static-chunk-size-results-5.5.txt' |
1832 | --- t/pt-table-checksum/samples/static-chunk-size-results-5.5.txt 2013-02-19 01:51:14 +0000 |
1833 | +++ t/pt-table-checksum/samples/static-chunk-size-results-5.5.txt 2014-08-08 14:10:17 +0000 |
1834 | @@ -21,7 +21,7 @@ |
1835 | 0 0 0 1 0 mysql.time_zone_transition |
1836 | 0 0 0 1 0 mysql.time_zone_transition_type |
1837 | 0 0 2 1 0 mysql.user |
1838 | -0 0 38 1 0 percona_test.checksums |
1839 | +0 0 22 1 0 percona_test.checksums |
1840 | 0 0 1 1 0 percona_test.load_data |
1841 | 0 0 1 1 0 percona_test.sentinel |
1842 | 0 0 200 1 0 sakila.actor |
1843 | |
1844 | === modified file 't/pt-table-checksum/samples/static-chunk-size-results-5.6.txt' |
1845 | --- t/pt-table-checksum/samples/static-chunk-size-results-5.6.txt 2013-02-19 01:51:44 +0000 |
1846 | +++ t/pt-table-checksum/samples/static-chunk-size-results-5.6.txt 2014-08-08 14:10:17 +0000 |
1847 | @@ -20,7 +20,7 @@ |
1848 | 0 0 0 1 0 mysql.time_zone_transition |
1849 | 0 0 0 1 0 mysql.time_zone_transition_type |
1850 | 0 0 2 1 0 mysql.user |
1851 | -0 0 37 1 0 percona_test.checksums |
1852 | +0 0 22 1 0 percona_test.checksums |
1853 | 0 0 1 1 0 percona_test.load_data |
1854 | 0 0 1 1 0 percona_test.sentinel |
1855 | 0 0 200 1 0 sakila.actor |