Merge lp:~percona-toolkit-dev/percona-toolkit/partial-fix-1063912-ptc-pxc-slaves into lp:~percona-toolkit-dev/percona-toolkit/2.1.5-release

Proposed by Daniel Nichter
Status: Merged
Merged at revision: 398
Proposed branch: lp:~percona-toolkit-dev/percona-toolkit/partial-fix-1063912-ptc-pxc-slaves
Merge into: lp:~percona-toolkit-dev/percona-toolkit/2.1.5-release
Diff against target: 905 lines (+586/-33)
11 files modified
bin/pt-query-digest (+208/-6)
bin/pt-stalk (+113/-4)
bin/pt-table-checksum (+30/-7)
lib/Cxn.pm (+3/-1)
lib/RowChecksum.pm (+5/-0)
t/lib/Pingback.t (+67/-7)
t/lib/RowChecksum.t (+45/-4)
t/pt-stalk/plugin.t (+74/-0)
t/pt-stalk/samples/plugin001.sh (+21/-0)
t/pt-table-checksum/bugs.t (+16/-0)
t/pt-table-checksum/run_time.t (+4/-4)
To merge this branch: bzr merge lp:~percona-toolkit-dev/percona-toolkit/partial-fix-1063912-ptc-pxc-slaves
Reviewer Review Type Date Requested Status
Daniel Nichter Approve
Review via email: mp+128566@code.launchpad.net
To post a comment you must log in.
405. By Daniel Nichter

Add note to PXC limiations that complex repl setups aren't supported.

Revision history for this message
Daniel Nichter (daniel-nichter) :
review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'bin/pt-query-digest'
--- bin/pt-query-digest 2012-09-24 19:24:36 +0000
+++ bin/pt-query-digest 2012-10-08 18:50:28 +0000
@@ -15160,8 +15160,10 @@
15160may actually include different kinds of attributes (for example, you may have a15160may actually include different kinds of attributes (for example, you may have a
15161server with the Percona patches).15161server with the Percona patches).
1516215162
15163For a full list of attributes, see15163See L<"ATTRIBUTES REFERENCE"> near the end of this documentation for a list
15164L<http://code.google.com/p/maatkit/wiki/EventAttributes>.15164of common and L<"--type"> specific attributes. A familiarity with these
15165attributes is necessary for working with L<"--filter">,
15166L<"--ignore-attributes">, and other attribute-related options.
1516515167
15166With creative use of L<"--filter">, you can create new attributes derived15168With creative use of L<"--filter">, you can create new attributes derived
15167from existing attributes. For example, to create an attribute called15169from existing attributes. For example, to create an attribute called
@@ -15189,8 +15191,8 @@
15189These attributes are no different from slow log attributes, so you can use them15191These attributes are no different from slow log attributes, so you can use them
15190with L<"--[no]report">, L<"--group-by">, in a L<"--filter">, etc.15192with L<"--[no]report">, L<"--group-by">, in a L<"--filter">, etc.
1519115193
15192These attributes and more are documented at15194See the memcached section of L<"ATTRIBUTES REFERENCE"> for a list of
15193L<http://code.google.com/p/maatkit/wiki/EventAttributes>.15195memcached-specific attributes.
1519415196
15195=head1 OUTPUT15197=head1 OUTPUT
1519615198
@@ -15835,8 +15837,8 @@
1583515837
15836It is permissible for the code to have side effects (to alter C<$event>).15838It is permissible for the code to have side effects (to alter C<$event>).
1583715839
15838You can find an explanation of the structure of $event at15840See L<"ATTRIBUTES REFERENCE"> for a list of common and L<"--type"> specific
15839L<http://code.google.com/p/maatkit/wiki/EventAttributes>.15841attributes.
1584015842
15841Here are more examples of filter code:15843Here are more examples of filter code:
1584215844
@@ -17069,6 +17071,206 @@
1706917071
17070Replace C<TOOL> with the name of any tool.17072Replace C<TOOL> with the name of any tool.
1707117073
17074=head1 ATTRIBUTES REFERENCE
17075
17076Events may have the following attributes. If writing a L<"--filter">,
17077be sure to check that an attribute is defined in each event before
17078using it, else the filter code may crash the tool with a
17079"use of uninitialized value" error.
17080
17081You can dump event attributes for any input like:
17082
17083 $ pt-query-digest \
17084 slowlog \
17085 --filter 'print Dumper $event' \
17086 --no-report \
17087 --sample 1
17088
17089That will produce a lot of output with "attribute => value" pairs like:
17090
17091 $VAR1 = {
17092 Query_time => '0.033384',
17093 Rows_examined => '0',
17094 Rows_sent => '0',
17095 Thread_id => '10',
17096 Tmp_table => 'No',
17097 Tmp_table_on_disk => 'No',
17098 arg => 'SELECT col FROM tbl WHERE id=5',
17099 bytes => 103,
17100 cmd => 'Query',
17101 db => 'db1',
17102 fingerprint => 'select col from tbl where id=?',
17103 host => '',
17104 pos_in_log => 1334,
17105 ts => '071218 11:48:27',
17106 user => '[SQL_SLAVE]'
17107 };
17108
17109=head2 COMMON
17110
17111These attribute are common to all input L<"--type"> and L<"--processlist">,
17112except where noted.
17113
17114=over
17115
17116=item arg
17117
17118The query text, or the command for admin commands like C<Ping>.
17119
17120=item bytes
17121
17122The byte length of the C<arg>.
17123
17124=item cmd
17125
17126"Query" or "Admin" for all except memcached. For memcached it's
17127the memcached command: get, set, etc.
17128
17129=item db
17130
17131The current database, except for memcached. The value comes from USE
17132database statements. By default, C<Schema> is an alias which is automatically
17133changed to C<db>; see L<"--attribute-aliases">.
17134
17135=item fingerprint
17136
17137An abstracted form of the query. See L<"FINGERPRINTS">.
17138
17139=item host
17140
17141Client host which executed the query.
17142
17143=item pos_in_log
17144
17145The byte offset of the event in the log or tcpdump,
17146except for L<"--processlist">.
17147
17148=item Query_time
17149
17150The total time the query took, including lock time.
17151
17152=item ts
17153
17154The timestamp of when the query ended.
17155
17156=back
17157
17158=head2 SLOW, GENERAL, AND BINARY LOGS
17159
17160Events have all available attributes from the log file. Therefore, you only
17161need to look at the log file to see which events are available, but remember:
17162not all events have the same attributes.
17163
17164Percona Server adds many attributes to the slow log; see
17165http://www.percona.com/doc/percona-server/5.1/diagnostics/slow_extended.html#changes-to-the-log-format
17166for more information.
17167
17168=head2 TCPDUMP
17169
17170These attributes are available when parsing L<"--type"> tcpdump.
17171
17172=over
17173
17174=item Error_no
17175
17176The MySQL error number if the query caused an error.
17177
17178=item ip
17179
17180The client's IP address. Certain log files may also contain this attribute.
17181
17182=item No_good_index_used
17183
17184Yes or No if no good index existed for the query (flag set by server).
17185
17186=item No_index_used
17187
17188Yes or No if the query did not use any index (flag set by server).
17189
17190=item port
17191
17192The client's port number.
17193
17194=item Warning_count
17195
17196The number of warnings, as otherwise shown by C<SHOW WARNINGS>.
17197
17198=back
17199
17200=head2 PROCESSLIST
17201
17202If using L<"--processlist">, an C<id> attribute is available for
17203the process ID, in addition to the common attributes.
17204
17205=head2 MEMCACHED
17206
17207These attributes are available when parsing L<"--type"> memcached.
17208
17209=over
17210
17211=item exptime
17212
17213Expiration time.
17214
17215=item key
17216
17217The key used by cmd.
17218
17219=item key_print
17220
17221An abstracted form of the key.
17222
17223=item Memc_add
17224
17225Yes/No if the command is add.
17226
17227=item Memc_append
17228
17229Yes/No if the command is append.
17230
17231=item Memc_cas
17232
17233Yes/No if the command is cas.
17234
17235=item Memc_error
17236
17237Yes/No if command caused an error. Currently, the only error is when
17238a retrieval command is interrupted.
17239
17240=item Memc_get
17241
17242Yes/No if the command is get.
17243
17244=item Memc_gets
17245
17246Yes/No if the command is gets.
17247
17248=item Memc_miss
17249
17250Yes/No if the command tried to access a nonexistent key.
17251
17252=item Memc_prepend
17253
17254Yes/No if the command is prepend.
17255
17256=item Memc_replace
17257
17258Yes/No if the command is replace.
17259
17260=item Memc_set
17261
17262Yes/No if the command is set.
17263
17264=item res
17265
17266Result of cmd.
17267
17268=item val
17269
17270The return value of cmd, if any.
17271
17272=back
17273
17072=head1 AUTHORS17274=head1 AUTHORS
1707317275
17074Baron Schwartz and Daniel Nichter17276Baron Schwartz and Daniel Nichter
1707517277
=== modified file 'bin/pt-stalk'
--- bin/pt-stalk 2012-09-20 13:59:16 +0000
+++ bin/pt-stalk 2012-10-08 18:50:28 +0000
@@ -933,6 +933,30 @@
933ITER=1933ITER=1
934934
935# ###########################################################################935# ###########################################################################
936# Plugin hooks
937# ###########################################################################
938
939before_stalk() {
940 :
941}
942
943before_collect() {
944 :
945}
946
947after_collect() {
948 :
949}
950
951after_collect_sleep() {
952 :
953}
954
955after_stalk() {
956 :
957}
958
959# ###########################################################################
936# Subroutines960# Subroutines
937# ###########################################################################961# ###########################################################################
938962
@@ -1016,12 +1040,12 @@
10161040
1017oktorun() {1041oktorun() {
1018 if [ $OKTORUN -eq 0 ]; then1042 if [ $OKTORUN -eq 0 ]; then
1019 EXIT_REASON="OKTORUN is false"1043 [ -z "$EXIT_REASON" ] && EXIT_REASON="OKTORUN is false"
1020 return 1 # stop running1044 return 1 # stop running
1021 fi1045 fi
10221046
1023 if [ -n "$OPT_ITERATIONS" ] && [ $ITER -gt $OPT_ITERATIONS ]; then1047 if [ -n "$OPT_ITERATIONS" ] && [ $ITER -gt $OPT_ITERATIONS ]; then
1024 EXIT_REASON="no more iterations"1048 [ -z "$EXIT_REASON" ] && EXIT_REASON="no more iterations"
1025 return 1 # stop running1049 return 1 # stop running
1026 fi1050 fi
10271051
@@ -1136,6 +1160,9 @@
1136 log "pt-stalk ran with $RAN_WITH" >> "$OPT_DEST/$prefix-trigger"1160 log "pt-stalk ran with $RAN_WITH" >> "$OPT_DEST/$prefix-trigger"
1137 last_prefix="$prefix"1161 last_prefix="$prefix"
11381162
1163 # Plugin hook:
1164 before_collect
1165
1139 # Fork and background the collect subroutine which will1166 # Fork and background the collect subroutine which will
1140 # run for --run-time seconds. We (the parent) sleep1167 # run for --run-time seconds. We (the parent) sleep
1141 # while its collecting (hopefully --sleep is longer than1168 # while its collecting (hopefully --sleep is longer than
@@ -1143,7 +1170,11 @@
1143 (1170 (
1144 collect "$OPT_DEST" "$prefix"1171 collect "$OPT_DEST" "$prefix"
1145 ) >> "$OPT_DEST/$prefix-output" 2>&1 &1172 ) >> "$OPT_DEST/$prefix-output" 2>&1 &
1146 log "Collector PID $!"1173 local collector_pid=$!
1174 log "Collector PID $collector_pid"
1175
1176 # Plugin hook:
1177 after_collect $collector_pid
1147 else 1178 else
1148 # There will not be enough disk space, so do not collect.1179 # There will not be enough disk space, so do not collect.
1149 warn "Collect canceled because there will not be enough disk space after collecting another $margin MB"1180 warn "Collect canceled because there will not be enough disk space after collecting another $margin MB"
@@ -1156,6 +1187,9 @@
1156 ITER=$((ITER + 1))1187 ITER=$((ITER + 1))
1157 cycles_true=01188 cycles_true=0
1158 sleep_ok "$OPT_SLEEP" "Sleeping $OPT_SLEEP seconds after collect"1189 sleep_ok "$OPT_SLEEP" "Sleeping $OPT_SLEEP seconds after collect"
1190
1191 # Plugin hook:
1192 after_collect_sleep
1159 else1193 else
1160 # Trigger/check/value is ok, sleep until next check.1194 # Trigger/check/value is ok, sleep until next check.
1161 sleep_ok "$OPT_INTERVAL"1195 sleep_ok "$OPT_INTERVAL"
@@ -1178,7 +1212,7 @@
1178 # Note: $$ is the parent's PID, but we're a child proc.1212 # Note: $$ is the parent's PID, but we're a child proc.
1179 # Bash 4 has $BASHPID but we can't rely on that. Consequently,1213 # Bash 4 has $BASHPID but we can't rely on that. Consequently,
1180 # we don't know our own PID. See the usage of $! below.1214 # we don't know our own PID. See the usage of $! below.
1181 RAN_WITH="--function=$OPT_FUNCTION --variable=$OPT_VARIABLE --threshold=$OPT_THRESHOLD --match=$OPT_MATCH --cycles=$OPT_CYCLES --interval=$OPT_INTERVAL --iterations=$OPT_ITERATIONS --run-time=$OPT_RUN_TIME --sleep=$OPT_SLEEP --dest=$OPT_DEST --prefix=$OPT_PREFIX --notify-by-email=$OPT_NOTIFY_BY_EMAIL --log=$OPT_LOG --pid=$OPT_PID"1215 RAN_WITH="--function=$OPT_FUNCTION --variable=$OPT_VARIABLE --threshold=$OPT_THRESHOLD --match=$OPT_MATCH --cycles=$OPT_CYCLES --interval=$OPT_INTERVAL --iterations=$OPT_ITERATIONS --run-time=$OPT_RUN_TIME --sleep=$OPT_SLEEP --dest=$OPT_DEST --prefix=$OPT_PREFIX --notify-by-email=$OPT_NOTIFY_BY_EMAIL --log=$OPT_LOG --pid=$OPT_PID --plugin=$OPT_PLUGIN"
11821216
1183 log "Starting $0 $RAN_WITH"1217 log "Starting $0 $RAN_WITH"
11841218
@@ -1190,9 +1224,15 @@
1190 # Make a secure tmpdir.1224 # Make a secure tmpdir.
1191 mk_tmpdir1225 mk_tmpdir
11921226
1227 # Plugin hook:
1228 before_stalk
1229
1193 # Stalk while oktorun.1230 # Stalk while oktorun.
1194 stalk1231 stalk
11951232
1233 # Plugin hook:
1234 after_stalk
1235
1196 # Clean up.1236 # Clean up.
1197 rm_tmpdir1237 rm_tmpdir
1198 remove_pid_file "$OPT_PID"1238 remove_pid_file "$OPT_PID"
@@ -1217,6 +1257,15 @@
1217 option_error "Invalid --function value: $OPT_FUNCTION"1257 option_error "Invalid --function value: $OPT_FUNCTION"
1218 fi1258 fi
12191259
1260 # Verify and source the --plugin.
1261 if [ "$OPT_PLUGIN" ]; then
1262 if [ -f "$OPT_PLUGIN" ]; then
1263 . "$OPT_PLUGIN"
1264 else
1265 option_error "Invalid --plugin value: $OPT_PLUGIN is not a file"
1266 fi
1267 fi
1268
1220 if [ -z "$OPT_STALK" -a "$OPT_COLLECT" ]; then1269 if [ -z "$OPT_STALK" -a "$OPT_COLLECT" ]; then
1221 # Not stalking; do immediate collect once.1270 # Not stalking; do immediate collect once.
1222 OPT_ITERATIONS=11271 OPT_ITERATIONS=1
@@ -1611,6 +1660,66 @@
16111660
1612Create a PID file when daemonized.1661Create a PID file when daemonized.
16131662
1663=item --plugin
1664
1665type: string
1666
1667Load a plugin to hook into the tool and extend is functionality.
1668The specified file does not need to be executable, nor does its first line
1669need to be shebang line. It only needs to define one or more of these
1670Bash functions:
1671
1672=over
1673
1674=item before_stalk
1675
1676Called before stalking.
1677
1678=item before_collect
1679
1680Called when the stalk condition is triggered, before running a collector
1681process as a backgrounded subshell.
1682
1683=item after_collect
1684
1685Called after running a collector process. The PID of the collector process
1686is passed as the first argument. This hook is called before
1687C<after_collect_sleep>.
1688
1689=item after_collect_sleep
1690
1691Called after sleeping L<"--sleep"> seconds for the collector process to finish.
1692This hook is called after C<after_collect>.
1693
1694=item after_stalk
1695
1696Called after stalking. Since pt-stalk stalks forever by default,
1697this hook is only called if L<"--iterations"> is specified.
1698
1699=back
1700
1701For example, a very simple plugin that touches a file when a collector
1702process is triggered:
1703
1704 before_colllect() {
1705 touch /tmp/foo
1706 }
1707
1708Since the plugin is completely sourced (imported) into the tool's namespace,
1709be careful not to define other functions or global variables that already
1710exist in the tool. You should prefix all plugin-specific functions and
1711global variables with C<plugin_> or C<PLUGIN_>.
1712
1713Plugins have access to all command line options but they should not modify
1714them. Each option is a global variable like C<$OPT_DEST> which corresponds
1715to L<"--dest">. Therefore, the global variable for each command line option
1716is C<OPT_> plus the option name in all caps with hyphens replaced by
1717underscores.
1718
1719Plugins can stop the tool by setting the global variable C<OKTORUN>
1720to C<1>. In this case, the global variable C<EXIT_REASON> should also
1721be set to indicate why the tool was stopped.
1722
1614=item --prefix 1723=item --prefix
16151724
1616type: string1725type: string
16171726
=== modified file 'bin/pt-table-checksum'
--- bin/pt-table-checksum 2012-10-05 22:28:45 +0000
+++ bin/pt-table-checksum 2012-10-08 18:50:28 +0000
@@ -3320,7 +3320,9 @@
3320 PTDEBUG && _d($sql);3320 PTDEBUG && _d($sql);
3321 my $row = $self->{dbh}->selectrow_arrayref($sql);3321 my $row = $self->{dbh}->selectrow_arrayref($sql);
3322 PTDEBUG && _d(defined $row ? @$row : 'undef');3322 PTDEBUG && _d(defined $row ? @$row : 'undef');
3323 $self->{is_cluster_node} = $row && $row->[0] ? 1 : 0;3323 $self->{is_cluster_node} = $row && $row->[1]
3324 ? ($row->[1] eq 'ON' || $row->[1] eq '1')
3325 : 0;
3324 3326
3325 return $self->{is_cluster_node};3327 return $self->{is_cluster_node};
3326}3328}
@@ -5146,6 +5148,9 @@
5146 my $func = $args{func} || uc($o->get('function'));5148 my $func = $args{func} || uc($o->get('function'));
5147 my $cols = $self->get_checksum_columns(%args);5149 my $cols = $self->get_checksum_columns(%args);
51485150
5151 die "all columns are excluded by --columns or --ignore-columns"
5152 unless @{$cols->{select}};
5153
5149 my $query;5154 my $query;
5150 if ( !$args{no_cols} ) {5155 if ( !$args{no_cols} ) {
5151 $query = join(', ',5156 $query = join(', ',
@@ -9340,11 +9345,20 @@
9340 # Make a nibble iterator for this table. This should only fail9345 # Make a nibble iterator for this table. This should only fail
9341 # if the table has no indexes and is too large to checksum in9346 # if the table has no indexes and is too large to checksum in
9342 # one chunk.9347 # one chunk.
9343 my $checksum_cols = $rc->make_chunk_checksum(9348 my $checksum_cols = eval {
9344 dbh => $master_cxn->dbh(),9349 $rc->make_chunk_checksum(
9345 tbl => $tbl,9350 dbh => $master_cxn->dbh(),
9346 %crc_args9351 tbl => $tbl,
9347 );9352 %crc_args
9353 );
9354 };
9355
9356 if ( $EVAL_ERROR ) {
9357 warn ts("Skipping table $tbl->{db}.$tbl->{tbl} because "
9358 . "$EVAL_ERROR\n");
9359 return;
9360 }
9361
9348 my $nibble_iter;9362 my $nibble_iter;
9349 eval {9363 eval {
9350 $nibble_iter = new OobNibbleIterator(9364 $nibble_iter = new OobNibbleIterator(
@@ -10664,7 +10678,8 @@
1066410678
10665short form: -c; type: array; group: Filter10679short form: -c; type: array; group: Filter
1066610680
10667Checksum only this comma-separated list of columns.10681Checksum only this comma-separated list of columns. If a table doesn't have
10682any of the specified columns it will be skipped.
1066810683
10669=item --config10684=item --config
1067010685
@@ -10780,6 +10795,8 @@
10780type: Hash; group: Filter10795type: Hash; group: Filter
1078110796
10782Ignore this comma-separated list of columns when calculating the checksum.10797Ignore this comma-separated list of columns when calculating the checksum.
10798If a table has all of its columns filtered by --ignore-columns, it will
10799be skipped.
1078310800
10784=item --ignore-databases10801=item --ignore-databases
1078510802
@@ -11313,6 +11330,12 @@
11313cannot be detected automatically. The lag check (see L<"REPLICA CHECKS">)11330cannot be detected automatically. The lag check (see L<"REPLICA CHECKS">)
11314is not performed for cluster nodes.11331is not performed for cluster nodes.
1131511332
11333Mixed replication setups are not currently supported. For example, the tool
11334does not work completely if the master host is replicating to a cluster,
11335or if the cluster is replicating to another cluster. In short, the only
11336supported setup is a single cluster with nodes optionally having traditional
11337replication slaves.
11338
11316=back11339=back
1131711340
11318=head1 BUGS11341=head1 BUGS
1131911342
=== modified file 'lib/Cxn.pm'
--- lib/Cxn.pm 2012-08-29 22:35:17 +0000
+++ lib/Cxn.pm 2012-10-08 18:50:28 +0000
@@ -203,7 +203,9 @@
203 PTDEBUG && _d($sql);203 PTDEBUG && _d($sql);
204 my $row = $self->{dbh}->selectrow_arrayref($sql);204 my $row = $self->{dbh}->selectrow_arrayref($sql);
205 PTDEBUG && _d(defined $row ? @$row : 'undef');205 PTDEBUG && _d(defined $row ? @$row : 'undef');
206 $self->{is_cluster_node} = $row && $row->[0] ? 1 : 0;206 $self->{is_cluster_node} = $row && $row->[1]
207 ? ($row->[1] eq 'ON' || $row->[1] eq '1')
208 : 0;
207 209
208 return $self->{is_cluster_node};210 return $self->{is_cluster_node};
209}211}
210212
=== modified file 'lib/RowChecksum.pm'
--- lib/RowChecksum.pm 2012-06-07 03:36:07 +0000
+++ lib/RowChecksum.pm 2012-10-08 18:50:28 +0000
@@ -67,6 +67,11 @@
67 my $func = $args{func} || uc($o->get('function'));67 my $func = $args{func} || uc($o->get('function'));
68 my $cols = $self->get_checksum_columns(%args);68 my $cols = $self->get_checksum_columns(%args);
6969
70 # Skip tables that have all their columns skipped; See
71 # https://bugs.launchpad.net/percona-toolkit/+bug/1016131
72 die "all columns are excluded by --columns or --ignore-columns"
73 unless @{$cols->{select}};
74
70 # Prepend columns to query, resulting in "col1, col2, FUNC(..col1, col2...)",75 # Prepend columns to query, resulting in "col1, col2, FUNC(..col1, col2...)",
71 # unless caller says not to. The only caller that says not to is76 # unless caller says not to. The only caller that says not to is
72 # make_chunk_checksum() which uses this row checksum as part of a larger77 # make_chunk_checksum() which uses this row checksum as part of a larger
7378
=== modified file 't/lib/Pingback.t'
--- t/lib/Pingback.t 2012-09-13 13:39:04 +0000
+++ t/lib/Pingback.t 2012-10-08 18:50:28 +0000
@@ -126,9 +126,19 @@
126 );126 );
127 }127 }
128128
129 my $expect_post;
130 if ( $args{post} ) {
131 $expect_post = join("\n",
132 map { "$_->{id};$_->{item};$_->{val}" }
133 sort {
134 $a->{item} cmp $b->{item} ||
135 $a->{id} cmp $b->{id}
136 } @{$args{post}});
137 $expect_post .= "\n";
138 }
129 is(139 is(
130 $post ? ($post->{content} || '') : '',140 $post ? ($post->{content} || '') : '',
131 $args{post},141 $expect_post || '',
132 "$args{name} client response"142 "$args{name} client response"
133 );143 );
134144
@@ -152,7 +162,18 @@
152 }162 }
153 ],163 ],
154 # client should POST this164 # client should POST this
155 post => "$general_id;Data::Dumper;$dd_ver\n$general_id;Perl;$perl_ver\n",165 post => [
166 {
167 item => 'Data::Dumper',
168 id => $general_id,
169 val => $dd_ver,
170 },
171 {
172 item => 'Perl',
173 id => $general_id,
174 val => $perl_ver,
175 },
176 ],
156 # Server should return these suggetions after the client posts177 # Server should return these suggetions after the client posts
157 sug => [178 sug => [
158 'Data::Printer is nicer.',179 'Data::Printer is nicer.',
@@ -174,7 +195,18 @@
174 content => "",195 content => "",
175 }196 }
176 ],197 ],
177 post => "$general_id;Data::Dumper;$dd_ver\n$general_id;Perl;$perl_ver\n",198 post => [
199 {
200 item => 'Data::Dumper',
201 id => $general_id,
202 val => $dd_ver,
203 },
204 {
205 item => 'Perl',
206 id => $general_id,
207 val => $perl_ver,
208 },
209 ],
178 sug => undef,210 sug => undef,
179);211);
180212
@@ -184,7 +216,7 @@
184 name => "No response to GET",216 name => "No response to GET",
185 response => [],217 response => [],
186 no_response => 1,218 no_response => 1,
187 post => "",219 post => undef,
188 sug => undef,220 sug => undef,
189);221);
190222
@@ -199,7 +231,18 @@
199 content => "Perl;perl_version;PERL_VERSION\nData::Dumper;perl_module_version\n",231 content => "Perl;perl_version;PERL_VERSION\nData::Dumper;perl_module_version\n",
200 },232 },
201 ],233 ],
202 post => "$general_id;Data::Dumper;$dd_ver\n$general_id;Perl;$perl_ver\n",234 post => [
235 {
236 id => $general_id,
237 item => 'Data::Dumper',
238 val => $dd_ver,
239 },
240 {
241 id => $general_id,
242 item => 'Perl',
243 val => $perl_ver,
244 },
245 ],
203 sug => undef,246 sug => undef,
204);247);
205248
@@ -223,7 +266,13 @@
223 }266 }
224 ],267 ],
225 # client should POST this268 # client should POST this
226 post => "$master_id;MySQL;$mysql_ver $mysql_distro\n",269 post => [
270 {
271 id => $master_id,
272 item => 'MySQL',
273 val => "$mysql_ver $mysql_distro",
274 }
275 ],
227 # Server should return these suggetions after the client posts276 # Server should return these suggetions after the client posts
228 sug => ['Percona Server is fast.'],277 sug => ['Percona Server is fast.'],
229 );278 );
@@ -440,7 +489,18 @@
440 }489 }
441 ],490 ],
442 # client should POST this491 # client should POST this
443 post => "$slave1_id;MySQL;$mysql_ver $mysql_distro\n$master_id;MySQL;$mysql_ver $mysql_distro\n",492 post => [
493 {
494 id => $slave1_id,
495 item => 'MySQL',
496 val => "$mysql_ver $mysql_distro",
497 },
498 {
499 id => $master_id,
500 item => 'MySQL',
501 val => "$mysql_ver $mysql_distro",
502 }
503 ],
444 # Server should return these suggetions after the client posts504 # Server should return these suggetions after the client posts
445 sug => [505 sug => [
446 'Percona Server is fast.',506 'Percona Server is fast.',
447507
=== modified file 't/lib/RowChecksum.t'
--- t/lib/RowChecksum.t 2012-06-03 17:54:32 +0000
+++ t/lib/RowChecksum.t 2012-10-08 18:50:28 +0000
@@ -26,9 +26,6 @@
26if ( !$dbh ) {26if ( !$dbh ) {
27 plan skip_all => "Cannot connect to sandbox master";27 plan skip_all => "Cannot connect to sandbox master";
28}28}
29else {
30 plan tests => 29;
31}
3229
33$sb->create_dbs($dbh, ['test']);30$sb->create_dbs($dbh, ['test']);
3431
@@ -421,9 +418,53 @@
421 'Ignores specified columns'418 'Ignores specified columns'
422);419);
423420
421# #############################################################################
422# crash with --columns if none match / --ignore-columns if everything is ignored
423# https://bugs.launchpad.net/percona-toolkit/+bug/1016131
424# #############################################################################
425# Re-using the $tbl from the previous test!
426local @ARGV = ('--ignore-columns', 'a,b,c');
427$o->get_opts();
428local $EVAL_ERROR;
429eval {
430 $c->make_row_checksum(
431 tbl => $tbl,
432 func => 'CRC32',
433 );
434};
435
436like(
437 $EVAL_ERROR,
438 qr/all columns are excluded by --columns or --ignore-columns/,
439 "Dies if all columns are ignored by --ignore-columns"
440);
441
442
443$tbl = {
444 db => 'mysql',
445 tbl => 'user',
446 tbl_struct => $tp->parse($tp->get_create_table($dbh, 'mysql', 'user')),
447};
448local @ARGV = qw(--columns some_column_that_doesnt_exist);
449$o->get_opts();
450local $EVAL_ERROR;
451eval {
452 $c->make_row_checksum(
453 tbl => $tbl,
454 func => 'SHA1',
455 );
456};
457
458like(
459 $EVAL_ERROR,
460 qr/all columns are excluded by --columns or --ignore-columns/,
461 'Dies if all columns are ignored by --columns'
462);
463
424# ############################################################################464# ############################################################################
425# Done.465# Done.
426# ############################################################################466# ############################################################################
427$sb->wipe_clean($dbh);467$sb->wipe_clean($dbh);
428ok($sb->ok(), "Sandbox servers") or BAIL_OUT(__FILE__ . " broke the sandbox");468ok($sb->ok(), "Sandbox servers") or BAIL_OUT(__FILE__ . " broke the sandbox");
429exit;469
470done_testing;
430471
=== added file 't/pt-stalk/plugin.t'
--- t/pt-stalk/plugin.t 1970-01-01 00:00:00 +0000
+++ t/pt-stalk/plugin.t 2012-10-08 18:50:28 +0000
@@ -0,0 +1,74 @@
1#!/usr/bin/env perl
2
3BEGIN {
4 die "The PERCONA_TOOLKIT_BRANCH environment variable is not set.\n"
5 unless $ENV{PERCONA_TOOLKIT_BRANCH} && -d $ENV{PERCONA_TOOLKIT_BRANCH};
6 unshift @INC, "$ENV{PERCONA_TOOLKIT_BRANCH}/lib";
7};
8
9use strict;
10use warnings FATAL => 'all';
11use English qw(-no_match_vars);
12use Test::More;
13use Time::HiRes qw(sleep);
14
15use PerconaTest;
16use DSNParser;
17use Sandbox;
18
19my $dp = new DSNParser(opts=>$dsn_opts);
20my $sb = new Sandbox(basedir => '/tmp', DSNParser => $dp);
21my $dbh = $sb->get_dbh_for('master');
22
23if ( !$dbh ) {
24 plan skip_all => 'Cannot connect to sandbox master';
25}
26
27my $cnf = "/tmp/12345/my.sandbox.cnf";
28my $pid_file = "/tmp/pt-stalk.pid.$PID";
29my $log_file = "/tmp/pt-stalk.log.$PID";
30my $dest = "/tmp/pt-stalk.collect.$PID";
31my $output;
32my $retval;
33my $pid;
34
35diag(`rm $pid_file 2>/dev/null`);
36diag(`rm $log_file 2>/dev/null`);
37diag(`mkdir $dest`);
38
39# We'll have to watch Uptime since it's the only status var that's going
40# to be predictable.
41my (undef, $uptime) = $dbh->selectrow_array("SHOW STATUS LIKE 'Uptime'");
42my $threshold = $uptime + 2;
43
44$retval = system("$trunk/bin/pt-stalk --iterations 1 --dest $dest --variable Uptime --threshold $threshold --cycles 1 --run-time 2 --pid $pid_file --plugin $trunk/t/pt-stalk/samples/plugin001.sh -- --defaults-file=$cnf >$log_file 2>&1");
45
46PerconaTest::wait_until(sub { !-f $pid_file });
47
48is(
49 $retval >> 8,
50 0,
51 "Exit 0"
52);
53
54foreach my $hook (qw(
55 before_stalk
56 before_collect
57 after_collect
58 after_collect_sleep
59 after_stalk
60)) {
61 ok(
62 -f "$dest/$hook",
63 "$hook hook called"
64 );
65}
66
67# #############################################################################
68# Done.
69# #############################################################################
70diag(`rm $pid_file 2>/dev/null`);
71diag(`rm $log_file 2>/dev/null`);
72diag(`rm -rf $dest 2>/dev/null`);
73ok($sb->ok(), "Sandbox servers") or BAIL_OUT(__FILE__ . " broke the sandbox");
74done_testing;
075
=== added file 't/pt-stalk/samples/plugin001.sh'
--- t/pt-stalk/samples/plugin001.sh 1970-01-01 00:00:00 +0000
+++ t/pt-stalk/samples/plugin001.sh 2012-10-08 18:50:28 +0000
@@ -0,0 +1,21 @@
1#!/bin/sh
2
3before_stalk() {
4 date >> "$OPT_DEST/before_stalk"
5}
6
7before_collect() {
8 date >> "$OPT_DEST/before_collect"
9}
10
11after_collect() {
12 date >> "$OPT_DEST/after_collect"
13}
14
15after_collect_sleep() {
16 date >> "$OPT_DEST/after_collect_sleep"
17}
18
19after_stalk() {
20 date >> "$OPT_DEST/after_stalk"
21}
022
=== modified file 't/pt-table-checksum/bugs.t'
--- t/pt-table-checksum/bugs.t 2012-07-27 17:52:24 +0000
+++ t/pt-table-checksum/bugs.t 2012-10-08 18:50:28 +0000
@@ -176,6 +176,22 @@
176);176);
177177
178# #############################################################################178# #############################################################################
179# pt-table-checksum can crash with --columns if none match
180# https://bugs.launchpad.net/percona-toolkit/+bug/1016131
181# #############################################################################
182
183($output) = full_output(
184 sub { pt_table_checksum::main(@args, '--tables', 'mysql.user,mysql.host',
185 '--columns', 'some_fale_column') },
186);
187
188like(
189 $output,
190 qr/\QSkipping table mysql.user because all columns are excluded by --columns or --ignore-columns/,
191 "Bug 1016131: ptc should skip tables where all columns are excluded"
192);
193
194# #############################################################################
179# Done.195# Done.
180# #############################################################################196# #############################################################################
181$sb->wipe_clean($master_dbh);197$sb->wipe_clean($master_dbh);
182198
=== modified file 't/pt-table-checksum/run_time.t'
--- t/pt-table-checksum/run_time.t 2012-07-18 16:07:27 +0000
+++ t/pt-table-checksum/run_time.t 2012-10-08 18:50:28 +0000
@@ -38,16 +38,16 @@
38my $output;38my $output;
39my $exit_status;39my $exit_status;
4040
41# On my 2.4 GHz with SSD this takes a little more than 3s,41# On my 2.4 GHz with SSD this takes a little more than 5s,
42# so no test servers should be faster, hopefully.42# so no test servers should be faster, hopefully.
43my $t0 = time;43my $t0 = time;
44$exit_status = pt_table_checksum::main(@args,44$exit_status = pt_table_checksum::main(@args,
45 qw(--quiet --quiet -d sakila --chunk-size 100 --run-time 1));45 qw(--quiet --quiet -d sakila --chunk-size 50 --run-time 1));
46my $t = time - $t0;46my $t = time - $t0;
4747
48ok(48ok(
49 $t >= 1.5 && $t <= 2.0,49 $t >= 1.5 && $t <= 2.5,
50 "Run in roughly --run-time 1 second"50 "Ran in roughly --run-time 1 second"
51) or diag("Actual run time: $t");51) or diag("Actual run time: $t");
5252
53my $rows = $master_dbh->selectall_arrayref("SELECT DISTINCT CONCAT(db, '.', tbl) FROM percona.checksums ORDER by db, tbl");53my $rows = $master_dbh->selectall_arrayref("SELECT DISTINCT CONCAT(db, '.', tbl) FROM percona.checksums ORDER by db, tbl");

Subscribers

People subscribed via source and target branches

to all changes: