Note this is currently a work-in-progress, pushed for
potential early feedback.
It addresses:
1) The original thread_id of the master is propagated into
the thread_id of the slave-binlogged GTID thread_id,
rather than the thread_id of the applier thread.
2) When seeing Q_DUMMY, rather than loop through each
status var byte, skip to the end of the status var
section.
3) Added comments to explain the purpose of Q_DUMMY
4) Simplified Query_log_event::begin_event to always use
Q_DUMMY, rather than keeping the time zone str as well
5) Fixed DBUG_ASSERT to be part of the conditoin to read in
the event thread id, as to not read unowned memory
6) Removed invalid ASSERTIONS for begin_event()
- --parallel=N with or without --single-transaction
- Error cases (too many connections, emulate error on one connection)
- Windows specific test for named pipe connections
MDEV-32216 Connection pool with asynchronous query execution.
Parallelism is achieved by using mysql_send_query on multiple connections
without waiting for results, and using IO multiplexing (poll/IOCP) to
wait for completions.
Refresh libmariadb to pick up CONC-676 (fixes for IOCP use with named pipe)
- make connect_to_db() return MYSQL*, we'll reuse the function for
connection pool.
- Remove variable 'mysql_connection', duplicated by variable 'mysql'
- do not attempt to start slave if connection did not succeed,#
and fix mysqldump.result
Improve the performance of slave connect using B+-Tree indexes on each binlog
file. The index allows fast lookup of a GTID position to the corresponding
offset in the binlog file, as well as lookup of a position to find the
corresponding GTID position.
This eliminates a costly sequential scan of the starting binlog file
to find the GTID starting position when a slave connects. This is
especially costly if the binlog file is not cached in memory (IO
cost), or if it is encrypted or a lot of slaves connect simultaneously
(CPU cost).
The size of the index files is generally less than 1% of the binlog data, so
not expected to be an issue.
Most of the work writing the index is done as a background task, in
the binlog background thread. This minimises the performance impact on
transaction commit. A simple global mutex is used to protect index
reads and (background) index writes; this is fine as slave connect is
a relatively infrequent operation.
Here are the user-visible options and status variables. The feature is on by
default and is expected to need no tuning or configuration for most users.
binlog_gtid_index
On by default. Can be used to disable the indexes for testing purposes.
binlog_gtid_index_page_size (default 4096)
Page size to use for the binlog GTID index. This is the size of the nodes
in the B+-tree used internally in the index. A very small page-size (64 is
the minimum) will be less efficient, but can be used to stress the
BTree-code during testing.
binlog_gtid_index_span_min (default 65536)
Control sparseness of the binlog GTID index. If set to N, at most one
index record will be added for every N bytes of binlog file written.
This can be used to reduce the number of records in the index, at
the cost only of having to scan a few more events in the binlog file
before finding the target position
Two status variables are available to monitor the use of the GTID indexes:
Binlog_gtid_index_hit
Binlog_gtid_index_miss
The "hit" status increments for each successful lookup in a GTID index.
The "miss" increments when a lookup is not possible. This indicates that the
index file is missing (eg. binlog written by old server version
without GTID index support), or corrupt.
Signed-off-by: Kristian Nielsen <email address hidden>
