We introduce a new set of synchronisation primitives ("query ready")
that are signaled when the server is ready to take queries, and make
spider initialisation use them instead of the primitives associated
with server start.
During spider initialisation, if the server is query ready, proceed
with the initialisation synchronously (requiring the sql
service). Otherwise, leave the init to the first sts bg thread as
usual.
When creating a spider handler before the spider init is done, if the
server is not query ready, just give up and return NULL. If the server
is query ready, then wait for a specific amount of time (5s) for the
spider init to be done, and on timeout give up and return NULL.
How this fixes / prevents various init bugs:
MDEV-27233 (spider installed and used in init file). Before -
spider_create_handler() called by read_init_file() waits for bg thread
to finish init, whereas the bg thread is waiting for server
start. After - spider init is run synchronously because the server is
query ready when read_init_file(). No funny business with concurrency.
MDEV-29904 (spider loaded with plugin-load-add). Before (before the
original patch to MDEV-27233 was reverted) - fully synchronous spider
init fails due to dependencies on Aria / acl init. After - spider init
is run async, and as soon as the server is query ready, the init
proceeds and succeeds.
MDEV-22979 (mysqld --bootstrap --plugin-load-add=ha_spider). Before -
bootstrap queries causes call to spider_create_handler(), which waits
for bg thread to finish init, whereas bg thread waits for server
start. After - spider init is run async, waiting for server to be
query ready. spider_create_hander() notices bg thread still trying to
init spider, and server is not query ready so it gives up.
MDEV-28218 (spider is loaded in a query after server start,
immediately before another query dropping a spider table). Before - bg
thread is created to init spider async, while the server proceeds to
the next query that tries to drop a spider system table. race
condition causes the situation where the server holds the mdl lock to
the table and waits for the bg thread in spider_create_handler(), and
the bg thread waiting for the mdl lock to create the table. After -
since server has already started, spider init is run synchronously,
thus no concurrency issues. Same applies to the variant of MDEV-28218
represented by mdev_28218_init_file where the queries are in an init
file.
Another variant of MDEV-28218 represented by mdev_28218_mixed loads
spider with plugin-load-add and executes the drop table query in an
init_file. Spider init is async because the server was not query
ready. As soon as the the server becomes query ready the server tries
to execute the drop table query. It waits on the init in
spider_create_handler(), but because of the deadlock it times out
after 5 seconds. The drop table query did not find any table, but
spider init then proceeds as it finally acquires the mdl lock, and
creates the table. The moral of the story is to advise users not to
mix plugin-load-add=ha_spider with init_file containing queries
related to spider system tables. Queries using spider engine to create
tables are still ok because there's no deadlock.
MDEV-30370 (mysqld --wsrep-recover --plugin-load-add=spider). The
timed wait in the patch for MDEV-30370 still applies, so it still
passes.
More on the design:
Compared to the server start primitives, the query ready primitives
are signaled earlier ("query ready point") to avoid unnecessary wait
and deadlocks.
The spider initialisation does not only require other plugins like
Aria to be initialised first, but also acl init (due to its creation
of udfs) as well as the creation of the `mysql` database (due to its
system tables being in this database). Thus the point where it can go
ahead with the initialisation should be after the bootstrap.
By the time the server can accept queries by reading from init files,
it can definitely accept all spider init queries. So the point of
spider initialisation can be before this point.
Looking at the lines between boostrap and reading init file, the most
natural position is just before the call to read_init_file(), where we
can also give the primitives a most meaningful name "query ready".
Removing procedures that were created and dropped during init.
This also fixes a race condition where mtr test with
plugin-load-add=ha_spider.so causes post test check to fail as it
expects the procedures to still be there.
MDEV-27095 installing one spider plugin should not trigger others
There are several plugins in ha_spider: spider, spider_alloc_mem,
spider_wrapper_protocols, spider_rewrite etc.
INSTALL PLUGIN foo SONAME ha_spider causes all the other ones to be
installed by the init queries where foo is any of the plugins.
This introduces unnecessary complexiy. For example it reads
mysql.plugins to find all other plugins, causing the hack of moving
spider plugin init to a separate thread.
To install all spider related plugins, install soname ha_spider should
be used instead.
This also fixes spurious rows in mysql.plugin when installing say only
the spider plugin with `plugin-load-add=SPIDER=ha_spider.so`:
select * from mysql.plugin;
name dl
spider_alloc_mem ha_spider.so # should not be here
spider_wrapper_protocols ha_spider.so # should not be here
Adapted from part of the reverted commit
c160a115b8b6dcd54bb3daf1a751ee9c68b7ee47.
MDEV-31147 json_normalize does not work correctly with MSAN build
json_normalize_number(): Avoid accessing str past str_len.
The function would seem to work incorrectly when some digits are
not followed by a decimal point (.) or an exponent (E or e).
MDEV-29401 InnoDB history list length increased in 10.6 compared to 10.5
The InnoDB buffer pool and locking were heavily refactored in
MariaDB Server 10.6. Among other things, dict_sys.mutex was removed,
and the contended lock_sys.mutex was replaced with a combination of
lock_sys.latch and distributed latches in hash tables. Also, a
default value was changed to innodb_flush_method=O_DIRECT to improve
performance in write-heavy workloads.
One thing where an adjustment was missing is around the parameters
innodb_max_purge_lag (number of committed transactions waiting to
be purged), and innodb_max_purge_lag_delay
(maximum number of microseconds to delay a DML operation).
purge_coordinator_state::do_purge(): Pass the history_size to trx_purge()
and reset srv_dml_needed_delay if the history is empty.
Keep executing the loop non-stop as long as srv_dml_needed_delay is set.
trx_purge_dml_delay(): Made part of trx_purge().
Set srv_dml_needed_delay=0 when nothing can be purged (!n_pages_handled).
row_mysql_delay_if_needed(): Mimic the logic of
innodb_max_purge_lag_wait_update().
Cleanup of MDEV-14974: --port ignored for --host=localhost
The old code added to 10.6 was inconsisting in how TCP/IP and
socket connection was chosen. One got also a confusing warning
in some cases.
Examples:
> ../client/mysql --print-defaults
../client/mysql would have been started with the following arguments:
--socket=/tmp/mariadbd.sock --port=3307 --no-auto-rehash
> ../client/mysql
ERROR 2002 (HY000): Can't connect to local server through socket '/tmp/mariadbd.sock' (2)
> ../client/mysql --print-defaults
../client/mysql would have been started with the following arguments:
--socket=/tmp/mariadbd.sock --port=3307 --no-auto-rehash
> ../client/mysql --port=3333
WARNING: Forcing protocol to TCP due to option specification. Please explicitly state intended protocol.
ERROR 2002 (HY000): Can't connect to server on 'localhost' (111)
> ../client/mysql --port=3333 --socket=sss
ERROR 2002 (HY000): Can't connect to local server through socket 'sss' (2)
> ../client/mysql --socket=sss --port=3333
ERROR 2002 (HY000): Can't connect to local server through socket 'sss' (2)
Some notable things:
- One gets a warning if one uses just --port if config file sets socket
- Using port and socket gives no warning
- Using socket and then port still uses socket
This patch changes things the following ways:
If --port= is given on the command line, the the protocol is automatically
changed to "TCP/IP".
- If --socket= is given on the command line, the protocol is automatically
changed to "socket".
- The last option wins
- No warning is given if protocol changes automatically.
dh_missing: warning: usr/share/mysql/mysql-test/mysql-test-run exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/share/mysql/mysql-test/mtr exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/share/mysql/mysql-test/mariadb-test-run exists in debian/tmp but is not installed to anywhere
dh_missing: warning: usr/share/mysql/mysql-test/mysql-test-run.pl exists in debian/tmp but is not installed to anywhere
Add all to mariadb-test.install and remove mariadb-test.links
