Author: Nayuta Yanagisawa
Author Date: 2022-06-29 16:58:57 UTC

MDEV-27172 Prefix indices on Spider tables may lead to wrong query results

Spider converts HA_READ_KEY_EXACT to the equality (=) in the
function spider_db_append_key_where_internal() but the conversion
is not necessarily correct for tables with prefix indices.

We fix the bug by converting HA_READ_KEY_EXACT to 'LIKE "foo%"' when
a target key is a prefix key. The fix is partly inspired by FEDERATED.
See ha_federated::create_where_from_key() for more details.

Author: Julius Goryavsky
Author Date: 2022-06-30 01:44:50 UTC

MDEV-28758: Mariabackup copies binary logs to backup directory

This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, as it was
before MDEV-27524 - it transfers one last binlog file by default.

The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.

Author: Julius Goryavsky
Author Date: 2022-06-29 20:25:05 UTC

MDEV-28758: Mariabackup copies binary logs to backup directory

This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, as it was
before MDEV-27524 - it transfers one last binlog file by default.

The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.

Author: Nikita Malyavin
Author Date: 2022-06-29 22:49:18 UTC

MDEV-28771 control Cache_flip_event_log lifetime with reference count

Author: Nikita Malyavin
Author Date: 2022-06-29 17:16:19 UTC

MDEV-28930 ALTER TABLE Deadlocks with parallel TL_WRITE

ALTER ONLINE TABLE acquires table with TL_READ. Myisam normally acquires
TL_WRITE for DML, which makes it hang until table is freed.

We deadlock once ALTER upgrades its MDL lock.

Solution:
Unlock table earlier. We don't need to hold TL_READ once we finished
copying. Relay log replication requires no data locks on `from` table.

Author: midenok
Author Date: 2022-06-29 21:26:40 UTC

TMP: versioning.foreign debugging

Author: Julius Goryavsky
Author Date: 2022-06-29 20:25:05 UTC

MDEV-28758: Mariabackup copies binary logs to backup directory

This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, as it was
before MDEV-27524 - it transfers one last binlog file by default.

The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.

Author: midenok
Author Date: 2022-06-29 19:53:29 UTC

MDEV-28567 Assertion `0' in open_tables upon function-related operation

DBUG_ASSERT(0) was added by MDEV-17554 (auto-create history
partitions) as an experimental measure. Testing has shown this
conditional branch of can_recover_from_failed_open() can be possible
due to MDL deadlock.

The fix replaces DBUG_ASSERT with more specific one for
!OT_ADD_HISTORY_PARTITION.

Test case was synchronized to reproduce deadlock always and commented
with execution path of MDL locking for Good (no deadlock) and Bad
(deadlock). The logging was done with the help of preceding patch for
debug MDL tracing.

Author: Vladislav Vaintroub
Author Date: 2022-06-29 17:44:04 UTC

MDEV-28982- race condition in charset_client_win.

Do not use chcp when mtr is running in parallel mode, since it possibly
resets the console charset, shared by all mtr processes and threads.

Author: Vladislav Vaintroub
Author Date: 2022-06-29 17:44:04 UTC

MDEV-28982- race condition in charset_client_win.

Do not use chcp when mtr is running in parallel mode, since it possibly
resets the console charset, shared by all mtr processes and threads.

Author: Brandon Nesterenko
Author Date: 2022-06-27 18:29:10 UTC

MDEV-27161: Add option for SQL thread to limit maximum execution time per query replicated

New Feature:
============
This patch adds a new system variable, slave_max_statement_time,
which limits the execution time of s slave’s events, as well as a
new column for SHOW ALL SLAVES STATUS, Slave_statement_timeouts,
which indicates the number of statements which have timed out on a
slave.

Reviewed By:
============
<TODO>

Author: Sergei Golubchik
Author Date: 2022-06-29 16:01:20 UTC

separate online_alter_cache_data from binlog_cache_data

Author: Vlad Lesin
Author Date: 2022-06-29 15:51:19 UTC

MDEV-28709 unexpected X lock on Supremum in READ COMMITTED

The lock is created during page splitting after moving records and
locks(lock_move_rec_list_(start|end)()) to the new page, and inheriting
the locks to the supremum of left page from the successor of the infimum
on right page. There is no need in such inheritance for READ COMMITTED
isolation level and not-gap locks.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-06-29 12:56:10 UTC

MDEV-28838 password_reuse_check plugin mixes username and password

To prevent the problem of mixing user name and password and
host name and user name we add length of the hostname and user name
to the hash.

Author: qggcs
Author Date: 2022-06-25 14:03:08 UTC

json_type should consider the charset mbmaxlen

Author: Vlad Lesin
Author Date: 2022-06-29 14:03:56 UTC

MDEV-28709 unexpected X lock on Supremum in READ COMMITTED

The lock is created during page splitting after moving records and
locks(lock_move_rec_list_(start|end)()) to the new page, and inheriting
the locks to the supremum of left page from the successor of the infimum
on right page. There is no need in such inheritance for READ COMMITTED
isolation level and not-gap locks.

Author: Oleg Smirnov
Author Date: 2022-06-18 13:54:39 UTC

MDEV-26427 Draft

1. Fix server crash (but inserted result is still incorrect).
2. Implement correct execution of queries like "insert into t select sum(1)"

Author: Nayuta Yanagisawa
Author Date: 2021-10-13 13:45:26 UTC

MDEV-26544 Assertion `part_share->auto_inc_initialized' failed in ha_partition::get_auto_increment on INSERT

The partition storage engine ignores return (error) values of
handler::info(). As a result, a query that should be aborted is
not aborted and then the server violates the assertion.

Author: Sergei Golubchik
Author Date: 2022-05-30 14:04:09 UTC

C/C 3.3

Author: Sergei Golubchik
Author Date: 2022-05-30 14:04:09 UTC

C/C 3.3

Author: Marko Mäkelä
Author Date: 2022-06-29 13:22:45 UTC

Merge 10.9 into 10.10

Author: Marko Mäkelä
Author Date: 2022-06-29 13:22:22 UTC

Merge 10.8 into 10.9

Author: Marko Mäkelä
Author Date: 2022-06-29 13:17:14 UTC

Merge 10.6 into 10.7

Author: Monty
Author Date: 2022-06-03 10:24:18 UTC

Reduced size of POSITION

Replaced Cost_estimate prefix_cost with a double as prefix_cost was
only used to store and retrive total prefix cost.

This also speeds up things (a bit) as don't have to call
Cost_estimate::total_cost() for every access to the prefix_cost.

Sizeof POSITION decreased from 304 to 256.

Author: midenok
Author Date: 2022-04-07 12:42:01 UTC

Stopword upgrade/downgrade test

Plus some history generated to make stopword,vers_stopword better
tested.

Author: Julius Goryavsky
Author Date: 2022-06-29 03:05:57 UTC

MDEV-27524 addendum: additional tests

Author: Alexander Barkov
Author Date: 2022-06-08 11:12:03 UTC

MDEV-23287 The INET4 data type

Author: midenok
Author Date: 2022-06-17 23:19:41 UTC

MDEV-28567 MDL debug logging

Log MDL state transitions. Trace-friendly message
format. DBUG_LOCK_FILE replaced by thread-local storage.

Logged states legend:
  Seized lock was acquired without waiting
  Waiting lock is waiting
  Acquired lock was acquired after waiting
  Released lock was released
  Deadlock lock was aborted due to deadlock
  Timeout lock was aborted due to timeout >0
  Nowait lock was aborted due to zero timeout
  Killed lock was aborted due to kill message
  OOM can not acquire because out of memory

Usage:
  mtr --mysqld=--debug=d,mdl,query:i:o,/tmp/mdl.log

Cleanup from garbage messages:
  sed -i -re \
  '/(mysql|performance_schema|sys|mtr)\// d; /MDL_BACKUP_/ d' \
  /tmp/mdl.log

Author: midenok
Author Date: 2022-06-17 23:19:41 UTC

MDEV-28567 MDL debug logging

Log MDL state transitions. Trace-friendly message
format. DBUG_LOCK_FILE replaced by thread-local storage.

Logged states legend:
  Seized lock was acquired without waiting
  Waiting lock is waiting
  Acquired lock was acquired after waiting
  Released lock was released
  Deadlock lock was aborted due to deadlock
  Timeout lock was aborted due to timeout >0
  Nowait lock was aborted due to zero timeout
  Killed lock was aborted due to kill message
  OOM can not acquire because out of memory

Usage:
  mtr --mysqld=--debug=d,mdl,query:i:o,/tmp/mdl.log

Cleanup from garbage messages:
  sed -i -re \
  '/(mysql|performance_schema|sys|mtr)\// d; /MDL_BACKUP_/ d' \
  /tmp/mdl.log

Author: Sergei Golubchik
Author Date: 2022-05-30 14:04:09 UTC

C/C 3.3

Author: Nikita Malyavin
Author Date: 2022-06-28 14:44:23 UTC

MDEV-28771 Assertion `table->in_use && tdc->flushed' failed after ALTER

Setting `table->s->tdc->flushed= 1` is unfortunately error-prone.
The three-way join made before ALTER TABLE remains tdc->ref_count equal 3.

Most likely, Monty's S3 fix at 2c8c15483 is also affected.

We lazily decrement ref_count, only when FLUSH is requested. In this case,
it represents a number of available cached TC instances, which are also
freed lazily, by a TDC_element::flush_unused call.

Calling flush_unused would be also incorrect.
If it will be called in parallel with following:
BEGIN
UPDATE t SET a = a+1;
// suppose flush_unused is done here
COMMIT

then it will release a share, though that transaction would still hold an
MDL lock. So it ends up with another SIGSEGV

More details on tdc->flushed:
In tc_release_table:
  if (table->needs_reopen() || table->s->tdc->flushed ||
      tc[i].records > tc_size)
  {
    tc[i].records--;
    mysql_mutex_unlock(&tc[i].LOCK_table_cache);
    tc_remove_table(table);
  }
This is called on table close. It releases only currently used table
instances, but not those two additional, used during 3-way join.

ref_count is incremented when new tdc is inserted in lf_hash, or when new
TC is acquired (which is also treated as a cache miss, see
tdc_acquire_share)

It is decremented mostly on FLUSH/REPAIR/etc, by a share->tdc->flush_unused
call, so it really represents a number of allocated TC elements. Anyway,
flush_unused call requires MDL_EXCLUSIVE to work properly.

Author: Marko Mäkelä
Author Date: 2022-06-28 13:30:14 UTC

WIP MDEV-21098: validate index records better

page_rec_get_nth(): Return nullptr on error.

page_dir_slot_get_rec_validate(): Like page_dir_slot_get_rec(),
but validate the pointer and return nullptr on error.

page_cur_search_with_match(), page_cur_search_with_match_bytes(),
page_dir_split_slot(): Indicate failure in a return value.

page_cur_search(): Replaced with page_cur_search_with_match().

TODO: check all return values of:
page_rec_get_next() in btr/ and gis/
page_rec_get_next_const()
page_cur_move_to_next()
rec_get_next_ptr_const() = page_rec_get_next_const()
rec_get_next_ptr() = page_rec_get_next()

Author: Monty
Author Date: 2022-06-28 12:57:41 UTC

MDEV-28897 Wrong table.get_ref_count() upon concurrent truncate and backup stage operation

The issue was that flush_tables() didn't take a MDL lock on cached
TABLE_SHARE before calling open_table() to do a HA_EXTRA_FLUSH call.
Most engines seams to have no issue with it, but apparantly this conflicts
with InnoDB in 10.6 when using TRUNCATE

Fixed by taking a MDL lock before trying to open the table in
flush_tables().

There is no test case as it hard to repeat the scheduling that causes
the error. I did run the test case in MDEV-28897 to verify
that the bug is fixed.

Author: midenok
Author Date: 2022-06-27 20:40:30 UTC

MDEV-28956 Locking is broken if CREATE OR REPLACE fails under LOCK TABLES

add_back_last_deleted_lock() was called when the lock was never
removed. Lock is removed in finalize_atomic_replace() in
close_all_tables_for_name(). finalize_atomic_replace() is done only
for successful operation.

In non-atomic codepath it drops the table first, if anything fails
later we don't need to return back the lock since there is no table
now. So the fix is required as well.

Author: Julius Goryavsky
Author Date: 2022-06-28 11:55:42 UTC

MDEV-27524 addendum: additional tests

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2022-06-28 10:21:05 UTC

MDEV-28919 Assertion `(((core_null) + 7) >> 3) == oindex.n_core_null_bytes || !not_redundant()' failed

- In case of discarded tablespace, InnoDB can't read the root page to
assign the n_core_null_bytes. Consecutive instant DDL fails because
of non-matching n_core_null_bytes.

Author: Alexander Barkov
Author Date: 2022-06-28 07:38:27 UTC

A cleanup for MDEV-25243 ASAN heap-use-after-free in Item_func_sp::execute_impl upon concurrent view DDL and I_S query with view and function

The test was reported to fail sporadicaly with this diff:

--- mysql-test/main/information_schema_tables.result
+++ mysql-test/main/information_schema_tables.reject
@@ -21,6 +21,8 @@
 disconnect con1;
 connection default;
 DROP VIEW IF EXISTS vv;
+Warnings:
+Note 4092 Unknown VIEW: 'test.vv'

in the "The originally reported non-deterministic test" part.
Disabling warnings around the DROP VIEW statement.

Author: Julius Goryavsky
Author Date: 2022-06-28 09:27:49 UTC

MDEV-28656: post-merge fixes

This commit fixes some inaccuracies introduced by
the automatic merge process when migrating changes
from previous versions to 10.9.

Author: Julius Goryavsky
Author Date: 2022-06-28 03:06:30 UTC

MDEV-28968: mariabackup SST doesn't properly handle --innodb-buffer-pool-filename

The mariabackup SST script cannot properly handle SST from
the joiner side if a non-standard filename is specified in
the "--innodb-buffer-pool-filename" option. Log analysis
shows that the received file is copied to the destination
directory under the original name "ib_buffer_pool", and not
under the name specified using the option on the command
line or in the configuration file.

This commit fixes this bug and adds an updated mtr tests
to check for correct handling of the non-standard filenames
specified using the "--innodb-buffer-pool-filename" option.

Author: Nayuta Yanagisawa
Author Date: 2021-10-11 05:21:10 UTC

MDEV-24343 Spider Left join failed Unknown column 't0.ID' in 'on clause'

The Spider mixes the comma join with other join types, and thus
ERROR 1054 occurs. This is well-known issue caused by the higher
precedence of JOIN over the comma (,).

We can fix the problem simply by using JOINs instead of commas.

Author: Igor Babaev
Author Date: 2022-06-18 23:28:48 UTC

MDEV-28883 Re-design the upper level of handling UPDATE and DELETE statements

This patch introduces a new way of handling UPDATE and DELETE commands at
the top level after the parsing phase. This new way of processing update
and delete statements can be seen in the implementation of the prepare()
and execute() methods from the new Sql_cmd_dml class. This class derived
from the Sql_cmd class can be considered as an interface class for processing
such commands as SELECT, INSERT, UPDATE, DELETE and other comands
manipulating data in tables.
With this patch processing of update and delete statements after parsing
proceeds by the following schema:
  - precheck of the access rights is performed for the used tables
  - the used tables are opened
  - context analysis phase is performed for the statement
  - the used tables are locked
  - the statement is optimized and executed
  - clean-up is performed for the statement
The implementation of the method Sql_cmd_dml::execute() adheres this schema.
The virtual functions of the class Sql_cmd_dml used for precheck of the
access rights, context analysis, optimization and execution allow to adjust
this schema for processing data manipulation statements of any types.

This schema of processing data manipulation statements is taken from the
current MySQL code. Moreover the definition the class Sql_cmd_dml introduced
in this patch is almost a full replica of such class in the existing MySQL.
However the implementation of the derived classes for update and delete
statements is quite different. This implementation employs the JOIN class
for all kinds of update and delete statements. It allows to perform main
bulk of context analysis actions by the function JOIN::prepare(). This
guarantees that characteristics and properties of the statement tree
discovered for optimization phase when doing context analysis are the same
for single-table and multi-table updates and deletes.

With this patch the following functions are gone:
  mysql_prepare_update(), mysql_multi_update_prepare(),
  mysql_update(), mysql_multi_update(),
  mysql_prepare_delete(), mysql_multi_delete_prepare(), mysql_delete().
The code within these functions have been used as much as possible though.
The functions mysql_test_update() and mysql_test_delete() are also not
needed anymore. The method Sql_cmd_dml::prepare() serves processing
  - update/delete statement
  - PREPARE stmt FROM "<update/delete statement>"
  - EXECUTE stmt when stmt is prepared from update/delete statement.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Author: Nayuta Yanagisawa
Author Date: 2022-06-21 15:56:16 UTC

MDEV-21310 AUTO_INCREMENT column throws range error on INSERT in partitioned table | Assertion `part_share->auto_inc_initialized || !can_use_for_auto_inc_init()' failed.

The bug is caused by a similar mechanism as MDEV-21027.

The function, check_insert_or_replace_autoincrement, failed to open
all the partitions on INSERT SELECT statements and it results in the
assertion error.

Author: Vanislavsky
Author Date: 2022-02-13 13:19:02 UTC

MDEV-25704 Add RANDOM_BYTES function

MySQL 5.6 added the RANDOM_BYTES function.
https://dev.mysql.com/doc/refman/5.6/en/encryption-functions.html#function_random-bytes

This is needed for compatibility purposes.

Author: Sergei Golubchik
Author Date: 2022-06-18 21:23:28 UTC

MDEV-27104 deprecate DES_ENCRYPT/DECRYPT functions

Author: Sergei Golubchik
Author Date: 2022-06-17 19:20:43 UTC

don't use ssl for windows named pipes - it doesn't work

Author: Nayuta Yanagisawa
Author Date: 2022-06-21 15:56:16 UTC

MDEV-21310 AUTO_INCREMENT column throws range error on INSERT in partitioned table | Assertion `part_share->auto_inc_initialized || !can_use_for_auto_inc_init()' failed.

This bug is caused by a similar mechanism as MDEV-21027.

The function, check_insert_or_replace_autoincrement, failed to open
all the partitions on INSERT SELECT statements and it results in the
assertion error.

Author: Nayuta Yanagisawa
Author Date: 2022-06-16 11:25:02 UTC

MDEV-28352 Spider: heap-use-after-free in ha_spider::lock_tables(), heap freed by spider_commit()

The heap-use-after-free is caused by the following mechanism:

  * In the execution of FLUSH TABLE WITH READ LOCK, the function
    spider_free_trx_conn() is called and the connections held by
    SPIDER_TRX::trx_conn_hash are freed.

  * Then, an instance of ha_spider maintains the freed connections
    because they are also referenced from ha_spider::conns.
    The ha_spider instance is kept in a lock structure until the
    corresponding table is unlocked.

  * Spider accesses ha_spider::conns on the implicit UNLOCK TABLE
    issued by BEGIN.

In the first place, when the connections have been freed, it means
that there are really no remote table locked by Spider.
Thus, there is no need for Spider to access ha_spider::cons on the
implicit UNLOCK TABLE.

We can fix the bug by removing the above mentioned access to
ha_spider::conns. We also modified spider_free_trx_conn() so that it
frees the connections only when no table is locked to reduce the
chance of another heap-use-after-free on ha_spider::conns.

Author: Nayuta Yanagisawa
Author Date: 2022-06-16 11:25:02 UTC

MDEV-28352 Spider: heap-use-after-free in ha_spider::lock_tables(), heap freed by spider_commit()

The heap-use-after-free is caused by the following mechanism:

  * In the execution of FLUSH TABLE WITH READ LOCK, the function
    spider_free_trx_conn() is called and the connections held by
    SPIDER_TRX::trx_conn_hash are freed.

  * Then, an instance of ha_spider maintains the freed connections
    because they are also referenced from ha_spider::conns.
    The ha_spider instance is kept in a lock structure until the
    corresponding table is unlocked.

  * Spider accesses ha_spider::conns on the implicit UNLOCK TABLE
    issued by BEGIN.

In the first place, when the connections have been freed, it means
that there are really no remote table locked by Spider.
Thus, there is no need for Spider to access ha_spider::cons on the
implicit UNLOCK TABLE.

We can fix the bug by removing the above mentioned access to
ha_spider::conns. We also modified spider_free_trx_conn() so that it
frees the connections only when no table is locked to reduce the
chance of another heap-use-after-free on ha_spider::conns.

Author: Alexander Barkov
Author Date: 2022-06-08 11:12:03 UTC

MDEV-23287 The INET4 data type

Author: Brandon Nesterenko
Author Date: 2022-06-27 13:39:19 UTC

MDEV-27655 Fix UBSAN's Findings in Replication

Tagged mysys/my_compare.c::ha_key_cmp() with attribute to ignore
UBSAN runtime checks on shift operations. In particular, it is
expected that this function can perform shift of greater length
than the number of bits in the underlying field variables.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-06-27 12:23:41 UTC

MDEV-27691: post probe to disable service connection

Author: Vladislav Vaintroub
Author Date: 2022-06-01 11:46:33 UTC

MDEV-11026 Make InnoDB number of IO write/read threads dynamic

Fix concurrency error - avoid accessing deleted memory, when io_slots is
resized. the deleted memory in this case was vftable pointer in
aiocb::m_internal_task

The fix avoids calling dummy release function, via a flag in task_group.

Author: Oleg Smirnov
Author Date: 2022-06-27 05:37:59 UTC

MDEV-28869 Eliminated tables are not shown in EXPLAIN

Author: Julius Goryavsky
Author Date: 2022-06-27 03:03:21 UTC

MDEV-28758: Mariabackup copies binary logs to backup directory

This commit changes the default settings so that, by default,
mariabackup will not copy binary logs files to the backup directory
and will transfer only one (most recet) binary log file between nodes
during SST. If the user wants to make a backup containing all binary
logs or wants to transfer them all between nodes, then user must add
the sst_max_binlogs=-1 option to the [sst] section, to one of the
server configuration sections, or must use the equivalent command
line option (when running mariabackup).

Author: Nayuta Yanagisawa
Author Date: 2022-04-25 09:29:01 UTC

MDEV-27233 Server hangs when using --init-file which loads Spider and creates a Spider table

Spider waits until the server initialization is complete
(i.e., mysqld_server_started == 1) before creating the Spider system
tables like mysql.spider_XX. Spider also wait until the tables are
created before instantiating ha_spider.

This results in dead lock when Spider is loaded and a non-system
Spider table is created by --in-file because queries passed by
--in-file are executed during the server initialization
(i.e., mysqld_server_started == 0).

In the first place, there seems to be no need for Spider to wait until
the server initialization is completed before creating the system
tables. Thus, we simply remove that synchronization.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-06-24 09:55:42 UTC

MDEV-22133 handle_fatal_signal (sig=11) on optimized builds in handle_grant_table instead of ERROR | Buffer overflow (on optimized builds)

Return an error if we can not read a table required for ACL.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2022-06-24 12:49:57 UTC

- Ignore the corrupted index during row_create_prebuilt()

Author: Alexey Botchkov
Author Date: 2022-02-15 09:37:59 UTC

MDEV-27832 disable binary logging for SQL SERVICE.

Binary logging is now disabled for the queries run by SQL SERVICE.
The binlogging can be turned on with the 'SET SQL_LOG_BIN=On' query.

Conflicts:
 sql/sql_prepare.cc

Author: Alexey Botchkov
Author Date: 2022-02-15 09:37:59 UTC

MDEV-27832 disable binary logging for SQL SERVICE.

Binary logging is now disabled for the queries run by SQL SERVICE.
The binlogging can be turned on with the 'SET SQL_LOG_BIN=On' query.

Conflicts:
 sql/sql_prepare.cc

Author: Nikita Malyavin
Author Date: 2022-06-23 22:45:44 UTC

sampling analysis

file=open(...)
for l in file:
     if l.startswith('SAMPLE') and not l.startswith('SAMPLE root'):
         print(l)
         n = dict((p.split('=')
           for p in l.replace('SAMPLE ', '').replace(' REJECT', '').split()
           ))
         if 'REJECT' in l:
             n['rej']=True
         nodes.append(n)

max(int(n['size']) for n in nodes)
max((float(n['p']), serialize(n)) for n in nodes)
max((float(n['size']), serialize(n)) for n in nodes if n['height'] == '0')

Author: midenok
Author Date: 2022-06-23 18:42:51 UTC

fts_get_max_doc_id() + memory leak fix

Author: Vladislav Vaintroub
Author Date: 2022-06-23 13:37:11 UTC

Revert "TSAN: data race on vptr (ctor/dtor vs virtual call)"

This reverts commit 78084fa747f373f2b404c3cb543d19f439100d9e.

This commit was done to please TSAN misreporting an error. But as
consequence, it could cause a real error, crash in os_aio_free on
shutdown

Author: Sergei Golubchik
Author Date: 2022-06-23 10:59:31 UTC

my_safe_process: try to kill the process softly first

first SIGTERM and if the process didn't die in 10 seconds, SIGKILL it.

This allows various tools like `rr`, `gcov`, `gprof`, etc to flush
their data to disk properly

Author: sachin
Author Date: 2020-09-14 14:27:02 UTC

MDEV-23264 Unique blobs allow duplicate values upon UPDATE

Problem:-
  We are able to insert duplicate value in table because cmp_binary_offset
  is not able to differentiate between NULL and empty string. So
  check_duplicate_long_entry_key is never called and we don't check for
  duplicate.
Solution
  Added a if condition with is_null() on field which can differentiate
  between NULL and empty string.

Author: Vladislav Vaintroub
Author Date: 2022-06-23 03:53:55 UTC

MDEV-28920 Rescheduling of innodb_stats_func() missing

Fixed tpool timer implementation on POSIX.
Prior to this patch, under some specific rare circumstances (concurrency
related), timer callback execution might be skipped.

Author: Nayuta Yanagisawa
Author Date: 2022-06-22 17:17:24 UTC

MDEV-19190 Assertion `part_share->auto_inc_initialized' failed in ha_partition::get_auto_increment

Author: Rucha Deodhar
Author Date: 2022-06-22 09:07:52 UTC

MDEV-28762: recursive call of some json functions without stack control

Analysis: Some recursive json functions dont check for stack control
Fix: Add check_stack_overrun(). The last argument is NULL because it is not
used

Author: Rucha Deodhar
Author Date: 2022-06-21 13:40:11 UTC

MDEV-28762: recursive call of some json functions without stack control

Analysis: Some recursive json functions dont check for stack control
Fix: Add check_stack_overrun(). The last argument is NULL because it is not
used

Author: Rucha Deodhar
Author Date: 2022-06-21 09:28:34 UTC

MDEV-28762: recursive call of some json functions without stack control

Analysis: Some recursive json functions dont check for stack control
Fix: Add check_stack_overrun(). The last argument is NULL because it is not
used

Author: Vladislav Vaintroub
Author Date: 2022-06-22 09:58:00 UTC

x

Author: Marko Mäkelä
Author Date: 2022-06-22 07:08:24 UTC

MDEV-28800 WIP: Avoid crashes on memory allocation failure

FIXME: heap-use-after-free in ./mtr --suite=innodb_gis
and failures of
innodb.full_crc32_import
innodb.innodb_defrag_concurrent
innodb.instant_alter

FIXME: Allocate locks upfront for page reorganize or split,
so that the operation can gracefully fail before any irreversible
persistent changes are performed.

FIXME: Remove buf_pool_t::running_out(), or defer the invocation
to when actually attempting to allocate lock objects.

Author: Julius Goryavsky
Author Date: 2022-06-21 11:29:45 UTC

MDEV-26562: galera-sst-mariabackup is failing due to missing xtrabackup_checkpoints

This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).

Also fixed broken logic of automatic generation of the Diffie-Hellman parameters
for socat version less than 1.7.3 (which defaults to 512-bit values instead of
2048-bit ones).

Author: Julius Goryavsky
Author Date: 2022-06-16 01:45:45 UTC

MDEV-28656: Inability to roll upgrade without stopping the Galera cluster

This commit fixes a divergence in the IST/SST script code between
the 10.3+ and 10.9+ branches, due to which the cluster cannot contain
nodes with different versions of the server (since in this case SST/IST
scripts may fail).

This commit also includes test fixes specific to 10.4+ branches and
a few manual fixes after automatic "git cherry-pick", as well as server
code change specific to 10.9+.

Author: Marko Mäkelä
Author Date: 2022-06-21 16:20:15 UTC

Merge 10.10

Author: Daniel Black
Author Date: 2022-06-19 19:42:22 UTC

MDEV-28866: mariadb-upgrade to 10.8 mysql.column_stats hist_type + histogram errors

The incorrect type of mysql.column_stats caused the server during the
upgrade of every other table to complain:

[ERROR] Incorrect definition of table mysql.column_stats: expected column 'hist_type' at position 9
and expected column 'histogram' at position 10 to have type longblob.

To prevent these verbose server errors, we upgrade the
mysql.column_stats table first.

Consequencely limit "Incorrect defination of table mysql.*" to the approprate test case

Author: Nikita Malyavin
Author Date: 2022-02-23 19:38:51 UTC

Add tests and code instrumentations

Author: Oleg Smirnov
Author Date: 2022-06-21 09:20:17 UTC

MDEV-28881 Server crashes in Dep_analysis_context::create_table_value

SELECT_LEX::first_select()->join may be NULL for some scenarios.
This commit adds a check for this.

Author: Julius Goryavsky
Author Date: 2022-06-17 11:56:03 UTC

MDEV-28583 postfix: fixing .result files after automatic merge

Author: Igor Babaev
Author Date: 2022-06-20 06:43:17 UTC

MDEV-7487 Semi-join optimization for single-table update/delete statements

This patch allows to use semi-join optimization at the top level of
single-table update and delete statements.
The problem of supporting such optimization became easy to resolve after
processing a single-table update/delete statement started using JOIN
structure. This allowed to use JOIN::prepare() not only for multi-table
updates/deletes but for single-table ones as well. This was done in the
patch for mdev-28883:
 Re-design the upper level of handling UPDATE and DELETE statements.

Note that JOIN::prepare() detects all subqueries that can be considered
as candidates for semi-join optimization. The code added by this patch
looks for such candidates at the top level and if such candidates are found
in the processed single-table update/delete the statement is handled in
the same way as a multi-table update/delete.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Author: Igor Babaev
Author Date: 2022-06-20 06:43:17 UTC

MDEV-7487 Semi-join optimization for single-table update/delete statements

This patch allows to use semi-join optimization at the top level of
single-table update and delete statements.
The problem of supporting such optimization became easy to resolve after
processing a single-table update/delete statement started using JOIN
structure. This allowed to use JOIN::prepare() not only for multi-table
updates/deletes but for single-table ones as well. This was done in the
patch for mdev-28883:
 Re-design the upper level of handling UPDATE and DELETE statements.

Note that JOIN::prepare() detects all subqueries that can be considered
as candidates for semi-join optimization. The code added by this patch
looks for such candidates at the top level and if such candidates are found
in the processed single-table update/delete the statement is handled in
the same way as a multi-table update/delete.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Author: Brandon Nesterenko
Author Date: 2022-06-20 16:56:13 UTC

MDEV-19801

Added optional parameter to keep gtid state in reset_slave.inc

Author: Nayuta Yanagisawa
Author Date: 2022-06-20 09:02:30 UTC

MDEV-28739 Trying to lock uninitialized mutex or hang upon shutdown after using Spider with query_cache

Author: Nayuta Yanagisawa
Author Date: 2022-06-20 09:02:30 UTC

MDEV-28739 Trying to lock uninitialized mutex or hang upon shutdown after using Spider with query_cache

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-10-20 09:37:14 UTC

MDEV-26647 (simple_password_check) Include password validation plugin information in the error message if the SQL statement is not satisfied password policy

Make the plugin reporting cause of the error.

Author: Nayuta Yanagisawa
Author Date: 2022-06-20 11:05:32 UTC

MDEV-27575 SIGSEGV in intern_plugin_lock on SHUTDOWN when setting Spider as default storage engine (temporary or global)

When Spider is set as the default storage engine, it is unloaded later
compared with other plugins. That is because Spider is referenced by
@@global.default_storage_engine and cannot be unloaded before the
variable is set to NULL.

spider_db_done() creates a new THD but, in the above mentioned case,
the new THD needs to lock its default_storage_engine and it does not
expect that the server is almost shut down and the variable is NULL.

Luckily, the newly created THD is used for nothing. Thus, we can just
remove the THD creation.

Author: Sergei Golubchik
Author Date: 2022-06-20 07:34:33 UTC

fixup! MDEV-28883 Re-design the upper level of handling UPDATE and DELETE statements

compilation failure (fall-through)
and ubsan failure (load of value 2779096485, which is not a valid value for type 'enum_lock_tables_state')

Author: Nikita Malyavin
Author Date: 2022-02-23 19:38:51 UTC

Add tests and code instrumentations

Author: Vlad Lesin
Author Date: 2022-06-06 11:31:19 UTC

MDEV-21136 InnoDB's records_in_range estimates can be way off

Get rid of BTR_ESTIMATE and btr_cur_t::path_arr.

Before the fix btr_estimate_n_rows_in_range_low() used two
btr_cur_search_to_nth_level() calls to create two arrays of tree path,
the array per border. And then it tried to estimate the number of rows
diving level-by-level with the array elements. As the path pages are
unlatched during the arrays iterating, the tree could be modified, the
estimation function called itself until the number of attempts exceed.

After the fix the estimation happens during search process. Roughly, the
algorithm is the following. Dive in the left page, then if there are pages
between left and right ones, read a few pages to the right, if the right
page is reached, fetch it and count the exact number of rows, otherwise
count the estimated number of rows, and fetch the right page.

The latching order corresponds to WL#6326 rules, i.e.:

(2.1) [same as (1.1)]: Page latches must be acquired in descending order
of tree level.

(2.2) When acquiring a node pointer page latch at level L, we must hold
the left sibling page latch (at level L) or some ancestor latch
(at level>L).

When we dive to the level down, the parent page is unlatched only after
the the current level page is latched. When we estimate the number of rows
on some level, we latch the left border, then fetch the next page, and
then fetch the next page unlatching the previous page after the current
page is latched until the right border is reached. I.e. the left sibling
is always latched when we acquire page latch on the same level. When we
reach the right border, the current page is unlatched, and then the right
border is latched. Following to (2.2) rule, we can do this because the
right border's parent is latched.

Author: Sergey Petrunia
Author Date: 2022-06-18 16:52:14 UTC

Cleanups for: 0762dd928 Improve pruning in greedy_search by sorting tables during search

Author: Jan Lindström
Author Date: 2022-06-12 07:39:15 UTC

Fixed regression on SST tests.

We incorrectly used my_thread_end(), which
sets mysys_var pointer to NULL, so the next THD::THD will crash.
Removed my_thread_init()/end() pairs and because
Wsrep_allowlist_service::allowlist_cb is not always called from a
new thread added a thread to do so.

Fix co-authored by Sergei Golubchik <serg@mariadb.org>
and mkaruza <mario.karuza@galeracluster.com>

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Alexander Barkov
Author Date: 2022-06-15 08:39:00 UTC

Switching libmariadb to "Libmariadb changes for MDEV-27009 Add UCA-14.0.0 collations"

Author: Sujatha Sivakumar
Author Date: 2021-08-24 11:45:04 UTC

MDEV-20122 Deprecate MASTER_USE_GTID=Current_Pos to favor new MASTER_DEMOTE_TO_SLAVE option

Deprecate the `Current_Pos` option of MASTER_USE_GTID to suggest
the safe alternative option MASTER_DEMOTE_TO_SLAVE=TRUE. See
the previous commit for the implementation and specifics of this
new option.

Reviewed By:
=============
Andrei Elkin <andrei.elkin@mariadb.com>

Author: Sergei Golubchik
Author Date: 2022-06-18 13:10:34 UTC

remove invalid test

it starts an EXPLAIN of a multi-table join and tries to KILL it.
no sync points.
depending on how fast the hareware is and optimizer development
it might kill EXPLAIN at some random point in time (generally unrelated
to the Bug#28598 it was supposed to test) or EXPLAIN might finish
before the KILL and the test will fail.

Author: Igor Babaev
Author Date: 2022-06-18 07:49:15 UTC

MDEV-27159 Re-design the upper level of handling DML commands

This patch allows to execute only single-table and multi-table
UPDATE and DELETE statements using the method Sql_cmd_dml::execute().
The code that handles INSERT statements has not been touched.

Author: Rucha Deodhar
Author Date: 2022-05-02 09:07:37 UTC

MDEV-27128: Implement JSON Schema Validation FUNCTION

Todo: Add comments, tests for object validation and commit message

Author: Julius Goryavsky
Author Date: 2022-06-17 02:59:29 UTC

MDEV-26562: galera-sst-mariabackup is failing due to missing xtrabackup_checkpoints

This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).

Author: Julius Goryavsky
Author Date: 2022-06-17 11:56:03 UTC

MDEV-28583 postfix: fixing .result files after automatic merge

Author: Igor Babaev
Author Date: 2022-06-17 06:45:38 UTC

MDEV-27159 Re-design the upper level of handling DML commands

This patch allows to execute only single-table and multi-table
UPDATE and DELETE statements using the method Sql_cmd_dml::execute().
The code that handles INSERT statements has not been touched.

Author: Sergei Golubchik
Author Date: 2022-06-16 20:39:35 UTC

fix spider.variable_deprecation test

Author: Vicențiu Ciorbaru
Author Date: 2022-06-16 13:35:55 UTC

INFORMATION_SCHEMA.views denies with show view and select

This commit fixes show view and select privilege denies.

Other cleanup work:
- TODO(cvicentiu) describe this when rebasing.
- get_table_grant -> Take in table strings, not a TABLE_LIST
- get_column_grant -> Take in sctx.
- check_some_grants_remain -> take in role denies.
- check_column_access_denied -> Utility function for show view.