Author: Nikita Malyavin
Author Date: 2025-01-02 23:55:05 UTC

Online alter: react softly to online log buffer write errors

Two points:
1. We don't have to fail a transaction if a write error has occurred at online
alter's commit, i.e. online log buffer write: in case of write error, alter
table will have to fail anyway, but it doesn't actually break the transaction
integrity.
The same point about failing the rest of other possible online alters -- we
should complete them as usual.

2. We shouldn't report any error, a warning is enough -- the transaction didn't
really fail, it is completed (all else being equal), though with some problems,
like ALTER TABLE being cancelled, which we can report as a warning.

Part of MDEV-35661

Author: ParadoxV5
Author Date: 2025-01-02 23:13:01 UTC

Remove dup check from `throw_bounds_warning`

MDEV-11917 merged a unified `const char*` overload for the
two numeric `throw_bounds_warning` overloads. The `my_error`-on-
`MODE_STRICT_ALL_TABLES` branch wasn’t removed from the two, though.

This commit removes this leftover duplication.

Author: Sergei Golubchik
Author Date: 2025-01-02 19:23:08 UTC

MDEV-32576 increase query length in he InnoDB deadlock output

* increase target buffer size to 3072
* remove the parameter, just use the buffer size as a limit

Author: Monty
Author Date: 2025-01-02 13:15:22 UTC

Only print "InnoDB: Transaction was aborted..." if log_warnings >= 4

This is a minor fixup for
MDEV-24035 Failing assertion UT_LIST_GET_LEN(lock.trx_locks) == 0
causing disruption and replication failure

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2025-01-02 06:56:26 UTC

MDEV-34898 Doublewrite recovery of innodb_checksum_algorithm=full_crc32 encrypted pages does not work

- InnoDB fails to recover the full crc32 encrypted page from
doublewrite buffer. The reason is that buf_dblwr_t::recover()
fails to identify the space id from the page because the page has
been encrypted from FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION bytes.

Fix:
===
buf_dblwr_t::recover(): preserve any pages whose space_id
does not match a known tablespace. These could be encrypted pages
of tablespaces that had been created with
innodb_checksum_algorithm=full_crc32.

buf_page_t::read_complete(): If the page looks corrupted and the
tablespace is encrypted and in full_crc32 format, try to
restore the page from doublewrite buffer.

recv_dblwr_t::recover_encrypted_page(): Find the page which
has the same page number and try to decrypt the page using
space->crypt_data. After decryption, compare the space id.
Write the recovered page back to the file.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2025-01-02 12:04:24 UTC

MDEV-34119 Assertion `page_dir_get_n_heap(new_page) == 2U' failed in dberr_t PageBulk::init()

Problem:
=======
- insert..select statement on partition table fails to use
bulk insert for the transaction.

Solution:
========
- Enable the bulk insert operation for insert..select
statement for partition table.

Author: Marko Mäkelä
Author Date: 2025-01-02 11:59:19 UTC

MDEV-35701 trx_t::autoinc_locks causes unnecessary dynamic memory allocation

trx_t::autoinc_locks: Use small_vector<lock_t*,4> in order to avoid any
dynamic memory allocation in the most common case (a statement is
holding AUTO_INCREMENT locks on at most 4 tables or partitions).

lock_cancel_waiting_and_release(): Instead of removing elements from
the middle, simply assign nullptr, like lock_table_remove_autoinc_lock().

small_vector_base::Capacity: Remove. This only needs to be a template
parameter N (compile-time constant) in small_vector.

Author: Marko Mäkelä
Author Date: 2025-01-02 09:29:21 UTC

Merge

Author: Marko Mäkelä
Author Date: 2025-01-02 09:27:30 UTC

squash! 089a880c87c5d43dd5ec9c43894d9fe4f80a07f5

btr_search_drop_page_hash_index(): Do not release part.latch if we
need to invoke multiple batches of rec_fold().

Author: ParadoxV5
Author Date: 2025-01-01 02:20:57 UTC

fixup: I kept putting `buff` first in `longlong10_to_str`

Author: ParadoxV5
Author Date: 2024-12-31 22:40:20 UTC

`mysqladmin.cc`: Do `strcspn` before `strmov`

Avoid scanning a number from the input string only
to convert it back to a string at another buffer.
This order locates the delimeter first and copies the entire substring,
including any non-numeric misinput, to the query and the error message.
(The previous code recognized `0`/`0x` prefixes,
ignored non-numeric suffixes, and converted invalid input to 0.)

Author: Monty
Author Date: 2024-12-30 15:41:26 UTC

Added missing drop procedure to log_state.test

Author: Kristian Nielsen
Author Date: 2024-12-30 14:24:59 UTC

MDEV-34705: Binlog-in-engine: Refactor InnoDB part

Move high-level binlog code to handler/handler0binlog.cc and low-level code
to fsp/fsp0binlog.cc.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Sergei Golubchik
Author Date: 2024-12-29 15:20:19 UTC

MDEV-35607 Compile error with gcc-15 (signal returns)

set policy CMP0067 to NEW, this will make ty_compile to use
project-wide CMAKE_C_STANDARD and CMAKE_CXX_STANDARD settings

Author: midenok
Author Date: 2024-12-30 10:59:14 UTC

MDEV-28130 MariaDB SEGV issue at tree_search_next

In case of error last_pos points to null_element and there is no any
other children. tree_search_next() walks the children from last_pos
until the leaves (null_element) ignoring the case the topmost parent
in search state is the leaf itself.

Author: Sergei Golubchik
Author Date: 2024-12-29 17:20:57 UTC

MDEV-35724 VEC_DISTANCE does not work in HAVING clause

Author: Sergei Golubchik
Author Date: 2024-12-29 16:54:52 UTC

MDEV-35417 InnoDB crashes under ... AND DISABLE CHECKPOINT

restore pre-11.7 behavior when InnoDB was ignoring the
"AND DISABLE CHECKPOINT" clause of FTWGRL.

InnoDB isn't ready for it yet

Author: Monty
Author Date: 2024-12-15 13:57:53 UTC

MDEV-20912 Add support for utf8mb4_0900_* collations in MariaDB Server

This is done by mapping most of the existing MySQL unicode 0900 collations
to MariadB 1400 unicode collations. The assumption is that 1400 is a super
set of 0900 for all practical purposes.

I also added a new function 'compare_collations()' and changed most code
to use this instead of comparing character sets directly.
This enables one to seamlessly mix-and-match the corresponding 0900 and
1400 sets. Field comparision and alter table treats the character sets
as identical.

All MySQL 8.0 0900 collations are supported except:
- utf8mb4_ja_0900_as_cs
- utf8mb4_ja_0900_as_cs_ks
- utf8mb4_ru_0900_as_cs
- utf8mb4_zh_0900_as_cs

These do not have corresponding entries in the MariadB 01400 collations.

Other things:
- Added COMMENT colum to information_schema.collations. For utf8mb4_0900
  colletions it contains the corresponding alias collation.

Author: Monty
Author Date: 2024-12-15 13:57:53 UTC

MDEV-20912 Add support for utf8mb4_0900_* collations in MariaDB Server

This is done by mapping most of the existing MySQL unicode 0900 collations
to MariadB 1400 unicode collations. The assumption is that 1400 is a super
set of 0900 for all practical purposes.

I also added a new function 'compare_collations()' and changed most code
to use this instead of comparing character sets directly.
This enables one to seamlessly mix-and-match the corresponding 0900 and
1400 sets. Field comparision and alter table treats the character sets
as identical.

All MySQL 8.0 0900 collations are supported except:
- utf8mb4_ja_0900_as_cs
- utf8mb4_ja_0900_as_cs_ks
- utf8mb4_ru_0900_as_cs
- utf8mb4_zh_0900_as_cs

These do not have corresponding entries in the MariadB 01400 collations.

Other things:
- Added COMMENT colum to information_schema.collations. For utf8mb4_0900
  colletions it contains the corresponding alias collation.

Author: Monty
Author Date: 2024-11-22 12:23:57 UTC

Change default MEM_ROOT allocations to decrease calls to malloc

This commit updates default memory allocations size used with MEM_ROOT
objects to minumize the number of calls to malloc().

Changes:
- Updated MEM_ROOT block sizes in sql_const.h
- Updated MALLOC_OVERHEAD to also take into account the extra memory
  allocated by my_malloc()
- Updated init_alloc_root() to only take MALLOC_OVERHEAD into acount as
  buffer size, not MALLOC_OVERHEAD + sizeof(USED_MEM).
- Reset mem_root->first_block_usage if and only if first block was used.
- Increase MEM_ROOT buffers sized used by my_load_defaults, plugin_init,
  Create_tmp_table, allocate_table_share, TABLE and TABLE_SHARE.
  This decreases number of malloc calls during queries.
- Use a small buffer for THD->main_mem_root in THD::THD. This avoids
  multiple malloc() call for new connections.

I tried the above changes on a complex select query with 12 tables.
The following shows the number of extra allocations that where used
to increase the size of the MEM_ROOT buffers.

Original code:
- Connection to MariaDB: 9 allocations
- First query run: 146 allocations
- Second query run: 24 allocations

Max memory allocated for thd when using with heap table: 61,262,408
Max memory allocated for thd when using Aria tmp table: 419,464

After changes:
Connection to MariaDB: 0 allocations
- First run: 25 allocations
- Second run: 7 allocations

Max memory allocated for thd when using with heap table: 61,347,424
Max memory allocated for thd when using Aria table: 529,168

The new code uses slighly more memory, but avoids memory fragmentation
and is slighly faster thanks to much fewer calls to malloc().

Author: VladislavVaintroub
Author Date: 2024-12-27 12:07:04 UTC

MDEV-35575 Fix memory leak, when installing auth_gssapi plugin fails.

Make sure to release memory, which was allocated by gss_import_name().
Also when plugin_init() fails.

Author: Oleg Smirnov
Author Date: 2024-12-26 10:30:11 UTC

MDEV-32411 Item_sum arguments incorrectly reset to temp table fields which causes crash

The issue is caused by a logic error in Item_sum::get_tmp_table_item() method:
it resets arguments of the item to point to the result fields during
change_ref_to_tmp_fields() call. However, Item_sum arguments must not be modified.
It is enough for Item_sum objects to call ancestor's implementation
Item::get_tmp_table_item().

This fix is in accordance with MySQL commit 2e3dc09087c24798c90e05163ed3d931f6b93db3

Author: Vlad Lesin
Author Date: 2024-10-10 12:43:36 UTC

MDEV-34877 Port "Bug #11745929 Change lock priority so that the transaction holding S-lock gets X-lock first" fix from MySQL to MariaDB

This commit implements
mysql/mysql-server@7037a0bdc83196755a3bf3e935cfb3c0127715d5
functionality, i.e. if some transaction A holds not-gap S-lock on some
record, and some other transactions B={b1, b2, ..., bn} have not-gap
X-locks waiting for the S-lock of transaction A, and transaction A
requests not-gap and not insert intention X-lock which conflicts with
the X-locks of transactions B and does not conflict with another locks
in the queue, then grant the X-lock to transaction A.

MySQL's commit contains the following explanation of why insert-intention
locks must not overtake a waiting ordinary or gap locks:

"It is important that this decission rule doesn't allow
INSERT_INTENTION locks to overtake WAITING locks on gaps (`S`, `S|GAP`,
`X`, `X|GAP`), as inserting a record into a gap would split such WAITING
lock, violating the invariant that each transaction can have at most
single WAITING lock at any time."

I would add to the explanation the following. Suppose we have trx 1 which
holds ordinary X-lock on some record. And trx 2 executes "DELETE FROM t"
or "SELECT * FOR UPDATE" in RR(see lock_delete_updated.test and
MDEV-27992), i.e. it creates waiting ordinary X-lock on the same record.
And then trx 1 wants to insert some record just before the locked record.
It requests insert-intention lock, and if the lock overtakes trx 2 lock,
there will be phantom records for trx 2 in RR. lock_delete_updated.test
shows how "DELETE" allows to insert some records in already scanned gap
and misses some records to delete.

The current implementation differs from MySQL implementation. There are
two key differences:

1. Lock queue ordering. In MySQL all waiting locks precede all granted
   locks. A new waiting lock is added to the head of the queue, a new
   granted lock is added to the end of the queue, if some waiting lock
   is granted, it's moved to the end of the queue. In MariaDB any new
   lock is added to the end of the queue and waiting lock does not change
   its position in the queue where the lock is granted. The rule is that
   blocking lock must be located before blocked lock in lock queue. We
   maintain the rule with inserting bypassing lock just before bypassed
   one.

2. MySQL implementation uses some object(locksys::Trx_locks_cache) which
   can be passed to consecutive calls to rec_lock_has_to_wait() for the
   same trx and heap_no to cache the result of checking if trx has a
   granted lock which is blocking the waiting lock(see
   locksys::Trx_locks_cache::has_granted_blocker()). The current
   implementation does not use such object, because it looks for such
   granted lock on the level of lock_rec_other_has_conflicting() and
   lock_rec_has_to_wait_in_queue(). I.e. there is no need in additional
   lock queue iteration in
   locksys::Trx_locks_cache::has_granted_blocker(), as we already iterate
   it in lock_rec_other_has_conflicting() and
   lock_rec_has_to_wait_in_queue().

During the testing the following case was found. Suppose we have
delete-marked record and going to do inplace insert into
that delete-marked record. Usually we don't create explicit lock if
there are no conlicting with not gap X-lock locks(see
lock_clust_rec_modify_check_and_lock(), btr_cur_update_in_place()). The
implicit lock will be converted to explicit one by demand.

That can happen during INSERT, the not-gap S-lock can
be acquired on searching for duplicates(see
row_ins_duplicate_error_in_clust()), and, if delete-marked record is
found, inplace insert(see btr_cur_upd_rec_in_place()) modifies the
record, what is treated as implicit lock.

But there can be a case when some transaction trx1 holds not-gap S-lock,
another transaction trx2 creates waiting X-lock, and then trx2 tries to
do inplace insert. Before the fix the waiting X-lock of trx2 would be
conflicting lock, and trx1 would try to create explicit X-lock, what
would cause deadlock, and one of the transactions whould be rolled back.
But after the fix, trx2 waiting X-lock is not treated as conflicting
with trx1 X-lock anymore, as trx1 already holds S-lock. If we don't create
explicit lock, then some other transaction trx3 can create it during
implicit to explicit lock conversion and place it at the end of the
queue. So there can be the following locks order in the queue:

S1(granted) X2(waiting) X1(granted)

The above queue is not valid, because all granted trx1 locks must be
placed before waiting trx2 lock. Besides, lock_rec_release_try() can
remove S(granted, trx1) lock and grant X lock to trx 2, and there can be
two granted X-locks on the same record:

X2(granted) X1(granted)

Taking into account that lock_rec_release_try() can release cell and
lock_sys latches leaving some locks unreleased, the queue validation
function can fail in any unexpected place.

It can be fixed with two ways:

1) Place explicit X(granted, trx1) lock before X(waiting, trx2) lock
   during implicit to explicit lock conversion. This option is implemented
   in MySQL, as granted lock is always placed at the top of locks queue,
   and waiting locks are placed at the bottom of the queue. MariaDB does
   not do this, and implementing this variant would require conflicting
   locks search before converting implicit to explicit lock, what, in
   turns, would require cell and/or lock_sys latch acquiring.

2) Create and place X(granted, trx1) lock before X(waiting, trx2) during
   inplace INSERT, i.e. when lock_rec_lock() is invoked from
   lock_clust_rec_modify_check_and_lock() or
   lock_sec_rec_modify_check_and_lock(), if X(waiting, trx2) is
   bypassed. Such a way we don't need in additional conflicting locks
   search, as they are searched anyway in lock_rec_low().

This fix implements the second variant(see the changes around
c_lock_info.insert_after in lock_rec_lock). I.e. if some record was
delete-marked and we do inplace insert in such a record, and some lock for
bypass was found, create explicit lock to avoid conflicting lock search on
each implicit to explicit lock conversion. We can remove it if MDEV-35624
is implemented.

lock_rec_other_has_conflicting(), lock_rec_has_to_wait_in_queue():
search locks to bypass along with conflicting locks searching in the
same loop. The result is returned in conflicting_lock_info object.
There can be several locks to bypass, only the first one is returned to
limit lock_rec_find_similar_on_page() with the first bypassed lock to
preserve "blocking before blocked" invariant. conflicting_lock_info also
contains a pointer to the lock, after which we can insert bypassing
lock. This lock precedes bypassed one.

Bypassing lock can be next-key lock, and the following cases are
possible:

1. S1(not-gap, granted) II2(granted) X3(waiting for S1),

   When new X1(ordinary) lock is acquired, there will be the following
   locks queue:

   S1(not-gap, granted) II2(granted) X1(ordinary, granted) X3(waiting for
   S1)

   If we had inserted new X1 lock just after S1, and S1 had been released
   on transaction commit or rollback, we would have the following
   sequence in the locks queue:

   X1(ordinary, granted) II2(granted) X3(waiting for X1)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   This is not a real issue as II lock once granted can be
   ignored but it could possibly hit some assert(taking into account
   that lock_release_try() can release lock_sys latch, and other threads
   can acquire the latch and validate lock queue) as it breaks our design
   constraint that any granted lock in the queue should not conflict
   with locks ahead in the queue. But lock_rec_queue_validate() does not
   check the above contraint. We place new bypassing lock just before
   bypassed one, but there still can be the case when lock bitmap is used
   instead of creating new lock object(see lock_rec_add_to_queue() and
   lock_rec_find_similar_on_page()), and the lock, which owns the
   bitmap, can precede II2(granted). We can either disable
   lock_rec_find_similar_on_page() space optimization for bypassing locks
   or treat "X1(ordinary, granted) II2(granted)" sequence as valid. As
   we don't currently have the function which would fail on the above
   sequence, let treat it as valid for the case, when lock_release()
   execution is in process.

2. S1(ordinary, granted) II2(waiting for S1) X3(waiting for S1)

   When new X1(ordinary) lock is acquired, there will be the following
   locks queue:

   S1(ordinary, granted) II2(waiting for S1) X1(ordinary, granted)
   X3(waiting for S1).

   After S1 releasing there will be:

   II2(granted) X1(ordinary, granted) X3(waiting for S1)
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

   The above queue is valid because ordinary lock does not conflict with
   II-lock(see lock_rec_has_to_wait()).

lock_rec_create_low(): insert new lock to the position which
lock_rec_other_has_conflicting(), lock_rec_has_to_wait_in_queue()
return if the lock is bypassing.

lock_rec_find_similar_on_page(): add ability to limit similiar lock search
with the certain lock to preserve "blocking before blocked" invariant for
all bypassed locks.

lock_rec_add_to_queue(): don't treat bypassed locks as waiting ones to
let lock bitmap reusing for bypassing locks.

lock_rec_lock(): fix inplace insert case, explained above.

lock_rec_dequeue_from_page(), lock_rec_rebuild_waiting_queue: move
bypassing lock to the correct place to preserve "blocking before blocked"
invariant.

Author: ParadoxV5
Author Date: 2024-12-24 23:53:32 UTC

1

Author: midenok
Author Date: 2024-12-24 12:14:19 UTC

tmp

Author: Oleg Smirnov
Author Date: 2024-12-20 07:58:33 UTC

MDEV-35680 Table number > MAX_TABLES causes overflow of table_map at main.join test

Fix a regression introduced by commit d98ac851 (MDEV-29935, MDEV-26247) causing
MAX_TABLES overflow in `setup_table_map()`. The check for MAX_TABLES was moved
outside of the loop that increments table numbers, allowing overflows during
loop iterations. Since setup_table_map() operates on a 64-bit bitmap, table
numbers exceeding 64 triggered the UBSAN check.
This commit returns the overflow check within the loop and adds a debug
assertion to `setup_table_map()` to ensure no bitmap overrun occurs.

Author: Oleg Smirnov
Author Date: 2024-12-20 07:58:33 UTC

MDEV-35680 Table number > MAX_TABLES causes overflow of table_map at main.join test

Fix a regression introduced by commit d98ac851 (MDEV-29935, MDEV-26247) causing
MAX_TABLES overflow in `setup_table_map()`. The check for MAX_TABLES was moved
outside of the loop that increments table numbers, allowing overflows during
loop iterations. Since setup_table_map() operates on a 64-bit bitmap, table
numbers exceeding 64 triggered the UBSAN check.
This commit returns the overflow check within the loop and adds a debug
assertion to `setup_table_map()` to ensure no bitmap overrun occurs.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-12-23 21:36:01 UTC

MDEV-35326: Memory Leak in init_io_cache_ext upon SHUTDOWN

'result' from m_lex of an instruction cleared in 3 places (1) cleanup()
(2) send_eof() (3) destructor but in case of deleting procedure just after
error non of them called.
(1) called before instruction execution of an instruction
(2) called at the end of successful execution of an instruction or
conventional execution
(3) called unconditionally at very end of conventional execution

In conventional execution it freed by direct call of the destructor in
THD::end_statement(), so lets call it when we destruct procedure (all
other cases will be covered by (1) and (2).

Author: Nikita Malyavin
Author Date: 2024-12-22 16:25:58 UTC

MDEV-35661 Assertion `!cache_list ||...' fails in online_alter_close_connection

After MDEV-35182 (88adcbf3) fix, online_alter still wasn't fully ready for
`XA PREPARE` to be skipped.

In fact, according to XA standard, even xa_commit() call may happen without
preceding xa_prepare(), so this stage is completely optional.

This commit removes any relying on xa_prepare() call. The only thing that will
be done there is an equivalent of "statement-level commit", which basically just
flushes a pending rows event to an online alter cache.

This is done by always storing an online alter cache in the XA_data object
whenever XA is active, rather than in THD::ha_data.

Author: VladislavVaintroub
Author Date: 2024-12-23 18:39:16 UTC

MDEV-33978 P_S.THREADS is not showing all server threads

This 10.6-specific change only makes uring IO completion thread appear
in P_S.THREADS

Author: VladislavVaintroub
Author Date: 2024-12-23 14:58:55 UTC

MDEV-33978 P_S.THREADS is not showing all server threads

This patch only makes sure Linux getevents thread is shown in PS

Author: ParadoxV5
Author Date: 2024-12-22 20:43:22 UTC

MDEV-35646: Limit `pseudo_thread_id` to `UINT32_MAX`

Although the `my_thread_id` type is 64 bits, binlog format specs
limits it to 32 bits in practice. (See also: MDEV-35706)

The writable SQL variable `pseudo_thread_id` didn’t realize this though
and had a range of `ULONGLONG_MAX` (at least `UINT64_MAX` in C/C++).
It consequentially accepted larger values silently, but only the lower
32 bits of whom gets binlogged; this could lead to inconsistency.

Author: Dave Gosselin
Author Date: 2024-12-20 21:01:03 UTC

MDEV-35126 Wrong results from st_isvalid for multipolygon

Implements a polygon triangulation algorithm for determining when convex
polygons overlap. Finds polygon intersections by testing for edge
intersections.

Concave polygons are not well-supported and some valid arrangements are detected
as overlapping. For example, a five-pointed star with a small box between two
ears of the star.

Author: Yuchen Pei
Author Date: 2024-12-19 05:58:51 UTC

MDEV-34813 A simple implementation of ha_partition::compare_key_parts

Similar to can_convert_nocopy

Author: Yuchen Pei
Author Date: 2024-12-20 03:14:28 UTC

MDEV-34925 Fix segv on thd in spider recovery.

Create temporary thd whenever needed in spider xa commit / recovery.
This prevents crash during recovery.

Author: Yuchen Pei
Author Date: 2024-11-26 00:28:57 UTC

MDEV-34911 Sargable substr(col, 1, n) = str

Make Item_func_eq of the following forms sargable by updating the relevant range
analysis methods:

1. substr(col, 1, n) = str
2. str = substr(col, 1, n)
3. left(col, n) = str
4. str = left(col, n)

where col is a indexed column and str is a const and inexpensive item
of length n.

We do this by factoring out Item_func_like::get_mm_leaf() and apply it
to a string obtained from escaping str and then appending a wildcard
"%" to it.

The addition of the two Functype enums, LEFT_FUNC and SUBSTR_FUNC,
requires changes in the spider group by handler to continue handling
LEFT and SUBSTR correctly.

Co-authored-by: Yuchen Pei <ycp@mariadb.com>
Co-authored-by: Sergei Petrunia <sergey@mariadb.com>

Author: Yuchen Pei
Author Date: 2024-11-26 00:28:57 UTC

MDEV-34911 Sargable substr(col, 1, n) = str

Make Item_func_eq of the following forms sargable by updating the relevant range
analysis methods:

1. substr(col, 1, n) = str
2. str = substr(col, 1, n)
3. left(col, n) = str
4. str = left(col, n)

where col is a indexed column and str is a const and inexpensive item
of length n.

We do this by factoring out Item_func_like::get_mm_leaf() and apply it
to a string obtained from escaping str and then appending a wildcard
"%" to it.

The addition of the two Functype enums, LEFT_FUNC and SUBSTR_FUNC,
requires changes in the spider group by handler to continue handling
LEFT and SUBSTR correctly.

Co-authored-by: Yuchen Pei <ycp@mariadb.com>
Co-authored-by: Sergei Petrunia <sergey@mariadb.com>

Author: Lena Startseva
Author Date: 2024-12-10 06:21:23 UTC

MDEV-31005: Make working cursor-protocol

Excluded cases in main.vector,
 main.ctype_utf8,
 main.type_timestamp,
 main.log_slow_always_query_time

Fix for v.11.7

Author: Lena Startseva
Author Date: 2024-12-10 06:21:23 UTC

MDEV-31005: Make working cursor-protocol

Excluded cases in main.vector,
 main.ctype_utf8,
 main.type_timestamp,
 main.log_slow_always_query_time

Fix for v.11.7

Author: Dave Gosselin
Author Date: 2024-12-18 17:02:10 UTC

MDEV-35117 Improve error message on unexpected geometries for st_distance_sphere

When invoking st_distance_sphere with unexpected geometries, the error
message now given is:
  ERROR HY000: Internal error: Point or multipoint geometries expected
This commit fixes a few formatting issues in the affected function.

Author: Daniele Sciascia
Author Date: 2024-12-17 12:34:02 UTC

MDEV-35660 Assertion `trx->xid.is_null()' failed

The assertion fails during wsrep recovery step, in function
innobase_rollback_by_xid(). The transaction's xid is normally
cleared as part of lookup by xid, unless the transaction has
a wsrep specific xid.
This is a regression from MDEV-24035 (commit ddd7d5d8e34c3edd)
which removed the part clears xid before rollback for transaction
with a wsrep specific xid.

Author: Rex Johnston
Author Date: 2024-12-18 00:15:13 UTC

MDEV-35673 Test coverage for outer references

This is an umbrella task for good coverage for testing outer references.
An attempt at quantifying dimensionality is made in order to cover as
many different test cases as possible without gap or overlap. We attempt
to cover the cases where the magnitude of the dimension is zero (no or
degenerate case), one (a single example) and two (multiple examples of the
characteristic).
We also attempt to measure the output for each different test case, executed
in different ways.
1) Normally
2) prepared statement
3) wrapped in a derived table
4) defined in a view and executed
5) defined in a CTE and executed
6) defined in a procedure and executed
Note that running --ps-protocol and --view-protocol on top of these produces
expected results.

Result sets are compared for execution when run these different ways.
This is a work in progress, currently we have to manually check the result
file output for each statement and rely on MTR will compare results.
Care has been made to ensure that results are present and filtered/transformed
by each test case.

Author: ParadoxV5
Author Date: 2024-12-17 02:56:09 UTC

MDEV-35665 Potential Buffer Overrun in Gtid_log_event::write()

Two-Phase ALTER added a sa_seq_no field, but `Gtid_log_event::write()`'s
size calculation doesn't have an addend in its name.

This patch resizes the buffer to match `write()`'s code.

Author: Dave Gosselin
Author Date: 2024-12-18 15:26:33 UTC

MDEV-35102 CREATE TABLE AS SELECT ST_collect ... does not work

Update max_length field during fix_fields to ensure that the column
has the correct length during CREATE TABLE. st_collect now returns
a geometry type rather than a blob.

Author: Daniele Sciascia
Author Date: 2024-09-26 14:07:13 UTC

MDEV-35018 MDL BF-BF conflict on DROP TABLE

DROP TABLE on child and UPDATE of parent table can cause an MDL BF-BF
conflict when applied concurrently.
DROP TABLE takes MDL locks on both child and its parent table, however
it only it did not add certification keys for the parent table.
This patch adds the following:
 * Append certification keys corresponding to all parent tables
   before DROP TABLE replication.
 * Fix wsrep_append_fk_parent_table() so that it works when it is
   given a table list containing temporary tables.
 * Make sure function wsrep_append_fk_parent_table() is only called
   for local transaction. That was not the case for ALTER TABLE.
 * Add a test case that verifies that UPDATE parent depends on
   preceeding DROP TABLE child.
 * Adapt galera_ddl_fk_conflict test to work with DROP TABLE as well.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: ParadoxV5
Author Date: 2024-12-18 00:56:16 UTC

MDEV-35519 Populate thread_id of Backward Compatibility Query event

This commit bubbles `info->fdev` to `Query_log_event::begin_event`
so it can reconstruct the GTID event from binlog for *real* conversion.

`Gtid_log_event`s and `Query_log_event`s had no common fields previously,
but that changed when MDEV-7850 added Thread ID records to GTID events.

Author: Kristian Nielsen
Author Date: 2024-11-30 20:27:29 UTC

MDEV-33239: mysqlbinlog always stops at timestamp 0xffffffff

Do not use the magic value 0xffffffff as meaning "no --stop-datetime option
specified", as this is a valid timestamp value. Use an explicit boolean flag
instead.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Dave Gosselin
Author Date: 2024-12-16 20:57:17 UTC

MDEV-35103 CREATE TABLE AS SELECT ST_VALIDATE creates a column with datatype point, not geometry

Changes the type handler for geometry validate from 'point' to 'geometry'

Author: Kristian Nielsen
Author Date: 2024-12-03 16:18:23 UTC

MDEV-35233: RBR does not work with CSV tables

Handle null bits for record comparison in row events the same way as in
handler::calculate_checksum(), forcing bits that can be undefined to 1.
These bits are the trailing unused bits, as well as the first bit for
tables not using HA_OPTION_PACK_RECORD.

The csv storage engine leaves these bits at 0, while the row-based
replication has them set to 1, which otherwise cause can't find record error.

Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Sergei Golubchik
Author Date: 2024-12-17 15:17:01 UTC

Revert "MDEV-31340 fixup: clang++-20 -Wdeprecated-literal-operator"

This reverts commit 4d9548876e1654f86301d284af1b795c6b5bacfc.
for centos7 bintar builder with gcc 4.8.5

Author: Sergei Golubchik
Author Date: 2024-12-17 15:17:01 UTC

Revert "MDEV-31340 fixup: clang++-20 -Wdeprecated-literal-operator"

This reverts commit 4d9548876e1654f86301d284af1b795c6b5bacfc.
for centos7 bintar builder with gcc 4.8.5

Author: Kristian Nielsen
Author Date: 2024-12-17 14:05:20 UTC

MDEV-35643: Update testcase now that utf8mb4_0900_* collations are supported

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Kristian Nielsen
Author Date: 2024-12-17 12:01:22 UTC

Review patch for MDEV-35207 / PR3664

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Oleg Smirnov
Author Date: 2024-12-17 06:47:22 UTC

MDEV-35654 Re-evaluation of constant subquery triggers debug assertion

Constant subqueries, once evaluated during optimization, should not
be re-evaluated neither during optimization, nor during execution.
Nevertheless, at some scenarios re-evaluation was
possible, since previously evaluated constant IN-predicates were
not properly marked. This commit implements proper marking of subqueries
within constant conditions, preventing their re-evaluation

Author: Yuchen Pei
Author Date: 2024-12-17 04:24:35 UTC

MDEV-35662 Optimize subqueries before sending EXPLAIN output in single table update

The placement of calls to optimize_unflattened_subqueries(false) in
MDEV-25008 4b6922a315fa5411665ac99c0b40fd7238093403 reversed this
order, causing subquery plans to be chosen after sending EXPLAIN
output when the former would have errored out, which causes assertion
failure when attempting to emit an error after send_eof.

Note that there's no equivalent problem in single table deletion
because Sql_cmd_delete::delete_from_single_table() does not invoke
send_explain().

Author: Daniel Black
Author Date: 2024-12-17 04:03:11 UTC

Merge branch '10.5' into 10.5-mdev-35648

Author: Nikita Malyavin
Author Date: 2024-11-05 12:38:03 UTC

MDEV-34309 [2/2] CHECK TABLE: implement referential integrity check

Author: Nikita Malyavin
Author Date: 2024-11-05 12:38:03 UTC

MDEV-34309 [2/2] CHECK TABLE: implement referential integrity check

Author: Dave Gosselin
Author Date: 2024-12-11 20:20:30 UTC

MDEV-35568 Reintroduce delete_while_scanning for multi_delete

Reintroduces delete_while_scanning optimization for multi_delete.
Reverse some test changes from the initial feature devlopment now
that we delete-on-the-fly once again.

Author: VladislavVaintroub
Author Date: 2024-12-16 13:06:46 UTC

MDEV-34979 - generate CPEs

Security team informs, that for the purpose of vulnerability scanning,
we need to add "cpe" element in addition to "purl"

Hardcode a handful of known github name -> CPE ID mappings,
and use use this information to populate SBOM with CPEs.

Author: Yuchen Pei
Author Date: 2024-12-16 04:24:09 UTC

MDEV-35326 Clean up lex->result after a failing exec of sp core function

If the execution of the sp core function fails during optimization
stage, the exec stage will not take place, where the cleanup normally
happens through calls to join->result->send_eof().

Given that the cleanup of lex->result happened through a call to
reinit_stmt_before_use() previously in
sp_lex_keeper::reset_lex_and_exec_core(), if we need to do another
cleanup after failure, it is natural to do it in the same function.

Author: Yuchen Pei
Author Date: 2024-12-15 23:26:21 UTC

MDEV-35641 bring call to use_all_columns() forward when reading from mysql.servers

TABLE::use_all_columns turn on all bits of read_set, which is
interpreted by innodb as a request to read all columns. Without doing
so before calling init_read_record(), innodb will not retrieve any
columns if mysql.servers table has been altered to use innodb as the
engine, and any foreign servers stored in the table are "lost".

Author: Sergei Golubchik
Author Date: 2024-12-04 16:51:23 UTC

MDEV-9158 post-merge fixes

* format error messages (spaces, "bytes")
* speed up "length too large" test from 12s to 70ms
* fix it for --parallel
* fix "named pipe" test to actually test a named pipe
* add the standard header to tests, enable result log
* fix for ASAN

Author: Vicențiu Ciorbaru
Author Date: 2024-12-14 14:40:58 UTC

more readme updates

Author: Kristian Nielsen
Author Date: 2024-12-13 19:14:42 UTC

MDEV-35643 Add support for MySQL 8.0 binlog events

Add test case and fix a small typo in post_header_len initialization.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Sergei Golubchik
Author Date: 2024-12-13 09:40:04 UTC

MDEV-35046 update test results

followup for 3f114a09306

Author: Marko Mäkelä
Author Date: 2024-12-13 06:11:54 UTC

Merge 10.6-MDEV-35049

Author: Yuchen Pei
Author Date: 2024-12-13 03:54:27 UTC

MDEV-35326 Clean up lex->result after a failing select

Author: Rex Johnston
Author Date: 2024-08-21 23:00:06 UTC

MDEV-32294 2nd execution problem with inconsistent outer context paths

Fields referencing tables more outward (i.e. closer to the top most
query) embedded within subqueries can sometimes require further name
resolution after query merges during optimization.
An example of this is

select * from
(
  select * from t3
    where c in
    (
      select a from t1
        where ( select b from t2 where a > 0 and b < 3 ) <> 0
    )
) dt

here field _a_ in a single row subselect has a context that points to
the select "select a from t1".
After optimization, this select lex has been merged with it's parent.
If run as a prepared statement, on second execution, name resolution on
this field will reference a context points to this (now invalid)
select_lex.
Further name resolution can also be invoked on the first execution when
conditions are moved from having to where, such as this

select * from
(
  select * from t1
     where a in
     (
       select a from t1
       where ( select b from t2 where b < 3 group by a having a > 0 ) <> 0
     )
) dt

Fixed by implementing a smart Name_resolution_context::outer_select()
method that skips merged SELECT_LEXs and disabling direct access to
these members of Name_resolution_context.

Also fixed: incorrectly trying to alter select dependency during 2nd
execution and NOT fixing select dependency when the attached able is
defined inside a view.

When calculating the Item_subselect::used_tables_cache the current code
relies on upper_refs, this is populated by Item_subselect::mark_as_dependent()
which gets called during fix_fields on a Field/Reference that is an
outer reference.
This list contains an item contained within the Item_subselect somewhere
and a select_lex where that item is resolved.
For each item in this list, Item_subselect::update_used_tables() searches
outwards until it encounters 'new_parent'. If it does encounter this
select_lex, then the tables used in this item are records as being used
in this Item_subselect. If it doesn't encounter this select_lex, then
it sets OUTER_BIT_TABLE_REF indicating that the somewhere within lies
and item resolved above our new parent.

This new code code turns this upside down by directly referencing items
resolved in a SELECT_LEX, in that SELECT_LEX. Later, when recalculating
Item_subselect::used_tables_cache, rather than looking though all items
below that might be resolved within the SELECT_LEX of interest
(new_parent), we can iterate through the items resolved in 'new_parent'
and decide if they are relevant to our Item_subquery. We do this by
starting in the select_lex where the item is defined, walking outwards
on outer_select() and seeing if the master_unit() is the same as
Item_subselect::unit. If it is, this item 'belongs' to our
Item_subselect.

Changes:

We add SELECT_LEX::resolved_here List<Item>
Item because we can gather Item_field or Item_direct_view_ref
We add to this list on statement memory only during prepare or first execution.

When looking at items in 'new_parent' we can determine where this item is
used in this Item_subselect by walking outwards from where the item is
defined up to 'new_parent' looking to see if the enclosing unit is the
unit attached to out Item_subselect.
If we have a match then we know that we add the bit
field_item->field->table->map into our bitmap (used_tables_cache).

Test to see if this Item belongs to our unit implemented in
Item_belongs_to()

When we merge 2 queries together (either a derived table merge or a
subquery to semijoin merge), we need to manage the resolved_here list
for both SELECT_LEXs.
The SELECT_LEX that is uppermost must have it's resolved_here list
pruned of items that are belong to the SELECT_LEX to be merged. These
pruned items will no longer be outer references.

Author: Julius Goryavsky
Author Date: 2024-12-12 09:56:36 UTC

MDEV-34820: wsrep_sst_mariabackup get_footprint() portability and accuracy fix

Added get_footprint() implementation for FreeBSD (and for other
non-Linux systems), and added "apparent file size" mode for Linux
to take into account the real file size (without compression) when
used with filesystems like ZFS.

Author: VladislavVaintroub
Author Date: 2024-12-12 16:21:12 UTC

MDEV-34740 - fixed memory leak

Author: Marko Mäkelä
Author Date: 2024-12-12 09:16:56 UTC

Improve d76f5774fe70c51577cfbfdddcbfb3309d51f06e

It would be great if the actual warnings would have been listed at
least somewhere.

Author: Dmitry Shulga
Author Date: 2024-12-12 03:43:34 UTC

MDEV-34958: after Trigger doesn't work correctly with bulk insert

This bug has the same nature as the issues
  MDEV-34718: Trigger doesn't work correctly with bulk update
  MDEV-24411: Trigger doesn't work correctly with bulk insert

To fix the issue covering all use cases, resetting the thd->bulk_param
temporary to the value nullptr before invoking triggers and restoring
its original value on finishing execution of a trigger is moved to the method
  Table_triggers_list::process_triggers
that be invoked ultimately for any kind of triggers.

Author: Julius Goryavsky
Author Date: 2024-09-26 02:27:56 UTC

MDEV-34822 addendum: minor test corrections after fix

Author: Julius Goryavsky
Author Date: 2024-09-26 02:27:56 UTC

MDEV-34822 addendum: minor test corrections after fix

Author: Dave Gosselin
Author Date: 2024-12-05 19:53:48 UTC

MDEV-35587 unit.innodb_sync leaks memory on mac

unit.innodb_sync calls my_end to cleanup its memory

Author: Dmitry Shulga
Author Date: 2024-12-11 17:21:33 UTC

Follow-up patch to add version marker at the end of tests

Author: Vicențiu Ciorbaru
Author Date: 2024-12-11 15:19:24 UTC

MDEV-21829: Use packed sort keys in Unique objects

The task deals with packing the values stored in the Unique tree for each record.

The changes brought by this feature is:

  1) Unique tree can have dynamic length keys
  2) Format of keys looks like
     <key_length> <packed_value1> <packed_value2> ....... <packed_valueN>

Unique class is currently used in

1) agg_func(DISTINCT col)
   Here most aggregate functions like SUM, AVG accept only fixed size arguments
   so it is not beneficial to use packing for these. Packing is done for
   COUNT and GROUP_CONCAT (or JSON_ARRAYAGG) aggregate function as these are meaningful

2) index-merge stores row-ids
   index merge stores row-ids which are of fixed size, so packing is not required

3) Engine Independent Table statistics
   Packing is done here for variable length data types

This task is an extension to MDEV-21580.

Author: Yuchen Pei
Author Date: 2024-12-11 07:59:39 UTC

MDEV-35557 Fix SEGV when reading from ALTERed mysql.servers table

Also factor out the json parsing steps into a function.

Author: Sergey Petrunia
Author Date: 2024-11-26 12:50:41 UTC

MDEV-35616: Add basic optimizer support for virtual column

(Review input addressed)

After this patch, the optimizer can handle virtual column expressions
in WHERE/ON clauses. If the table has an indexed virtual column:

  ALTER TABLE t1
    ADD COLUMN vcol INT AS (col1+1),
    ADD INDEX idx1(vcol);

and the query uses the exact virtual column expression:

  SELECT * FROM t1 WHERE col1+1 <= 100

then the optimizer will be able use index idx1 for it.

This is achieved by walking the WHERE/ON clauses and replacing instances
of virtual column expression (like "col1+1" above) with virtual column's
Item_field (like "vcol"). The latter can be processed by the optimizer.

Replacement is considered (and done) only in items that are potentially
usable to the range optimizer.

Author: Yuchen Pei
Author Date: 2024-12-10 06:31:03 UTC

MDEV-22241 Check for LIMIT ROWS EXAMINED in optimization

By Varun Gupta

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2023-07-17 14:29:20 UTC

MDEV-27861: Creating partitioned tables should not be allowed with wsrep_osu_method=TOI and wsrep_strict_ddl=ON

Problem was incorrect handling of partitioned tables,
because db_type == DB_TYPE_PARTITION_DB
wsrep_should_replicate_ddl incorrectly marked
DDL as not replicatable. However, in partitioned
tables we should check implementing storage engine
from table->file->partition_ht() if available because
if partition handler is InnoDB all DDL should be allowed
even with wsrep_strict_ddl. For other storage engines
DDL should not be allowed and error should be issued.

This is 10.5 version of the fix.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Daniel Black
Author Date: 2024-12-09 23:17:00 UTC

Merge branch '10.5' into 10.5-mdev-35583

Author: Marko Mäkelä
Author Date: 2024-12-09 15:03:11 UTC

WIP MDEV-35000 dict_table_close() is a performance hog

FIXME: innodb.innodb_stats_fetch result difference, and probably much more

Author: Rucha Deodhar
Author Date: 2024-12-05 21:48:43 UTC

check tests + cleanup

Author: Sergey Petrunia
Author Date: 2024-12-09 08:26:45 UTC

Code cleanup:
- Make print_warn() not be a template.
- Introduce Printable_parser_rule interface, make grammar rules that
  emit warnings implement it ( and print_warn invokes its function).

- Remove Parser::Hint::append_args() as it is not used anywhere.
  (it used to be necessary call print_warn(... (Parser::Hint*)NULL);

Author: Sergey Petrunia
Author Date: 2024-12-09 08:26:45 UTC

Code cleanup:
- Make print_warn() not be a template.
- Introduce Printable_parser_rule interface, make grammar rules that
  emit warnings implement it ( and print_warn invokes its function).

- Remove Parser::Hint::append_args() as it is not used anywhere.
  (it used to be necessary call print_warn(... (Parser::Hint*)NULL);

Author: Sergey Petrunia
Author Date: 2024-12-09 08:26:45 UTC

Code cleanup:
- Make print_warn() not be a template.
- Introduce Printable_parser_rule interface, make grammar rules that
  emit warnings implement it ( and print_warn invokes its function).

- Remove Parser::Hint::append_args() as it is not used anywhere.
  (it used to be necessary call print_warn(... (Parser::Hint*)NULL);

Author: Rucha Deodhar
Author Date: 2024-02-27 14:13:51 UTC

MDEV-32854: Make JSON_DEPTH_LIMIT configurable

Create server variable to store limit for json depth with default value of
32, minimum 0 and maximum 100. Replace use of the previous direct or
indirect use JSON_DEPTH_LIMIT examples in array.

Author: Rucha Deodhar
Author Date: 2024-12-05 21:48:43 UTC

check tests + cleanup

Author: Yuchen Pei
Author Date: 2024-12-06 07:33:25 UTC

MDEV-35153 [demo] Changing socket length in mysql.servers causes rpl failure

Fails in the following way, but not for 10.6

$ mtr --no-reorder rpl.rpl_mysql_upgrade,stmt rpl.rpl_mysqldump_gtid_slave_pos,stmt
...
rpl.rpl_mysql_upgrade 'stmt' [ pass ] 1585
rpl.rpl_mysqldump_gtid_slave_pos 'stmt' [ fail ]
        Test ended at 2024-12-06 17:47:51

CURRENT_TEST: rpl.rpl_mysqldump_gtid_slave_pos
--- /home/ycp/source/mariadb-server/10.5/src/mysql-test/suite/rpl/r/rpl_mysqldump_gtid_slave_pos.result 2024-12-06 17:35:41.269072001 +1100
+++ /home/ycp/source/mariadb-server/10.5/src/mysql-test/suite/rpl/r/rpl_mysqldump_gtid_slave_pos.reject 2024-12-06 17:47:51.028049337 +1100
@@ -35,12 +35,7 @@
 include/sync_with_master_gtid.inc
 select * from mysql.gtid_slave_pos;
 domain_id sub_id server_id seq_no
-0 2 1 1
-0 3 1 2
-0 4 1 3
-0 5 1 4
-0 6 1 5
-0 7 1 6
+0 2 1 6
 select @@global.gtid_slave_pos as "after initial slave got in sync";
 after initial slave got in sync
 0-1-6

Result length mismatch

Author: Yuchen Pei
Author Date: 2024-12-06 04:39:41 UTC

MDEV-35153 10.6

Author: Yuchen Pei
Author Date: 2024-08-07 07:27:15 UTC

MDEV-34716 Fix mysql.servers socket max length too short

The limit of socket length on unix according to libc is 108, see
sockaddr_un::sun_path, but in the table it is a string of max length
64, which results in truncation of socket and failure to connect by
plugins using servers such as spider.

Author: Dave Gosselin
Author Date: 2024-06-24 21:16:27 UTC

MDEV-23278 Incorrect Calculation while using Avg Function

The decimal_precision virtual override for Item_func_case is
necessary to accurately compute precision of values populated
into decimal fields. This method was removed apparently during
a refactor of the Item_func_case family of classes between 10.2
and 10.5.

Author: Kristian Nielsen
Author Date: 2024-10-21 14:56:35 UTC

MDEV-29744: Fix incorrect locking order of LOCK_log/LOCK_commit_ordered and LOCK_global_system_variables

The LOCK_global_system_variables must not be held when taking mutexes
such as LOCK_commit_ordered and LOCK_log, as this causes inconsistent
mutex locking order that can theoretically cause the server to
deadlock.

To avoid this, temporarily release LOCK_global_system_variables in two
system variable update functions, like it is done in many other
places.

Enforce the correct locking order at server startup, to more easily
catch (in debug builds) any remaining wrong orders that may be hidden
elsewhere in the code.

Note that when this is merged to 11.4, similar unlock/lock of
LOCK_global_system_variables must be added in update_binlog_space_limit()
as is done in binlog_checksum_update() and fix_max_binlog_size(), as this
is a new function added in 11.4 that also needs the same fix. Tests will
fail with wrong mutex order until this is done.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

fixup_locking_order

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Kristian Nielsen
Author Date: 2024-12-03 14:02:06 UTC

MDEV-24959: ER_BINLOG_ROW_LOGGING_FAILED (1534: Writing one row to the row-based binary log failed)

The Write_rows_log_event originally allocated the m_rows_buf up-front, and
thus is_valid() checks that the buffer is allocated correctly. But at some
point this was changed to allocate the buffer lazily on demand. This means
that a a valid event can now have m_rows_buf==NULL. The is_valid() code was
not changed, and thus is_valid() could return false on a valid event.

This caused a bug for REPLACE INTO t() VALUES(), () which generates a
write_rows event with no after image; then the m_rows_buf was never
allocated and is_valid() incorrectly returned false, causing an error in
some other parts of the code.

Also fix a couple of missing special cases in the code for mysqlbinlog to
correctly decode (in comments) row events with missing after image.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Rex Johnston
Author Date: 2024-12-05 02:14:35 UTC

MDEV-35565 Server crashes simplifying group by <subquery>

During optimization, we may call update_depend_map_for_order()
on a group by list containing an expression with an outer reference.
The table map for this outer reference will be collected by
Item_subselect::recalc_used_tables and used to return a depend_map.
This map will indicate the presence of tables not in this select and
can cause erroneous results or crashes. We update the class used to
collect these bitmaps to check whether the containing select_lex is
applicable, and if not, set the OUTER_BIT_TABLE_REF bit in the table map.

Author: Sergey Vojtovich
Author Date: 2024-11-28 19:02:29 UTC

MDEV-34699 - mhnsw: support aarch64 SIMD instructions

SIMD implementations of bloom filters and dot product calculation.

A microbenchmark shows 1.7x dot product performance improvement compared to
regular -O2/-O3 builds and 2.4x compared to builds with auto-vectorization
disabled.

Performance improvement (microbenchmark) for bloom filters is less exciting,
within 10-30% ballpark depending on compiler options and load.

Misc implementation notes:
CalcHash: no _mm256_shuffle_epi8(), use explicit XOR/shift.
CalcHash: no 64bit multiplication, do scalar multiplication.
ConstructMask/Query: no _mm256_i64gather_epi64, access array elements explicitly.
Query: no _mm256_movemask_epi8, accumulate bits manually.

Author: VladislavVaintroub
Author Date: 2024-12-04 17:58:23 UTC

wip

Author: Sergey Petrunia
Author Date: 2024-12-04 15:30:11 UTC

Address the rest of review input