Author: Vicențiu Ciorbaru
Author Date: 2024-07-26 18:14:46 UTC

S390 fix

Author: Sergei Golubchik
Author Date: 2024-07-26 17:12:57 UTC

windows ugh

Author: Galina Shalygina
Author Date: 2024-07-24 11:55:55 UTC

MDEV-23983: Crash caused by query containing constant having clause

Before this patch the crash occured when a single row dataset is used and
Item::remove_eq_conds() is called for HAVING. This function is not supposed
to be called after the elimination of multiple equalities.

To fix this problem instead of Item::remove_eq_conds() Item::val_int() is
used. In this case the optimizer tries to evaluate the condition for the
single row dataset and discovers impossible HAVING immediately. So, the
execution phase is skipped.

Author: Sergei Golubchik
Author Date: 2024-07-18 13:20:42 UTC

localization-friendly names in ER_BAD_FIELD

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-26 11:48:03 UTC

MDEV-32401 expression cache lead to crash if table of wrong type created and the cache switched off

1) take into account TMP_TABLE_ALL_COLUMNS when
   we are modifying agg_item->result_field
2) remove unused now "bool materialized_subquery;"

Author: Lena Startseva
Author Date: 2024-07-26 08:29:28 UTC

Fix

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-26 08:52:34 UTC

MDEV-14231 MATCH() AGAINST( IN BOOLEAN MODE), results mismatch

- Added plugin_debug.test, multiple_index.test to innodb_fts suite
from mysql-5.7.
- multiple_index result changes due to stat_n_rows
value. Rollback properly sets stat_n_rows in MariaDB.
- plugin_debug result changes is due to position wasn't stored
in fts default parser. Simple parser doesn't support phrase
search query.

Author: Sergey Petrunia
Author Date: 2024-07-26 07:15:50 UTC

MDEV-34380: Set optimizer_switch='cset_narrowing=on' by default

Author: Julius Goryavsky
Author Date: 2024-07-26 07:46:58 UTC

MDEV-34647: experimental fix

Author: Rucha Deodhar
Author Date: 2024-07-17 20:00:15 UTC

check upgrading to 11.5 CS from 11.4 ES

Check that after upgrading to 11.5 CS from 11.4 ES
after applying patches for MDEV-32188 there is no
table corruption seen, only (unnecessary) repair of table.

Author: Monty
Author Date: 2024-05-14 20:47:59 UTC

MDEV-33856: Alternative Replication Lag Representation via Received/Executed Master Binlog Event Timestamps

This commit adds 3 new status variables to 'show all slaves status':

- Master_last_event_time ; timestamp of the last event read from the
  master by the IO thread.
- Slave_last_event_time ; Master timestamp of the last event committed
  on the slave.
- Master_Slave_time_diff: The difference of the above two timestamps.

All the above variables are NULL until the slave has started and the
slave has read one query event from the master that changes data.

- Added information_schema.slave_status, which allows us to remove:
   - show_master_info(), show_master_info_get_fields(),
     send_show_master_info_data(), show_all_master_info()
   - class Sql_cmd_show_slave_status.
   - Protocol::store(I_List<i_string_pair>* str_list) as it is not
     used anymore.
- Changed old SHOW SLAVE STATUS and SHOW ALL SLAVES STATUS to
  use the SELECT code path, as all other SHOW ... STATUS commands.

Other things:
- Xid_log_time is set to time of commit to allow slave that reads the
  binary log to calculate Master_last_event_time and
  Slave_last_event_time.
  This is needed as there is not 'exec_time' for row events.
- Fixed that Load_log_event calculates exec_time identically to
  Query_event.
- Updated RESET SLAVE to reset Master/Slave_last_event_time
- Updated SQL thread's update on first transaction read-in to
  only update Slave_last_event_time on group events.
- Fixed possible (unlikely) bugs in sql_show.cc ...old_format() functions
  if allocation of 'field' would fail.

Reviewed By:
Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Monty
Author Date: 2024-05-14 20:47:59 UTC

MDEV-33856: Alternative Replication Lag Representation via Received/Executed Master Binlog Event Timestamps

This commit adds 3 new status variables to 'show all slaves status':

- Master_last_event_time ; timestamp of the last event read from the
  master by the IO thread.
- Slave_last_event_time ; Master timestamp of the last event committed
  on the slave.
- Master_Slave_time_diff: The difference of the above two timestamps.

All the above variables are NULL until the slave has started and the
slave has read one query event from the master that changes data.

- Added information_schema.slave_status, which allows us to remove:
   - show_master_info(), show_master_info_get_fields(),
     send_show_master_info_data(), show_all_master_info()
   - class Sql_cmd_show_slave_status.
   - Protocol::store(I_List<i_string_pair>* str_list) as it is not
     used anymore.
- Changed old SHOW SLAVE STATUS and SHOW ALL SLAVES STATUS to
  use the SELECT code path, as all other SHOW ... STATUS commands.

Other things:
- Xid_log_time is set to time of commit to allow slave that reads the
  binary log to calculate Master_last_event_time and
  Slave_last_event_time.
  This is needed as there is not 'exec_time' for row events.
- Fixed that Load_log_event calculates exec_time identically to
  Query_event.
- Updated RESET SLAVE to reset Master/Slave_last_event_time
- Updated SQL thread's update on first transaction read-in to
  only update Slave_last_event_time on group events.
- Fixed possible (unlikely) bugs in sql_show.cc ...old_format() functions
  if allocation of 'field' would fail.

Reviewed By:
Brandon Nesterenko <brandon.nesterenko@mariadb.com>
Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Oleg Smirnov
Author Date: 2024-07-24 14:17:48 UTC

MDEV-33755 Some test cause MSAN alarms due to uninitialized Item_func::not_null_tables_cache

Commit 09d991d01c47e22030879e5bf0c7a4893a598199 resolved false
MSAN alarms by initializing `Item_func::not_null_tables_cache` with 0
during construction. However, this solution inadvertently disabled
Valgrind diagnostics, which we want to retain to detect potential issues
(e.g., failing to set not_null_tables_cache after calling
Item_func::quick_fix_field()).

This commit introduces the macro `MEM_UNDEFINED_VALGRIND_ONLY`, enabling
Valgrind diagnostic without triggering MSAN.
Now, Item_func::not_null_tables_cache is marked as undefined during
construction using this macro

Author: Daniel Black
Author Date: 2024-06-14 04:05:48 UTC

MDEV-19052 Range-type window frame supports only numeric datatype

When there is no bounds on the upper or lower part of the window,
it doesn't matter if the type is numeric.

It also doesn't matter how many ORDER BY items there are in the
query.

Reviewers: Sergei Petrunia and Oleg Smirnov

Author: Daniel Black
Author Date: 2024-07-25 07:08:31 UTC

MDEV-34099: stack overrun - postfix -test

Move ER_STACK_OVERRUN_NEED_MORE testis to not asan.

Author: Vicențiu Ciorbaru
Author Date: 2024-07-24 19:01:53 UTC

Vec_FromString and Vec_ToString

Author: Souradeep Saha
Author Date: 2024-06-27 00:18:39 UTC

Cleanup Whitespace in unittest/ directory

Cleanup unnecessary whitespace at the end of lines and end of files
in the unittest/ directory. Note that all code changes are
non-functional.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

Author: Dave Gosselin
Author Date: 2024-07-22 15:13:06 UTC

MDEV-27037 mysqlbinlog emits a warning when reaching EOF before stop-datetime

Emit a warning in the event that we finished processing input files
before reaching the boundary indicated by --stop-datetime.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-24 13:28:26 UTC

MDEV-34181 Instant table aborts after discard tablespace

- commit 85db5347311340e39753b0200fb9d459a5024535 (MDEV-33400)
avoid the clearing of the instant alter metadata during discard
tablespace. But it fails to remove the assignment of
n_core_null_bytes with total nullable fields exist in the
index for discarded tablespace.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-24 09:27:05 UTC

disabling view protcol untill fix

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-24 09:27:05 UTC

disabling view protcol untill fix

Author: Dmitry Shulga
Author Date: 2024-07-24 09:01:38 UTC

MDEV-34517: Memory leak on re-compilation of a failing statement inside a stored routine

SP instructions, consisting a body of a stored routine, had the same memory
root as an instance of the class sp_head, representing abstraction for stored
routine itself. It resulted in memory leaks on re-parsing a failed statement
of a stored routine in case the statement re-compilation has to be performed
by the reason of changes in metadata of tables, triggers, etc. the stored
routine depends on.

To fix this kind of memory leaks, every SP instruction requiring access to
a LEX object must do re-parsing of a failed statement on its own memory root.
These memory roots are allocated on sp_head's memory root and every instance of
the sp_lex_instr class has a pointer to allocated memory root in case re-parsing
of the correspondiong SP instruction was requested. On every subsequent
re-parsing of the failed statement, a memory allocated on SP instruction's
memory root is released and the memory root re-initialized. Following memory
allocations taken place on re-parsing the SP instruction's statement
is performed on the dedicated memory root. So, no memory leaks will happen on
SP statement re-parsing.

Author: Yuchen Pei
Author Date: 2024-07-18 09:26:27 UTC

MDEV-34589 Do not execute before queries in spider_db_mbase::rollback()

Rollback is not supposed to fail. This prevents false failures in
spider rollback.

Author: Alexey Botchkov
Author Date: 2024-07-09 20:50:44 UTC

MDEV-20250

The Insert_prelocking_strategy class added that prunes partition for the
INSERT INTO.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-23 16:05:27 UTC

MDEV-34066 Output of SHOW ENGINE INNODB STATUS uses the nanoseconds suffix for microseconds

- This issue is caused by commit e71e6133535da8d5eab86e504f0b116a03680780
(MDEV-24671). Change the output of transaction lock wait
time in microseconds suffix.

Author: Alexey Botchkov
Author Date: 2024-07-23 14:57:05 UTC

test2

Author: Oleg Smirnov
Author Date: 2024-07-23 08:34:23 UTC

MDEV-34634 Types mismatch when cloning items causes debug assertion

New runtime diagnostic introduced with MDEV-34490 has detected
that `Item_int_with_ref` incorrectly returns an instance of its ancestor
class `Item_int`. This commit fixes that.

In addition, this commit reverts a part of the diagnostic related
to `clone_item()` checks. As it turned out, `clone_item()` is not required
to return an object of the same class as the cloned one. For example,
look at `Item_param::clone_item()`: it can return objects of `Item_null`,
`Item_int`, `Item_string`, etc, depending on the object state.
So the runtime type diagnostic is not applicable to `clone_item()` and
is disabled with this commit.

As the similar diagnostic failures are expected to appear again
in the future, this commit introduces a new test file in the main suite:
item_types.test, and new test cases may be added to this file

Reviewer: Oleksandr Byelkin <sanja@mariadb.com>

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-23 12:07:34 UTC

MDEV-34357 InnoDB: Assertion failure in file ./storage/innobase/page/page0zip.cc line 4211

During InnoDB root page split, InnoDB does the following
1) First move the root records to the new page(p1)
2) Empty the root, insert the node pointer to the root page
3) Split the new page and make it as child nodes.
4) Finds the split record, allocate another new page(p2)
to the index
5) InnoDB stores the record(ret) predecessor to the supremum
record of the page (p2).
6) In page_copy_rec_list_start(), move the records from p1 to p2
upto the split record
6) Given table is a compressed row format page, InnoDB attempts to
compress the page p2 and failed (due to innodb_compression_level = 0)
7) Since the compression fails, InnoDB gets the number of preceding
records(ret_pos) of a record (ret) on the page (p2)
8) Page (p2) is a new page, ret points to infimum record.
ret_pos can be 0. InnoDB have wrong condition that ret_pos shouldn't
be 0 and returns corruption. InnoDB has similar wrong check in
page_copy_rec_list_end()

Author: Sergey Petrunia
Author Date: 2024-07-23 11:58:24 UTC

Remove unused Table_function_json_table::m_text_literal_cs

Author: Sergey Petrunia
Author Date: 2024-07-23 11:33:33 UTC

MDEV-25822 JSON_TABLE: default values should allow non-string literals

(Polished initial patch by Alexey Botchkov)
Make the code handle DEFAULT values of any datatype

- Make Json_table_column::On_response::m_default be Item*, not LEX_STRING.
- Change the parser to use string literal non-terminals for producing
  the DEFAULT value
-- Also, stop updating json_table->m_text_literal_cs for the DEFAULT
   value literals as it is not used.

Author: Rucha Deodhar
Author Date: 2024-07-23 10:39:10 UTC

MDEV-27412: JSON_TABLE doesn't properly unquote strings

Analysis:
The value gets appended as string instead of unescaped json value

Fix:
Append the value of json in a temporary string and then store it in the
field instead of directly storing as string.

Author: Sergey Petrunia
Author Date: 2024-07-23 08:32:42 UTC

Remove Table_function_json_table::m_text_literal_cs - it is not used.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-23 08:56:28 UTC

Merge branch '11.1' into 11.2

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-23 08:51:51 UTC

Merge branch '10.11' into 11.1

Author: Yuchen Pei
Author Date: 2024-07-22 09:37:06 UTC

MDEV-25008 [poc] Estimate outer_lookup_keys later in mysql_delete and mysql_update

This helps make better choice of materialization vs in-to-exists

Author: Yuchen Pei
Author Date: 2024-07-23 06:49:16 UTC

MDEV-25008 Some further mtr --record on top of monty's patch

main.derived_opt still failing an assertion

Also adding MDEV-25008 testcase

Author: Yuchen Pei
Author Date: 2024-07-23 04:22:33 UTC

unit->distinct in mysql_derived_prepare

Author: Andrei
Author Date: 2024-07-22 17:52:26 UTC

MDEV-15393 post-push: complete rpl_mysqldump_gtid_slave_pos fixes.

Added a missed
  --source include/save_master_gtid.inc
by the previous commit.

Author: Vlad Lesin
Author Date: 2024-06-27 09:15:37 UTC

MDEV-34466 XA prepare don't release unmodified records for some cases

There is no need to exclude exclusive non-gap locks from the procedure
of locks releasing on XA PREPARE execution in
lock_release_on_prepare_try() after commit
17e59ed3aad481918f1a01d8afbc071c316d5930 (MDEV-33454), because
lock_rec_unlock_unmodified() should check if the record was modified
with the XA, and release the lock if it was not.

lock_release_on_prepare_try(): don't skip X-locks, let
lock_rec_unlock_unmodified() to process them.

lock_sec_rec_some_has_impl(): add template parameter for not acquiring
trx_t::mutex for the case if a caller already holds the mutex, don't
crash if lock's bitmap is clean.

row_vers_impl_x_locked(), row_vers_impl_x_locked_low(): add new argument
to skip trx_t::mutex acquiring.

rw_trx_hash_t::validate_element(): don't acquire trx_t::mutex if the
current thread already holds it.

Thanks to Andrei Elkin for finding the bug.
Reviewed by Marko Mäkelä, Debarun Banerjee.

Author: Yuchen Pei
Author Date: 2024-07-22 09:05:11 UTC

MDEV-25008 [poc] Estimate outer_lookup_keys later in mysql_delete

TODO:
- better estimate? not just row numbers
- UPDATE

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-22 08:03:15 UTC

MDEV-34608 Temporary tablespace information missing in INFORMATION SCHEMA.INNODB_TABLESPACES_ENCRYPTION

- INNODB_TABLESPACES_ENCRYPTION table in information schema
should display innodb temporary tablespace with redo log
default encryption ID when innodb_encrypt_temporary_tables
is enabled.

Author: Alexey Botchkov
Author Date: 2024-07-22 07:29:15 UTC

mysql_prepare_insert run in prelocking.

Author: Lena Startseva
Author Date: 2024-05-23 01:54:14 UTC

MDEV-31005: Make working cursor-protocol

Updated tests: cases with bugs or which cannot be run
with the cursor-protocol were excluded with
"--disable_cursor_protocol"/"--enable_cursor_protocol"

Fix for v.10.5

Author: Rex Johnston
Author Date: 2024-07-02 01:27:41 UTC

MDEV-34506 2nd execution name resolution problem with pushdown into unions

Statements affected by this bug need all the following to be true
1) a derived table table or view whose specification contains a set
     operation at the top level.
2) a grouping operator (such as having) operating on a column alias
     other than in the first select of the union/intersect
3) an outer condition that will be pushed into all selects in this
     union/intersect

When pushing a condition into all selects of a unit with more than one
select, pushdown_cond_for_derived() renames items so we can re-use the
condition being pushed.
These names need to be saved and reset for correct name resolution on
second execution of prepared statements.

Author: Oleg Smirnov
Author Date: 2024-07-17 09:50:14 UTC

MDEV-33281 Fix code review comments

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-20 06:12:42 UTC

Merge branch '10.5' into 10.6

Author: Andrei
Author Date: 2024-07-15 14:50:37 UTC

MDEV-15393 gtid_slave_pos duplicate key errors after mysqldump restore

When mysqldump is run to dump the `mysql` system database, it generates
INSERT statements into the table `mysql.gtid_slave_pos`.
After running the backup script
those inserts did not produce the expected gtid state on slave. In
particular the maximum of mysql.gtid_slave_pos.sub_id did not make
into
   rpl_global_gtid_slave_state.last_sub_id

an in-memory object that is supposed to match the current state of the
table. And that was regardless of whether --gtid option was specified
or not. Later when the backup recipient server starts as slave
in *non-gtid* mode this desychronization may lead to a duplicate key
error.

This effect is corrected for --gtid mode mysqldump/mariadb-dump only
as the following. The fixes ensure the insert block of the dump
script is followed with a "summing-up" SET @global.gtid_slave_pos
assignment.

For the implemenation part, note a deferred print-out of
SET-gtid_slave_pos and associated comments is prefered over relocating
of the entire blocks if (opt_master,slave_data &&
do_show_master,slave_status) ... because of compatiblity
concern. Namely an error inside do_show_*() is handled in the new code
the same way, as early as, as before.

A regression test can be run in how-to-reproduce mode as well.
One affected mtr test observed.
rpl_mysqldump_slave.result "mismatch" shows now the new deferring print
of SET-gtid_slave_pos policy in action.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-19 13:00:10 UTC

new libfmt 11.0.1

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-19 12:17:08 UTC

New columnstore 23.10.2

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-19 11:46:35 UTC

New CC 3.3

Author: Alexander Barkov
Author Date: 2024-07-19 09:33:07 UTC

MDEV-34580 Assertion `(key_part->key_part_flag & 4) == 0' failed key_hashnr

The patch for MDEV-34417 erroneously assumed in DBUG_ASSERT() that BNLH joins
cannot be used on prefix indexes. In fact, BNHL can work with prefix indexes,
but the tests did not cover this scenario.

Fixing the asserts from "the key part does not have prefix segments" to
"the key value does not need truncation".

Author: Oleg Smirnov
Author Date: 2024-07-19 08:11:56 UTC

MDEV-33281 Fix code review comments

Author: Yuchen Pei
Author Date: 2024-07-19 03:11:26 UTC

Merge remote-tracking branch 'upstream/bb-10.6-release' into 10.11

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-07-18 14:25:33 UTC

Merge branch '10.5' into 10.6

Author: Yuchen Pei
Author Date: 2024-07-18 09:26:27 UTC

MDEV-34589 Do not execute before queries in spider_db_mbase::rollback()

Rollback is not supposed to fail. This prevents false failures in
spider rollback.

Author: Yuchen Pei
Author Date: 2024-07-18 09:16:51 UTC

[demo][debug] spider: debugging MDEV-27438

Author: Alexander Barkov
Author Date: 2024-07-18 04:15:43 UTC

Additional tests for MDEV-28345 ASAN: use-after-poison or unknown-crash in my_strtod_int from charset_info_st::strntod or test_if_number

Author: Brandon Nesterenko
Author Date: 2024-07-17 18:56:45 UTC

MDEV-33921: Fix rpl_xa_empty_transaction.test

The test was missing a save_master_gtid.inc on the master,
leading to the slave thinking it was in sync after executing
sync_with_master_gtid.inc, despite not having executed the
latest transaction. This skipped transaction, XA COMMIT,
was supposed to error-to-be-ignored because its XID could not
be found, but be thrown out because the replication filters
would filter out the target database. However, if the slave
was able to stop before executing the transaction, then
the replication filer is reset (to empty), and when the
slave is later restarted, that transactions error would
no longer be ignored.

Additionally, as the test cases added in MDEV-33921 rely
on GTID synchronization, the test cases now force
master_use_gtid=slave_pos for consistency

Author: Sergei Golubchik
Author Date: 2024-07-09 13:11:18 UTC

MDEV-34353 Revert "don't wait indefinitely for signal handler in --bootstrap"

This reverts commit 938b92937247. Not needed after 90d376e01710.

Author: Yuchen Pei
Author Date: 2024-07-17 02:38:49 UTC

Merge branch '10.5' into 10.6

Author: Yuchen Pei
Author Date: 2024-07-17 00:54:28 UTC

MDEV-30408 Reset explicit_limit in exists2in

Item_exists_subselect::fix_length_and_dec() sets explicit_limit to 1.
In the exists2in transformation it resets select_limit to NULL. For
consistency we should reset explicity_limit too.

This fixes a bug where spider table returns wrong results for queries
that gets through exists2in transformation when semijoin is off.

Author: Ian Gilfillan
Author Date: 2024-07-16 12:41:56 UTC

Add missing options to the mariadb-dump man page

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-16 20:26:42 UTC

MDEV-34392 Modification of the column fails to check foreign key constraint

Don't allow the referencing key column from NULL TO NOT NULL
when

1) Foreign key constraint type is ON UPDATE SET NULL
2) Foreign key constraint type is ON DELETE SET NULL
3) Foreign key constraint type is UPDATE CASCADE and referenced
column declared as NULL

Don't allow the referenced key column from NOT NULL to NULL
when foreign key constraint type is UPDATE CASCADE
and referencing key columns doesn't allow NULL values

check_foreign_constraints_nullability(): Introduced this handler
function to check the foreign key constraint nullability for
the alter operation

fk_check_column_changes(): Enforce the above rules for COPY
algorithm

innobase_check_foreign_drop_col(): Checks whether the dropped
column exists in existing foreign key relation

innobase_check_foreign_low() : Enforce the above rules for
INPLACE algorithm

Author: Brandon Nesterenko
Author Date: 2024-07-16 12:41:35 UTC

MDEV-34571: Fix funcs_1.is_columns_is_embedded

Result file needed re-recording to account for the
new information_schema columns

Author: Sergey Petrunia
Author Date: 2024-07-16 09:43:53 UTC

MDEV-30623: Fix the testcase

- Fix view-protocol: long expressions in SELECT
  list should have "expr AS column_name".

- Also, moved the test from subselect*test to
  suite/json/t/json_table.test.

Author: Oleg Smirnov
Author Date: 2024-04-01 07:03:46 UTC

MDEV-33010 Crash when pushing condition with CHARSET()/COERCIBILITY() into derived table

Based on the current logic, objects of classes Item_func_charset and
Item_func_coercibility (responsible for CHARSET() and COERCIBILITY()
functions) are always considered constant.
However, SQL syntax allows their use in a non-constant manner, such as
CHARSET(t1.a), COERCIBILITY(t1.a).

In these cases, the `used_tables()` parameter corresponds to table names
in the function parameters, creating an inconsistency: the item is marked
as constant but accesses tables. This leads to crashes when
conditions with CHARSET()/COERCIBILITY() are pushed into derived tables.

This commit addresses the issue by setting `used_tables()` to 0 for
`Item_func_charset` and `Item_func_coercibility`. Additionally, the items
now store the return values during the preparation phase and return
them during the execution phase. This ensures that the items do not call
its arguments methods during the execution and are truly constant.

Reviewer: Alexander Barkov <bar@mariadb.com>

Author: VladislavVaintroub
Author Date: 2024-07-16 07:30:20 UTC

MDEV-33265 mariadb-secure-installation fails with --defaults-group-suffix

Reason for the error is that --defaults-group-suffix is passed twice
to the command line client, and option parser is not prepared for this.

Double occurence of comes from 2 invocations of parse_arguments(), which
appends unparsed arguments each time it is called.

Fixed by treating --defaults-group-suffix like other "--defaults-"
(--defaults-file/--defaults-extra-file). it will be parsed, and thus
passed only once to the command line client.

Author: Alexander Barkov
Author Date: 2024-07-16 04:52:39 UTC

Moving a part of sql_lex.h into other *.h files

- Lex_ident_cli* into a new file sql/lex_ident_cli.h
- Lex_ident_sys* into a new file sql/lex_ident_sys.h
- Well_formed_prefix into include/m_ctype.h

This change is needed to the optimizer hint parser coming soon.

Author: Alexander Barkov
Author Date: 2024-07-16 04:52:39 UTC

Moving a part of sql_lex.h into other *.h files

- Lex_ident_cli* into a new file sql/lex_ident_cli.h
- Lex_ident_sys* into a new file sql/lex_ident_sys.h
- Well_formed_prefix into include/m_ctype.h

This change is needed to the optimizer hint parser coming soon.

Author: Yuchen Pei
Author Date: 2024-07-16 03:26:37 UTC

MDEV-34589 spider: unconditionally restore da status in rollback

The documentation of ha_rollback_trans() suggests that
handlerton::rollback() should never return error:

      if ((err= ht->rollback(ht, thd, all)))
      {
        // cannot happen

Further, any error returned from ha_rollback_trans() is discarded in
trans_rollback_stmt(), and the code path in mysql_admin_table() after
calling trans_rollback_stmt() calls my_eof() which asserts clear
dastatus. So we clear the dastatus in spider_rollback() on error.

Author: Yuchen Pei
Author Date: 2024-07-16 02:00:07 UTC

MDEV-34589 pass error from ha_rollback_trans() to trans_rollback_stmt()

Storage engine handler rollback may fail, in which case
ha_rollback_trans() should indicate failure too, otherwise the
dastatus may be inconsistent.

Author: Robin Newhouse
Author Date: 2024-05-01 19:44:59 UTC

Update test upgrade script for use with latest Fedora

Fedora 40 introdced wget2 as the default wget program, which caused a
break in the functionality of the test_upgrade.sh script. Modified the
archive.mariadb.org check so that it uses a one-line `curl` check to
identify the correct repository URL.

Additionally added rpm sources for the boost-program-options and openssl
1.1 and 1.0.2. This is necessary when building older versions of MariaDB
(e.g., 10.4) on newer linux distributions (e.g., Fedora 39) that do not
always have access to all required dependencies. In the above example
older MariaDB versions are not compatible with OpenSSL 3.0+, so require
an older version.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services.

Author: Robin Newhouse
Author Date: 2024-05-01 19:44:59 UTC

Update test upgrade script for use with latest Fedora

Fedora 40 introdced wget2 as the default wget program, which caused a
break in the functionality of the test_upgrade.sh script. Modified the
archive.mariadb.org check so that it uses a one-line `curl` check to
identify the correct repository URL.

Additionally added rpm sources for the boost-program-options and openssl
1.1 and 1.0.2. This is necessary when building older versions of MariaDB
(e.g., 10.4) on newer linux distributions (e.g., Fedora 39) that do not
always have access to all required dependencies. In the above example
older MariaDB versions are not compatible with OpenSSL 3.0+, so require
an older version.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services.

Author: Brandon Nesterenko
Author Date: 2024-07-15 13:54:38 UTC

MDEV-34348: MariaDB is violating clang-16 -Wcast-function-type-strict

This branch is a WIP and does not compile.

This HEAD commit just makes cast-function-type-strict fatal.
It will stay at the HEAD, and prior commits contain the
actual fixes, organized by the part of the code that the fix
targets. Future changes will be force pushed to have their
fixes come before this HEAD.

This branch cherry-picks patches from daniel@mariadb.org
(in bb-10.5-MDEV-34508-ubsan-errors) which initially
incorporate the flag into the build, as well as fix some of
its findings.

Author: Oleg Smirnov
Author Date: 2024-07-03 08:42:21 UTC

MDEV-34490 get_copy() and build_clone() may return an instance of an ancestor class instead of a copy/clone

The `Item` class methods `get_copy()`, `build_clone()`, and `clone_item()`
face an issue where they may be defined in a descendant class
(e.g., `Item_func`) but not in a further descendant (e.g., `Item_func_child`).
This can lead to scenarios where `build_clone()`, when operating on an
instance of `Item_func_child` with a pointer to the base class (`Item`),
returns an instance of `Item_func` instead of `Item_func_child`.

Since this limitation cannot be resolved at compile time, this commit
introduces runtime type checks for the copy/clone operations.
A debug assertion will now trigger in case of a type mismatch.

`get_copy()`, `build_clone()`, and `clone_item()` are no more virtual,
but virtual `do_get_copy()`, `do_build_clone()`, and `do_clone_item()`
are added to the protected section of the class `Item`.

Additionally, const qualifiers have been added to certain methods
to enhance code reliability.

Reviewer: Oleksandr Byelkin <sanja@mariadb.com>

Author: Georg Richter
Author Date: 2024-07-15 09:42:37 UTC

Use C/C 3.4-tls branch

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-07-15 09:14:59 UTC

MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in ha_innobase::estimate_rows_upper_bound

- Fixing the compilation issue.

Author: Yuchen Pei
Author Date: 2024-07-15 03:18:48 UTC

MDEV-34588 Check for thd != current_thd in spider_get_trx()

Seems to be the same cause as MDEV-33031

Author: Yuchen Pei
Author Date: 2024-07-03 02:22:13 UTC

MDEV-34449 Default ha_spider::bulk_size to 0.

This fixes a valgrind failure where the bulk_size is used before
initialised in ha_spider::end_bulk_insert().

Author: Julius Goryavsky
Author Date: 2024-07-13 02:38:10 UTC

MDEV-34505: galera.mariadb_tzinfo_to_sql fails deterministically on Ubuntu 24.04

Fixed a sorting order condition that in its previous form could lead
to the formation of an incorrect pattern for comparing strings.

Author: Vicențiu Ciorbaru
Author Date: 2024-07-12 11:12:21 UTC

sp-no-valgrind runs succesfully if ASAN support is compiled in

not_valgrind_build.inc (used by sp-no-valgrind.test) should only be
checking for valgrind as part of version string.

In fact, we do not have a specific version suffix with -asan, unless one
specifically changes the server suffix manually, during cmake run.

Author: Nikita Malyavin
Author Date: 2024-07-08 23:35:42 UTC

Architecture + extract function ptr from vtable

This patch fixes the architectural problem that was in the previous one:
* class handler is not anymore abused with extra methods that are needed only
locally during update, and belong to sql layer.

* stack is not abused with extra thread_local variables.

The prototype the function must follow now is:
int (*)(T*, const uchar*, const uchar*);

A pointer to member is converted into a function pointer. For that we need to
extract the correct one from vtable. See rtti.h. vtable and pointer to member
formats are implementation-defined, so RTTI::Method_intrn will have to be
implemented for every abi.

The two const uchar* arguments are still excessive for some cases. This only can
be improved with true JIT code generation.

class Exec_plan (exec_plan.h) handles storing, modifying and executing the plan.
class Update_execution_plan hold all the members to call and all the parameters
needed.

Update_execution_plan replaces some variables in update_single_table. It lies on
stack so the performance will be the same.

Author: Yuchen Pei
Author Date: 2024-07-12 08:48:16 UTC

MDEV-30408 Reset explicit_limit in exists2in

Item_exists_subselect::fix_length_and_dec() sets explicit_limit to 1.
In the exists2in transformation it resets select_limit to NULL. For
consistency we should reset explicity_limit too.

This fixes a bug where spider table returns wrong results for queries
that gets through exists2in transformation when semijoin is off.

Author: Monty
Author Date: 2024-07-11 11:52:23 UTC

MDEV-34571 Add page accessed and pages read from disk to table_stats

Trivial batch, using the handler statistics already collected for
the slow query log.

The reason for the changes in test cases was mainly to change to use
select TABLE_SCHEMA ... from information_schema.table_statistics instead
of 'show table_statistics' to avoid future changes to test results
if we add more columns to table_statistics.

Author: Yuchen Pei
Author Date: 2024-07-12 05:54:52 UTC

MDEV-27419 [demo/check-ci] choose desc key for desc ordering

A key may be chosen in test_if_skip_sort_order(), after calling
test_if_cheaper_ordering(). The latter iterates over the keys, and
call test_if_order_by_key() to determine whether the key can be used
in ORDER BY.

We make it so that if the result of test_if_order_by_key() indicates
the current key under consideration would result in a better matching
ordering (e.g. ORDER BY DESC matches a desc key better than an asc
key), then we choose that key, given that the range_cost is the same.

TODO: consider whether it is possible to quantify the order matching
by passing direction to get_range_limit_read_cost.

Author: Yuchen Pei
Author Date: 2024-07-12 05:38:05 UTC

MDEV-27419 [demo/check-ci] choose desc ordering in test_if_cheaper_ordering

Author: Brandon Nesterenko
Author Date: 2024-07-11 12:55:45 UTC

MDEV-34554: rpl_change_master_demote sporadically fails on buildbot

MDEV-34274 did not fix the test failure. The test has a START SLAVE
UNTIL condition, where we can't use sync_with_master_gtid.inc,
wait_for_slave_to_start.inc, or wait_for_slave_to_stop.inc because
our MTR connection thread races with the start/stop of the SQL/IO
threads. So instead, for slave start, we prove the threads started
by waiting for the connection count to increase by 2; and for slave
stop, we wait for the processlist count to return to its pre start
slave number.

Author: Monty
Author Date: 2024-07-11 10:52:14 UTC

Added supporession of server restart message to events.events_restart

Author: Oleg Smirnov
Author Date: 2024-05-04 12:50:55 UTC

MDEV-34041 Display additional information for materialized subqueries in EXPLAIN/ANALYZE FORMAT=JSON

This commits adds the "materialization" block to the output of
EXPLAIN/ANALYZE FORMAT=JSON when materialized subqueries are involved
into processing. In the case of ANALYZE additional runtime information
is displayed, such as:
  - chosen strategy of materialization
  - number of partial match/index lookup loops
  - sizes of partial match buffers

Author: Daniel Black
Author Date: 2024-06-20 05:34:16 UTC

MDEV-19123 Debian configuration - no explicit configuration for ut8mb4

There is no need for a character-set-server configuration when utf8mb4
is now the server default.

Also remove the character-set-collations as its no longer required and
the uca1400_ai_ci is now the default for all character sets that support
it. ref: MDEV-25829 / MDEV-34430.

Author: Yuchen Pei
Author Date: 2024-07-08 08:22:50 UTC

MDEV-27902 Spider check trx and get conn in rnd_next()

This allows creation of SPIDER_CONN on demand, if the previous one was
freed.

Also, the first_link_idx's are reset during
spider_check_trx_and_get_conn(), which in the case of remote HANDLER
commands, might not match the link to use correct first_link_idx for
remote HANDLER statement that was later set in
ha_spider::rnd_handler_init() (causing testing regressions in the
spider/handler suite). Therefore we fix the first_link_idx there.

Author: Yuchen Pei
Author Date: 2024-07-11 03:16:46 UTC

MDEV-34541 Clean up spider self reference check

SPIDER_CONN::loop_check_meraged_first is useless, because all
SPIDER_CONN_LOOP_CHECKs are in SPIDER_CONN::loop_check_queue, which in
spider_db_conn::fin_loop_check() is iterated over.

This fixes the use-after-free issue when there are three spider tables
sharing the same remote, and their corresponding
SPIDER_CONN_LOOP_CHECKs getting merged in
spider_conn_queue_and_merge_loop_check()

This also fixes MDEV-34555

Author: Yuchen Pei
Author Date: 2024-07-05 08:32:54 UTC

MDEV-29962 Spider: creates connections if needed before lock_tables()

Same cause as MDEV-31996. This prevents reuse of conns freed
previously, e.g. during the commit of previous statement.

It does not occur in 10.4 because of the commit for MDEV-19002 removed
the call to spider_check_trx_and_get_conn().

Author: Yuchen Pei
Author Date: 2024-07-11 05:48:48 UTC

MDEV-32492 Delete and remove trx_ha on spider share mismatch

A SPIDER_TRX_HA associated with a SPIDER_TRX could have longer
lifetime than its associated SPIDER_SHARE. And it is identified with
the associated table name. When the SPIDER_SHARE no longer valid, e.g.
when the associated spider table has been dropped and recreated, the
SPIDER_TRX_HA should be reset too.

Since spider could create a new SPIDER_SHARE with the exact same
address of a freed SPIDER_SHARE, we try to mark all SPIDER_TRX_HAs
associated with a SPIDER_SHARE invalid when the SPIDER_SHARE is about
to be freed.

Author: Daniel Black
Author Date: 2024-06-20 05:34:16 UTC

MDEV-19123 Debian configuration - no explicit configuration for ut8mb4

There is no need for a character-set-server configuration when utf8mb4
is now the server default.

Also remove the character-set-collations as its no longer required and
the uca1400_ai_ci is now the default for all character sets that support
it. ref: MDEV-25829 / MDEV-34430.

Author: Brandon Nesterenko
Author Date: 2024-06-20 18:21:48 UTC

MDEV-33921: Replication breaks when filtering two-phase XA transactions

There are two problems.

First, replication fails when XA transactions are used where the
slave has replicate_do_db set and the client has touched a different
database when running DML such as inserts. This is because XA
commands are not treated as keywords, and are thereby not exempt
from the replication filter. The effect of this is that during an XA
transaction, if its logged “use db” from the master is filtered out
by the replication filter, then XA END will be ignored, yet its
corresponding XA PREPARE will be executed in an invalid state,
thereby breaking replication.

Second, if the slave replicates an XA transaction which results in
an empty transaction, the XA START through XA PREPARE first phase of
the transaction won’t be binlogged, yet the XA COMMIT will be
binlogged. This will break replication in chain configurations.

The first problem is fixed by treating XA commands in
Query_log_event as keywords, thus allowing them to bypass the
replication filter. Note that Query_log_event::is_trans_keyword() is
changed to accept a new parameter to define its mode, to either
check for XA commands or regular transaction commands, but not both.
In addition, mysqlbinlog is adapted to use this mode so its
--database filter does not remove XA commands from its output.

The second problem fixed by overwriting the XA state in the XID
cache to be XA_ROLLBACK_ONLY, so at commit time, the server knows to
rollback the transaction and skip its binlogging. If the xid cache
is cleared before an XA transaction receives its completion command
(e.g. on server shutdown), then before reporting ER_XAER_NOTA when
the completion command is executed, the filter is first checked if
the database is ignored, and if so, the error is ignored.

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>
Andrei Elkin <andrei.elkin@mariadb.com>

Author: Brandon Nesterenko
Author Date: 2024-06-20 18:21:48 UTC

MDEV-33921: Replication breaks when filtering two-phase XA transactions

There are two problems.

First, replication fails when XA transactions are used where the
slave has replicate_do_db set and the client has touched a different
database when running DML such as inserts. This is because XA
commands are not treated as keywords, and are thereby not exempt
from the replication filter. The effect of this is that during an XA
transaction, if its logged “use db” from the master is filtered out
by the replication filter, then XA END will be ignored, yet its
corresponding XA PREPARE will be executed in an invalid state,
thereby breaking replication.

Second, if the slave replicates an XA transaction which results in
an empty transaction, the XA START through XA PREPARE first phase of
the transaction won’t be binlogged, yet the XA COMMIT will be
binlogged. This will break replication in chain configurations.

The first problem is fixed by treating XA commands in
Query_log_event as keywords, thus allowing them to bypass the
replication filter. Note that Query_log_event::is_trans_keyword() is
changed to accept a new parameter to define its mode, to either
check for XA commands or regular transaction commands, but not both.
In addition, mysqlbinlog is adapted to use this mode so its
--database filter does not remove XA commands from its output.

The second problem fixed by overwriting the XA state in the XID
cache to be XA_ROLLBACK_ONLY, so at commit time, the server knows to
rollback the transaction and skip its binlogging. If the xid cache
is cleared before an XA transaction receives its completion command
(e.g. on server shutdown), then before reporting ER_XAER_NOTA when
the completion command is executed, the filter is first checked if
the database is ignored, and if so, the error is ignored.

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>
Andrei Elkin <andrei.elkin@mariadb.com>

Author: Alexander Barkov
Author Date: 2024-07-10 09:30:01 UTC

Merge remote-tracking branch 'origin/11.5' into 11.6

Author: Monty
Author Date: 2024-05-14 20:47:59 UTC

MDEV-33856 New definition for Seconds_Behind_Master

This commit adds 3 new status variables to 'show all slaves status':

- Master_last_event_time ; timestamp of the last event read from the
  master by the IO thread.
- Slave_last_event_time ; Master timestamp of the last event committed
  on the slave.
- Master_Slave_time_diff: The difference of the above two timestamps.

All the above variables are NULL until the slave has started and the
slave has read one query event from the master that changes data.

- Added information_schema.slave_status, which allows us to remove:
   - show_master_info(), show_master_info_get_fields(),
     send_show_master_info_data(), show_all_master_info()
   - class Sql_cmd_show_slave_status.
   - Protocol::store(I_List<i_string_pair>* str_list) as it is not
     used anymore.
- Changed old SHOW SLAVE STATUS and SHOW ALL SLAVES STATUS to
  use the SELECT code path, as all other SHOW ... STATUS commands.

Other things:
- Xid_log_time is set to time of commit to allow slave that reads the
  binary log to calculate Master_last_event_time and
  Slave_last_event_time.
  This is needed as there is not 'exec_time' for row events.
- Fixed that Load_log_event calculates exec_time identically to
  Query_event.
- Updated RESET SLAVE to reset Master/Slave_last_event_time
- Updated SQL thread's update on first transaction read-in to
  only update Slave_last_event_time on group events.
- Fixed possible (unlikely) bugs in sql_show.cc ...old_format() functions
  if allocation of 'field' would fail.

Author: Daniel Black
Author Date: 2024-07-09 08:29:11 UTC

MDEV-34348: Add cast-function-type-strict to warnings