lp:maria

Author: Alexander Barkov
Author Date: 2024-04-24 12:54:00 UTC

MDEV-21058 CREATE TABLE with generated column and RLIKE results in sigabrt

Regexp_processor_pcre::fix_owner() called Regexp_processor_pcre::compile(),
which could fail on the regex syntax error in the pattern and put
an error into the diagnostics area. However, the callers:
  - Item_func_regex::fix_length_and_dec()
  - Item_func_regexp_instr::fix_length_and_dec()
still returned "false" in such cases, which made the code
crash later inside Diagnostics_area::set_ok_status().

Fix:

- Change the return type of fix_onwer() from "void" to "bool"
  and return "true" whenever an error is put to the DA
  (e.g. on the syntax error in the pattern).
- Fixing fix_length_and_dec() of the mentioned Item_func_xxx
  classes to return "true" if fix_onwer() returned "true".

Author: Nikita Malyavin
Author Date: 2024-04-27 23:10:36 UTC

innodb: change DB_FOREIGN_DUPLICATE_KEY with DB_DUPLICATE_KEY during row insert

DB_FOREIGN_DUPLICATE_KEY in row_ins_duplicate_error_in_clust was motivated by
handling the cascade changes during versioned operations.

It was found though, that certain row_update_for_mysql calls could return
DB_FOREIGN_DUPLICATE_KEY, even if there's no foreign relations.

Change DB_FOREIGN_DUPLICATE_KEY to DB_DUPLICATE_KEY in
row_ins_duplicate_error_in_clust.

It will be later converted to DB_FOREIGN_DUPLICATE_KEY in
row_ins_check_foreign_constraint if needed.

Additionally, ha_delete_row should return neither. Ensure it by an assertion.

Author: mkaruza
Author Date: 2020-07-02 14:56:29 UTC

MDEV-18590: galera.versioning_trx_id: Test failure: mysqltest: Result content mismatch

Replicated events have time associated with them from originating
node which will be used for commit timestamp. Associated time can
be set in past before event is even applied.

For WSREP replication we don't need to use time information from
event.

Addressed review comments:
   Jan Lindström <jan.lindstrom@galeracluster.com>

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: mkaruza
Author Date: 2020-07-02 14:56:29 UTC

MDEV-18590: galera.versioning_trx_id: Test failure: mysqltest: Result content mismatch

Replicated events have time associated with them from originating
node which will be used for commit timestamp. Associated time can
be set in past before event is even applied.

For WSREP replication we don't need to use time information from
event.

Addressed review comments:
   Jan Lindström <jan.lindstrom@galeracluster.com>

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Sergei Golubchik
Author Date: 2024-04-27 13:15:37 UTC

MDEV-25102 UNIQUE USING HASH error after ALTER ... DISABLE KEYS

on disable_indexes(HA_KEY_SWITCH_NONUNIQ_SAVE) the engine does
not know that the long unique is logically unique, because on the
engine level it is not. And the engine disables it,

Change the disable_indexes/enable_indexes API. Instead of the enum
mode, send a key_map of indexes that should be enabled. This way the
server will decide what is unique, not the engine.

Author: Alexander Barkov
Author Date: 2024-04-12 05:25:13 UTC

MDEV-20610 Assertion failed or btr_validate_index(..) in row_upd_sec_index_entry on a time_zone change

Author: Alexander Barkov
Author Date: 2024-04-26 07:57:25 UTC

MDEV-33534 UBSAN: Negation of -X cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in my_double_round from sql/item_func.cc|

The negation in this line:
  ulonglong abs_dec= dec_negative ? -dec : dec;
did not take into account that 'dec' can be the smallest possible
signed negative value -9223372036854775808. Its negation is
an operation with an undefined behavior.

Fixing the code to use Longlong_hybrid, which implements a safe
method to get an absolute value.

Author: Alexander Barkov
Author Date: 2024-04-26 07:57:25 UTC

MDEV-33534 UBSAN: Negation of -X cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself in my_double_round from sql/item_func.cc|

The negation in this line:
  ulonglong abs_dec= dec_negative ? -dec : dec;
did not take into account that 'dec' can be the smallest possible
signed negative value -9223372036854775808. Its negation is
an operation with an undefined behavior.

Fixing the code to use Longlong_hybrid, which implements a safe
method to get an absolute value.

Author: Alexander Barkov
Author Date: 2024-04-23 08:57:36 UTC

MDEV-12668 SRID is not preserved in UNION, VIEW, MIN, MAX

Fixing the problem that an operation involving a mix of
two or more GEOMETRY operands did not preserve their SRIDs.
Now SRIDs are preserved by hybrid functions, subqueries, TVCs, UNIONs, VIEWs.

Incompatible change:
  An attempt to mix two different SRIDs now raises an error.

Details:

- Adding a new class Type_extra_attributes. It's a generic
  container which can store very specific data type attributes.
  For now it can store one uint32 and one const pointer attribute
  (for GEOMETRY's SRID and for ENUM/SET TYPELIB respectively).
  In the future it can grow as needed.

  Type_extra_attributes will also be reused soon to store "const Type_zone*"
  pointers for the TIMESTAMP's "WITH TIME ZONE 'tz'" attribute
  (a timestamp data type with a fixed time zone independent from @@time_zone).
  The time zone attribute will be stored in exactly the same way like
  a TYPELIB pointer is stored by ENUM/SET.

- Removing Column_definition_attributes members "interval" and "srid".
  Deriving Column_definition_attributes from the generic attribute container
  Type_extra_attributes instead.

- Adding a new class Type_typelib_attributes, to store
  the TYPELIB of the ENUM and SET data types. Deriving Field_enum from it.
  Removing the member Field_enum::typelib.

- Adding a new class Type_geom_attributes, to store
  the GEOMETRY related attributes. Deriving Field_geom from it.
  Removing the member Field_geom::srid.

- Removing virtual methods:
    Field::get_typelib()
    Type_all_attributes::get_typelib() and
    Type_all_attributes::set_typelib()
  They were very specific to TYPELIB.
  Adding more generic virtual methods instead:
  * Field::type_extra_attributes() - to get extra attributes
  * Type_all_attributes::type_extra_attributes() - to get extra attributes
  * Type_all_attributes::type_extra_attributes_addr() - to set extra attributes

- Removing Item_type_holder::enum_set_typelib. Deriving Item_type_holder
  from the generic attribute container Type_extra_attributes instead.
  This makes it possible for UNION to preserve SRID
  (in addition to preserving TYPELIB).

- Deriving Item_hybrid_func from Type_extra_attributes.
  This makes it possible for hybrid functions (e.g. CASE, COALESCE,
  LEAST, GREATEST etc) to preserve SRID.

- Deriving Item_singlerow_subselect from Type_extra_attributes and
  overriding methods:
  * Item_cache::type_extra_attributes()
  * subselect_single_select_engine::fix_length_and_dec()
  * Item_singlerow_subselect::type_extra_attributes()
  * Item_singlerow_subselect::type_extra_attributes_addr()
  This is needed to preserve SRID in subqueries and TVCs

- Cleanup: fixing the data type of members
  * Binlog_type_info::m_enum_typelib
  * Binlog_type_info::m_set_typelib
  from "TYPELIB *" to "const TYPELIB *"

Author: Alexander Barkov
Author Date: 2024-04-23 08:57:36 UTC

MDEV-12668 SRID is not preserved in UNION, VIEW, MIN, MAX

Fixing the problem that an operation involving a mix of
two or more GEOMETRY operands did not preserve their SRIDs.
Now SRIDs are preserved by hybrid functions, subqueries, TVCs, UNIONs, VIEWs.

Incompatible change:
  An attempt to mix two different SRIDs now raises an error.

Details:

- Adding a new class Type_extra_attributes. It's a generic
  container which can store very specific data type attributes.
  For now it can store one uint32 and one const pointer attribute
  (for GEOMETRY's SRID and for ENUM/SET TYPELIB respectively).
  In the future it can grow as needed.

  Type_extra_attributes will also be reused soon to store "const Type_zone*"
  pointers for the TIMESTAMP's "WITH TIME ZONE 'tz'" attribute
  (a timestamp data type with a fixed time zone independent from @@time_zone).
  The time zone attribute will be stored in exactly the same way like
  a TYPELIB pointer is stored by ENUM/SET.

- Removing Column_definition_attributes members "interval" and "srid".
  Deriving Column_definition_attributes from the generic attribute container
  Type_extra_attributes instead.

- Adding a new class Type_typelib_attributes, to store
  the TYPELIB of the ENUM and SET data types. Deriving Field_enum from it.
  Removing the member Field_enum::typelib.

- Adding a new class Type_geom_attributes, to store
  the GEOMETRY related attributes. Deriving Field_geom from it.
  Removing the member Field_geom::srid.

- Removing virtual methods:
    Field::get_typelib()
    Type_all_attributes::get_typelib() and
    Type_all_attributes::set_typelib()
  They were very specific to TYPELIB.
  Adding more generic virtual methods instead:
  * Field::type_extra_attributes() - to get extra attributes
  * Type_all_attributes::type_extra_attributes() - to get extra attributes
  * Type_all_attributes::type_extra_attributes_addr() - to set extra attributes

- Removing Item_type_holder::enum_set_typelib. Deriving Item_type_holder
  from the generic attribute container Type_extra_attributes instead.
  This makes it possible for UNION to preserve SRID
  (in addition to preserving TYPELIB).

- Deriving Item_hybrid_func from Type_extra_attributes.
  This makes it possible for hybrid functions (e.g. CASE, COALESCE,
  LEAST, GREATEST etc) to preserve SRID.

- Deriving Item_singlerow_subselect from Type_extra_attributes and
  overriding methods:
  * Item_cache::type_extra_attributes()
  * subselect_single_select_engine::fix_length_and_dec()
  * Item_singlerow_subselect::type_extra_attributes()
  * Item_singlerow_subselect::type_extra_attributes_addr()
  This is needed to preserve SRID in subqueries and TVCs

- Cleanup: fixing the data type of members
  * Binlog_type_info::m_enum_typelib
  * Binlog_type_info::m_set_typelib
  from "TYPELIB *" to "const TYPELIB *"

Author: Nikita Malyavin
Author Date: 2023-01-09 18:17:42 UTC

MDEV-30046 Refactor write_record and fix idempotent replication

Idempotent write_row works same as REPLACE: if there is a duplicating
record in the table, then it will be deleted and re-inserted, with the
same update optimization.

The code in Rows:log_event::write_row was basically copy-pasted from
write_record.

What's done:
REPLACE operation was unified across replication and sql. It is now
representred as a Write_record class, that holds the whole state, and allows
re-using some resources in between the row writes.

Replace, IODKU and single insert implementations are split across different
methods, reluting in a much cleaner code.

The entry point is preserved as a single Write_record::write_record() call.
The implementation to call is chosen on the constructor stage.

This allowed several optimizations to be done:
1. The table key list is not iterated for every row. We find last unique key in
the order of checking once and preserve it across the rows. See last_uniq_key().
2. ib_handler::referenced_by_foreign_key acquires a global lock. This call was
done per row as well. Not all the table config that allows optimized replace is
folded into a single boolean field can_optimize. All the fields to check are
even stored in a single register on a 64-bit platform.
3. DUP_REPLACE and DUP_UPDATE cases now have one less level of indirection
4. modified_non_trans_tables is checked and set only when it's really needed.
5. Obsolete bitmap manipulations are removed.

Also:
* Unify replace initialization step across implementations:
  add prepare_for_replace and finalize_replace
* alloca is removed in favor of mem_root allocation. This memory is reused
  across the rows.
* An rpl-related callback is added to the replace branch, meaning that an extra
check is made per row replace even for the common case. It can be avoided with
templates if considered a problem.

Author: mkaruza
Author Date: 2024-04-27 16:40:58 UTC

MDEV-18590: galera.versioning_trx_id: Test failure: mysqltest: Result content mismatch

Replicated events have time associated with them from originating
node which will be used for commit timestamp. Associated time can
be set in past before event is even applied.

For WSREP replication we don't need to use time information from
event.

Addressed review comments:
   Jan Lindström <jan.lindstrom@galeracluster.com>

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Monty
Author Date: 2024-04-27 10:35:48 UTC

Fixed slow bootstrap introduced in 10.6

The problem was that the signal thread was not killed when using
unireg_abort().

The bug was introduced by:
MDEV-30260: Slave crashed:reload_acl_and_cache during shutdown

Other things fixed:
- Don't produce memory leaks with safemalloc if all threads was not
  ended properly (not useful)

Author: Alexander Barkov
Author Date: 2024-04-27 04:54:38 UTC

MDEV-30646 View created via JSON_ARRAYAGG returns incorrect json object

Backporting add782a13e58 from 10.6, this fixes the problem.

Author: Monty
Author Date: 2024-04-26 07:20:24 UTC

squash! 9831521fc852f72317024778c723e28d121635e6

- Change variable names in commit to
max_tmp_session_space_usage and max_tmp_total_space_usage

Author: VladislavVaintroub
Author Date: 2024-04-26 21:32:28 UTC

pcre2 - fix CMAKE_C_FLAGS for MSVC for external project

Author: Monty
Author Date: 2024-04-26 12:30:23 UTC

fixup! ddab8e0840f1b11c44bb05accae1291b1191aa1a

Author: Marko Mäkelä
Author Date: 2024-04-26 14:00:07 UTC

Add some more unit tests

Author: Monty
Author Date: 2024-04-26 13:49:03 UTC

fixup! 3a7a3e9b3be3504e1bfc32c931e14ed47a317d9b

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-26 13:27:27 UTC

Merge branch '11.0' into 11.1

Author: Sergei Golubchik
Author Date: 2024-04-26 12:12:47 UTC

sporadic failures of rpl.rpl_parallel_multi_domain_xa

it's a slow test, the slave needs to catch up, reading >1500
transactions. A default MASTER_GTID_WAIT() timeout in
sync_with_master_gtid.inc is 120 seconds, which might be not
enough for a slow/overloaded slave.

Let's wait forever or until ./mtr --testcase-timeout,
whatever comes first.

Author: Sergei Golubchik
Author Date: 2024-04-26 12:12:47 UTC

sporadic failures of rpl.rpl_parallel_multi_domain_xa

it's a slow test, the slave needs to catch up, reading >1500
transactions. A default MASTER_GTID_WAIT() timeout in
sync_with_master_gtid.inc is 120 seconds, which might be not
enough for a slow/overloaded slave.

Let's wait forever or until ./mtr --testcase-timeout,
whatever comes first.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-26 11:50:03 UTC

Merge branch 'merge-zlib' (1.3.1) into 10.4

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-26 11:50:03 UTC

Merge branch 'merge-zlib' (1.3.1) into 10.4

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-26 10:37:09 UTC

Merge branch '10.11' into 11.0

Author: Yuchen Pei
Author Date: 2024-04-26 09:59:28 UTC

MDEV-28992 Spider group by handler: Push down TIMESTAMPDIFF function

The macro ITEM_FUNC_TIMESTAMPDIFF_ARE_PUBLIC was removed in a previous
commit (1128b54aa4577da7e31c739ad9601bb11d476c72). We do not revert it
separately because it is too old and it is cleaner to "squash" the
changes here.

Similar to pr#2225, with the testcase adapted from that patch:

--8<---------------cut here---------------start------------->8---
From 884f7c6df16236748ca975339e0b1c267e195309 Mon Sep 17 00:00:00 2001
From: "Norio Akagi (norakagi)" <norakagi@amazon.com>
Date: Wed, 3 Aug 2022 23:30:34 -0700
Subject: [PATCH] [MDEV-28992] Push down TIMESTAMP_DIFF in spider

This changes so that TIMESTAMP_DIFF function in a query is pushed down and works natively in Spider.
Instead of directly accessing item's member, now we can rely on a public accessor method to make it work.
Unit tests are added under spider.pushdown_timestamp_diff.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
--8<---------------cut here---------------end--------------->8---

Author: Daniele Sciascia
Author Date: 2024-04-25 13:21:16 UTC

Fixup 0ccdf54b644352f42e1768bc660be7ab50c1e9d

0ccdf54 removed stack allocated THD objects from functions
Wsrep_schema::replay_transaction(). However, it inadvertedly
anticipated the destruction of the THD, causing assertions and usage
of THD after it was destroyed.
The fix consists in extracting the original function into a separate
function, and leave the allocation and destruction of the THD object
in Wsrep_schema::replay_transaction(), making sure that using the heap
allocated THD has no side effects.
Same for Wsrep_schema::recover_sr_transactions().

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Daniele Sciascia
Author Date: 2024-04-25 13:21:16 UTC

Fixup 0ccdf54b644352f42e1768bc660be7ab50c1e9d

0ccdf54 removed stack allocated THD objects from functions
Wsrep_schema::replay_transaction(). However, it inadvertedly
anticipated the destruction of the THD, causing assertions and usage
of THD after it was destroyed.
The fix consists in extracting the original function into a separate
function, and leave the allocation and destruction of the THD object
in Wsrep_schema::replay_transaction(), making sure that using the heap
allocated THD has no side effects.
Same for Wsrep_schema::recover_sr_transactions().

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Yuchen Pei
Author Date: 2024-04-26 07:30:15 UTC

MDEV-34002 Initialise fields in spider_db_handler

Otherwise it may result in nonsensical values like 190 for a boolean.

Author: Yuchen Pei
Author Date: 2024-04-26 07:24:03 UTC

MDEV-34002 Initialise fields in spider_db_handler

Otherwise it may result in nonsensical values like 190 for a boolean.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-26 06:26:56 UTC

Merge branch 'bb-10.11-release' into bb-11.0-release

# Conflicts:
# mysql-test/main/ssl_timeout.test
# sql/sql_parse.cc
# storage/archive/ha_archive.cc
# storage/innobase/btr/btr0btr.cc
# storage/innobase/btr/btr0cur.cc
# storage/innobase/buf/buf0buf.cc
# storage/innobase/ibuf/ibuf0ibuf.cc
# storage/spider/spd_db_mysql.cc
# storage/spider/spd_sys_table.cc

Author: Yuchen Pei
Author Date: 2024-04-26 06:22:36 UTC

MDEV-29363 replace subqueries with their values before adding to item_equal

The optimizer constructs multiple equalities, and swap them in during
equality propagation. In doing so, it may create multiple pointers to
the same item. When the item contain subqueries, bad things can
happen. For example, in pushdown of HAVING to WHERE, if the HAVING
conditions contain multiple equalities with a const subquery, the
subquery will not be excluded from pushdown into WHERE. And if there
are multiple pointers to the same subquery, it may unfix the subquery
and destroy and rebuild its join, which could cause further
use-after-free if there are external pointers to items in the
join (e.g. tmp table fields as in the main case in MDEV-29363).

This patch also fixes cases in MDEV-32424, MDEV-32608, MDEV-32539.

Author: Sergei Golubchik
Author Date: 2024-04-25 18:26:06 UTC

MDEV-33492 fix installation of rpm/deb packages

followup for 0271517495e2

Author: Yuchen Pei
Author Date: 2024-04-26 00:06:11 UTC

MDEV-30929 [demo] further debugging suggestions from andrei

Author: Rex Johnston
Author Date: 2024-04-25 20:08:31 UTC

MDEV-28621 group by optimization incorrectly removing subquery where subject buried in a function

Minimal patch to stop crashes. Do not remove the unit/select_lex from tree of unit/select_lex.

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-04-12 07:10:29 UTC

MDEV-33896 : Galera test failure on galera_3nodes.MDEV-29171

Based on logs we might start SST before donor has reached
Primary state. Because this test shutdowns all nodes we
need to make sure when we start nodes that previous nodes
have reached Primary state and joined the cluster.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-23 09:37:11 UTC

MDEV-33790 Incorrect DEFAULT expression evaluated in UPDATE

The problem was that Item_default_value::associate_with_target_field
assigned passed as argument field as an argument which changed argument
in case of default() call with certain field (i.e. deault(field)).

There is no way to get wrong field in constructor so we will not reassign
parameter.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-04-25 08:40:00 UTC

MDEV-22855 Assertion `!field->prefix_len || field->fixed_len == field->prefix_len' failed in btr_node_ptr_max_size

Problem:
========
- InnoDB wrongly calulates the record size in
btr_node_ptr_max_size() when prefix index of
the column has to be stored externally.

Fix:
====
- InnoDB should add the maximum field size to
record size when the field is a fixed length one.

Author: Kristian Nielsen
Author Date: 2024-04-23 19:02:08 UTC

MDEV-33602: Sporadic test failure in rpl.rpl_gtid_stop_start

The test could fail with a duplicate key error because switching to non-GTID
mode could start at the wrong old-style position. The position could be
wrong when the previous GTID connect was stopped before receiving the fake
GTID list event which gives the old-style position corresponding to the GTID
connected position.

Work-around by injecting an extra event and syncing the slave before
switching to non-GTID mode.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Vlad Lesin
Author Date: 2024-04-24 18:05:08 UTC

Testing the idea of purge_sys_t::choose_next_log() refactoring.

Author: mariadb-DebarunBanerjee
Author Date: 2024-04-24 14:23:13 UTC

MDEV-33669 mariabackup --backup hangs

This is a server hang and not an issue with backup. While concurrent
DDLs in server gets in hanged state, mariabackup waits for DDLs to
finish trying to acquire MDL_BACKUP_BLOCK_DDL.

The server hang is serious in nature and caused by thread pool state
being incorrectly set to thread creation pending state while no creation
is actually pending. Once a thread pool reaches such state no new thread
gets created in the pool.

While it could possibly affect all thread pools in server, the innodb
thread pool is the victim in current bug where IO job gets blocked when
the pool is stuck with much less number of threads than intended.
Available workers are blocked in purge waiting for page lock to be
released by IO write (SX lock) causing a complete deadlock.

The issue is caused by the state variable m_thread_creation_pending
introduced by MDEV-31095: 9e62ab7aafbf. We check and set the variable
early while attempting to create a new thread in pool but fail to reset
it if we exit the flow for other reasons like maximum threads reached
or get into thread creation throttling path.

Fix: The simple fix is to make sure that the state is reset back in case
we don't actually attempt to create the thread.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2024-04-24 13:49:54 UTC

MDEV-33087 ALTER TABLE...ALGORITHM=COPY should build indexes more efficiently

- During copy algorithm, InnoDB should use bulk insert operation
for row by row insert operation. By doing this, copy algorithm
can effectively build indexes. This optimization is disabled
for temporary table, versioning table and table which has
foreign key relation.

Introduced the variable innodb_alter_copy_bulk to allow
the bulk insert operation for copy alter operation
inside InnoDB. This is enabled by default

ha_innobase::extra(): HA_EXTRA_END_ALTER_COPY mode tries to apply
the buffered bulk insert operation, updates the non-persistent
table stats.

row_merge_bulk_t::write_to_index(): Update stat_n_rows after
applying the bulk insert operation

row_ins_clust_index_entry_low(): In case of copy algorithm,
switch to bulk insert operation.

copy_data_error_ignore(): Handles the error while copying
the data from source to target file.

Author: Rucha Deodhar
Author Date: 2024-03-15 18:58:48 UTC

MDEV-22141: JSON_REMOVE returns NULL on valid arguments

Analysis:
When we scan json to get to a beginning according to the path, we end up
scanning json even if we have exhausted it. When eventually returns error.

Fix:
Continue scanning json only if we have not exhausted it and return result
accordingly.

Author: Rucha Deodhar
Author Date: 2024-03-15 18:58:48 UTC

MDEV-22141: JSON_REMOVE returns NULL on valid arguments

Analysis:
When we scan json to get to a beginning according to the path, we end up
scanning json even if we have exhausted it. When eventually returns error.

Fix:
Continue scanning json only if we have not exhausted it and return result
accordingly.

Author: Dave Gosselin
Author Date: 2024-04-15 16:15:04 UTC

MDEV-33616 Tests failing on macOS

compat/oracle.sp-inout now uses lowercase object names for test
compatibility with both case-insensitive and case-sensitive
filesystems.

Author: Sergey Petrunia
Author Date: 2024-04-24 11:55:26 UTC

MDEV-28621: group by optimization incorrectly removing subquery

Test patch: Remove the subqueries at optimizer phase.
- The removed subqueries are still visible in EXPLAIN.
- The check whether a subquery is a candidate for conversion
  into semi-join now allows GROUP BY (but no aggregates, etc)
- There is only one meaningful query plan difference:
  subqueries that have child subqueries are considered expensive.

  A subquery that's cheap but has a removable GROUP BY was not
  considered expensive before. Now it is considered expensive.
  Looks like and edge case.

Author: Rucha Deodhar
Author Date: 2024-03-18 18:34:47 UTC

MDEV-32287: JSON_EXTRACT not returning multiple values for same path

Analysis:
When scanning json and getting the exact path at each step, if a path
is reached, we end up adding the item in the result and immediately get the
next item which results in current path changing.
Fix:
Instead of immediately returning the item, count the occurences of the path
in argument and append in the result as needed.

Author: Brandon Nesterenko
Author Date: 2024-04-18 21:45:27 UTC

MDEV-33924: Gtid_log_event thread_id should be 32-bit

In accordance with MDEV-15089, and to be consistent with
Query_log_event::thread_id, the Gtid_log_event::thread_id
should also be 32-bit when writing/reading to the binary
log.

Author: Lena Startseva
Author Date: 2024-04-24 06:19:35 UTC

Removing MDEV-27871 from tastcases because it is not a bug
Fix for v.11.0

Author: Lena Startseva
Author Date: 2024-04-24 04:16:39 UTC

 Removing MDEV-27871 from tastcases because it is not a bug
 Fix for v.10.11

Author: Yuchen Pei
Author Date: 2024-04-24 04:08:56 UTC

MDEV-29363 replace subqueries with their values before adding to item_equal

The optimizer constructs multiple equalities, and swap them in during
equality propagation. In doing so, it may create multiple pointers to
the same item. When the item contain subqueries, bad things can
happen. For example, in pushdown of HAVING to WHERE, if the HAVING
conditions contain multiple equalities with a const subquery, the
subquery will not be excluded from pushdown into WHERE. And if there
are multiple pointers to the same subquery, it may unfix the subquery
and destroy and rebuild its join, which could cause further
use-after-free if there are external pointers to items in the
join (e.g. tmp table fields as in the main case in MDEV-29363).

This patch also fixes cases in MDEV-32424, MDEV-32608, MDEV-32539.

Author: Yuchen Pei
Author Date: 2024-04-18 23:42:07 UTC

[demo] spider.spider_fixes_part: wait and retry

Author: Daniel Black
Author Date: 2024-04-23 01:17:59 UTC

MDEV-33967 hander socket writes do not result in binary log entries

As handlersocket bypasses the sql layer that prepares and causes
binary log entries to be created, we need to do this in the
plugin directly to create binary log entries.

Reported thanks to Yutaro Iwasaki on Zulip.

Author: Sergey Petrunia
Author Date: 2024-04-22 17:39:31 UTC

MDEV-18478 ANALYZE for statement should show selectivity of ICP, part#3

Fix the previous patch:
- Only enable handler_stats if thd->should_collect_handler_stats()==true.
- Make handler_index_cond_check() work when handler_stats are not enabled.

Author: Kristian Nielsen
Author Date: 2024-04-23 19:02:08 UTC

MDEV-33602: Sporadic test failure in rpl.rpl_gtid_stop_start

The test could fail with a duplicate key error because switching to non-GTID
mode could start at the wrong old-style position. The position could be
wrong when the previous GTID connect was stopped before receiving the fake
GTID list event which gives the old-style position corresponding to the GTID
connected position.

Work-around by injecting an extra event and syncing the slave before
switching to non-GTID mode.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Lena Startseva
Author Date: 2024-04-23 15:53:58 UTC

Removing MDEV-27871 from tastcases because it is not a bug
Fix for v. 10.6

Author: Brandon Nesterenko
Author Date: 2024-04-18 21:45:27 UTC

MDEV-33924: Gtid_log_event thread_id should be 32-bit

In accordance with MDEV-15089, and to be consistent with
Query_log_event::thread_id, the Gtid_log_event::thread_id
should also be 32-bit when writing/reading to the binary
log.

Author: Lena Startseva
Author Date: 2024-04-22 10:53:06 UTC

Removing MDEV-27871 from tastcases because it is not a bug

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-04-12 07:10:29 UTC

MDEV-33896 : Galera test failure on galera_3nodes.MDEV-29171

Based on logs we might start SST before donor has reached
Primary state. Because this test shutdowns all nodes we
need to make sure when we start nodes that previous nodes
have reached Primary state and joined the cluster.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Sergei Golubchik
Author Date: 2024-04-22 20:51:23 UTC

MDEV-32973 SHOW TABLES LIKE shows temporary tables with non-matching names

* compare both db and table name
* use the correct charset

Author: Sergei Golubchik
Author Date: 2024-04-22 20:51:23 UTC

MDEV-32973 SHOW TABLES LIKE shows temporary tables with non-matching names

* compare both db and table name
* use the correct charset

Author: Alexey Botchkov
Author Date: 2024-04-23 09:27:44 UTC

check if it breaks tests.

Author: Dmitry Shulga
Author Date: 2024-04-23 03:46:53 UTC

MDEV-33769: Memory leak found in the test main.rownum run with --ps-protocol against a server built with the option -DWITH_PROTECT_STATEMENT_MEMROOT

Author: Daniel Black
Author Date: 2024-04-23 01:17:59 UTC

MDEV-33967 hander socket writes do not result in binary log entries

As handlersocket bypasses the sql layer that prepares and causes
binary log entries to be created, we need to do this in the
plugin directly to create binary log entries.

Reported thanks to Yutaro Iwasaki on Zulip.

Author: Yuchen Pei
Author Date: 2024-04-23 00:56:19 UTC

MDEV-24610 MEMORY SE: check overflow in info calls with HA_STATUS_AUTO

Author: Monty
Author Date: 2024-04-22 14:42:09 UTC

fixup! f551fa75e13dd57b057117c570f2e2414ec6d500

Author: Sergei Golubchik
Author Date: 2024-04-22 15:22:11 UTC

fix failing large_tests.maria_recover_encrypted

update results

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-04-18 12:41:30 UTC

MDEV-33928 : Assertion failure on wsrep_thd_is_aborting

Problem was assertion assuming we always hold
THD::LOCK_thd_data mutex that is not true.
In most cases this is true but function is
also used from InnoDB lock manager and
there we can't take THD::LOCK_thd_data to
obey mutex ordering. Removed assertion as
wsrep transaction state can't change even
that case.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Monty
Author Date: 2024-04-22 14:42:09 UTC

fixup! f551fa75e13dd57b057117c570f2e2414ec6d500

Author: Elena Stepanova
Author Date: 2024-04-22 14:15:22 UTC

fixup MDEV-33145 Add FLUSH GLOBAL STATUS

Transferring commit c2213a6aae1ddc1cca5390e8b6eef9e102c2298e

Author: Sergei Golubchik
Author Date: 2024-03-15 17:42:06 UTC

mysqltest: support MARIADB_OPT_RESTRICTED_AUTH

C/C 3.4 disables mysql_old_password by default, so

add an option for the `connect` command to support specifying
allowed authentication plugins (MARIADB_OPT_RESTRICTED_AUTH).

use it to enable mysql_old_password when needed for testing

Author: Alexey Botchkov
Author Date: 2024-04-22 12:47:33 UTC

Test case showing the problem.

Author: Yuchen Pei
Author Date: 2024-04-22 04:52:39 UTC

MDEV-24610 Check for partition auto increment overflow

When ha_partition::info() is called with a flag containing
HA_STATUS_AUTO, it may iterate partitions and call info() on them with
HA_STATUS_AUTO, and update its own auto increment value with the
maximum of the partitions.

When info() is called on the MEMORY SE, it could cause an overflow
when its internal auto increment value is already ULONGLONG_MAX,
thereby setting the table stats auto increment value to 0.

If such an overflow happens, we return an error in
ha_partition::info() indicating so.

Note that callsites of ha_partition::update_next_auto_inc_val(), which
calls ha_partition::info() with HA_STATUS_AUTO, do not handle any
returned error.

Author: Yuchen Pei
Author Date: 2024-04-22 04:52:39 UTC

MDEV-24610 Check for partition auto increment overflow

When ha_partition::info() is called with a flag containing
HA_STATUS_AUTO, it may iterate partitions and call info() on them with
HA_STATUS_AUTO, and update its own auto increment value with the
maximum of the partitions.

When info() is called on the MEMORY SE, it could cause an overflow
when its internal auto increment value is already ULONGLONG_MAX,
thereby setting the table stats auto increment value to 0.

If such an overflow happens, we return an error in
ha_partition::info() indicating so.

Note that callsites of ha_partition::update_next_auto_inc_val(), which
calls ha_partition::info() with HA_STATUS_AUTO, do not handle any
returned error.

Author: Elena Stepanova
Author Date: 2024-04-20 20:02:12 UTC

MDEV-33622 Server crashes when the UPDATE statement (which has duplicate key) is run after setting a low thread_stack

Added the missing test case

Author: Alexander Barkov
Author Date: 2024-04-12 05:25:13 UTC

MDEV-20610 Assertion failed or btr_validate_index(..) in row_upd_sec_index_entry on a time_zone change

Author: Julius Goryavsky
Author Date: 2024-04-20 01:51:31 UTC

HASHICORP_KEY_MANAGEMENT: MDEV-33420 post-fix

Compilation correction after MDEV-33420.

Author: Julius Goryavsky
Author Date: 2024-04-20 01:51:31 UTC

HASHICORP_KEY_MANAGEMENT: MDEV-33420 post-fix

Compilation correction after MDEV-33420.

Author: VladislavVaintroub
Author Date: 2024-03-04 22:12:34 UTC

MDEV-33625 Add option --dir to mariadb-dump

New option works just like --tab, wrt output (sql file for table definition
and tab-separated for data, same options, e.g --parallel)

Compared to --tab it allows --databases and --all-databases.
When --dir is used , it creates directory structure in the output directory,
pointed to by --dir. For every database to be dumped, there will be a
directory with database name.

All options that --tab supports, are also supported by --dir, in particular
--parallel

Author: Sergei Golubchik
Author Date: 2024-03-17 14:00:09 UTC

reset cached client plugin when it's no longer needed

it's a pointer into the net buffer, so it might be overwritten by the
next read or write. And the next plugin switch (in multi-auth) will
try to compare it (in send_plugin_request_packet) which is normally
harmless but fails the assert with Lex_ident::is_valid_ident()

Author: Vicențiu Ciorbaru
Author Date: 2024-04-18 10:15:04 UTC

Bug fixes - on top of Hugo's patch

Author: Sergey Petrunia
Author Date: 2024-03-13 08:53:31 UTC

MDEV-12404: Add assertions about Index Condition Pushdown use

Add assertions about limitations one has when using Index Condition
Pushdown:
- add handler::assert_icp_limitations()
- call this function from functions that may attempt violations.

Verified that assert_icp_limitations() as well as calls to it are
compiled away in release build.

Author: Yuchen Pei
Author Date: 2024-04-18 07:16:44 UTC

[merge fixup] spider: restore extern declarations of some vars

How did they get removed during the merge?? The removal did not appear
in the resolution diff at all.

Author: Yuchen Pei
Author Date: 2024-04-18 01:25:41 UTC

[demo] spider.spider_fixes_part: sleep until not running?

Author: Anel Husakovic
Author Date: 2024-04-12 06:55:27 UTC

MDEV-33420: HASHICORP_KEY_MANAGEMENT fails on Windows with libcurl installed

- When `libcurl` is installed in path out of default path, like on
Windows, `include_directories` failed to find `curl/curl.h`.
- Fix `cmake` by using modern syntax with imported target and
`find_package`
- Fix warnings treated as the errors
  - Remove `HASHICORP_HAVE_EXCEPTIONS` macro and related code
- Add package to `Server` component in Windows
- Tested with `$ ./mysql-test/mtr --suite=vault`
- Closes PR #3068
- Reviewer: <wlad@mariadb.com>
            <julius.goryavsky@mariadb.com>

Author: Brandon Nesterenko
Author Date: 2024-04-17 20:38:40 UTC

Restore thread existence check

Author: Yuchen Pei
Author Date: 2024-04-17 07:26:41 UTC

MDEV-30727 Check spider_hton_ptr in spider udfs

We have to #undef my_error and find it from udfs when spider is not
installed.

Author: VladislavVaintroub
Author Date: 2024-04-16 20:04:59 UTC

MDEV-33840 - fixup.

Be more careful, when switching on/off maintenance timer.

As long as there are active tasks and not enough threads, this will now
trigger the periodic deadlock check.

Previously, pool could block, without timer even starting.

Author: Igor Babaev
Author Date: 2024-04-16 22:05:30 UTC

MDEV-30073 Wrong result on 2nd execution of PS for query with NOT EXISTS

This is a preliminary patch to check possible failures in buildbot.
Variant 7

Author: Sergey Petrunia
Author Date: 2024-04-16 09:40:47 UTC

MDEV-33752: EXPLAIN UPDATE acquires locks on table rows

Make EXPLAIN commands behave like SELECTs when it comes to locking:
- In the SQL parser, make EXPLAIN DELETE and EXPLAIN UPDATE not to
  request write MDL or table locks.
- Same as above in multi_delete_set_locks_and_link_aux_tables().
- InnoDB analyzes the return value of thd_sql_command() and sets its
  locking behavior accordingly.
  EXPLAIN UPDATE|DELETE have thd_sql_command()=SQLCOM_UPDATE|DELETE.
  Introduce also thd_sql_is_explain() and if that returns true, InnoDB
  now will work as if thd_sql_command()==SQLCOM_SELECT.

Author: VladislavVaintroub
Author Date: 2024-04-16 07:16:52 UTC

MDEV-16944 followup

- Better comments for do_write_line
- Add EOL to the line to be printed like "echo" does

Author: Dave Gosselin
Author Date: 2024-04-16 01:09:28 UTC

remove unused function should_collect_handler_stats

Author: Rex Johnston
Author Date: 2024-04-03 00:55:24 UTC

MDEV-28621 group by optimization incorrectly removing subquery where subject buried in a function

A group by query optimization technique, where the a group by clause in
a subquery can be removed is incorrectly removing references to subquery
items that may be defined and used elsewhere.

This is the initial implementation of a reference counter in the unit
attached to an Item_subquery.

Author: Dave Gosselin
Author Date: 2024-04-15 19:17:30 UTC

MDEV-33616 Tests failing on macOS

These tests rely on THR_KEY_mysys but it is not initialized. On
Linux, the corresponding thread variable is null, but on macOS it has a
nonzero value. In all cases, initialize the variable explicitly. We could
call MY_INIT, however some memory is leaked on test program exit in that
case. Instead, factor just enough to be able to initialize THR_KEY_mysys
from the test code (as well as from the library code).

Author: Elena Stepanova
Author Date: 2024-04-15 18:12:26 UTC

MDEV-33913 TABLE_STATISTICS increments ROWS_CHANGED twice upon UPDATE and does not count DELETE

Author: Kristian Nielsen
Author Date: 2024-04-15 16:54:30 UTC

Fix windows build failure

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Dave Gosselin
Author Date: 2024-03-07 20:31:26 UTC

MTR flag to mark tests as incompatible with macOS

Introduces a new MTR include, not_mac.inc, which when included
at the top of a test, prevents that test from running on macOS.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-04-12 12:43:21 UTC

and even more fixes

Author: Dave Gosselin
Author Date: 2024-03-18 17:03:33 UTC

MDEV-32777 DDL operations very slow under macOS

macOS is a development platform for our product but calls to
fcntl(F_FULLFSYNC) are too slow for developer needs. macOS offers
at least three different disk flush calls, making tradeoffs between
reliability and performance (see analysis notes on this MDEV).
On macOS only and as a compromise, we will back out the change to
call fcntl(F_FULLFSYNC), sacrificing a full flush in favor of more
performant writes.

Author: Monty
Author Date: 2024-04-12 15:59:59 UTC

Added Lock_time_ms and Table_catalog columns to metadata_lock_info

If compiled for debugging, LOCK_DURATION is also filled in.