lp:maria

Author: midenok
Author Date: 2024-02-27 14:53:07 UTC

Repeat without restart enabled in test

But it fails after ~100 repetitions because SHOW ENGINE INNODB STATUS
inserts " into output:

mysqltest: At line 132: query 'let $i= `select instr(\"\$s\", "LATEST
DETECTED DEADLOCK")`' failed: ER_PARSE_ERROR (1064): You have an error
in your SQL syntax; check the manual that corresponds to your MariaDB
server version for the right syntax to use near ';;
 2: len 7; hex 7f000000320110; asc 2 ;;
 3: len 4; hex 80000001; asc ...' at line 1

Author: midenok
Author Date: 2024-02-27 12:13:45 UTC

mysqltest intelligible error message about empty variable name

Author: Sergei Golubchik
Author Date: 2024-02-25 18:00:48 UTC

MDEV-33429 compilation of MariaDB 10.11.7 fails on arm32, sizeof(MYSQL) is wrong

Author: Monty
Author Date: 2023-12-02 15:58:26 UTC

Some changes to prepare for updated maria-backup

- Updated prototype for is_binary_frm_header().
- Added extra argument to ma_control_file_open().
- Added ma_control_file_open_or_create() for usage by tests.
  (to make test a bit simpler).

Author: Igor Babaev
Author Date: 2024-02-23 06:58:52 UTC

MDEV-31276 Wrong warnings on 2-nd execution of PS for query with GROUP_CONCAT

If a query with GROUP_CONCAT is executed then the server reports a warning
every time when the length of the result of this function exceeds the set
value of the system variable group_concat_max_len. This bug led to the set
of warnings from the second execution of the prepared statement that did
not coincide with the one from the first execution if the executed query
was a grouping query over a join of tables using GROUP_CONCAT function and
join cache was not allowed to be employed.
The descrepancy of the sets of warnings was due to lack of cleanup for
Item_func_group_concat::row_count after execution of the query.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Author: Kristian Nielsen
Author Date: 2024-02-25 16:41:50 UTC

Binlog in Engine: Very first sketch, able to create and write an InnoDB tablespace

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Monty
Author Date: 2024-02-26 11:01:38 UTC

Moved test for online alter in connect to separate test

This was needed as mtr --embedded connect.alter failed as online is
ignored for embedder server.
This fixes a wrong commit 30c965f8664cd0aad44847c0b99b618d385c2ffe

Author: Julius Goryavsky
Author Date: 2024-02-19 17:17:36 UTC

galera: correction after wsrep-lib update

Correction to ensure compatibility with the updated wsrep-lib library.

Author: Marko Mäkelä
Author Date: 2024-02-23 08:09:14 UTC

fixup! 4d04025ef98ed82b41aecb4a17c66d96a4a65de6

Improve tests

Author: Marko Mäkelä
Author Date: 2024-02-22 11:53:17 UTC

WIP fix MDEV-26643

Author: VladislavVaintroub
Author Date: 2024-01-24 18:28:51 UTC

MDEV-32841 Add tests for new counters.

Only add innodb_async_reads/writes_total_count here.
For all other counters, writing reproducible tests is rather tricky.

Author: Nikita Malyavin
Author Date: 2024-02-21 16:11:09 UTC

MDEV-23729 INFORMATION_SCHEMA for users locked due to max_password_errors

Author: Georg Richter
Author Date: 2024-02-21 15:59:50 UTC

test fixes:

Since TLSv1.0 is no longer supported in C/C 3.4, TLS1.0 tests which
were expected to fail are removed.

Author: Marko Mäkelä
Author Date: 2024-02-21 10:57:48 UTC

Merge

Unresolved conflicts: storage/innobase/os/os0file.cc
FIXME: Unchecked return value of log_write_and_flush() in
mtr_t::commit_file()

Author: Yuchen Pei
Author Date: 2024-02-21 01:19:26 UTC

MDEV-31789 Deprecate spider_casual_read

The corresponding table param was deprecated as part of MDEV-28861

Author: Yuchen Pei
Author Date: 2024-01-04 01:12:50 UTC

MDEV-28152 Features for sequences

- Add `as <int_type>` to sequence creation options
  - int_type can be signed or unsigned integer types, including
    tinyint, smallint, mediumint, int and bigint
  - Limitation: when alter sequence as <new_int_type>, cannot have any
    other alter options in the same statement
  - Limitation: increment remains signed longlong, and the hidden
    constraint (cache_size x abs(increment) < longlong_max) stays for
    unsigned types. This means for bigint unsigned, neither
    abs(increment) nor (cache_size x abs(increment)) can be between
    longlong_max and ulonglong_max
- Truncating maxvalue and minvalue from user input to the nearest max
  or min value of the type, plus or minus 1. When the truncation
  happens, a warning is emitted
- Information schema table for sequences

Author: Alexander Barkov
Author Date: 2023-09-25 17:48:01 UTC

MDEV-12252 ROW data type for stored function return values

Adding support for the ROW data type in the stored function RETURNS clause:

- explicit ROW(..members...) for both sql_mode=DEFAULT and sql_mode=ORACLE

  CREATE FUNCTION f1() RETURNS ROW(a INT, b VARCHAR(32)) ...

- anchored "ROW TYPE OF [db1.]table1" declarations for sql_mode=DEFAULT

  CREATE FUNCTION f1() RETURNS ROW TYPE OF test.t1 ...

- anchored "[db1.]table1%ROWTYPE" declarations for sql_mode=ORACLE

  CREATE FUNCTION f1() RETURN test.t1%ROWTYPE ...

Adding support for anchored scalar data types in RETURNS clause:

- "TYPE OF [db1.]table1.column1" for sql_mode=DEFAULT

  CREATE FUNCTION f1() RETURNS TYPE OF test.t1.column1;

- "[db1.]table1.column1" for sql_mode=ORACLE

  CREATE FUNCTION f1() RETURN test.t1.column1%TYPE;

Details:

- Adding a new sql_mode_t parameter to
    sp_head::create()
    sp_head::sp_head()
    sp_package::create()
    sp_package::sp_package()
  to guarantee early initialization of sp_head::m_sql_mode.
  Before this change, this member was not initialized at all during
  CREATE FUNCTION/PROCEDURE/PACKAGE statements, and was not used.
  Now it needs to be initialized to write properly the
  mysql.proc.returns column, according to the create time sql_mode.

- Moving the code from the loop body in sp_rcontext::init_var_items()
  into a separate member Spvar_definition::make_item_field_row(),
  to reuse this code for variable/parameter definitions
  and for the RETURNS clause.

- Changing the data type of sp_head::m_return_field_def
  from Column_definition to Spvar_definition.
  So now it supports not only SQL column field types,
  but also explicit ROW and anchored ROW data types.

- Modifying sql_yacc.yy to support the new grammar.
  Adding new helper methods:
    * sf_return_fill_definition_row()
    * sf_return_fill_definition_rowtype_of()
    * sf_return_fill_definition_type_of()

- Fixing tests in:
  * Virtual_tmp_table::setup_field_pointers() in sql_select.cc
  * Send_field::normalize() in field.h
  to prevent calling Type_handler_row::field_type(),
  which is implemented a DBUG_ASSERT(0).
  Before this patch the affected methods were called only
  for scalar data types. Now ROW is also possible.

- Adding a new virtual method Field::cols()

- Overriding methods:
   Item_func_sp::cols()
   Item_func_sp::element_index()
   Item_func_sp::check_cols()
   Item_func_sp::bring_value()
  to support the ROW data type.

- Extending the rule sp_return_type to support
  * explicit ROW and anchored ROW data types
  * anchored scalar data types

- Overriding Field_row::sql_type() to print
  the data type of an explicit ROW.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-02-20 08:32:26 UTC

Merge branch '11.3' into mariadb-11.3.2

Author: VladislavVaintroub
Author Date: 2024-02-20 01:26:50 UTC

x

Author: Sergey Petrunia
Author Date: 2024-02-19 22:38:37 UTC

Trivial fix in my_bitmap.c

Author: Alexander Barkov
Author Date: 2023-10-07 15:35:18 UTC

MDEV-32380 Array data type for stored routnes

Author: Alexey Botchkov
Author Date: 2024-02-19 10:04:39 UTC

MDEV-20250

preliminary solution.

Author: Yuchen Pei
Author Date: 2024-02-19 04:12:16 UTC

MDEV-33434 spider direct sql: Check length before memcpy

similar to MDEV-30981

Author: Daniel Bartholomew
Author Date: 2024-02-17 22:18:13 UTC

Merge branch 'bb-11.4-bumpversion' of github.com:MariaDB/server into bb-11.4-bumpversion

Author: Daniel Bartholomew
Author Date: 2024-02-16 21:33:35 UTC

Merge branch 'bb-11.3-bumpversion' of github.com:MariaDB/server into bb-11.3-bumpversion

Author: VladislavVaintroub
Author Date: 2024-02-16 12:54:47 UTC

speed up SSL connection time by approx factor of 2, for WolfSSL.

Optimize common RSA math operations (WOLFSSL_HAVE_SP_RSA),
Use intel assembly optimizations for it.

"slow" ssl tests, such as main.ssl-big and main.ssl_connect now execute
in half the time, with this patch.

Author: Marko Mäkelä
Author Date: 2024-02-16 15:58:35 UTC

MDEV-32175 References to buf_page_t::frame may cost some performance

This is work in progress, for performance testing.

Based on the outcome, we might want to revert changes that cause
additional calls to page_offset() or page_align().

Author: Oleg Smirnov
Author Date: 2024-02-16 07:48:26 UTC

MDEV-33436 Transform OUTER JOIN to INNER JOIN in subqueries used with IN operator

This commit adds logging of the transformations made to joins
to the optimizer trace

Author: Andrei
Author Date: 2024-02-15 18:48:58 UTC

Another increment to Xlle support.

- xid:s of Xlle are treated as if they are XAP:s at the start of
  the (recovery) initial binlog file
- xid_cache_insert() needs an explicit "was binlogged" argument
  when xid is recovered (into prepared state)
- Xid_list_log_event::Xid_list_log_event() made to compute
  the size of the list inside the ctor's body; its
  List_log_event base is inited with zero.
  That's 'cos thd_arg may be also null and then a THD instance
  has to be dynamically created, to be passed to methods dealing
  with the list, also at time of `mysqld_server_started` is still false.

Author: Sergei Golubchik
Author Date: 2024-02-14 15:19:01 UTC

MDEV-31857 fix galera.MW-284

followup for abcd23add202

Author: midenok
Author Date: 2023-10-11 15:40:40 UTC

MDEV-30973 Run-time configuration and fixes

Logging slave retries is only needed for short period of time during
measurement. Requiring server restart makes this feature inconvenient
(or impossible in some cases).

This patch provides run-time configuration which requires concurrency
protection between writing output file and switching logging off.

ConfigFactory.pm provides correctly suffixed logname for master and
slave.

FLUSH ERROR LOGS flushes slave retries log as well.

Configuration options (Global):

  log_slave_retries
  log_slave_retries_file
  log_slave_retries_max

log_slave_retries enables or disables logging of slave retries.

log_slave_retries_file specifies path to log file.
Defaults to mysqld-retries.err in the default error log location.

log_slave_retries_max specifies maximum retry attempt at which the
repeated message is printed (see below).

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-02-13 08:24:32 UTC

fix view protocol in MDEV-29179

Author: Yuchen Pei
Author Date: 2024-02-12 10:08:22 UTC

MDEV-33441 Do not deinit plugin variables when retry requested

After MDEV-31400, plugins are allowed to ask for retries when failing
initialisation. However, such failures also cause plugin system
variables to be deleted (plugin_variables_deinit()) before retrying
and are not re-added during retry.

We fix this by checking that if the plugin has requested a retry the
variables are not deleted. Because plugin_deinitialize() also calls
plugin_variables_deinit(), if the retry fails, the variables will
still be deleted.

Alternatives considered:

- remove the plugin_variables_deinit() from plugin_initialize() error
handling altogether. We decide to take a more conservative approach
here.

- re-add the system variables during retry. It is more complicated
than simply iterating over plugin->system_vars and call
my_hash_insert(). For example we will need to assign values to
the test_load field and extract more code from test_plugin_options(),
if that is possible.

Author: Yuchen Pei
Author Date: 2024-02-12 13:11:21 UTC

MDEV-33434 spider direct sql: Check length before memcpy

similar to MDEV-30981

Author: Yuchen Pei
Author Date: 2024-02-12 10:08:22 UTC

MDEV-33441 Do not deinit plugin variables when retry requested

After MDEV-31400, plugins are allowed to ask for retries when failing
initialisation. However, such failures also cause plugin system
variables to be deleted (plugin_variables_deinit()) before retrying
and are not re-added during retry.

We fix this by checking that if the plugin has requested a retry the
variables are not deleted. Because plugin_deinitialize() also calls
plugin_variables_deinit(), if the retry fails, the variables will
still be deleted.

Alternatives considered:

- remove the plugin_variables_deinit() from plugin_initialize() error
handling altogether. We decide to take a more conservative approach
here.

- re-add the system variables during retry. It is more complicated
than simply iterating over plugin->system_vars and call
my_hash_insert(). For example we will need to assign values to
the test_load field and extract more code from test_plugin_options(),
if that is possible.

Author: Sergei Golubchik
Author Date: 2024-01-17 14:32:45 UTC

initial support for vector indexes

MDEV-33407 Parser support for vector indexes

The syntax is

  create table t1 (... vector index (v) ...);

limitation:
* v is a binary string and NOT NULL
* only one vector index per table
* temporary tables are not supported

MDEV-33404 Engine-independent indexes: subtable method

added support for so-called "high level indexes", they are not visible
to the storage engine, implemented on the sql level. For every such
an index in a table, say, t1, the server implicitly creates a second
table named, like, t1#i#05 (where "05" is the index number in t1).
This table has a fixed structure, no frm, not accessible directly,
doesn't go into the table cache, needs no MDLs.

MDEV-33406 basic optimizer support for k-NN searches

for a query like SELECT ... ORDER BY func() optimizer will use
item_func->part_of_sortkey() to decide what keys can be used
to resolve ORDER BY.

Author: VladislavVaintroub
Author Date: 2024-02-09 01:18:32 UTC

MDEV-33430 - Fix self-signed certificate errors on Windows

Adjust test after fixing the C/C.

On Windows, use --host=127.0.0.2 to fake "insecure" transport
with TCP connection for test purposes. 127.0.0.2 is loopback address,
that can be used instead of usual 127.0.0.1

Unfortunately, this technique does not work on all *nixes the same,
notably neither on BSDs nor Solaris. Thus default --host=localhost
remains "insecure" transport,when TCP is used. but it is not that critical,
the "self-signed" is not nearly as annoying on *nixes as it is on Windows.

Author: Brandon Nesterenko
Author Date: 2024-02-08 16:55:02 UTC

MDEV-14357: rpl.rpl_domain_id_filter_io_crash failed in buildbot with wrong result

A race condition with the SQL thread, where depending on if it was
killed before or after it had executed the fake/generated IGN_GTIDS
Gtid_list_log_event, may or may not update gtid_slave_pos with the
position of the ignored events. Then, the slave would be restarted
while resetting IGNORE_DOMAIN_IDS to be empty, which would result in
the slave requesting different starting locations, depending on
whether or not gtid_slave_pos was updated. And, because previously
ignored events could now be requested and executed (no longer
ignored), their presence would fail the test.

This patch fixes this in two ways. First, to use GTID positions for
synchronization rather than binlog file positions. Then second, to
synchronize the SQL thread’s gtid_slave_pos with the ignored events
before killing the SQL thread.

To consistently reproduce the test failure, the following patch can
be applied:

diff --git a/sql/log_event_server.cc b/sql/log_event_server.cc
index f51f5b7deec..de62233acff 100644
--- a/sql/log_event_server.cc
+++ b/sql/log_event_server.cc
@@ -3686,6 +3686,12 @@ Gtid_list_log_event::do_apply_event(rpl_group_info *rgi)
     void *hton= NULL;
     uint32 i;

+ sleep(1);
+ if (rli->sql_driver_thd->killed || rli->abort_slave)
+ {
+ return 0;
+ }
+

Author: Alexander Barkov
Author Date: 2024-02-08 11:58:26 UTC

MDEV-33428 Error messages ER_PACKAGE_ROUTINE_* are not good enough

Changing the format in error messages:
- ER_PACKAGE_ROUTINE_IN_SPEC_NOT_DEFINED_IN_BODY
- ER_PACKAGE_ROUTINE_FORWARD_DECLARATION_NOT_DEFINED

from
  "Subroutine 'db.pkg.f1' ..."

to a more clear:
  "FUNCTION `db.pkg.f1` ..."
  "PROCEDURE `db.pkg.p1` ..."

Author: Yuchen Pei
Author Date: 2024-02-08 07:32:34 UTC

MDEV-29363 avoid adding items with subqueries to item_equal

The optimizer constructs multiple equalities, and swap them in during
equality propagation. In doing so, it may create multiple pointers to
the same item. When the item contain subqueries, bad things can
happen. For example, in pushdown of HAVING to WHERE, if the HAVING
conditions contain multiple equalities with a const subquery, the
subquery will not be excluded from pushdown into WHERE. And if there
are multiple pointers to the same subquery, it may unfix the subquery
and destroy and rebuild its join, which could cause further
use-after-free if there are external pointers to items in the
join (e.g. tmp table fields as in the main case in MDEV-29363).

This patch also fixes cases in MDEV-32424, MDEV-32608, MDEV-32539.

Author: Dmitry Shulga
Author Date: 2024-02-08 05:17:02 UTC

MDEV-15703: Crash in EXECUTE IMMEDIATE 'CREATE OR REPLACE TABLE t1 (a INT DEFAULT ?)' USING DEFAULT

This patch fixes the issue with passing the DEFAULT or IGNORE values to
positional parameters for some kind of SQL statements to be executed
as prepared statements.

The main idea of the patch is to associate an actual value being passed
by the USING clause with the positional parameter represented by
the Item_param class. Such association must be performed on execution of
UPDATE statement in PS/SP mode. Other corner cases that results in
server crash is on handling CREATE TABLE when positional parameter
placed after the DEFAULT clause or CALL statement and passing either
the value DEFAULT or IGNORE as an actual value for the positional parameter.
This case is fixed by checking whether an error is set in diagnostics
area at the function pack_vcols() on return from the function pack_expression()

Author: Daniel Black
Author Date: 2024-02-08 01:49:20 UTC

Merge branch '11.4' into bb-11.4-MDEV-18478

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-02-07 07:50:01 UTC

Merge branch '11.2' into mariadb-11.2.3

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-02-07 07:47:37 UTC

Merge branch '11.1' into mariadb-11.1.4

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-02-07 07:21:29 UTC

Merge branch '11.0' into mariadb-11.0.5

Author: Alexander Barkov
Author Date: 2024-02-07 05:05:28 UTC

MDEV-33392 Server crashes when using RANDOM_BYTES function and GROUP BY clause on a column with a negative value

Item_func_random_bytes did not set its NULL-ability flag.

Author: Yuchen Pei
Author Date: 2024-02-06 13:30:25 UTC

MDEV-32640 allow sys_var_charptr to update session value inplace

If true, it allocates the session value once, and updates it with
strmake(), instead of freeing existing char* and creating new ones.

Make default_master_connection such a variable, so that the session
updates syncs automatically with statement ones. This fixes a
regression introduced by MDEV-15935.

Author: Daniel Bartholomew
Author Date: 2024-02-06 13:29:54 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2024-02-06 13:28:35 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2024-02-06 13:27:16 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2024-02-06 13:25:30 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2024-02-06 13:23:21 UTC

bump the VERSION

Author: Daniel Bartholomew
Author Date: 2024-02-06 13:21:25 UTC

bump the VERSION

Author: Yuchen Pei
Author Date: 2024-02-06 11:26:01 UTC

MDEV-32640 [wip] add an in_place_update field to sys_var_charptr

Author: Oleg Smirnov
Author Date: 2024-02-05 10:43:09 UTC

MDEV-32034: Second SELECT from view created from INFORMATION_SCHEMA.OPTIMIZER_TRACE gives NULL

Function list_has_optimizer_trace_table() is responsible for checking
whether the query includes I_S.OPTIMIZER_TRACE. However, the function
is called at an early stage, before references are resolved,
resulting in the omission of cases where a SELECT is made from a view
constructed from I_S.OPTIMIZER_TRACE.

To address this, an supplementary call to
list_has_optimizer_trace_table() is added at the end of tracing,
when all the information is available. That allows to identify cases
in which the SELECT is made from a view or a stored procedure
containing I_S.OPTIMIZER_TRACE

Author: Brandon Nesterenko
Author Date: 2024-02-05 12:56:53 UTC

Revert "MDEV-7850: Extend GTID Binlog Events with Thread Id"

This reverts commit c37b2087b4abe576f1b0391c8d379dba6299dcb5.

In c37b20887, when re-binlogging a GTID event on a replica,
it will overwrite the thread_id from the primary to be the
value of the slave applier (SQL thread or parallel worker).

This should be the value of the original thread_id on the
master connection though, to both help track temporary
tables, and be consistent with Query_log_event.

Reverting the commit to re-target 11.5, so we can re-test
with the corrected thread_id.

Author: Alexander Barkov
Author Date: 2024-02-05 10:41:44 UTC

MDEV-33386 Wrong error message on `GRANT .. ON PACKAGE no_such_package ..`

When displaying the ER_SP_DOES_NOT_EXIST error, use
Sp_handler::type_lex_cstring() to the the underlying
object type:
- PROCEDURE
- FUNCTION
- PACKAGE
- PACKAGE BODY
instead of hard-coded "FUNCTION or PROCEDURE".

Author: Yuchen Pei
Author Date: 2024-02-05 09:56:51 UTC

[fixup] Spider: the MDEV-29718 fix requires use of spider_same_server_link

...for spider testcases where the remote data node is specified by a
server on the same server

Author: Sergei Golubchik
Author Date: 2023-10-15 14:03:14 UTC

MDEV-32473 --disable-ssl doesn't disable it

Author: Nikita Malyavin
Author Date: 2024-02-03 05:40:06 UTC

MDEV-33348 ALTER TABLE lock waiting stages are indistinguishable

Several points of synchronization during ALTER TABLE COPY looked identical
in the progress report query. Besides, if its the late lock upgrade stage,
the data would be:
STAGE 0
MAX_STAGE 0
PROGRESS 0.000

which looks irrelevant.

This patch moves thd_progress_deinit call after the last lock upgrade.

Also, for online alter, if there is nothing to replicate, the
progress and max_progress values would be 0, which discard the result data
on the side of sql_show, see processlist_callback in sql_show.cc.
So now the minimal max_progress will be 1. To avoid 0% progress in the
report, minimax progress value is also set to 1, so we will see 100% if
there's nothing to replicate.

Author: Igor Babaev
Author Date: 2024-02-03 04:33:40 UTC

MDEV-31361 Wrong result on 2nd execution of PS for query with derived table

This bug led to wrong result sets returned by the second execution of
prepared statements from selects using mergeable derived tables pushed
into external engine. Such derived tables are always materialized. The
decision that they have to be materialized is taken late in the function
mysql_derived_optimized(). For regular derived tables this decision is
usually taken at the prepare phase. However in some cases for some derived
tables this decision is made in mysql_derived_optimized() too. It can be
seen in the code of mysql_derived_fill() that for such a derived table it's
critical to change its translation table to tune it to the fields of the
temporary table used for materialization of the derived table and this
must be done after each refill of the derived table. The same actions are
needed for derived tables pushed into external engines.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Author: Kristian Nielsen
Author Date: 2024-02-01 22:29:55 UTC

MDEV-31949: XA review, test cases for binlog recovery

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: Rex Johnston
Author Date: 2023-08-18 03:51:54 UTC

MDEV-31466 Add optional correlation column list for derived tables

extend Mariadb to support column name assignment in derived sytax
(some expression) [as|=] ident [(comma separated column name list)]

Author: Sergey Petrunia
Author Date: 2024-01-26 13:54:35 UTC

MDEV-33314: Crash in calculate_cond_selectivity_for_table() with many columns

Variant#3: moved the logic out of create_key_parts_for_pseudo_indexes

Range Analyzer (get_mm_tree functions) can only process up to MAX_KEY=64
indexes. The problem was that calculate_cond_selectivity_for_table used
it to estimate selectivities for columns, and since a table can
have > MAX_KEY columns, would invoke Range Analyzer with more than MAX_KEY
"pseudo-indexes".

Fixed by making calculate_cond_selectivity_for_table() to run Range
Analyzer with at most MAX_KEY pseudo-indexes. If there are more
columns to process, Range Analyzer will be invoked multiple times.

Also made this change:
- param.real_keynr[0]= 0;
+ MEM_UNDEFINED(&param.real_keynr, sizeof(param.real_keynr));

Range Analyzer should have no use on real_keynr when it is run with
pseudo-indexes.

Author: Alexander Barkov
Author Date: 2024-02-01 11:20:37 UTC

Add a replication MTR test cloning the slave with mariadb-backup

Part2: adding a test for Galera->MariaDB replication

Author: Igor Babaev
Author Date: 2024-02-01 07:50:41 UTC

MDEV-31305 Crash caused by query with aggregation over materialized derived

This bug was fixed by the patch for bug MDEV-30706.
Only a test case is added in this commit.

Author: Brandon Nesterenko
Author Date: 2024-01-30 14:46:38 UTC

MDEV-7850: Fix galera_sr.MDEV-18585

Binlog position changed due to presence of GTID thread_id

Author: midenok
Author Date: 2024-01-31 12:59:44 UTC

grep.inc fixes

Wrong comment and unclear error message leads to usage troubles.

Author: Sergei Golubchik
Author Date: 2024-01-30 15:39:28 UTC

regression introduced by MDEV-14448

Author: Brandon Nesterenko
Author Date: 2024-01-30 14:46:38 UTC

MDEV-7850: Fix galera_sr.MDEV-18585

Binlog position changed due to presence of GTID thread_id

Author: Denis Protivensky
Author Date: 2023-11-30 09:52:53 UTC

MDEV-22232 Post-fix test after changing behavior of ALTER DROP FOREIGN KEY

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Denis Protivensky
Author Date: 2023-11-30 09:52:53 UTC

MDEV-22232 Fix test after changing behavior of ALTER DROP FOREIGN KEY

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2023-09-04 09:22:51 UTC

MDEV-25731 : Assertion `mode_ == m_local' failed in wsrep::client_state::streaming_params()

Problem was that if wsrep_load_data_splitting was used
streaming replication (SR) parameters were set
for MyISAM table. Galera does not currently support SR for
MyISAM.

Fix is to ignore wsrep_load_data_splitting setting (with
warning) if table is not InnoDB table.

This is 10.4-10.5 case of fix.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Monty
Author Date: 2024-01-29 13:51:29 UTC

MDEV-21472: ALTER TABLE ... ANALYZE PARTITION ... with EITS reads and locks all rows

This was fixed in 10.2 in 2020 but merging the code to 10.3 caused the
bug to come back.

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-01-04 14:46:47 UTC

MDEV-33173 : Galera test case galera_sr_kill_slave_before_apply unstable

Add wait_condition to make sure tables are created before next
operations.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-01-04 14:46:47 UTC

MDEV-33173 : Galera test case galera_sr_kill_slave_before_apply unstable

Add wait_condition to make sure tables are created before next
operations.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Yuchen Pei
Author Date: 2024-01-29 23:18:53 UTC

[fixup] Spider: the MDEV-29718 fix requires use of spider_same_server_link

...for spider testcases where the remote data node is specified by a
server on the same server

Author: VladislavVaintroub
Author Date: 2023-12-06 14:09:44 UTC

MDEV-32216 Option --parallel in mariadb-import

This is done for symmetry with mariadb-dump, which does not use threads
but allows parallelism via --parallel

Traditional --use-threads can still be used, it is synonymous
with --parallel

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-01-29 09:48:16 UTC

cc

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2023-09-04 09:22:51 UTC

MDEV-25731 : Assertion `mode_ == m_local' failed in wsrep::client_state::streaming_params()

Problem was that if wsrep_load_data_splitting was used
streaming replication (SR) parameters were set
for MyISAM table. Galera does not currently support SR for
MyISAM.

Fix is to ignore wsrep_load_data_splitting setting (with
warning) if table is not InnoDB table.

This is 10.6+ case of fix.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Yuchen Pei
Author Date: 2024-01-29 05:29:36 UTC

MDEV-33174 Fixing nondeterministic self-referencing test result

Author: Kristian Nielsen
Author Date: 2023-09-08 11:12:49 UTC

MDEV-4991: GTID binlog indexing

Improve the performance of slave connect using B+-Tree indexes on each binlog
file. The index allows fast lookup of a GTID position to the corresponding
offset in the binlog file, as well as lookup of a position to find the
corresponding GTID position.

This eliminates a costly sequential scan of the starting binlog file
to find the GTID starting position when a slave connects. This is
especially costly if the binlog file is not cached in memory (IO
cost), or if it is encrypted or a lot of slaves connect simultaneously
(CPU cost).

The size of the index files is generally less than 1% of the binlog data, so
not expected to be an issue.

Most of the work writing the index is done as a background task, in
the binlog background thread. This minimises the performance impact on
transaction commit. A simple global mutex is used to protect index
reads and (background) index writes; this is fine as slave connect is
a relatively infrequent operation.

Here are the user-visible options and status variables. The feature is on by
default and is expected to need no tuning or configuration for most users.

binlog_gtid_index
  On by default. Can be used to disable the indexes for testing purposes.

binlog_gtid_index_page_size (default 4096)
  Page size to use for the binlog GTID index. This is the size of the nodes
  in the B+-tree used internally in the index. A very small page-size (64 is
  the minimum) will be less efficient, but can be used to stress the
  BTree-code during testing.

binlog_gtid_index_span_min (default 65536)
  Control sparseness of the binlog GTID index. If set to N, at most one
  index record will be added for every N bytes of binlog file written.
  This can be used to reduce the number of records in the index, at
  the cost only of having to scan a few more events in the binlog file
  before finding the target position

Two status variables are available to monitor the use of the GTID indexes:

  Binlog_gtid_index_hit
  Binlog_gtid_index_miss

The "hit" status increments for each successful lookup in a GTID index.
The "miss" increments when a lookup is not possible. This indicates that the
index file is missing (eg. binlog written by old server version
without GTID index support), or corrupt.

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Author: VladislavVaintroub
Author Date: 2024-01-26 09:48:54 UTC

update C/C

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-01-05 09:33:53 UTC

MDEV-33138 : Galera test case MW-336 unstable

Add more inserts before wsrep_slave_threads is set to 1 and
add wait_condition to wait all of them are replicated before
wait_condition about number of wsrep_slave_threads.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Alexander Barkov
Author Date: 2024-01-23 14:04:31 UTC

MDEV-33299 Assertion `(tm->tv_usec % (int) log_10_int[6 - dec]) == 0' failed in void my_timestamp_to_binary(const timeval*, uchar*, uint)

This original query:

(1) SELECT ts0 FROM t1
     WHERE DATE(ts0) <= '2024-01-23';

was rewritten (by MDEV-8320) to:

(2) SELECT ts0 FROM t1
     WHERE ts0 <= '2024-01-23 23:59.59.999999';
     -- DATETIME comparison, Item_datetime on the right side

which was further optimized (by MDEV-32148) to:

(3) SELECT ts0 FROM t1
     WHERE ts0 <= TIMESTAMP/* WITH LOCAL TIME ZONE*/ '2024-01-23 23:59.59.999999';
     -- TIMESTAMP comparison, Item_timestamp_literal on the right side

The origin of the problem was in (2) - in the MDEV-8320 related code.
The recent new code for MDEV-32148 revealed this problem.

Item_datetime on step (2) was always created in an inconsistent way:
- with Item::decimals==0
- with ltime.second_part==999999,
  without taking into account the precision of the left side
  (e.g. ts0 in the above example)

On step (3), Item_timestamp_literal was created in an inconsistent way too,
because it copied the inconsistent data from step (2):
- with Item::decimals==0 (copied from Item_datetime::decimals)
- with m_value.tv_usec==999999 (copied from ltime.second_part of Item_datetime)

Later, the Item_timestamp_literal performed save_in_field()
and crashed in my_timestamp_to_binary() on a DBUG_ASSERT checking
consistency between the fractional precision and the fractional seconds value.

Fix:

On step (2) create Item_datetime with truncating maximum possible
second_part value of 999999 according to the the left side fractional
second precision. So for example it sets second_part as follows:
- 000000 for TIMESTAMP(0)
- 999000 for TIMESTAMP(3)
- 999999 for TIMESTAMP(6)

This automatically makes the code create a consistent Item_timestamp_literal
on step (3).

This also makes TIMESTAMP comparison work faster, because now
Item_timestamp_literal is created with Item::decimals value equal
to the Item_field (which is on the other side of the comparison),
so the low level function Type_handler_timestamp_common::cmp_native()
goes the fastest execution path optimized for the case when both sides
have equal fractional precision.

Adding a helper class TimeOfDay to reuse the code when populating:
- the last datetime point for YEAR()
- the last datetime point for DATE()
with a given fractional precision.

This class also helped to unify the equal code in create_start_bound()
and create_end_bound() into a single method create_bound().

Author: Rucha Deodhar
Author Date: 2024-01-25 19:53:18 UTC

temp

Author: Brandon Nesterenko
Author Date: 2024-01-25 18:13:30 UTC

MDEV-29369: Switch to stop_slave_io

Instead of waiting for an inevitable IO thread error,
pre-empt the error and stop the IO thread immediately

Author: VladislavVaintroub
Author Date: 2024-01-24 19:39:52 UTC

MDEV-32537 due to Linux, restrict thread name 15 characters, also in PS.

Rename some threads to workaround this restrictions,
e.g "rpl_parallel_thread"->"rpl_parallel",
"slave_background" -> "slave_bg" etc.

Author: Sergei Golubchik
Author Date: 2023-12-22 08:51:17 UTC

fix sporadic failures of main.lock_sync

wait for all connections to disconnect before the cleanup

Author: Yuchen Pei
Author Date: 2024-01-24 07:53:48 UTC

spider: disable spider/bugfix.mdev_28739_simple

Disable it until MDEV-32997 is fixed, and we already have
spider/bugfix.mdev_28739.

Author: Julius Goryavsky
Author Date: 2024-01-17 15:20:45 UTC

MDEV-30456: compilation fixes

Author: =?utf-8?q?Jan_Lindstr=C3=B6m?=
Author Date: 2024-01-05 09:33:53 UTC

MDEV-33138 : Galera test case MW-336 unstable

Add more inserts before wsrep_slave_threads is set to 1 and
add wait_condition to wait all of them are replicated before
wait_condition about number of wsrep_slave_threads.

Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Author: Yuchen Pei
Author Date: 2024-01-24 01:18:18 UTC

MDEV-28861 Deprecate spider table options by comment/connection

Also deprecating table params not implemented in MDEV-28856.

Author: Vicențiu Ciorbaru
Author Date: 2024-01-23 16:38:22 UTC

Improve READLINE_V5 detection

More in depth check to cover all used readline functions.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2024-01-12 16:34:19 UTC

MDEV-29587 Allowing insert into a view with columns that are not part the table

We can check only fields which take part in inserts.

Author: Brandon Nesterenko
Author Date: 2023-07-10 15:53:19 UTC

MDEV-7850: Extend GTID Binlog Events with Thread Id

This patch augments Gtid_log_event with the user thread-id.
In particular that compensates for the loss of this info in
Rows_log_events.

Gtid_log_event::thread_id gets visible in mysqlbinlog output like

  #231025 16:21:45 server id 1 end_log_pos 537 CRC32 0x1cf1d963 GTID 0-1-2 ddl thread_id=10

as 64 bit unsigned integer.

While the size of Gtid event has grown by 8-9 bytes
replication from OLD <-> NEW is not affected by it.

This work was started by the late Sujatha Sivakumar.
Brandon Nesterenko took it over, reviewed initial patches and extended
the work.

Reviewed-by: <andrei.elkin@mariadb.com>

Author: Alexander Barkov
Author Date: 2024-01-22 12:53:54 UTC

MDEV-27666 User variable not parsed as geometry variable in geometry function

Adding a test that a 10.4 slave understands a 10.5.24 binary log with a POINT
user varible.

This test proves that:

- The 10.5->10.4 replication does not break

- The extra data type name chunk, indicated by the new 10.5
  User_var_log_event chunk code CHUNK_DATA_TYPE_NAME==2,
  is just ignored by 10.4, so a POINT user variable is seen
  as a LONGBLOB user variable. Hence, a 10.4 slave creates a LONGBLOB
  column instead of a POINT column. This is how a pre-MDEV-27666 version
  worked.
  After merging of this patch to 10.5, the SHOW CREATE TABLE statement
  in the test output is expected to report a POINT column rather than
  a LONGBLOB column.

Author: Yuchen Pei
Author Date: 2024-01-22 04:54:25 UTC

MDEV-32999 Spider: only warn of connection string deprecation when creating a table

Author: Yuchen Pei
Author Date: 2024-01-22 01:54:35 UTC

MDEV-29363 Do not change an item to another with subquery when propagating equal fields

Author: Monty
Author Date: 2024-01-21 17:10:37 UTC

Fixed some mtr tests that failed on windows

Most things where wrong in the test suite.
The one thing that was a bug was that table_map_id was in some places
defined as ulong and in other places as ulonglong. On Linux 64 bit this
is not a problem as ulong == ulonglong, but on windows this caused failures.
Fixed by ensuring that all instances of table_map_id are ulonglong.