Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-02-26 13:32:26 UTC

MDEV-24997 Assertion mtr->is_named_space(page_id.space()) in ibuf0ibuf.cc:624

- This is caused by commit deadec4e689c9435e20ebb89fd8f84d3f0f90ff3
(MDEV-24569). InnoDB fails to set the tablespace associated with
mini-transacton while resetting the change buffer bitmap bits of
the page.

Author: Marko Mäkelä
Author Date: 2021-02-26 13:02:13 UTC

Merge 10.5 into 10.6

Author: Sachin Setiya
Author Date: 2021-02-21 09:30:20 UTC

MDEV-7409 On RBR, extend the PROCESSLIST info to include at least the name of the recently used table

When RBR is used, add the db name to db Field and table name to Status
Field of the "SHOW FULL PROCESSLIST" command for SQL thread.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-02-26 11:07:24 UTC

MDEV-14180 Automatically disable key rotation checks for file_key_managment plugin

- InnoDB iterates the fil_system space list to encrypt the
tablespace in case of key rotation.But it is not necessary
for any encryption plugin which doesn't do any key rotation.
InnoDB should avoid key rotation for file key management
encryption plugin.

Introduce new encryption service called encryption_set_no_rotation()
and encryption_get_rotation(). Encryption plugin should
call encryption_set_no_rotation() in case it doesn't
support key rotation. Introduce a new variable called
srv_encrypt_rotate and it should be assigned via
encryption_get_rotation(). If it doesn't do key rotation
then InnoDB should add the tablespace to rotation list.

Author: Eugene
Author Date: 2021-02-26 10:59:41 UTC

review fixes

Author: varun
Author Date: 2021-02-26 05:01:24 UTC

Updating test results

Author: varun
Author Date: 2021-02-26 05:01:24 UTC

Updating test results

Author: Alexey Botchkov
Author Date: 2021-02-26 09:26:00 UTC

MDEV-24965 With ALTER USER ...IDENTIFIED BY command, password doesn't replaced by asterisks in audit log.

Check for the ALTER USER command added.

Author: midenok
Author Date: 2021-02-25 13:21:16 UTC

MDEV-16546 System versioning setting to allow history modification

1. force_fields_visible session variable makes system-invisible and
user-invisible fields visible on next table open. FLUSH TABLES
required for already opened tables.

2. If secure_timestamp allows to modify timestamp variable then
following DML commands: INSERT, INSERT..SELECT and LOAD DATA can
specify row_start and row_end system field values.

3. Cleaned up select_insert::send_data() from setting vers_write as
this parameter is now set on TABLE initialization.

Author: Igor Babaev
Author Date: 2021-02-26 07:11:03 UTC

MDEV-22786 Crashes with nested table value constructors

The bug caused crashes of the server when processing queries with nested
table value constructors (TVC) . It happened because the grammar rules to
parse TVC used the same global lists for both nested TVC and nesting TVC.
This patch provides its own lists structures for each TVC nest level.

Author: Marko Mäkelä
Author Date: 2021-02-25 16:10:31 UTC

fixup 522fd37c3e0a8d38d9796dceb349393f0e4b9e44

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-02-25 15:21:30 UTC

MDEV-24863 AHI entries mismatch with the index while reloading the evicted tables.

In btr_search_drop_page_hash_index(), InnoDB should take
the exclusive lock on the AHI latch if index is already
freed to avoid the freed memory access during
buf_pool_resize()

Author: varun
Author Date: 2021-02-25 11:17:21 UTC

review addressal

Author: Sergei Golubchik
Author Date: 2021-02-25 13:10:25 UTC

mtr --gdb: fix for --rr and for a warning

use _RR_TRACE_DIR=dir instead of -o dir, as the former can store
multiple traces in dir (if, e.g., the test restarts mysqld)

suppress uninitialized warning when $exe is undefined (--manual-XXX)

Author: Marko Mäkelä
Author Date: 2021-02-25 10:31:26 UTC

Merge 10.3 into 10.4

Author: Monty
Author Date: 2021-02-25 09:55:08 UTC

MENT-1098 Crash during update on 10.4.17 after upgrade from 10.4.10

The reason for the crash was that there was not a write lock to
protect against file rotations in the server_audit plugin after an
audit plugin patch to changed audit mutexes to read & write locks.

The fixes are:
* Moving server_audit.c to use read & write locks (which improves
  performance).
* Added functionality in file_logger.c to not do file rotations until
  it is allowed by the caller (done without any interface changes for
  the logging service).
* Move checking of file size limit to server_audit.c and if it is time to
  do a rotation change the read lock to a write lock and tell file_logger
  that it is now allowed to rotate the log files.

Author: Dmitry Shulga
Author Date: 2021-02-25 07:20:11 UTC

MDEV-24860: Incorrect behaviour of SET STATEMENT in case it is executed as a prepared statement

Running statements with SET STATEMENT FOR clause is handled incorrectly in
case the whole statement is executed in prepared statement mode.
For example, running of the following statement
  SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR CREATE TABLE t1 AS SELECT CONCAT('abc') AS c1;
results in different definition of the table t1 depending on whether
the statement is executed as a prepared or as a regular statement.

In first case the column c1 is defined as
  `c1` varchar(3) DEFAULT NULL
in the last case the column c1 is defined as
  `c1` varchar(3) NOT NULL

Different definition for the column c1 arise due to the fact that
a value of the data memeber Item_func_concat::maybe_null depends on
whether strict mode is on or off. Below is definition of the method
fix_fields() of the class Item_str_func that is base class for the
class Item_func_concat that is created on parsing the
SET STATEMENT FOR clause.

bool Item_str_func::fix_fields(THD *thd, Item **ref)
{
  bool res= Item_func::fix_fields(thd, ref);
  /*
    In Item_str_func::check_well_formed_result() we may set null_value
    flag on the same condition as in test() below.
  */
  maybe_null= maybe_null || thd->is_strict_mode();
  return res;
}

Although the clause SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR
is parsed on PREPARE phase during processing of the prepared statement,
real setting of the sql_mode system variable is done on EXECUTION phase.
On the other hand, the method Item_str_func::fix_fields is called on PREPARE
phase. In result, thd->is_strict_mode() returns true during calling the method
Item_str_func::fix_fields(), the data member maybe_null is assigned the value
true and column c1 is defined as DEFAULT NULL.

To fix the issue the system variables listed in the SET STATEMENT FOR clause
are set at the beginning of handling the PREPARE phase just right before
calling the function check_prepared_statement() and their original values
restored immediate after return from this function.

Additionally, to avoid code duplication the source code used in the function
mysql_execute_command for setting variables, specified by SET STATEMENT
clause, were extracted to the standalone functions
run_set_statement_if_requested(). This new function is called from
the function mysql_execute_command() and the method
Prepared_statement::prepare().

Author: Daniel Black
Author Date: 2021-02-17 05:12:06 UTC

MDEV-23510: lf_dynarray_lvalue to return aligned address

Aligned to sizeof(void *).

This ensures that arm64 use of lf_hash remains stable.

Remove abuse of volatile in maria/lockman because it wasn't
necessary or advisable.

Author: varun
Author Date: 2021-02-25 02:34:35 UTC

Attempt to have sort-nest as a bush

Author: varun
Author Date: 2021-02-25 02:34:35 UTC

Attempt to have sort-nest as a bush

Author: Vladislav Vaintroub
Author Date: 2021-02-24 21:55:18 UTC

Replace slave with manager

Author: Anel Husakovic
Author Date: 2021-02-24 11:48:10 UTC

MDEV-6899:Database Dump has Syntax Error When Importing

- Adding the test case for MDEV-24200: Syntax error in mysqldump

Author: Jan Lindström
Author Date: 2020-11-03 08:44:26 UTC

MENT-411 : Implement wsrep_replicate_aria

Introduced two new wsrep_mode options
* REPLICATE_MYISAM
* REPLICATE_ARIA

Depracated wsrep_replicate_myisam parameter and we use
wsrep_mode = REPLICATE_MYISAM instead.

This required small refactoring of wsrep_check_mode_after_open_table
so that both MyISAM and Aria are handled on required DML cases.
Similarly, added Aria to wsrep_should_replicate_ddl to handle DDL
for Aria tables using TOI. Added test cases and improved MyISAM testing.
Changed use of wsrep_replicate_myisam to wsrep_mode = REPLICATE_MYISAM

Author: Jan Lindström
Author Date: 2021-02-24 10:02:54 UTC

MDEV-24964 : Heap-buffer-overflow on wsrep_schema.cc ::remove_fragments

Problem was that we used heap allocated key using too small
array. Fixed by using dynamic memory allocation using actual
needed size.

Author: Sergey Petrunia
Author Date: 2021-02-23 20:38:57 UTC

MDEV-24953: 10.5.9 crashes with large IN() list

The problem was in and_all_keys(), the code of MDEV-9759 which calculates
the new tree weight:

First, it didn't take into account the case when

(next->next_key_part=tmp) == NULL

and dereferenced a NULL pointer when getting tmp->weight.

Second, "if (param->alloced_sel_args > SEL_ARG::MAX_SEL_ARGS) break"
could leave the loop with incorrect value of weight.

Fixed by introducing SEL_ARG::update_weight_locally() and calling it
at the end of the function. This allows to avoid caring about all the
above cases.

Author: Alexey Botchkov
Author Date: 2021-02-23 19:07:42 UTC

MDEV-17399 JSON_TABLE.

Fis for table elimination.

Author: Sergei Golubchik
Author Date: 2021-02-23 12:01:27 UTC

Merge branch '10.4' into 10.5

Author: Sergei Golubchik
Author Date: 2021-02-23 12:01:27 UTC

Merge branch '10.4' into 10.5

Author: Sergei Golubchik
Author Date: 2021-02-22 23:56:14 UTC

Merge branch '10.3' into 10.4

Author: Vlad Lesin
Author Date: 2021-02-22 17:10:47 UTC

MDEV-24940: dict_hdr_get_new_id() changes table id in dictionary header, but does not write log record for it

Take a look dict_hdr_get_new_id(). If a table is temporary, log mode is
set to MTR_LOG_NO_REDO. But then table id is increased and written back
to space 0. But it's not logged due to log mode.

The proper fix is:
MDEV-17794 Do not assign persistent ID for temporary tables

But we don't want to backport a half of 10.[345] commits in order to fix
all bugs properly for this certain custom build. So this commit is just a
simple workaround for the bug fix.

Author: Larysa Sherepa
Author Date: 2021-02-22 14:44:40 UTC

reenable partition_open_files_limit.test, as MDEV-18360 is fixed

Author: Marko Mäkelä
Author Date: 2021-01-27 14:43:29 UTC

MDEV-24709 Assertion !recv_no_ibuf_operations failed in ibuf_page_low()

recv_recovery_from_checkpoint_start(): Clear the recv_no_ibuf_operations
flag at the same time when we enabled writes to the log.

The failure to clear the flag might have caused some missed
change buffer merges, at least to the secondary index of SYS_TABLES
that were accessed by trx_resurrect_table_locks() while the last
recovery batch was in progress.

Thanks to Thirunarayanan Balathandayuthapani for suggesting this fix.

Author: Jan Lindström
Author Date: 2021-02-10 07:53:37 UTC

MDEV-24830 : Write a warning to error log if Galera replicates InnoDB table with no primary key

Two new features for Galera
* Write a warning to error log if Galera replicates table with storage engine not supported by Galera (at the moment only InnoDB is supported
** Warning is pushed to client also
** MyISAM is allowed if wsrep_replicate_myisam=ON
* Write a warning to error log if Galera replicates table with no primary key
** Warning is pushed to client also
** MyISAM is allowed if wsrep_relicate_myisam=ON
* In both cases apply flood control if > 10 same warning is writen to error log
(requires log_warnings > 1), flood control will suppress warnings for 300 seconds

Author: Monty
Author Date: 2021-02-20 12:46:19 UTC

MDEV-24929 Server crash in thr_multi_unlock or in get_schema_tables_result

This was caused by two different bugs:
1) Information_schema tables where not locked by lock_tables, but
   get_lock_data() was not filtering these out. This caused a crash when
   mysql_unlock_some_tables() tried to unlock tables early, including
   not locked information schema tables.

Fixed by not locking SYSTEM_TMP_TABLES

2) In some cases the optimizer will notice that we do not need to read
   the information_schema tables at all. In this case
   join_tab->read_record is not set, which caused a crash in
   get_schema_tables_result()

Fixed by ignoring const tables in get_schema_tables_result()

Author: Daniel Black
Author Date: 2021-02-17 05:15:19 UTC

innodb/extrabackup: use global my_{[cm]alloc,free}_aligned

Add error handling to most of the mariabackup/innodb
paths to handle failure to allocate memory.

Author: Nikita Malyavin
Author Date: 2020-01-21 09:48:47 UTC

Period: add versioning combination

Author: Nikita Malyavin
Author Date: 2020-01-21 09:48:47 UTC

Period: add versioning combination

Author: Robert Bindar
Author Date: 2020-12-15 15:39:24 UTC

MDEV-24114 SHOW CREATE USER doesnt display correct password expiry status

Given PASSWORD EXPIRE and PASSWORD EXPIRE [NEVER|INTERVAL x DAY] are
two different mechanisms, SHOW CREATE USER should display all the
information required to restore the state of an account which
includes both a manual expired state and an automatic policy.

The solution proposed here keeps a CREATE USER ... PASSWORD EXPIRE
statement and adds an aditional
ALTER USER .. PASSWORD EXPIRE [NEVER|INTERVAL x DAY] when necessary

This way a tool can restore almost the complete state of an account
as it was before a dump. The only information left still is the
value of the password_last_changed column from mysql.global_priv

Author: midenok
Author Date: 2021-02-19 15:41:03 UTC

MDEV-20865 foreign keys and partitioning

Author: Sergei Golubchik
Author Date: 2021-02-18 22:17:50 UTC

Merge branch 'bb-10.3-release' into bb-10.4-release

Author: Sergei Golubchik
Author Date: 2021-02-18 21:09:53 UTC

Merge branch 'bb-10.2-release' into bb-10.3-release

Author: Sergei Golubchik
Author Date: 2021-02-18 13:20:48 UTC

make @@wsrep_provider and @@wsrep_notify_cmd read-only

this should simplify run-time cluster management

Author: Marko Mäkelä
Author Date: 2021-02-18 16:20:50 UTC

MDEV-24917 Page cleaner wrongly remains idle

commit a9933105938d4d809ea33ac3196e681cd581374f (MDEV-24537)
introduced the regression that the page cleaner will keep sleeping
even if there is work to do.

innodb_max_dirty_pages_pct_update(): Always wake up the page cleaner
on any SET GLOBAL innodb_max_dirty_pages_pct= assignment.

buf_flush_page_cleaner(): If innodb_max_dirty_pages_pct is nonzero,
consult only that parameter when determining whether there is work
to do. Else, consult innodb_max_dirty_pages.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2020-11-25 16:22:45 UTC

MDEV-24312 Server audit fix of master_host has 60 character limit, increase to 255 bytes

Based on Alexey Botchkov <holyfoot@askmonty.org>

Author: Vicențiu Ciorbaru
Author Date: 2021-02-18 13:30:03 UTC

Remove race condition during `make dist`

Introduced by 85828b8f22e7f4dfa6a5d4b0a1ab9e133e7feea7

This is running 2 git processes in parallel, which, if unlucky can cause
either of them to fail with "File already exists" error.

Author: Jan Lindström
Author Date: 2021-02-18 08:09:30 UTC

Suppress warning on galera_ssl_upgrade test.

Author: Marko Mäkelä
Author Date: 2021-02-17 17:39:05 UTC

Merge 10.5 into 10.6

Author: Vlad Lesin
Author Date: 2021-02-09 08:07:52 UTC

MDEV-22110 InnoDB unnecessarily writes unmodified pages

This is backport from 10.5. Strictly speaking this is not a backport because
in 10.5 all the code which writes to mtr was moved to mtr_t class,
and the corresponding mtr_t functions have block_t parameter,
and that is why it's quite easy to mark the block as modified for mtr
in 10.5, it's enough just to call the correcponding function from the
corredponding mtr_t functions, which modify the block.

In 10.2 the code which writes in mtr_t is spread among the code base,
and that is why we can't backport MDEV-22110 as is. The common idea is
to find all mlog_write_initial_log_record_fast() and
mlog_write_initial_log_record_low() calls, and mark blocks as modified
there. We can't do this inside of mlog_write_initial_log_record_fast()
or mlog_write_initial_log_record_low() because when log mode is
MTR_LOG_NONE or MTR_LOG_NO_REDO, the functions are not invoked(see also
mlog_open() and it's invocation).

The downside of such approach is performace impact due to frequent
mtr_t::memo_modify_page() calls from mlog_write_*() and mlog_log_string()
functions.

Author: Robert Bindar
Author Date: 2021-02-17 14:30:50 UTC

MDEV-24887 Tests fail on macos because mysqltest can't use nonblock API

Author: Monty
Author Date: 2021-01-30 12:03:23 UTC

Remove some usage of Check_level_instant_set and Sql_mode_save

The reason for the removal are:
- Generates more code
  - Storing and retreving THD
  - Causes extra code and daata to be generated to handle possible throw
    exceptions (which never happens in MariaDB code)
- Uses more stack space

Other things:
- Changed convert_const_to_int() to use item->save_in_field_no_warnings(),
  which made the code shorter and simpler.
- Removed not needed code in Sp_handler::sp_create_routine()
- Added thd as argument to store_key.copy() to make function simpler
- Added thd as argument to some subselect* constructor that inherites
  from Item_subselect.

Author: Julius Goryavsky
Author Date: 2021-02-17 03:45:53 UTC

MDEV-21039: Server fails to start with unknown mysqld_safe options

Adding any unknown option to the "[mysqld_safe]" section makes
mysqld impossible to start with mysqld_multi. For example, after
adding the unknown option "numa_interleave" to the "[mysqld_safe]"
section, mysqld_multi exits with the following diagnostics:

[ERROR] /usr/local/mysql/bin/mysqld: unknown option '--numa_interleave'

To get rid of this behavior, this patch adds the passing of the default
group suffix from mysqld_multi to the mysqld_safe side.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-02-16 13:56:59 UTC

Merge branch 'bb-10.5-release' into bb-10.6-release

Author: Jan Lindström
Author Date: 2021-02-16 06:46:14 UTC

MDEV-24865 : Server crashes when truncate mysql user table

For truncate we try to find out possible foreign key tables
using open_tables. However, table_list was not cleaned up
properly and there was no error handling. Fixed by cleaning
table_list and adding proper error handling.

Author: Dmitry Shulga
Author Date: 2021-02-16 05:20:34 UTC

MDEV-16708: Unsupported commands for prepared statements

- Updated comments for some functions
- Relaxed assert condition in the function lex_end_stage1() by adding
  the command SQLCOM_ALTER_EVENT to a list of supported command
- Removed raising of the error ER_UNSUPPORTED_PS in the function
  check_prepared_statement() for the ALTER VIEW command

Author: Dmitry Shulga
Author Date: 2021-02-15 13:14:11 UTC

MDEV-24860: Incorrect behaviour of SET STATEMENT in case it is executed as a prepared statement

Running statements with SET STATEMENT FOR clause is handled incorrectly in
case the whole statement is executed in prepared statement mode.
For example, execution the following statement
  SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR CREATE TABLE t1 AS SELECT CONCAT('abc') AS c1;
results in different definition of the table t1 depending on whether
the statement is executed as a prepared or as a regular statement.

In first case the column c1 is defined as
  `c1` varchar(3) DEFAULT NULL
in the last case the column c1 is defined as
  `c1` varchar(3) NOT NULL

Different definition for the column c1 arise due to the fact that
a value of the data memeber Item_func_concat::maybe_null depends on
whether strict mode is on or off. Below is definition of the method
fix_fields() of the class Item_str_func that is base class for the
class Item_func_concat that is created on parsing the
SET STATEMENT FOR clause.

bool Item_str_func::fix_fields(THD *thd, Item **ref)
{
  bool res= Item_func::fix_fields(thd, ref);
  /*
    In Item_str_func::check_well_formed_result() we may set null_value
    flag on the same condition as in test() below.
  */
  maybe_null= maybe_null || thd->is_strict_mode();
  return res;
}

Although the clause SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR
is parsed on PREPARE phase during processing of the prepared statement,
real setting of the sql_mode system variable is done on EXECUTION phase.
On the other hand, the method Item_str_func::fix_fields is called on PREPARE
phase. In result, thd->is_strict_mode() returns true during calling the method
Item_str_func::fix_fields(), the data member maybe_null is assigned the value
true and column c1 is defined as DEFAULT NULL.

To fix the issue the system variables listed in the SET STATEMENT FOR clause
are set at the beginning of handling the PREPARE phase just right before
calling the function check_prepared_statement() and their original values
restored immediate after return from this function.

Additionally, to avoid code duplication the source code used in the function
mysql_execute_command for setting variables, specified by SET STATEMENT
clause, were extracted to the standalone functions
run_set_statement_if_requested(). This new function is called from
the function mysql_execute_command() and the method
Prepared_statement::prepare().

Author: Dmitry Shulga
Author Date: 2021-02-15 13:04:12 UTC

MDEV-24860: Incorrect behaviour of SET STATEMENT in case it is executed as a prepared statement

Running statements with SET STATEMENT FOR clause is handled incorrectly in
case the whole statement is executed in prepared statement mode.
For example, execution the following statement
  SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR CREATE TABLE t1 AS SELECT CONCAT('abc') AS c1;
results in different definition of the table t1 depending on whether
the statement is executed as a prepared or as a regular statement.

In first case the column c1 is defined as
  `c1` varchar(3) DEFAULT NULL
in the last case the column c1 is defined as
  `c1` varchar(3) NOT NULL

Different definition for the column c1 arise due to the fact that
a value of the data memeber Item_func_concat::maybe_null depends on
whether strict mode is on or off. Below is definition of the method
fix_fields() of the class Item_str_func that is base class for the
class Item_func_concat that is created on parsing the
SET STATEMENT FOR clause.

bool Item_str_func::fix_fields(THD *thd, Item **ref)
{
  bool res= Item_func::fix_fields(thd, ref);
  /*
    In Item_str_func::check_well_formed_result() we may set null_value
    flag on the same condition as in test() below.
  */
  maybe_null= maybe_null || thd->is_strict_mode();
  return res;
}

Although the clause SET STATEMENT sql_mode = 'NO_ENGINE_SUBSTITUTION' FOR
is parsed on PREPARE phase during processing of the prepared statement,
real setting of the sql_mode system variable is done on EXECUTION phase.
On the other hand, the method Item_str_func::fix_fields is called on PREPARE
phase. In result, thd->is_strict_mode() returns true during calling the method
Item_str_func::fix_fields(), the data member maybe_null is assigned the value
true and column c1 is defined as DEFAULT NULL.

To fix the issue the system variables listed in the SET STATEMENT FOR clause
are set at the beginning of handling the PREPARE phase just right before
calling the function check_prepared_statement() and their original values
restored immediate after return from this function.

Additionally, to avoid code duplication the source code used in the function
mysql_execute_command for setting variables, specified by SET STATEMENT
clause, were extracted to the standalone functions
run_set_statement_if_requested(). This new function is called from
the function mysql_execute_command() and the method
Prepared_statement::prepare().

Author: varun
Author Date: 2021-02-15 11:00:55 UTC

MDEV-23291: SUM column from a derived table returns invalid values

The issue here was the read_set bitmap was not set for a field which
was used as a reference in an inner select.
We need to make sure that if we are in an inner select and we have
references from outer select then we update the table bitmaps for
such references.

Introduced a function in the class Item_subselect that would
update bitmaps of table for the references within a
subquery that are defined in outer selects.

Author: Daniel Black
Author Date: 2021-02-15 01:31:31 UTC

MDEV-23510: arm64 lf_hash

volatile has no memory barrier schemantics, its for mmaped IO
so lets allow some optimizer gains and stop pretending it helps
with memory atomicity.

The MDEV lists a SEGV an assumption is made that an address was
partially read. As C packs structs strictly and on arm64 the
cache line size is 128, move the 32bit hashnr to the end so
we don't get the *key pointer split across two cache lines.

Author: Marko Mäkelä
Author Date: 2021-02-15 07:37:01 UTC

MDEV-24864 Fatal error in buf_page_get_low() / fseg_page_is_free()

The fix of MDEV-24569 and MDEV-24695 introduced a race condition when
a table is being rebuilt or dropped during the fseg_page_is_free() check.
The server would occasionally crash during the execution of the test
encryption.create_or_replace.

The fil_space_t::STOPPING flag can be set by DDL operations. Normally,
such concurrent operations are prevented by a metadata lock (MDL).
However, neither the change buffer merge nor the fil_crypt_thread() are
protected by MDL.

fil_crypt_read_crypt_data(), xdes_get_descriptor_const(): Pass the
BUF_GET_POSSIBLY_FREED flag to avoid the fatal error in buf_page_get_low()
if a DDL operation was just initiated.

Author: midenok
Author Date: 2021-02-14 20:26:12 UTC

MDEV-22562 Assertion `next_insert_id == 0' upon UPDATE on system-versioned table

Don't update autoinc counter on history row insert. Uniqueness is kept
due to merge with row_end.

Author: Vladislav Vaintroub
Author Date: 2021-02-14 17:30:39 UTC

MDEV-24341 Innodb - do not block in foreground thread in log_write_up_to(

Author: Andrei Elkin
Author Date: 2021-02-12 20:47:15 UTC

Followup to binlog_optimize_thread_scheduling := OFF that

now also simulates no explicit engine prepare().

Author: Marko Mäkelä
Author Date: 2021-02-12 15:43:10 UTC

MDEV-24643: Assertion failed in rw_lock::update_unlock()

mtr_defer_drop_ahi(): Upgrade the U lock to X lock and downgrade
it back to U lock in case the adaptive hash index needs to be dropped.

This regression was introduced in
commit 03ca6495df31313c96e38834b9a235245e2ae2a8 (MDEV-24142).

Author: Julius Goryavsky
Author Date: 2021-02-12 14:58:26 UTC

MDEV-24853: Duplicate key generated during cluster configuration change

Incorrect processing of an auto-incrementing field in the
WSREP-related code during applying transactions results in
a duplicate key being created. This is due to the fact that
at the beginning of the write_row() and update_row() functions,
the values of the auto-increment parameters are used, which
are read from the parameters of the current thread, but further
along the code other values are used, which are read from global
variables (when applying a transaction). This can happen when
the cluster configuration has changed while applying a transaction
(for example in the high_priority_service mode for Galera 4).
Further during IST processing duplicating key is detected, and
processing of the DB_DUPLICATE_KEY return code (inside innodb,
in the write_row() handler) results in a call to the
wsrep_thd_self_abort() function.

Author: Julius Goryavsky
Author Date: 2021-02-12 14:58:26 UTC

MDEV-24853: Duplicate key generated during cluster configuration change

Incorrect processing of an auto-incrementing field in the
WSREP-related code during applying transactions results in
a duplicate key being created. This is due to the fact that
at the beginning of the write_row() and update_row() functions,
the values of the auto-increment parameters are used, which
are read from the parameters of the current thread, but further
along the code other values are used, which are read from global
variables (when applying a transaction). This can happen when
the cluster configuration has changed while applying a transaction
(for example in the high_priority_service mode for Galera 4).
Further during IST processing duplicating key is detected, and
processing of the DB_DUPLICATE_KEY return code (inside innodb,
in the write_row() handler) results in a call to the
wsrep_thd_self_abort() function.

Author: Rucha Deodhar
Author Date: 2021-02-08 09:22:04 UTC

MDEV-22010: use executables MariaDB named in scripts

As a part of this MDEV following changes were made:
1) Mariadb named executables used instead of mysql named executables in scripts
2) renamed mysql-test-run and mysql-stress-test to mariadb-test-run and
mariadb-stress-test and created a symlink.

Author: Julius Goryavsky
Author Date: 2021-02-10 13:04:25 UTC

MDEV-19950: Galera test failure on galera_ssl_upgrade

The test requires adaptation for MariaDB, which is done
in this patch. In addition, this patch includes a fix for
the SST script startup code that adds escaping for special
characters on the command line (in case they are contained
in the arguments to mysqld). The fix does not require
separate tests, as the required tests are already part
of the mtr suite for Galera.

Author: Julius Goryavsky
Author Date: 2021-02-10 13:04:25 UTC

MDEV-19950: Galera test failure on galera_ssl_upgrade

The test requires adaptation for MariaDB, which is done
in this patch. In addition, this patch includes a fix for
the SST script startup code that adds escaping for special
characters on the command line (in case they are contained
in the arguments to mysqld). The fix does not require
separate tests, as the required tests are already part
of the mtr suite for Galera.

Author: Alexey Botchkov
Author Date: 2021-02-10 12:25:55 UTC

MDEV-17399 Add support for JSON_TABLE.

The specific table handler for the table functions was introduced,
and used to implement JSON_TABLE.

Author: David Hall
Author Date: 2021-02-09 19:15:22 UTC

update for columnstore-5.5.1-2

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-02-09 10:40:46 UTC

MDEV-14180 Automatically disable key rotation checks for file_key_management plugin

- Addressed marko's review comments

Author: Daniel Black
Author Date: 2021-02-08 05:42:18 UTC

MDEV-24344: BINLOG REPLAY privilege is missing from SHOW PRIVILEGES

Was added in 10.5.2 (MDEV-21975)

Author: Vlad Lesin
Author Date: 2021-02-09 08:07:52 UTC

MDEV-22110 InnoDB unnecessarily writes unmodified pages

This is just a draft which purpose is to have some worked version, so
don't pay attention to some mess in the code.

The following tests are still failed:

 encryption.innodb_encrypt_temporary_tables
 rpl.rpl_mixed_drop_create_temp_table innodb.innodb
 binlog.binlog_innodb_row innodb.blob-crash-failed
 main.ctype_utf8mb4_innodb innodb.u
 pdate_time innodb.innodb_mysql rpl.rpl_non_direct_row_mixing_engines
 innodb.encryption_threads_shutdown
 rpl.rpl_non_direct_stm_mixing_engines rpl.rpl_mixed_mixing_engines
 rpl.rpl_parallel_optimis
 tic innodb.innodb_defrag_concurrent main.partition_innodb
 rpl.rpl_stm_drop_create_temp_table perfschema.table_name
 innodb.innodb-fk-warnings handler.innodb
 mariabackup.ddl_incremental_encrypted m
 ain.flush_read_lock main.read_only_innodb innodb.innodb_bug54044
 rpl.rpl_row_mixing_engines innodb_zip.wl6501_scale_1
 rpl.rpl_non_direct_mixed_mixing_engines
 rpl.rpl_row_drop_create_temp_table rp
 l.rpl_stm_mixing_engines main.alter_table_online
 main.partition_explicit_prune innodb.temp_table_savepoint
 mariabackup.mlog_index_load

Author: varun
Author Date: 2021-02-09 06:11:30 UTC

Enabled the order by limit optimization by default

Author: varun
Author Date: 2021-02-07 14:00:30 UTC

Enabling order by limit optimization by default

Author: Alexey Botchkov
Author Date: 2021-02-08 20:59:55 UTC

MDEV-15888 Implement FLUSH TABLES tbl_name [, tbl_name] ... WITH READ LOCK for views.

Kind of 'naive' solution that surprisingly worked so trying to
understand why.

Author: Monty
Author Date: 2021-02-08 19:03:04 UTC

MDEV-24087 s3.replication_partition fails in buildbot wiht replication failure

A few of the failures was because of missing sync_slave_to_master in
the test suite.

However, the biggest reason for most faulures was that in case of
ALTER PARTITION the master writes the query to the binary log before
it has updated the .frm and .par files. This causes a problem for an
S3 slave as it will start execute the ALTER PARTITION but get old .frm and
.par files from S3 which causes "open table" to fail, either with an error
or in some case with a crash.
Fixed

Author: Sujatha Sivakumar
Author Date: 2020-04-09 15:15:45 UTC

MDEV-21117: refine the server binlog-based recovery for semisync

Problem:
=======
When the semisync master is crashed and restarted as slave it could
recover transactions that former slaves may never have seen.
A known method existed to clear out all prepared transactions
with --tc-heuristic-recover=rollback does not care to adjust
binlog accordingly.

Fix:
===
The binlog-based recovery is made to concern of the slave semisync role of
post-crash restarted server.
No changes in behaviour is done to the "normal" binloggging server
and the semisync master.

When the restarted server is configured with
  --rpl-semi-sync-slave-enabled=1
the refined recovery attempts to roll back prepared transactions
and truncate binlog accordingly.
In case of a partically committed (that is committed at least
in one of the engine participants) such transaction gets committed.
It's guaranteed no (partially as well) committed transactions
exist beyond the truncate position.
In case there exists a non-transactional replication event
(being in a way a committed transaction) past the
computed truncate position the recovery ends with an error.

To facilite the failover on the slave side
conditions to accept own events (having been discarded by the above recovery)
are relaxed to let so for the semisync slave that connects to master
in gtid mode. gtid_strict_mode is further recommended to secure
from inadvertent re-applying out of order gtids in general.
Non-gtid mode connected semisync slave would require
--replicate-same-server-id (mind --log-slave-updates must be OFF then).

Author: Alexander Barkov
Author Date: 2021-02-05 15:50:05 UTC

MDEV-24790 CAST('0e1111111111' AS DECIMAL(38,0)) returns a wrong result

Author: Sujatha Sivakumar
Author Date: 2021-02-05 09:21:11 UTC

MDEV-19371: Implement binlog_expire_logs_seconds for purging of binary logs

Backporting upstream changes.

commit a7e1ef858ee82493dd8ad9a76bc9c22fe3b8c05b
Author: Neha Kumari <neha.n.kumari@oracle.com>
Date: Thu Mar 2 12:30:08 2017 +0530

WL#9237: Add a new variable binlog_expire_logs_seconds

This worklog adds a system variable binlog_expire_logs_seconds in
addition to the existing expire_logs_days variable.
The ultimate requirement is that the user can set expire periods
smaller than one day by providing another extra variable.
The new variable binlog_expire_logs_seconds, will be set in those cases
where the expire period is not a integer multiple of days
like 1 day 2 hours and 32 minute.

@ sql/binlog.cc

  The purge time now also considers the value of
  binlog_expire_logs_seconds.

@ sql/mysqld.h

  Declared the new variable binlog_expire_logs_seconds

@ sql/mysqld.cc

  The purge time now also considers the value of
  binlog_expire_logs_seconds.

@ sql/sys_vars.cc

  Added the description for binlog_expire_logs_seconds, and
  modified the description for expire_logs_days.

Added test case to test the new variable:

- binlog_expire_logs_seconds: Ensure that when any of the two expire
  variable, expire_logs_days or binlog_expire_logs_seconds is set
  the purge happens on server restart and in cases where a force
  rotation of binlog happens.

Extended the existing test cases:
- expire_logs_days_basic: Added code to test the system variable
  binlog_expire_logs_seconds.

commit 9d591e29af1654b99a670d975ad8b9c95b011747
Author: Neha Kumari <neha.n.kumari@oracle.com>
Date: Fri Feb 2 09:12:08 2018 +0530

Bug#27252658 :CHANGE DEFAULT VALUE OF BINLOG_EXPIRE_LOGS_SECONDS

Problem:
At present the default value of binlog_expire_logs_seconds is 0, so in future
when expire_logs_days is removed the default expiration time will be set to
'0', which will make the logs to be not purged at all, where in expectation
they should have been purged after 30 days.

Fix:
To avoid the above scenario these are the three combinations regarding when
to consider the default value of what.

1) If user provides values for both options, both variables are set
   accordingly and 'binlog_expire_logs_seconds' takes effect.

2) If user provides a value for only one of the options, that particular
   variable is set accordingly, and the other variable defaults to 0.

3) If user does not provide a value for any of the options, then
   'binlog_expire_logs_seconds' defaults to 30 * 24 * 60 * 60 and 'expire_logs_days' to 0.

This patch also changes the default value of expire_logs_days to 0.

BUG#27699608: WARNING FOR CONFLICTING EXPIRE-LOGS-[DAYS|SECONDS] SUPPRESSED WITH SKIP-LOG-BIN

Problem:

 1. When --expire_logs_days and --binlog-expire-logs-seconds are set
    to conflicting values, and the binary log is disabled, the warning
    for the conflicting values was not issued and the values were not
    corrected.

 2. There was no warning when using either of these options when the
    binary log is disabled. This differs from other options that are
    void when the binary log is disabled, such as --binlog-format and
    --log-slave-updates

Fix:

 1. Check the conflicting values of these options even when the binary
    log is disabled.

 2. Generate a warning if any of these options is used when the binary
    log is disabled.

Author: Alexander Barkov
Author Date: 2021-02-05 15:50:05 UTC

MDEV-24790 CAST('0e1111111111' AS DECIMAL(38,0)) returns a wrong result

Author: Andrei Elkin
Author Date: 2021-02-06 17:54:57 UTC

MDEV-21117 incremental commit. Cleanup and unit-testing.

Author: Daniel Black
Author Date: 2021-02-06 20:37:57 UTC

MDEV-24557: mysqldump --system=user to version grants

The grants MariaDB has are in 'SHOW PRIVILEGES'.
This is extended to include a version number.

As mariadb-dump could be used against a MySQL-8.0
server, we assume any grants we don't recognised are
mysql-8.0 and apply that as a version comment.

This facilitiates the migration from MySQL-8.0 servers.

Author: varun
Author Date: 2021-02-05 18:24:28 UTC

Using recurisive descent instead of walk to check if the join cardinality estimates are precise

Author: Sujatha Sivakumar
Author Date: 2021-02-05 11:15:35 UTC

MDEV-24773: slave_compressed_protocol doesn't work properly with semi-sync replication

Back port upstream fix

commit 1800b015a1d487330f7b15f2020b887be348a66b
Author: Venkatesh Duggirala <venkatesh.duggirala@oracle.com>
Date: Fri Sep 8 20:29:22 2017 +0530

Bug#26027024 SLAVE_COMPRESSED_PROTOCOL DOESN'T WORK WITH
SEMI-SYNC REPLICATION IN MYSQL-5.7

Analysis: In mysql-5.6, dump thread (the thread that is created
on Master after Slave requested for a binlog dump) is also used
to receive acknowledgements from the Slave and act on them accordingly.
For performance reasons, a special thread called Ack Receiver thread
is added in mysql-5.7 Semi synchronous replication plugin.
This thread does not have special handling to receive acknowledgements
if Slave has enabled compression in the protocol. Hence Master is
unable to handle any slave if Slave_compressed_protocol is enabled
on it.

Fix: Enable compress flag on the communication channels if the Slave
has Slave_compressed_protocol ON.

Author: Roman Nozdrin
Author Date: 2021-02-04 11:58:40 UTC

MDEV-24298 MDB doesn't use Select Handler with INSERT..SELECT + UNION anymore

Author: Eugene
Author Date: 2021-02-04 11:23:25 UTC

BUF_GET_IF_IN_POOL

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-02-04 08:53:01 UTC

MDEV-24781 Assertion `mode == 16 || mode == 12 || fix_block->page.status != buf_page_t::FREED' failed in buf_page_get_low

This is caused by commit 3cef4f8f0fc88ae5bfae4603d8d600ec84cc70a9
(MDEV-515). dict_table_t::clear() frees all the blob during
rollback of bulk insert.But online log tries to read the
freed blob while applying the log. It can be fixed if we
truncate the online log during rollback of bulk insert operation.

Author: Roman Nozdrin
Author Date: 2021-02-03 16:45:50 UTC

Update MCS ref

Author: Vicențiu Ciorbaru
Author Date: 2021-02-03 16:16:03 UTC

wip2

Author: Sujatha Sivakumar
Author Date: 2021-01-18 08:30:13 UTC

MDEV-22741: *SAN: ERROR: AddressSanitizer: use-after-poison on address in instrings/strmake.c:36 from change_master (on optimized builds)

Problem:
========
CHANGE MASTER TO MASTER_USER='root', MASTER_SSL=0, MASTER_SSL_CA='',
  MASTER_SSL_CERT='', MASTER_SSL_KEY='', MASTER_SSL_CRL='',
  MASTER_SSL_CRLPATH='';

CHANGE MASTER TO MASTER_USER='root', MASTER_PASSWORD='', MASTER_SSL=0;

use-after-poison is reported for lex_mi->ssl_crl

File: sql_repl.cc

if (lex_mi->ssl_crl)
  strmake_buf(mi->ssl_crl, lex_mi->ssl_crl);

Analysis:
========
At the end of CHANGE MASTER statement execution, the LEX_MASTER_INFO
parameters are reset so that the next query will have a clean state. But
'ssl_crl' and 'ssl_crl_path' members of LEX_MASTER_INFO object are not
cleared during 'LEX_MASTER_INFO::reset'. Hence when a new CHANGE MASTER
statement is executed, the stale value of lex_mi->ssl_crl is used, so ASAN
reports use-after-poison.

Fix:
===
Clear 'ssl_crl' and 'ssl_crl_path' as part of 'reset'.

Author: Sujatha Sivakumar
Author Date: 2021-01-18 08:30:13 UTC

MDEV-22741: *SAN: ERROR: AddressSanitizer: use-after-poison on address in instrings/strmake.c:36 from change_master (on optimized builds)

Problem:
========
CHANGE MASTER TO MASTER_USER='root', MASTER_SSL=0, MASTER_SSL_CA='',
  MASTER_SSL_CERT='', MASTER_SSL_KEY='', MASTER_SSL_CRL='',
  MASTER_SSL_CRLPATH='';

CHANGE MASTER TO MASTER_USER='root', MASTER_PASSWORD='', MASTER_SSL=0;

use-after-poison is reported for lex_mi->ssl_crl

File: sql_repl.cc

if (lex_mi->ssl_crl)
  strmake_buf(mi->ssl_crl, lex_mi->ssl_crl);

Analysis:
========
At the end of CHANGE MASTER statement execution, the LEX_MASTER_INFO
parameters are reset so that the next query will have a clean state. But
'ssl_crl' and 'ssl_crl_path' members of LEX_MASTER_INFO object are not
cleared during 'LEX_MASTER_INFO::reset'. Hence when a new CHANGE MASTER
statement is executed, the stale value of lex_mi->ssl_crl is used, so ASAN
reports use-after-poison.

Fix:
===
Clear 'ssl_crl' and 'ssl_crl_path' as part of 'reset'.

Author: Sujatha Sivakumar
Author Date: 2021-01-18 08:30:13 UTC

MDEV-22741: *SAN: ERROR: AddressSanitizer: use-after-poison on address in instrings/strmake.c:36 from change_master (on optimized builds)

Problem:
========
CHANGE MASTER TO MASTER_USER='root', MASTER_SSL=0, MASTER_SSL_CA='',
  MASTER_SSL_CERT='', MASTER_SSL_KEY='', MASTER_SSL_CRL='',
  MASTER_SSL_CRLPATH='';

CHANGE MASTER TO MASTER_USER='root', MASTER_PASSWORD='', MASTER_SSL=0;

use-after-poison is reported for lex_mi->ssl_crl

File: sql_repl.cc

if (lex_mi->ssl_crl)
  strmake_buf(mi->ssl_crl, lex_mi->ssl_crl);

Analysis:
========
At the end of CHANGE MASTER statement execution, the LEX_MASTER_INFO
parameters are reset so that the next query will have a clean state. But
'ssl_crl' and 'ssl_crl_path' members of LEX_MASTER_INFO object are not
cleared during 'LEX_MASTER_INFO::reset'. Hence when a new CHANGE MASTER
statement is executed, the stale value of lex_mi->ssl_crl is used, so ASAN
reports use-after-poison.

Fix:
===
Clear 'ssl_crl' and 'ssl_crl_path' as part of 'reset'.

Author: Anel Husakovic
Author Date: 2021-02-01 13:25:51 UTC

MDEV-24135: Valgrind / ASAN warnings do not go to MTR XML

- Note 1: MDEV doesn't depend on Valgrind/*SAN warnings/error, but doesn't show warnings in general (example 1 - JIRA).
- Note 2: XML reports doesn't show message in case of failure, this patch will append this option too (example 2 - JIRA).
- Note 3: xml-report file is saved in basedir,in case of compiling
          out-of-source.
- Note 4: From 10.3+ you can control number of lines of failures with `--tail-lines=N`

Author: Vicențiu Ciorbaru
Author Date: 2021-02-02 18:30:56 UTC

Intel compiler: Fix myisamlog, use correct const chars where appropriate

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-02-02 16:55:53 UTC

Merge branch 'bb-10.3-release' into bb-10.4-release

Author: Nikita Malyavin
Author Date: 2021-02-01 19:18:52 UTC

calculate overlaps

Author: varun
Author Date: 2021-02-01 07:53:40 UTC

Added comments inside table_multi_eq_cond_selectivity

Author: varun
Author Date: 2021-01-27 07:28:20 UTC

MDEV-24740: Selectivity for equi-join predicates not involed in ref access is not taken into account for join cardinality estimation

First the function was checking if there is a keyuse only then we were
trying to use the function to get estimates for the equi-join predicated.
This function can calculate the estimates for selectivity for an non-indexed
equi-join condition also.

Second change is done when we try to get the avg frequency of a column,
then we need to make sure that statistics are available in the statistical
tables. This check was missing and so get_column_avg_frequency() was
returning zero for columns.

Updated few tests results.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-01-29 15:20:05 UTC

MDEV-24730 Insert log operation fails after purge resets n_core_fields

- Adding the test case which has more than 129 instant columns.