Author: Vladislav Vaintroub
Author Date: 2021-06-14 21:45:31 UTC

update libmariadb

Author: Andrei Elkin
Author Date: 2021-06-14 19:34:09 UTC

MDEV-21117 post-push part 2, S3 alter

Converts hton calls to static binlog_commit ones.

Author: Monty
Author Date: 2021-06-14 18:33:42 UTC

Updated libmarias3 to latest version

Author: Sergei Golubchik
Author Date: 2021-06-14 17:53:23 UTC

update C/C to 3.2.2-rc

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-06-14 16:15:44 UTC

MDEV-25872 InnoDB: Assertion failure in row_merge_read_clustered_index upon ALTER on table with indexed virtual columns

- InnoDB fails to check DB_COMPUTE_VALUE_FAILED error in
row_merge_read_clustered_index() and wrongly asserts that
the buffer shouldn't be ran out of memory.

Author: Marko Mäkelä
Author Date: 2021-06-14 15:41:56 UTC

MDEV-25801: buf_flush_dirty_pages() is very slow

In commit 7cffb5f6e8a231a041152447be8980ce35d2c9b8 (MDEV-23399)
the implementation of buf_flush_dirty_pages() was replaced with
a slow one, which would perform excessive scans of the
buf_pool.flush_list and make little progress.

buf_flush_list_space(): Replaces buf_flush_dirty_pages().
This is like buf_flush_list(), but operating on a single
tablespace at a time. Writes at most innodb_io_capacity
pages. Returns whether some of the tablespace might remain
in the buffer pool.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-06-14 15:29:52 UTC

MDEV-24713 Assertion `dict_table_is_comp(index->table)' failed in row_merge_buf_add()

- During online alter conversion from compact to redundant,
virtual column field length already set during
innobase_get_computed_value(). Skip the char(n) check for
virtual column in row_merge_buf_add()

Author: Anel Husakovic
Author Date: 2021-05-25 16:17:29 UTC

MDEV-20787: Script dgcov.pl does not work

Reviewed by: serg@mariadb.com

Author: Rucha Deodhar
Author Date: 2021-02-08 09:22:04 UTC

MDEV-22010: use executables MariaDB named in scripts

As a part of this MDEV following changes were made:
1) Mariadb named executables used instead of mysql named executables in scripts
2) renamed mysql-test-run and mysql-stress-test to mariadb-test-run and
mariadb-stress-test and created a symlink.

Author: Rucha Deodhar
Author Date: 2021-06-01 14:42:56 UTC

MDEV-17943: Unfortunate error code/message upon SET DEFAULT ROLE FOR

Analysis: Wrong error message string
Fix: Change error label and error message string

Author: Rucha Deodhar
Author Date: 2021-06-08 05:55:47 UTC

MDEV-24248: my_print_defaults is not taking all the values when using -e
option which is called from mysql.server (extra_args).

Fix: change mysql.server script to use --defaults-extra-file instead of -e

Author: Sergei Golubchik
Author Date: 2021-06-13 20:04:40 UTC

add force_and_permanent .cnf files for wrappers

to maintain old behavior when the server was dynamically linked
with a library. A library cannot be unloaded at runtime and
a missing library prevented the server from starting.

Author: Monty
Author Date: 2021-06-13 16:46:16 UTC

Fixed compiler warnings

- Usage of strncpy that didn't set an end \0
- Initialized some variables that caused compiler warnings
- Increased buffer that caused warnings from strncpy

One warning from InnoDB, others from Connect.

Author: midenok
Author Date: 2021-06-12 11:29:19 UTC

Removed revert from ddl_log_complete()

Author: Sergei Golubchik
Author Date: 2021-06-12 10:38:45 UTC

fix mysqlest crash on ./mtr --sp innodb_fts.innodb-fts-stopword

Author: Sergei Golubchik
Author Date: 2021-06-12 10:38:45 UTC

fix mysqlest crash on ./mtr --sp innodb_fts.innodb-fts-stopword

Author: Vlad Lesin
Author Date: 2021-06-11 15:57:34 UTC

MDEV-25899 intermediate files operations are not protected by backup
locks

Test case.

Author: Sachin Setiya
Author Date: 2021-06-11 13:26:19 UTC

MDEV-20131 Assertion `!pk->has_virtual ()' failed in instant_alter_column_possible

Problem:- Primary key on long unique columns is not allowed but when
`CREATE TABLE t2 (a TEXT, PRIMARY KEY(a(1871))) ENGINE=InnoDB;`
is executed with innodb_page_size=8k or 4k, primary key on DB_ROW_HASH_1 column
is created. Reason of this is in mysql_prepare_create_table we check against
max_key_part_length() to see whether key_part length is more then engine supported
key_part length , and if it is true error is thrown for primary key and long
unique index is created for unique key. But in this case max_key_part_length()
returns 3072 which is more the max_key_length() , which later in code triggers
creating of long unique index.

Solution:- max_supported_key_part_length() should return according to innodb
page size.

Author: Dmitry Shulga
Author Date: 2021-06-11 11:42:02 UTC

MDEV-16708: Unsupported commands for prepared statements

Within this task the following changes were made:
- Added sending of metadata info in prepare phase for the admin related
  command (check table, checksum table, repair, optimize, analyze).

- Refactored implmentation of HELP command to support its execution in
  PS mode

- Added support for execution of LOAD INTO and XA- related statements
  in PS mode

- Modified mysqltest to enable any statements except PREPARE FROM,
  EXECUTE, EXECUTE IMMEDIATE, DEALLOCATE to be run in PS mode.

- Fixed the following issues:
    The statement
      explain select (select 2)
    executed in regular and PS mode produces different results:

    MariaDB [test]> prepare stmt from "explain select (select 2)";
    Query OK, 0 rows affected (0,000 sec)
    Statement prepared
    MariaDB [test]> execute stmt;
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | 1 | PRIMARY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
    | 2 | SUBQUERY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    2 rows in set (0,000 sec)
    MariaDB [test]> explain select (select 2);
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    1 row in set, 1 warning (0,000 sec)

    In case the statement
      CREATE TABLE t1 SELECT * FROM (SELECT 1 AS a, (SELECT a+0)) a
    is run in PS mode it fails with the error
      ERROR 1054 (42S22): Unknown column 'a' in 'field list'.

- Uniform handling of read-only variables both in case the SET var=val
  statement is executed as regular or prepared statememt.

- Fixed assertion firing on handling LOAD DATA statement for temporary tables

- Relaxed assert condition in the function lex_end_stage1() by adding
  the commands SQLCOM_ALTER_EVENT, SQLCOM_CREATE_PACKAGE,
  SQLCOM_CREATE_PACKAGE_BODY to a list of supported command

- Removed raising of the error ER_UNSUPPORTED_PS in the function
  check_prepared_statement() for the ALTER VIEW command

- Added initialization of the data memember st_select_lex_unit::last_procedure
  (assign NULL value) in the constructor

  Without this change the test case main.ctype_utf8 fails with the following
  report in case it is run with the optoin --ps-protocol.
    mysqltest: At line 2278: query 'VALUES (_latin1 0xDF) UNION VALUES(_utf8'a' COLLATE utf8_bin)' failed: 2013: Lost connection

- The following bug reports were fixed:
    MDEV-24460: Multiple rows result set returned from stored
                routine over prepared statement binary protocol is handled incorrectly
    CONC-519: mariadb client library doesn't handle server_status and
              warnign_count fields received in the packet COM_STMT_EXECUTE_RESPONSE.

  Reasons for these bug reports have the same nature and caused by
  missing loop iteration on results sent by server in response to
  COM_STMT_EXECUTE packet.

  Enclosing of statements for processing of COM_STMT_EXECUTE response
  in the construct like
    do
    {
      ...
    } while (!mysql_stmt_next_result());
  fixes the above mentioned bug reports.

- Some tests has been disabled by the reason that they contains
  multi-statements that not supported in PS mode. Unfortunately,
  some of these multi-statements can't be converted to a sequence of
  single-statements by the reason that multi-statementness is
  a requirement for tests. Therefore the whole test file will be
  skipped in case it is run with --ps-protocol

Author: Dmitry Shulga
Author Date: 2021-06-11 11:42:02 UTC

MDEV-16708: Unsupported commands for prepared statements

Within this task the following changes were made:
- Added sending of metadata info in prepare phase for the admin related
  command (check table, checksum table, repair, optimize, analyze).

- Refactored implmentation of HELP command to support its execution in
  PS mode

- Added support for execution of LOAD INTO and XA- related statements
  in PS mode

- Modified mysqltest to enable any statements except PREPARE FROM,
  EXECUTE, EXECUTE IMMEDIATE, DEALLOCATE to be run in PS mode.

- Fixed the following issues:
    The statement
      explain select (select 2)
    executed in regular and PS mode produces different results:

    MariaDB [test]> prepare stmt from "explain select (select 2)";
    Query OK, 0 rows affected (0,000 sec)
    Statement prepared
    MariaDB [test]> execute stmt;
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | 1 | PRIMARY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
    | 2 | SUBQUERY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    2 rows in set (0,000 sec)
    MariaDB [test]> explain select (select 2);
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    | 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
    +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
    1 row in set, 1 warning (0,000 sec)

    In case the statement
      CREATE TABLE t1 SELECT * FROM (SELECT 1 AS a, (SELECT a+0)) a
    is run in PS mode it fails with the error
      ERROR 1054 (42S22): Unknown column 'a' in 'field list'.

- Uniform handling of read-only variables both in case the SET var=val
  statement is executed as regular or prepared statememt.

- Fixed assertion firing on handling LOAD DATA statement for temporary tables

- Relaxed assert condition in the function lex_end_stage1() by adding
  the commands SQLCOM_ALTER_EVENT, SQLCOM_CREATE_PACKAGE,
  SQLCOM_CREATE_PACKAGE_BODY to a list of supported command

- Removed raising of the error ER_UNSUPPORTED_PS in the function
  check_prepared_statement() for the ALTER VIEW command

- Added initialization of the data memember st_select_lex_unit::last_procedure
  (assign NULL value) in the constructor

  Without this change the test case main.ctype_utf8 fails with the following
  report in case it is run with the optoin --ps-protocol.
    mysqltest: At line 2278: query 'VALUES (_latin1 0xDF) UNION VALUES(_utf8'a' COLLATE utf8_bin)' failed: 2013: Lost connection

- The following bug reports were fixed:
    MDEV-24460: Multiple rows result set returned from stored
                routine over prepared statement binary protocol is handled incorrectly
    CONC-519: mariadb client library doesn't handle server_status and
              warnign_count fields received in the packet COM_STMT_EXECUTE_RESPONSE.

  Reasons for these bug reports have the same nature and caused by
  missing loop iteration on results sent by server in response to
  COM_STMT_EXECUTE packet.

  Enclosing of statements for processing of COM_STMT_EXECUTE response
  in the construct like
    do
    {
      ...
    } while (!mysql_stmt_next_result());
  fixes the above mentioned bug reports.

- Some tests has been disabled by the reason that they contains
  multi-statements that not supported in PS mode. Unfortunately,
  some of these multi-statements can't be converted to a sequence of
  single-statements by the reason that multi-statementness is
  a requirement for tests. Therefore the whole test file will be
  skipped in case it is run with --ps-protocol

Author: Sergey Petrunia
Author Date: 2021-06-10 20:54:14 UTC

MDEV-25858: Query results are incorrect when indexes are added

If test_if_skip_sort_order() decides to use an index to produce required
ordering, it should disable "Range Checked for each record" optimization.

This is because Range-Checked-for-each-record may decide to use an index
(or an index_merge) which will not produce the required ordering.

Author: Dmitry Shulga
Author Date: 2021-06-11 06:01:15 UTC

MDEV-16708: more test failures fixed (added --enable_prepared_warnings/--disable_prepared_warnings)

Author: Daniel Black
Author Date: 2021-06-11 04:30:42 UTC

MDEV-20162: fix connect-abstract test case

The check-testcase record uses a mysqltest connection
to the database to do the recording. With the server configured
as an abstract socket, the mysqltest client cannot connect and
fails.

We work around this by starting the server as normal and then
restart with an abstract socket and test this.

This didn't affect Windows as it just did a tcp connection.
So this did affect all unix socket based systems except Linux
as this was the only one that supported abstract sockets.

Author: midenok
Author Date: 2021-06-10 16:50:45 UTC

Review

Author: Andrei Elkin
Author Date: 2021-06-10 13:28:53 UTC

MDEV-18959: Correcting related to MDEV-21117

Identification of the truncate candidate in case of
no-binlog-offset engines requires to store this type
of capability into Gtid event even with just one engine
transaction, when the engine is such.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-06-10 10:08:14 UTC

MDEV-25872 InnoDB: Assertion failure in row_merge_read_clustered_index upon ALTER on table with indexed virtual columns

InnoDB fails to check DB_COMPUTE_VALUE_FAILED error in
row_merge_read_clustered_index() and wrongly asserts that
the buffer shouldn't be ran out of memory. Alter table
should give warning when the column value is being
truncated.

Author: Vladislav Vaintroub
Author Date: 2019-06-15 14:04:49 UTC

MDEV-19702 Refactor Bitmap<N> to be based on ulonglong, not on uint32

Removed Field_map, since it was used only in a single function.

Fixed is_indexed_agg_distinct(), since it relied on initialization of
Bitmap in constructor.

Fixes MDEV-25888 in 10.4

Author: Daniel Black
Author Date: 2021-03-30 06:32:36 UTC

MDEV-25282 Auto-shutdown on idle when socket-activated

Adds max_idle_execution system variable with a UINT_MAX
default value that corresponds to the time in seconds
under which the mariadbd executable will run in an idle state
with no connections.

Under systemd socket activiation this variable will get a 10 minute
default value. This will enable a service to be activated on
connection and fall back to a shutdown state after 10 minutes of
no queries. The systemd socket activation can restart the service
on the next connection transparently.

A global variable of server_last_activity is updated on
accepted connections (where max_idle_execution < UINT_MAX)
and when the connection count (standard or extra) is down
to <= 1 to keep the number of updates on a single variable
low (an not create another performance MDEV-21212 problem).

When the main accept loop times out on the max_idle_execution
seconds, and then the server_last_activity is checked along
with if current connection count (standard + extra) is 0
(in case a recently started connection hasn't finished
a query).

To make this neater, in main accept loop moved code to
handle_new_socket_connection that encompases accepting a connection
and the timeout mechanism has been separated too.

Changed when looping though possible connections, loop until
the end of the connection list and hereby assume two connection can
occur on the same poll/select call and both will be accepted.

Author: Elena Stepanova
Author Date: 2021-06-09 12:50:01 UTC

MDEV-25884 Tests use environment $USER variable without quotes

These are only 10.4+ tests. 10.2+ tests are pushed into 10.2
and will be merged into 10.4+ independently

Author: Elena Stepanova
Author Date: 2021-06-09 12:28:35 UTC

MDEV-25884 Tests use environment $USER variable without quotes

Author: Brandon Nesterenko
Author Date: 2021-06-09 17:03:03 UTC

MDEV-25607: Auto-generated DELETE from HEAP table can break replication

Problem:
=======
The special logic used by the memory storage engine
to keep slaves in sync with the master on a restart can
break replication. In particular, after a restart, the
master writes DELETE statements in the binlog for
each MEMORY-based table so the slave can empty its
data. If the DELETE is not executable, e.g. due to
invalid triggers, the slave will error and fail, whereas
the master will never see the problem.

Solution:
=======
Instead of DELETE statements, use TRUNCATE to
keep slaves in-sync with the master, thereby bypassing
triggers.

Reviewed By:
===========
<TODO>

Author: Andrei Elkin
Author Date: 2021-06-09 17:17:01 UTC

MDEV-21117 Address if the last review round.

Author: Sujatha Sivakumar
Author Date: 2021-06-09 17:07:36 UTC

MDEV-21117: New test case

Author: Sergei Golubchik
Author Date: 2021-06-09 11:17:58 UTC

MDEV-23933 main.failed_auth_unixsocket fails on arm

the idea of main.failed_auth_unixsocket was to have existing
user account (root) authenticate with unix_socket, then login with
non-existent user name, Non-existent user name forces the server
to perform the authentication in the name of some random existing
user. But it must still fail at the end, as the user name is wrong.

In 10.4 a second predefined user was added, mariadb.sys, so root
is not the only user in mysql.global_priv and unix_socket auth
must be forced for all existing user accounts, because we cannot
know what user account the server will randomly pick for non-existing
user auth.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-05-24 10:11:08 UTC

MDEV-25642 InnoDB rename table copy DDL fails while dropping the table

When doing a ALTER TABLE ... RENAME, MariaDB doesn't rename
original table to #sql-backup, which it does in other cases,
but insteads drops the original table directly. However
this optimization doesn't work in case of InnoDB table
with a foreign key constraint.

During copy algorithm, InnoDB fails to rename the foreign key
constraint(MDEV-25855). With this optimisation, InnoDB also
fails to drop the original table because the table has
FOREIGN Key constraint exist in INNODB_SYS_FOREIGN table.
This leads to orphan .ibd file in InnoDB dictionary.
so disabling this optimization when FK is involved.

Reviewer: monty@mariadb.org

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-05-24 10:11:08 UTC

MDEV-25642 InnoDB rename table copy DDL fails while dropping the table

When doing a ALTER TABLE ... RENAME, MariaDB doesn't rename
original table to #sql-backup, which it does in other cases,
but insteads drops the original table directly. However
this optimization doesn't work in case of InnoDB table
with a foreign key constraint.

During copy algorithm, InnoDB fails to rename the foreign key
constraint(MDEV-25855). With this optimisation, InnoDB also
fails to drop the original table because the table has
FOREIGN Key constraint exist in INNODB_SYS_FOREIGN table.
This leads to orphan .ibd file in InnoDB dictionary.
so disabling this optimization when FK is involved.

Reviewer: monty@mariadb.org

Author: Vladislav Vaintroub
Author Date: 2021-06-09 11:27:00 UTC

MDEV-22221: MariaDB with WolfSSL doesn't support AES-GCM cipher for SSL

Enable AES-GCM for SSL (only).

AES-GCM for encryption plugins remains disabled (aes-t fails, on some bug
in GCM or CTR padding)

Author: Sergey Petrunia
Author Date: 2021-05-22 12:53:33 UTC

MDEV-25630: Crash with window function in left expr of IN subquery

* Make Item_in_optimizer::fix_fields inherit the with_window_func
  attribute of the subquery's left expression (the subquery itself
  cannot have window functions that are aggregated in this select)

* Make Item_cache_wrapper::Item_cache_wrapper() inherit
  with_window_func attribute of the item it is caching.

Author: Marko Mäkelä
Author Date: 2021-06-09 11:23:20 UTC

MDEV-25886 CHECK TABLE crash with DB_MISSING_HISTORY if innodb_read_only

Occasionally, the test innodb.alter_copy would fail in MariaDB 10.6.1,
reporting DB_MISSING_HISTORY during CHECK TABLE. It started to occur during
the development of MDEV-25180, which introduced purge_sys.stop_SYS().
If we delay purge more during DDL operations, then the test would
almost always fail. The reason is that during startup we will restore
a purge view, and CHECK TABLE would still use REPEATABLE READ
even though innodb_read_only is set and other isolation levels
than READ UNCOMMITTED are not guaranteed to work.

ha_innobase::check(): Use READ UNCOMMITTED isolation level if
innodb_read_only is set or innodb_force_recovery exceeds 3.

dict_set_corrupted(): Do not update the persistent data dictionary
if innodb_force_recovery exceeds 3.

Author: Anel Husakovic
Author Date: 2020-10-13 12:36:25 UTC

MDEV-25878: mytop bugs: check for mysql driver and sockets

- Adding socket check for MariaDB/Mysql driver
- For RPM packaging use `Client` component for `mytop`.

Reviewed by: serg@mariadb.com

Author: Oleksandr "Sanja" Byelkin
Author Date: 2020-06-22 16:21:21 UTC

MDEV-21916: COM_STMT_BULK_EXECUTE with RETURNING insert wrong values

The problem is that array binding uses net buffer to read parameters for each
execution while each execiting with RETURNING write in the same buffer.

Solution is to allocate new net buffer to avoid changing buffer we are reading
from.

Author: Julius Goryavsky
Author Date: 2021-06-09 01:41:37 UTC

MDEV-25880: rsync may be mistakenly killed when overlapping SST

This commit fixes a bug was originally discovered during the
galera_nbo_sst_slave mtr test for 10.6 branch. However it is
relevant for all versions and can lead to intermittent SST
crashes via rsync on very fast server restarts - when a new
SST process (for example, after starting a new server instance)
overlaps the old SST process started by the previous, already
terminated server. This overlap can result in the new rsync
being killed instead of the old rsync, or the pid file from
the new rsync being killed, which then lead to problems.

Author: Marko Mäkelä
Author Date: 2021-06-08 13:18:46 UTC

Relax an assertion about fts_commit()

Author: Robert Bindar
Author Date: 2021-01-14 13:35:45 UTC

Typo fixes. Thanks @shinguz for the help!

Author: Igor Babaev
Author Date: 2021-06-08 02:51:57 UTC

Ported the test case for MDEV-25682 from 10.2

No fix for this bug is needed starting from version 10.4.

Author: Igor Babaev
Author Date: 2021-06-07 22:08:18 UTC

MDEV-25682 Explain shows an execution plan different from actually executed

If a select query contained an ORDER BY clause that followed a LIMIT clause
or an ORDER BY clause or ORDER BY with LIMIT the EXPLAIN output for the
query showed an execution plan different from that was actually executed.

Approved by Roman Nozdrin <roman.nozdrin@mariadb.com>

Author: Vladislav Vaintroub
Author Date: 2021-06-07 14:50:55 UTC

MDEV-25870 Windows - MSI generation cleanup, fix ARM64

- Support building ARM64 packages
- require InstallerVersion 5.0 -required for arm64, also
it is there already on Windows 7

Author: Monty
Author Date: 2021-06-07 14:40:30 UTC

MDEV-25866 Upgrade from pre-10.5.10 to 10.5.10 causes CHECK errors on encrypted Aria tables

Hard to do a test case, but tested by hand and verified that mysql_upgrade
will update the encrypted MariaDB tables.

Author: Marko Mäkelä
Author Date: 2021-06-07 14:46:59 UTC

MDEV-25869 Change buffer entries are lost on InnoDB restart

buf_read_ibuf_merge_pages(): If space->size is 0, invoke
fil_space_get_size() to determine the size of the tablespace
by reading the header page. Only after that proceed to delete
any entries that are beyond the end of the tablespace.
Otherwise, we could be deleting valid entries that actually
need to be applied.

This fixes a regression that had been introduced in
commit b80df9eba23b4eb9694e770a41135127c6dbc5df (MDEV-21069),
which aimed to avoid crashes during DROP TABLE of corrupted tables.

Author: Nikita Malyavin
Author Date: 2021-06-07 14:35:26 UTC

MDEV-24511 null field is created with CREATE..SELECT

The regression has been initially appeared by MDEV-12588 patch (441349aa).

The reason CREATE... SELECT NULL does not create null field is that
it calls handle_select(), where the field is created directly from Item,
by calling type_handler->type_handler_for_tmp_table(), instead of plain
type_handler.

In UNION case, we'll already have Item_field's containing UNION'ed tmp
table fields to the moment of select_create::prepare() call.

There, item's type_handler is used directly in
st_select_lex_unit::join_union_item_types(), instead of
type_handler_for_tmp_table() or type_handler_for_union().

Using those two involve too many changes in CREATE...SELECT...UNION
behavior, which may be reasonably undesired to be pushed in 10.3

However, patching Field_null::type_handler() directly seems to be accurate
enough.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-06-02 14:31:18 UTC

MDEV-14180 Automatically disable key rotation checks for file_key_managment plugin

Problem:
=======
- InnoDB iterates the fil_system space list to encrypt the
tablespace in case of key rotation. But it is not necessary
for any encryption plugin which doesn't do key version rotation.

Solution:
=========
- Introduce a new variable called srv_encrypt_rotate to
indicate whether encryption plugin does key rotation

fil_space_crypt_t::key_get_latest_version(): Enable the
srv_encrypt_rotate only once if current key version is higher than
innodb_encyrption_rotate_key_age

fil_crypt_must_default_encrypt(): Default encryption tables
should be added to default_encryp_tables list if
innodb_encyrption_rotate_key_age is zero and encryption
plugin doesn't do key version rotation

fil_space_create(): Add the newly created space to default_encrypt_tables
list if fil_crypt_must_default_encrypt() returns true

Removed the undetermined select from innodb-key-rotation-disable test.
By default, we add the tablespace to the rotation list and
background crypt thread does encryption of tablespace. So these
select doesn't give reliable results.

Author: Sachin Setiya
Author Date: 2021-05-14 07:11:13 UTC

Roll Forward recovery for binlog tests

Author: Otto Kekäläinen
Author Date: 2021-05-29 05:04:17 UTC

Revert "CONNECT: move jar files to /usr/share and include them in DEBs"

This partially reverts commit d7321893d8c50071632a102e17a7869da9cb03a5.

The *.jar files are not being built and all Debian builds are failing
as dh_install stops on missing files. To build them we would need to also
add new Java build dependencies.

In a stable release (10.2->10.5) we shouldn't add new files and certainly
not any new build dependencies, so reverting commit.

Also, the files are located in a different path, and already included
in the mariadb-test-data package:

  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JavaWrappers.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/JdbcMariaDB.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo2.jar
  /usr/share/mysql/mysql-test/plugin/connect/connect/std_data/Mongo3.jar

This change needs to be redesigned and applies only on 10.6 or newer.

Author: midenok
Author Date: 2021-05-19 14:51:34 UTC

MDEV-23639 Auto-create does not work under LOCK TABLES or inside triggers

Don't do tdc_remove_table() for OT_ADD_HISTORY_PARTITION because it is
not possible in locked tables mode.

LTM_LOCK_TABLES mode (and LTM_PRELOCKED_UNDER_LOCK_TABLES) works out
of the box as fast_alter_partition_table() can reopen tables via
locked_tables_list.

In LTM_PRELOCKED we reopen and relock table manually.

Author: midenok
Author Date: 2021-05-19 14:51:34 UTC

MDEV-23639 Auto-create does not work under LOCK TABLES or inside triggers

Don't do tdc_remove_table() for OT_ADD_HISTORY_PARTITION because it is
not possible in locked tables mode.

LTM_LOCK_TABLES mode (and LTM_PRELOCKED_UNDER_LOCK_TABLES) works out
of the box as fast_alter_partition_table() can reopen tables via
locked_tables_list.

In LTM_PRELOCKED we reopen and relock table manually.

Author: Marko Mäkelä
Author Date: 2021-06-04 11:29:51 UTC

MDEV-25783: Change buffer records are lost under heavy load

ibuf_read_merge_pages(): Disable some code that was added in MDEV-20394
in order to avoid a server hang if the change buffer is corrupted,
presumably due to the race condition in recovery that was later fixed in
MDEV-24449. The code will still be available in debug builds when
the command line option --debug=d,ibuf_merge_corruption is specified.

Author: Alexey Botchkov
Author Date: 2021-06-04 10:25:24 UTC

p1

Author: sjaakola
Author Date: 2021-04-29 06:54:24 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

The crash happened due to refactoring in MDEV-23328 fix to execute BF aborts
in bakcground rollbacker thread. This refactoring was done to sort out
potential mutex locking order violations between executions of BF aborting
vs manual kill command. As the result of this refactoring, the manual kill
command execution remains unaltered and BF aborting has been deviated to
work through a background thread. Executing BF aborting in background thread,
changes the time window for actual BF aborting, which is very sensitive to
changes. This has caused several issues so far (not only this one), and
should also affect to cluster performance with cluster conflicting use cases.

This commit reverts the changes done by MDEV-23328 (commit 29bbcac0ee841faaa68eeb09c86ff825eabbe6b6)
and modifies the manual kill command execution. The manual kill, in this
commit, is executed in similar way as BF conflict happening due to MDL
conflict. This happens by calling wsrep_abort_thd(). With this, the kill
command execution takes in consideration the state of execution of the
"killee" using different strategies for executing the kill dependind on
the state of the killee. The mutex ordering violation should be fixed, as
BF aborting happens now from inside of innodb both with BF aborting and
kill command execution.

As kill command can use different "kill states" for killing the killee,
BF aborting functions have been refactored to pass the kill state along.
BF aborting happening due to replication conflicts uses only KILL_QUERY state.

If server is configured with wsrep_provider, but a session has set
wsrep_on==OFF and then executes .e.g. a DDL statement, which has a MDL lock
conflict, then this session initiates BF abort for the MDL lock holder.
This should not happen, BF aborting is only for clustering use cases.
The fix in this commit, checks session's wsrep_on state, and skips calling
wsrep_grant_mdl_exception() if wsrep_on==OFF or wsrep_provider is not present.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-04-29 06:54:24 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

The crash happened due to refactoring in MDEV-23328 fix to execute BF aborts
in bakcground rollbacker thread. This refactoring was done to sort out
potential mutex locking order violations between executions of BF aborting
vs manual kill command. As the result of this refactoring, the manual kill
command execution remains unaltered and BF aborting has been deviated to
work through a background thread. Executing BF aborting in background thread,
changes the time window for actual BF aborting, which is very sensitive to
changes. This has caused several issues so far (not only this one), and
should also affect to cluster performance with cluster conflicting use cases.

This commit reverts the changes done by MDEV-23328 (commit 29bbcac0ee841faaa68eeb09c86ff825eabbe6b6)
and modifies the manual kill command execution. The manual kill, in this
commit, is executed in similar way as BF conflict happening due to MDL
conflict. This happens by calling wsrep_abort_thd(). With this, the kill
command execution takes in consideration the state of execution of the
"killee" using different strategies for executing the kill dependind on
the state of the killee. The mutex ordering violation should be fixed, as
BF aborting happens now from inside of innodb both with BF aborting and
kill command execution.

As kill command can use different "kill states" for killing the killee,
BF aborting functions have been refactored to pass the kill state along.
BF aborting happening due to replication conflicts uses only KILL_QUERY state.

If server is configured with wsrep_provider, but a session has set
wsrep_on==OFF and then executes .e.g. a DDL statement, which has a MDL lock
conflict, then this session initiates BF abort for the MDL lock holder.
This should not happen, BF aborting is only for clustering use cases.
The fix in this commit, checks session's wsrep_on state, and skips calling
wsrep_grant_mdl_exception() if wsrep_on==OFF or wsrep_provider is not present.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Brandon Nesterenko
Author Date: 2021-06-03 17:24:34 UTC

MDEV-25277: mysqlbinlog --verbose cannot read row events with compressed columns: Don't know how to handle column type: 140

Problem:
=======
Mysqlbinlog cannot show the type of a compressed
column when two levels of verbosity are provided.

Solution:
========
Extend the output to show that columns are compressed.

Behavioral Changes:
==================
  Old: When mysqlbinlog is called in verbose mode and
the database uses compressed columns, an error is
returned to the user.

  New: The output will now post-fix “ COMPRESSED”
on the type of compressed columns.

Reviewed By:
===========
<TODO>

Author: Eugene
Author Date: 2021-04-28 20:04:38 UTC

tmp

Author: midenok
Author Date: 2021-05-27 17:54:19 UTC

MDEV-24176 Server crashes after insert in the table with virtual
column generated using date_format() and if()

When table is reopened from cache vcol_info contains stale
expression. We refresh expression via TABLE::vcol_fix_exprs() but
first we must prepare a proper context (Vcol_expr_context) and do some
preparations which meet some requirements:

1. fields must not be fixed, we unfix them with cleanup() via
cleanup_processor.

2. Item_ident must have cached_table cleared. Again, this is stale
data which was already freed. cached_table interferes with
find_field_in_tables().

We cannot clear cached_table directly in Item_ident::cleanup()
method. Lots of spaghetti logic depends on cached_table existence even
after cleanup() done.

3. If Item_ident has table_name non-NULL we trigger expr update. That
is done via Item_ident::check_vcol_func_processor() and
VCOL_TABLE_ALIAS flag.

(4-7) The below conditions are prepared by Vcol_expr_context and used in
both fix_session_expr_for_read() and TABLE::vcol_fix_exprs():

4. Any fix_expr() must be done on expr_arena as there may be new items
created. It was a bug in fix_session_expr_for_read(). It was just not
reproduced because there were no second refix. Now refix is done for
more cases so it does reproduce. Tests affected: vcol.binlog

5. Also name resolution context must be narrowed to the single table
with all crutches in place. Tests affected: vcol.update

6. sql_mode must be clean and not fail expr update.

sql_mode such as MODE_NO_BACKSLASH_ESCAPES, MODE_NO_ZERO_IN_DATE, etc
must not affect vcol expression update. If the table was created
successfully any further evaluation must not fail. Tests affected:
main.func_like

7. Warnings are disabled during expr update. It is assumed that these
warnings were printed before during initialization phase or later
during execution phase. Tests affected: vcol.wrong_arena

Dictionary: refresh, update and refix are the synonyms for
vcol_fix_exprs() or fix_session_expr_for_read() calls.

Author: Daniel Black
Author Date: 2021-06-03 08:08:49 UTC

fix

Author: Monty
Author Date: 2021-06-02 11:05:12 UTC

Fixed that compile-pentium64-valgrind-max works

- Removed Tokudb (no need to test this anymore with valgrind)
- Added __attribute__(unused)) to a few places to be able to compile even
  if valgrind/memcheck.h is not installed.

Reviewer: Marko Mäkelä <marko.makela@mariadb.com>

Author: Igor Babaev
Author Date: 2021-06-02 15:40:30 UTC

MDEV-25635 Assertion failure when pushing from HAVING into WHERE of view

This bug could manifest itself after pushing a where condition over a
mergeable derived table / view / CTE DT into a grouping view / derived
table / CTE V whose item list contained set functions with constant
arguments such as MIN(2), SUM(1) etc. In such cases the field references
used in the condition pushed into the view V that correspond set functions
are wrapped into Item_direct_view_ref wrappers. Due to a wrong implementation
of the virtual method const_item() for the class Item_direct_view_ref the
wrapped set functions with constant arguments could be erroneously taken
for constant items. This could lead to a wrong result set returned by the
main select query in 10.2. In 10.4 where a possibility of pushing condition
from HAVING into WHERE had been added this could cause a crash.

Approved by Sergey Petrunya <sergey.petrunya@mariadb.com>

Author: Sergey Petrunia
Author Date: 2021-06-02 11:51:52 UTC

Change version to 10.7.0

Author: Alexey Botchkov
Author Date: 2021-05-31 09:48:09 UTC

MDEV-17399 JSON_TABLE.

Accept JSON values for the JSON fields.

Author: Julius Goryavsky
Author Date: 2021-05-29 17:54:25 UTC

MDEV-25818: RSYNC SST failed due to busy port

This commit reduces the likelihood of getting a busy port on
quick restarts with rsync SST (problem MDEV-25818) and fixes
a number of other flaws in SST scripts, adds new functionality,
and also synchronizes the xtrabackup-v2 script with the
mariabackup script (the latter applies only to the 10.2 branch):

 1) SST via rsync: rsync and stunnel does not always get the right
    time to complete by correctly handling SIGTERM. These utilities
    are now given more time to complete normally (via normal SIGTERM
    processing) before we move on to using "kill -9";
 2) SST via rsync: attempts to terminate an rsync or stunnel process
    (via "kill" utility) are only made if it did not terminated on
    its own;
 3) SST via rsync: if a combination of stunnel and rsync is used,
    then we need to wait for both utilities to finish or stop, not
    just one of them;
 4) The config file and pid file for stunnel are now deleted after
    successful completion of SST on the donor node;
 5) The configs and pid files from rsync and stunnel should not be
    deleted unless these utilities succeed (or are sucessfully
    terminated) on the joiner node;
 6) The configs and pid files now excluded from transfer via rsync;
 7) Spaces in paths are now valid for config files as well (when
    used with SST via rsync or mariabackup / xtrabackup[-v2]);
 8) SST via mariabackup: added preliminary verification of keys and
    certificates that are used when establishing a connection using
    SSL (to avoid long timeouts and improve diagnostics) - by analogy
    with how it is done for the xtrabackup-v2 (plus check for CA file),
    while that check is skipped if the user does not have openssl
    installed (or does not have diff utility);
 9) Added backup-threads=<n> configuration option which adds
    "--parallel=<n>" for mariabackup / xtrabackup at backup and
    move-back stages;
10) Added encrypt-threads and encrypt-chunk-size configuration
    options for xbcrypt management (when xbcrypt is used);
11) Small optimization: checking the socat version and adding
    a file with parameters for 2048-bit Diffie-Hellman (if necessary)
    is done only if the user has not specified "dhparam=" in the
    "sockopt" option value;
12) SST via rsync now supports "backup-threads" configuration option
    (in server-related sections or in the "[sst]");
13) Determining the number of available processors is now supported
    for FreeBSD + mariabackup/xtrabackup: before that we might have
    problems with "--compact" (rebuild indexes) or qpress on FreeBSD;
14) The check_pid() function should not raise an error state in
    the rare cases when the pid file was created, but it is empty,
    or if it is deleted right during the check, or when zero is read
    from the pid file;
15) Iproved templates that are used to check if a requested socket
    is "listening" when using the ss utility;
16) Shortened some other templates for socket state utilities;
17) Temporary files created by mariabackup / xtrabackup are moved
    to a separate subdirectory inside tmpdir (so they don't get
    mixed with other temporary files, which can make debugging
    more difficult);
18) 10.2 only: the script for SST via xtrabackup-v2 has been brought
    in full compliance with all the bugfixes made for mariabackup (as
    it previously contained many flaws compared to the updated script
    for mariabackup).

Author: Julius Goryavsky
Author Date: 2021-05-29 17:54:25 UTC

MDEV-25818: RSYNC SST failed due to busy port

This commit reduces the likelihood of getting a busy port on
quick restarts with rsync SST (problem MDEV-25818) and fixes
a number of other flaws in SST scripts, adds new functionality,
and also synchronizes the xtrabackup-v2 script with the
mariabackup script (the latter applies only to the 10.2 branch):

 1) SST via rsync: rsync and stunnel does not always get the right
    time to complete by correctly handling SIGTERM. These utilities
    are now given more time to complete normally (via normal SIGTERM
    processing) before we move on to using "kill -9";
 2) SST via rsync: attempts to terminate an rsync or stunnel process
    (via "kill" utility) are only made if it did not terminated on
    its own;
 3) SST via rsync: if a combination of stunnel and rsync is used,
    then we need to wait for both utilities to finish or stop, not
    just one of them;
 4) The config file and pid file for stunnel are now deleted after
    successful completion of SST on the donor node;
 5) The configs and pid files from rsync and stunnel should not be
    deleted unless these utilities succeed (or are sucessfully
    terminated) on the joiner node;
 6) The configs and pid files now excluded from transfer via rsync;
 7) Spaces in paths are now valid for config files as well (when
    used with SST via rsync or mariabackup / xtrabackup[-v2]);
 8) SST via mariabackup: added preliminary verification of keys and
    certificates that are used when establishing a connection using
    SSL (to avoid long timeouts and improve diagnostics) - by analogy
    with how it is done for the xtrabackup-v2 (plus check for CA file),
    while that check is skipped if the user does not have openssl
    installed (or does not have diff utility);
 9) Added backup-threads=<n> configuration option which adds
    "--parallel=<n>" for mariabackup / xtrabackup at backup and
    move-back stages;
10) Added encrypt-threads and encrypt-chunk-size configuration
    options for xbcrypt management (when xbcrypt is used);
11) Small optimization: checking the socat version and adding
    a file with parameters for 2048-bit Diffie-Hellman (if necessary)
    is done only if the user has not specified "dhparam=" in the
    "sockopt" option value;
12) SST via rsync now supports "backup-threads" configuration option
    (in server-related sections or in the "[sst]");
13) Determining the number of available processors is now supported
    for FreeBSD + mariabackup/xtrabackup: before that we might have
    problems with "--compact" (rebuild indexes) or qpress on FreeBSD;
14) The check_pid() function should not raise an error state in
    the rare cases when the pid file was created, but it is empty,
    or if it is deleted right during the check, or when zero is read
    from the pid file;
15) Iproved templates that are used to check if a requested socket
    is "listening" when using the ss utility;
16) Shortened some other templates for socket state utilities;
17) Temporary files created by mariabackup / xtrabackup are moved
    to a separate subdirectory inside tmpdir (so they don't get
    mixed with other temporary files, which can make debugging
    more difficult);
18) 10.2 only: the script for SST via xtrabackup-v2 has been brought
    in full compliance with all the bugfixes made for mariabackup (as
    it previously contained many flaws compared to the updated script
    for mariabackup).

Author: Julius Goryavsky
Author Date: 2021-05-29 17:54:25 UTC

MDEV-25818: RSYNC SST failed due to busy port

This commit reduces the likelihood of getting a busy port on
quick restarts with rsync SST (problem MDEV-25818) and fixes
a number of other flaws in SST scripts, adds new functionality,
and also synchronizes the xtrabackup-v2 script with the
mariabackup script (the latter applies only to the 10.2 branch):

 1) SST via rsync: rsync and stunnel does not always get the right
    time to complete by correctly handling SIGTERM. These utilities
    are now given more time to complete normally (via normal SIGTERM
    processing) before we move on to using "kill -9";
 2) SST via rsync: attempts to terminate an rsync or stunnel process
    (via "kill" utility) are only made if it did not terminated on
    its own;
 3) SST via rsync: if a combination of stunnel and rsync is used,
    then we need to wait for both utilities to finish or stop, not
    just one of them;
 4) The config file and pid file for stunnel are now deleted after
    successful completion of SST on the donor node;
 5) The configs and pid files from rsync and stunnel should not be
    deleted unless these utilities succeed (or are sucessfully
    terminated) on the joiner node;
 6) The configs and pid files now excluded from transfer via rsync;
 7) Spaces in paths are now valid for config files as well (when
    used with SST via rsync or mariabackup / xtrabackup[-v2]);
 8) SST via mariabackup: added preliminary verification of keys and
    certificates that are used when establishing a connection using
    SSL (to avoid long timeouts and improve diagnostics) - by analogy
    with how it is done for the xtrabackup-v2 (plus check for CA file),
    while that check is skipped if the user does not have openssl
    installed (or does not have diff utility);
 9) Added backup-threads=<n> configuration option which adds
    "--parallel=<n>" for mariabackup / xtrabackup at backup and
    move-back stages;
10) Added encrypt-threads and encrypt-chunk-size configuration
    options for xbcrypt management (when xbcrypt is used);
11) Small optimization: checking the socat version and adding
    a file with parameters for 2048-bit Diffie-Hellman (if necessary)
    is done only if the user has not specified "dhparam=" in the
    "sockopt" option value;
12) SST via rsync now supports "backup-threads" configuration option
    (in server-related sections or in the "[sst]");
13) Determining the number of available processors is now supported
    for FreeBSD + mariabackup/xtrabackup: before that we might have
    problems with "--compact" (rebuild indexes) or qpress on FreeBSD;
14) The check_pid() function should not raise an error state in
    the rare cases when the pid file was created, but it is empty,
    or if it is deleted right during the check, or when zero is read
    from the pid file;
15) Iproved templates that are used to check if a requested socket
    is "listening" when using the ss utility;
16) Shortened some other templates for socket state utilities;
17) Temporary files created by mariabackup / xtrabackup are moved
    to a separate subdirectory inside tmpdir (so they don't get
    mixed with other temporary files, which can make debugging
    more difficult);
18) 10.2 only: the script for SST via xtrabackup-v2 has been brought
    in full compliance with all the bugfixes made for mariabackup (as
    it previously contained many flaws compared to the updated script
    for mariabackup).

Author: Vladislav Vaintroub
Author Date: 2021-05-29 04:19:46 UTC

MDEV-25815 mariabackup crash or debug assert with --backup --databases-exclude

Fix regression (debug assertion or division by 0)
caused by cfd3d70ccbbfcf3fdec034be446317741dfae824

Author: Anel Husakovic
Author Date: 2021-05-28 15:15:28 UTC

MDEV-25802: mtr: race condition is the test quickly restarts twice

Author: Georg Richter
Author Date: 2021-05-28 08:33:08 UTC

Disable ssl_crl for server only (wolfssl)

Author: Sachin Setiya
Author Date: 2021-05-27 13:45:10 UTC

MDEV-25709 Replication IO_Thread Connecting Primary server handshake call "SELECT GTID_BINLOG_POS" even when MASTER_USE_GITD=NO and GTID is disabled

Remove the code

Author: Jan Lindström
Author Date: 2021-05-28 05:47:51 UTC

Fix test case galera_vote_rejoin_ddl

Author: Sergei Golubchik
Author Date: 2021-05-26 22:37:51 UTC

MDEV-25792 server hangs on early shutdown if InnoDB needs to purge indexed virtual columns

if the server isn't started and innodb initiated a shutdown process -
don't wait for the server to start, it won't

Author: Robert Bindar
Author Date: 2021-05-26 12:27:07 UTC

Revert "Add Pull Request template file to the MariaDB/server repository"

This reverts commit 17106c984b9649641068ddf108dcffd2c7773212.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2021-05-24 14:10:47 UTC

MDEV-25758 InnoDB spatial indexes miss large geometry fields after MDEV-25459

InnoDB should calculate the MBR for the first field of
spatial index and do the comparison with the clustered
index field MBR. Due to MDEV-25459 refactoring, InnoDB
calculate the length of the first field and fails with
too long column error.

Author: midenok
Author Date: 2021-05-27 04:45:49 UTC

Arena fix

Author: Vladislav Vaintroub
Author Date: 2021-05-27 06:44:50 UTC

MDEV-22010 amend

Author: Sachin Setiya
Author Date: 2021-05-24 10:23:03 UTC

MDEV-24667 LOAD DATA INFILE/inserted rows not written to binlog

Problem:- In master slave replication , when master is using statement format
and slave is using mixed format, sometime we can have binlog which is not
written. For example LOAD DATA INFILE is a unsafe statement , it is logged as
a row format, and if it is on temporary table , it wont be logged. If the
next statement is copy data from the tmp table to normal table, it is logged
as a statement format on the slave , because it thinks this statement is
unsafe. As we can see we did not log LOAD DATA INFILE and we simply logging
insert into from * into statement format , we have effectively lost the
tmp table data.

Solution:- On master in mixed format if tmp table is created thd->temporary_tables
is set to not null. This will make THD::reset_for_next_command to not change
the binlog format back to stmt. So till the tmp table exist all the logs will
be written in row format. On master THD::has_thd_temporary_tables() is used
to determine whether we have temporary tables or not. But this does not work
for the slave , slave shifts temporary tables from thd->temporary_tables to
rgi_slave->rli->save_temporary_tables, so this will make has_thd_temporary_tables()
check to return false on slave. So instead of this function I am calling
has_temporary_tables() , which also checks slave thread tmp tables.

Author: Monty
Author Date: 2021-05-26 11:53:26 UTC

Fixed results for main.delete_use_source to make it repeatable

Author: Daniel Black
Author Date: 2020-04-26 03:35:57 UTC

MDEV-25424: my_multi_malloc_large to use large pages

Change my_multi_malloc_large to use my_large_malloc.

The my_multi_malloc_large interface now takes a
final argument of a pointer to the size actually allocated.
This is needed because my_large_free(munmap) requires the
size.

Use size_t as its the native type for sizes of memory.

Return allocated amount with my_large_free, however first
adjusted with my_psi_key_free to ensure the right pointer
address is used.

Closes: #1513

Author: Igor Babaev
Author Date: 2021-05-26 00:13:17 UTC

MDEV-23886 Reusing CTE inside a function fails with table doesn't exist

In the code existed just before this patch binding of a table reference to
the specification of the corresponding CTE happens in the function
open_and_process_table(). If the table reference is not the first in the
query the specification is cloned in the same way as the specification of
a view is cloned for any reference of the view. This works fine for
standalone queries, but does not work for stored procedures / functions
for the following reason.
When the first call of a stored procedure/ function SP is processed the
body of SP is parsed. When a query of SP is parsed the info on each
encountered table reference is put into a TABLE_LIST object linked into
a global chain associated with the query. When parsing of the query is
finished the basic info on the table references from this chain except
table references to derived tables and information schema tables is put
in one hash table associated with SP. When parsing of the body of SP is
finished this hash table is used to construct TABLE_LIST objects for all
table references mentioned in SP and link them into the list of such
objects passed to a pre-locking process that calls open_and_process_table()
for each table from the list.
When a TABLE_LIST for a view is encountered the view is opened and its
specification is parsed. For any table reference occurred in
the specification a new TABLE_LIST object is created to be included into
the list for pre-locking. After all objects in the pre-locking have been
looked through the tables mentioned in the list are locked. Note that the
objects referenced CTEs are just skipped here as it is impossible to
resolve these references without any info on the context where they occur.
Now the statements from the body of SP are executed one by one that.
At the very beginning of the execution of a query the tables used in the
query are opened and open_and_process_table() now is called for each table
reference mentioned in the list of TABLE_LIST objects associated with the
query that was built when the query was parsed.
For each table reference first the reference is checked against CTEs
definitions in whose scope it occurred. If such definition is found the
reference is considered resolved and if this is not the first reference
to the found CTE the the specification of the CTE is re-parsed and the
result of the parsing is added to the parsing tree of the query as a
sub-tree. If this sub-tree contains table references to other tables they
are added to the list of TABLE_LIST objects associated with the query in
order the referenced tables to be opened. When the procedure that opens
the tables comes to the TABLE_LIST object created for a non-first
reference to a CTE it discovers that the referenced table instance is not
locked and reports an error.
Thus processing non-first table references to a CTE similar to how
references to view are processed does not work for queries used in stored
procedures / functions. And the main problem is that the current
pre-locking mechanism employed for stored procedures / functions does not
allow to save the context in which a CTE reference occur. It's not trivial
to save the info about the context where a CTE reference occurs while the
resolution of the table reference cannot be done without this context and
consequentially the specification for the table reference cannot be
determined.

This patch solves the above problem by moving resolution of all CTE
references at the parsing stage. More exactly references to CTEs occurred in
a query are resolved right after parsing of the query has finished. After
resolution any CTE reference it is marked as a reference to to derived
table. So it is excluded from the hash table created for pre-locking used
base tables and view when the first call of a stored procedure / function
is processed.
This solution required recursive calls of the parser. The function
THD::sql_parser() has been added specifically for recursive invocations of
the parser.

# Conflicts:
# sql/sql_cte.cc
# sql/sql_cte.h
# sql/sql_lex.cc
# sql/sql_lex.h
# sql/sql_view.cc
# sql/sql_yacc.yy
# sql/sql_yacc_ora.yy

Author: Nikita Malyavin
Author Date: 2021-05-25 18:08:08 UTC

MDEV-24511 Assertion failed in Field::ptr_in_record on ALTER TABLE

Assertion `l_offset >= 0 && table->s->rec_buff_length - l_offset > 0'
 failed in Field::ptr_in_record and errno: 168 "Unknown (generic) error
 from engine" from InnoDB on ALTER.

This

Author: Anel Husakovic
Author Date: 2021-05-25 16:17:29 UTC

MDEV-20787: Script dgcov.pl does not work

Author: Nikita Malyavin
Author Date: 2020-11-17 09:38:18 UTC

MDEV-19784 [4/4] fix problem with yacc and revert yacc changes

Author: Sachin Setiya
Author Date: 2020-06-11 08:03:48 UTC

MDEV-19157 Optimistic Parallel Replication fails when gtid_slave_pos table is non transnational and binlog events are transnational

Problem:- In Optimistic/Aggressive mode of parallel replication when we have
    conflict, we retry the transaction, But if gtid_slave_pos is non transactional
    In that case retry may result into duplicate key error for table because
    the conflicted transaction can not rollback the changes in gtid_slave_pos table
    So when we try the same trans again, We get the Duplicate Key error

Solution:- Users are advised to use transactional storage engine for gtid_slave_pos.
    In this patch we are throwing warning when we see that gtid_slave_pos in non
    transactional. In addition to throwing warnings Transactional event groups
    from master are not scheduled in Optimistic/Aggressive mode.

Author: Sergei Golubchik
Author Date: 2021-05-19 20:26:02 UTC

MDEV-25730: maria.repair test fails with valgrind

cherry-pick commit: 1fff2398ef3dda1a7e8404f18e4e165823bd4e0a
MDEV-22530 post push fixes from 10.6.

Followup. If the KILL happens - report it as a failure,
don't eat it up silently. Note that this has to be done after `table_name`
is populated, so that the error message could show it.

Author: Sergei Golubchik
Author Date: 2021-05-19 20:26:02 UTC

MDEV-25730: maria.repair test fails with valgrind

cherry-pick commit: 1fff2398ef3dda1a7e8404f18e4e165823bd4e0a
MDEV-22530 post push fixes from 10.6.

Modified 'maria.repair' test to make it stable on valgrind build.

Author: Sergei Golubchik
Author Date: 2021-05-19 20:26:02 UTC

MDEV-25730: maria.repair test fails with valgrind

cherry-pick commit: 1fff2398ef3dda1a7e8404f18e4e165823bd4e0a
MDEV-22530 post push fixes from 10.6.

Modified 'maria.repair' test to make it stable on valgrind build.

Author: Sergei Golubchik
Author Date: 2021-05-19 20:26:02 UTC

MDEV-25730: maria.repair test fails with valgrind

cherry-pick commit: 1fff2398ef3dda1a7e8404f18e4e165823bd4e0a
MDEV-22530 post push fixes from 10.6.

Modified 'maria.repair' test to make it stable on valgrind build.

Author: Marko Mäkelä
Author Date: 2021-05-25 07:47:25 UTC

MDEV-25113: Introduce a page cleaner mode before 'furious flush'

MDEV-23855 changed the way how the page cleaner is signaled by
user threads. If a threshold is exceeded, a mini-transaction commit
would invoke buf_flush_ahead() in order to initiate page flushing
before all writers would eventually grind to halt in
log_free_check(), waiting for the checkpoint age to reduce.

However, buf_flush_ahead() would always initiate 'furious flushing',
making the buf_flush_page_cleaner thread write innodb_io_capacity_max
pages per batch, and sleeping no time between batches, until the
limit LSN is reached. Because this could saturate the I/O subsystem,
system throughput could significantly reduce during these
'furious flushing' spikes.

With this change, we introduce a gentler version of flush-ahead,
which would write innodb_io_capacity_max pages per second until
the 'soft limit' is reached.

buf_flush_ahead(): Add a parameter to specify whether furious flushing
is requested.

buf_flush_async_lsn: Similar to buf_flush_sync_lsn, a limit for
the less intrusive flushing.

buf_flush_page_cleaner(): Keep working until buf_flush_async_lsn
has been reached.

log_close(): Suppress a warning message in the event that a new log
is being created during startup, when old logs did not exist.
Return what type of page cleaning will be needed.

mtr_t::finish_write(): Also when m_log.is_small(), invoke log_close().
Return what type of page cleaning will be needed.

mtr_t::commit(): Invoke buf_flush_ahead() based on the return value of
mtr_t::finish_write().

Author: Sergei Golubchik
Author Date: 2021-05-23 16:38:46 UTC

followup: rename generated files to have distinct names

Author: Julius Goryavsky
Author Date: 2021-05-24 15:59:15 UTC

MDEV-21192: SST failing when enabling IPV6

The following features have been added:

1) Automatic addition of the pf = ip6 option for socat
   when it can be recognized by the format of the connection
   address;
2) Automatically add or remove extra commas at the beginning
   and at the end of sockopt, for example, sockopt='pf=ip6'
   and sockopt=',pf=ip6' work equally well;

Also, due to interference in the code of the get_transfer()
function, I also refactored it and now:

3) encrypt = 4 is supported not only for xtrabackup-v2,
   but also for mariabackup - this can help with migration
   from Percona;
4) Improved setting of 'commonname' option for encrypt=3
   and encrypt=4 modes;

Author: Sergei Golubchik
Author Date: 2021-05-21 08:52:12 UTC

cmake: fewer Build-Depends in SRPM

don't require tar/gtar, git, getconf, groff/nroff, and ruby.

Author: Brandon Nesterenko
Author Date: 2021-05-21 20:53:43 UTC

MDEV-20215: binlog.show_concurrent_rotate failed in buildbot with wrong result

Problem:
=======
There are two issues that are addressed in this patch:
 1) SHOW BINARY LOGS uses caching to store
the binary logs that exist in the log directory; however,
if new events are written to the logs, the caching
strategy is unaware. This is okay for users, as it is
okay for SHOW to return slightly old data. The test,
however, can result in inconsistent data. It runs
two connections concurrently, where one shows the
logs, and the other adds a new file. The output of
SHOW BINARY LOGS then depends on when the
cache is built, with respect to the time that the
second connection rotates the logs.
 2) There is a race condition between RESET
MASTER and SHOW BINARY LOGS. More
specifically, where they both need the binary log
lock to begin, SHOW BINARY LOGS only needs
the lock to build its cache. If RESET MASTER
is issued after SHOW BINARY LOGS has built
its cache and before it has returned the results,
the presented data may be incorrect.

Solution:
=======
 1) As it is okay for users to see stale data, to
make the test consistent, use DEBUG_SYNC
to force the race condition (problem 2) to make
SHOW BINARY LOGS build a cache before
RESET MASTER is called. Then, use additional
logic from the next part of the solution to rebuild
the cache.
 2) Use an Atomic_counter to keep track of the
number of times RESET MASTER has been called.
If the value of the counter changes after building
the cache, the cache should be rebuilt and the
analysis should be restarted.

Author: Brandon Nesterenko
Author Date: 2021-05-21 04:11:07 UTC

MDEV-20215: binlog.show_concurrent_rotate failed in buildbot with wrong result

Problem:
=======
A race condition exists between two connections to
get the binary logs lock, where the first connection
tries to SHOW BINARY LOGS and uses the lock to
obtain all logs, and the second connection uses the
lock in RESET MASTER to purge the logs and restart
the rotation.

Solution:
=======
Where it is expected to have some amount of stale data
in the SHOW BINARY LOGS call, limit it to only the most
current binary log file size. More specifically, after going
through the cached file list (which can take up to 20
seconds), check to see if any new files have been created
in the directory. If so, analyze each new created file.
Additionally, the check for purged files only checks file
name, so to enhance the logic here, additionally
consider timestamp to reduce wasted time on file stats.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-05-20 09:42:00 UTC

fix of MDEV-24312

Move event table change to the group
of the event table changes after which
event scheduler can be started.

Author: Sachin Setiya
Author Date: 2021-05-19 14:46:57 UTC

MDEV-25444 mysql --binary-mode is not able to replay some mysqlbinlog outputs

Problem:- Some binary data is inserted into the table using Jconnector. When
binlog dump of the data is applied using mysql cleint it gives syntax error.

Reason:-
After investigating it turns out to be a issue of mysql client not able to properly
handle \\\0 <0 in binary>. In all binary files where mysql client fails to insert
these 2 bytes are commom (0x5c00)

Solution:-
I have changed mysql.cc to include for the possibility that binary string can
have \\\0 in it.