Author: Anel Husakovic
Author Date: 2022-07-04 13:27:36 UTC

MDEV-28455: CREATE TEMPORARY TABLES privilege is insufficient for SHOW COLUMNS

=========== Problem =============
- Invocation of `show column <temporary_tabel>` returns empty result,
even though user has *all* (`CREATE TEMPORARY TABLE`) grants.
- Invocation of `show [index,column] <tabel>` calls `SQLCOM_SELECT` as an `lex->sql_command` during `mysql_execute_command`.
- The function `check_table_access()` get called which is optimized for `show` commands,
  that is appending artificial `SELECT_ACL` (int 1) grant in order to call `check_show_access()`.
- `check_show_access()` is optimized for IS tables (`STATISTICS` and `COLUMN` for `show index` and `show column` respectively),
  that is calling `check_access()` and `check_grant()` for underlying table - so the user needs to have SELECT_ACL privilege on table level.
  There is special handling of temporary tables by setting `table->grant` to `TMP_TABLE_ACL` (int 12351), which includes all DML privileges (SELECT_ACL, and so on).
- When calling function pointer for `show index` (`get_schema_stat_record`) there is no checking of acces/grant/column.
- When calling function pointer for `show column` (`get_schema_column_record`) there is access check and column grant check.
- For the table (in this case temporary table) `show column` command is failing since column access is determined by bitwise operation of table grant privilege and `COL_ACLS`(int 2055).

=========== Solution =============
- Optimize call of `check_access()` only on table level, not for all
fields
- Append `REFERENCES_ACL` to be consistent with `COL_ACL` for temporary
table
- Note that `check_access()` that was called in early phase just checks for SELECT_ACL, but doesn't store the result, so one can think that calling the `check_access()` is redudant, but we have to call it in order to get table ACLs.
- Besides that, one solution is to check if the table is temporary and skip column grant
- Second solution is to check if the table is temporary and append `TMP_TABLE_ACL`, that is the superset of `COL_ACLS` to the `tables->grant.privilege`and check for column grants.

=========== Additionally =============
- Update comments for function `check_table_access` arguments

Reviewed by: <vicentiu@mariadb.org>

Author: Anel Husakovic
Author Date: 2022-07-12 16:24:11 UTC

Update results for 10.10

Author: midenok
Author Date: 2022-07-14 15:20:10 UTC

MDEV-29060 fix ps protocol

Author: mariadb-LuisLizardo
Author Date: 2022-07-14 10:06:00 UTC

Change logic to always close the writer after printing query blocks

Author: Igor Babaev
Author Date: 2022-07-13 18:13:06 UTC

MDEV-29088 Server crash upon CREATE VIEW with unknown column in ON condition

This bug caused crashes when the server executed such a CREATE VIEW
statement whose view specification contained a reference to an unknown
column in a subquery used in ON condition.
The cause of this bug is quite similar to the cause of the bug MDEV-26412.
The fix of this bug is quite similar to the fix for MDEV-26412.

Approved by Sergey Petrunia <sergey@mariadb.com>

Author: Oleg Smirnov
Author Date: 2022-07-07 15:57:38 UTC

MDEV-28881 Fix memory leak caused by STL usage

Dep_analysis_context::create_unique_pseudo_key_if_needed() was using
std::set which allocated memory outside MEM_ROOT and so required
explicit deallocation. This has been solved by replacing std::set with
MY_BITMAP which is allocated on MEM_ROOT

Author: Oleg Smirnov
Author Date: 2022-07-07 15:57:38 UTC

MDEV-28881 Fix memory leak caused by STL usage

Dep_analysis_context::create_unique_pseudo_key_if_needed() was using
std::set which allocated memory outside MEM_ROOT and so required
explicit deallocation. This has been solved by replacing std::set with
MY_BITMAP which is allocated on MEM_ROOT

Author: Dmitry Shulga
Author Date: 2022-07-12 10:18:48 UTC

MDEV-22001: Server crashes in st_select_lex_unit::exclude_level upon execution of SP

Running some statements that use IN subqueries outside context of a regular
query could result in server abnormal termination.

The reason for failure is that internal structures SELECT_LEX/SELECT_LEX_UNIT
created on behalf of parsed query were initialized incorrectly. Incorrect
initialization of the structures SELECT_LEX/SELECT_LEX_UNIT was introduced
by the commit de745ecf29721795710910a19bd0ea3389da804c
(MDEV-11953: support of brackets in UNION/EXCEPT/INTERSECT operations)
pushed into 10.4, that is the reason this bug report is not reproduced in 10.3.

To fix the issue the method SLECTE_LEX::register_unit is used for proper
initialization of the data structures SELECT_LEX/SELECT_LEX_UNIT. Additionally,
the method SELECT_LEX::get_slave() was removed from the source code base
since for those use cases where it is used it can be replaced by the method
first_inner_unit().

Author: midenok
Author Date: 2022-07-10 22:59:33 UTC

Cleanup: use normalized_path instead of build_table_filename()

Author: Daniel Black
Author Date: 2022-07-11 02:52:16 UTC

MDEV-29015 innodb os_file_set_size WSL workaround

WSL wasn't keeping stack of renamed files
(https://github.com/microsoft/WSL/issues/8443)
resulting in the fstat calls in os_file_set_size returning
ENOENT and no fallocate fallback being possible.

Users reported MySQL was ok, and it used my_seek to
determine the size. We copy this concept here to avoid
the WSL bug.

Author: Daniel Black
Author Date: 2022-07-11 02:52:16 UTC

MDEV-29015 innodb os_file_set_size WSL workaround

WSL wasn't keeping stack of renamed files
(https://github.com/microsoft/WSL/issues/8443)
resulting in the fstat calls in os_file_set_size returning
ENOENT and no fallocate fallback being possible.

Users reported MySQL was ok, and it used my_seek to
determine the size. We copy this concept here to avoid
the WSL bug.

Author: Monty
Author Date: 2022-06-30 11:02:53 UTC

Fix cost calculation in test_if_cheaper_ordering() to be cost based

The original code was mostly rule based and preferred clustered or
covering indexed independent of cost.

There where a few test changes:
- Some test changed from using filesort to index or table scan. This
  happened when most of the rows had to be sorted and the ORDER BY could
  use covering or a clustered index (innodb_mysql, create_spatial_index).
- Some test changed range to filesort. This where mainly because the range
  was scanning most of the rows or using index scan + row lookup and filesort
  with table scan is cheaper. (order_by).
- Change in join_cache was because sorting 2 rows is faster than retrieving
  10 rows.
- In selectivity_innodb.test one test changed to use a cheaper index.

Author: Alexey Botchkov
Author Date: 2022-07-08 09:54:10 UTC

Fix to quiet the compiler on Windows.

Author: Monty
Author Date: 2022-07-09 13:51:55 UTC

Fix mysqld--help.test

Disable printing of provider-... as these differ if
the modules are static

Author: Alexander Barkov
Author Date: 2022-06-08 11:12:03 UTC

MDEV-23287 The INET4 data type

Author: Alexander Barkov
Author Date: 2022-07-08 11:29:23 UTC

MDEV-29062 Wrong result set metadata for a mix of INT+ENUM

Author: midenok
Author Date: 2022-07-04 12:08:01 UTC

MDEV-29023 MTR hangs after multiple failures

Passing $opt_parallel as $childs is wrong: child can be killed before
it connects and you will never decrement $childs for this.

Another problem is (and that is the cause of this bug): child can be
killed and never close server socket. This can happen f.ex. after
unmaskable KILL signal. In such case the socket is closed by reaping
the child but that never happens inside reading the socket loop in
run_test_server().

The proper design is the waitless reap of children inside the socket
loop and if there is no more children we finish the socket loop. Since
there is Windows variation where we don't control the children via
waitpid(), all the clients must normally close the socket and only
this can finish the socket loop. For Unix variation we reckon that
case as all children closed the socket but not all yet died and for
that we do final waiting waitpid() (was done before the patch as
well).

To be more complete, we now handle 3 end-of-game scenarios in Unix:

   1. all children closed socket, all children died: everything is
      handled by the socket loop;

   2. all children closed socket, not all yet died: we wait for alive
      children to die after exiting the socket loop;

   3. not all children closed socket, all children died: everything is
      handled by the socket loop.

For Windows end-of-game scenario is only one:

   All children close the socket.

Author: midenok
Author Date: 2022-07-07 17:06:48 UTC

Tmp

Author: midenok
Author Date: 2022-07-07 14:27:34 UTC

MDEV-20088 Cleanup: unused Ed_connection, Execute_sql_statement, Protocol_local, etc

Removed needless classes.

Author: Nayuta Yanagisawa
Author Date: 2022-07-05 13:56:26 UTC

MDEV-29027 ASAN errors in spider_db_free_result after partition DDL

Spider calls ha_spider::close() at least twice on ALTER TABLE ... ADD
PARTITION. The first call frees wide_handler->trx and the second call
accesses wide_handler->trx->thd (heap-use-after-free).

In general, there seems to be no problem with using THD obtained by
the macro current_thd() except in background threads. Thus, we simply
replace wide_handler->trx->thd with current_thd().

Author: Brandon Nesterenko
Author Date: 2022-07-01 15:49:40 UTC

MDEV-27161 Fixes for affected tests

mysqld help options were extended with the new
option for slave_max_statement_time, and the
mysqld--help test result was re-recorded
respectively.

show all slaves status was extended with a new
variable and tests which use that full output
needed to be re-recorded.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2022-07-05 12:51:21 UTC

MDEV-20797 FULLTEXT search with apostrophe, and mandatory words

- InnoDB should ignore the single word followed by apostrophe while
tokenising the document. Example is that if the input string is O'brien
then right now, InnoDB seperates into two tokens as O, brien. But
after this patch, InnoDB can ignore the token 'O' and consider
only 'brien'.

Author: Nikita Malyavin
Author Date: 2022-07-04 22:19:12 UTC

MDEV-29013 ER_KEY_NOT_FOUND/lock timeout upon online alter with long unique

1. ER_KEY_NOT_FOUND
Some virtual columns were not updated because were not included in read_set
to the moment of calculation.

In Row_logs_event::do_apply_event some fields are excluded based on m_cols
value, the number of replicated rows.

We can't rely on this. Basically, we'd be satisfied, if all columns were
just set, at least for now.

2. ER_LOCK_WAIT_TIMEOUT
This is a long unique specific problem.

Sometimes, lookup_handler is created for to->file. To properly free it,
ha_reset should be called. It is usually done by calling
close_thread_table, but ALTER TABLE makes it differently. Hence, a single
ha_reset call is added to mysql_alter_table.

Also, the lifetime of lookup_handler is corrected: it's been allocated on
thd->mem_root, which in case of online alter
is event's mem_root, and can vary other ways.
It is changed to table->mem_root, which is more appropriate:
ha_reset is called only when table is closed, or in a few additional cases,
which correspond to a statement end. So memory leaks are unlikely here.

Author: Vladislav Vaintroub
Author Date: 2022-07-05 08:47:03 UTC

MDEV-24071 - repro

fixed a couple of assertions or crashes in the server.
Now Innodb is acting up

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-07-04 13:23:01 UTC

MDEV-29022 add_slave destroy child list and has dead code

Nowdays subquery in a UNION's ORDER BY placed correctly in fake select,
the only problem was incorrect Name_resolution_contect is fixed by this
patch in parsing, so we do not need scanning/reseting of ORDER BY of
a union.

(in 10.4 all work in parser correctly)

Author: Alexander Barkov
Author Date: 2022-07-05 07:39:40 UTC

MDEV-27015 Assertion `!is_null()' failed in FixedBinTypeBundle<FbtImpl>::Fbt FixedBinTypeBundle<FbtImpl>::Field_fbt::to_fbt()

The assuption that Field::is_null() is always false when
Field_fbt::val_native() or Field_fbt::to_fbt() are called
was wrong.

In some cases, e.g. when this helper Field method is called:

  inline String *val_str(String *str, const uchar *new_ptr)

we temporarily reset Field::ptr to some alternative record buffer
but don't reset null_ptr, so null_ptr still points to null flags
of the original record. In such cases it's meaningless to test
the original Field::null_ptr when Field::ptr is temporarily reset:
they don't relate to each other.

Removing the DBUG_ASSERT.

Author: Sergei Golubchik
Author Date: 2022-07-04 14:47:14 UTC

MDEV-28632 Change default of explicit_defaults_for_timestamp to ON

save it in binlog, so that CREATE TABLE could be replayed correctly
on the slave

Author: Oleg Smirnov
Author Date: 2022-06-27 05:37:59 UTC

MDEV-28869 Eliminated tables are not shown in EXPLAIN

Show tables that have been eliminated during optimization stage
in the output of EXPLAIN/ANALYZE [FORMAT=JSON] commands. Show both
regular and derived tables.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-07-04 13:23:01 UTC

MDEV-29022 add_slave destroy child list and has dead code

Nowdays subquery in a UNION's ORDER BY placed correctly in fake select,
the only problem was incorrect Name_resolution_contect is fixed by this
patch in parsing, so we do not need scanning/reseting of ORDER BY of
a union.

Author: Vicențiu Ciorbaru
Author Date: 2022-07-02 19:15:22 UTC

Implement cost_for_order_by function

The sort length is extracted similarly to how sortlength() function does
it. The function makes use of filesort_use_addons function to compute
the length of addon fields. Finally, by calling compute_sort_costs we
get the fastest_sort possible.

Other changes:
* Sort_param::using_addon_fields() assumes addon fields are already
  allocated. This makes the use of Sort_param unusable for
  compute_sort_costs *if* we don't want to allocate addon fields.

  As a preliminary fix, pass "with_addon_fields" as bool value to
  compute_sort_costs() and make the internal functions use that value
  instead of Sort_param::using_addon_fields() method.

  The ideal fix would be to define a "leaner" struct with only the
  necessary members, but this can be done as a separate commit.

* Remove unnecessary fields from Sort_costs::costs array.
  As we do not yet differentiate between merge sort with fixed length vs
  dynamic length fields, eliminate that differentiation from enum sort_type.
  It can be added at a later date when we indeed have a cost
  differentiation.

* Fixup comments

Author: Nayuta Yanagisawa
Author Date: 2022-07-01 12:41:45 UTC

MDEV-28996 ASAN errors in String::q_append / spider_string::q_append / spider_db_mbase_util::open_item_func

Fix the stack-use-after-scope caused by tmp_str.

Author: Anel Husakovic
Author Date: 2022-06-30 11:54:07 UTC

Format properly mtr report for the test case that is not completed

- Fixes 29e8c154172bd6b9dccd9757fddace93be6ad194 (MDEV-25857)
Review by: <@mariadb.com>

Author: midenok
Author Date: 2022-03-24 14:00:25 UTC

MDEV-25004 innodb_fts.versioning upgrade/downgrade test for system versioning

OLD_BINDIR must point to previous revision build for this test to work
correctly (see comment in versioning.test).

Author: sjaakola
Author Date: 2022-06-30 08:39:05 UTC

MDEV-26391 BF abortable mariabackup execution

Forcing old style node desync and pause, despite of WSREP_MODE_BF_MARIABACKUP
if node is operating as SST donor.

Author: Vlad Lesin
Author Date: 2022-06-29 14:03:56 UTC

MDEV-28709 unexpected X lock on Supremum in READ COMMITTED

The lock is created during page splitting after moving records and
locks(lock_move_rec_list_(start|end)()) to the new page, and inheriting
the locks to the supremum of left page from the successor of the infimum
on right page. There is no need in such inheritance for READ COMMITTED
isolation level and not-gap locks.

Author: Vlad Lesin
Author Date: 2022-06-29 15:51:19 UTC

MDEV-28709 unexpected X lock on Supremum in READ COMMITTED

The lock is created during page splitting after moving records and
locks(lock_move_rec_list_(start|end)()) to the new page, and inheriting
the locks to the supremum of left page from the successor of the infimum
on right page. There is no need in such inheritance for READ COMMITTED
isolation level and not-gap locks.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2022-06-28 10:21:05 UTC

MDEV-28919 Assertion `(((core_null) + 7) >> 3) == oindex.n_core_null_bytes || !not_redundant()' failed

- In case of discarded tablespace, InnoDB can't read the root page to
assign the n_core_null_bytes. Consecutive instant DDL fails because
of non-matching n_core_null_bytes.

Author: Nikita Malyavin
Author Date: 2022-06-30 11:37:22 UTC

speed up main.alter_table_online_debug

Author: qggcs
Author Date: 2022-06-25 14:03:08 UTC

json_type should consider the charset mbmaxlen

Author: qggcs
Author Date: 2022-06-25 14:03:08 UTC

json_type should consider the charset mbmaxlen

Author: Nayuta Yanagisawa
Author Date: 2022-06-29 16:58:57 UTC

MDEV-27172 Prefix indices on Spider tables may lead to wrong query results

Spider converts HA_READ_KEY_EXACT to the equality (=) in the
function spider_db_append_key_where_internal() but the conversion
is not necessarily correct for tables with prefix indices.

We fix the bug by converting HA_READ_KEY_EXACT to 'LIKE "foo%"' when
a target key is a prefix key. The fix is partly inspired by FEDERATED.
See ha_federated::create_where_from_key() for more details.

Author: Julius Goryavsky
Author Date: 2022-06-30 01:44:50 UTC

MDEV-28758: Mariabackup copies binary logs to backup directory

This commit restores defaults and functionality regarding binlogs
to the way it was prior to MDEV-27524. The mariabackup utility no
longer saves binlogs files as part of a backup without the --galera-info
option. However, since we use --galera-info during SST, the behavior
of mariabackup changes and, in combination with GTIDs support enabled,
mariabackup transfers one (most recent) binlog file obtained after
FLUSH BINARY LOGS. In other cases, binlogs are not transferred during
SST in mariabackup mode. As for SST in the rsync mode, as it was
before MDEV-27524 - it transfers one last binlog file by default.

The --sst-max-binlogs option for mariabackup and the sst_max_binlogs
parameter in the [sst] / server sections are no longer supported for
SST via mariabackup.

Author: Nikita Malyavin
Author Date: 2022-06-29 17:16:19 UTC

MDEV-28930 ALTER TABLE Deadlocks with parallel TL_WRITE

ALTER ONLINE TABLE acquires table with TL_READ. Myisam normally acquires
TL_WRITE for DML, which makes it hang until table is freed.

We deadlock once ALTER upgrades its MDL lock.

Solution:
Unlock table earlier. We don't need to hold TL_READ once we finished
copying. Relay log replication requires no data locks on `from` table.

Author: Vladislav Vaintroub
Author Date: 2022-06-29 17:44:04 UTC

MDEV-28982- race condition in charset_client_win.

Do not use chcp when mtr is running in parallel mode, since it possibly
resets the console charset, shared by all mtr processes and threads.

Author: Nayuta Yanagisawa
Author Date: 2021-10-13 13:45:26 UTC

MDEV-26544 Assertion `part_share->auto_inc_initialized' failed in ha_partition::get_auto_increment on INSERT

The partition storage engine ignores return (error) values of
handler::info(). As a result, a query that should be aborted is
not aborted and then the server violates the assertion.

Author: Sergei Golubchik
Author Date: 2022-05-30 14:04:09 UTC

C/C 3.3

Author: midenok
Author Date: 2022-04-07 12:42:01 UTC

Stopword upgrade/downgrade test

Plus some history generated to make stopword,vers_stopword better
tested.

Author: Julius Goryavsky
Author Date: 2022-06-29 03:05:57 UTC

MDEV-27524 addendum: additional tests

Author: Monty
Author Date: 2022-06-28 12:57:41 UTC

MDEV-28897 Wrong table.get_ref_count() upon concurrent truncate and backup stage operation

The issue was that flush_tables() didn't take a MDL lock on cached
TABLE_SHARE before calling open_table() to do a HA_EXTRA_FLUSH call.
Most engines seams to have no issue with it, but apparantly this conflicts
with InnoDB in 10.6 when using TRUNCATE

Fixed by taking a MDL lock before trying to open the table in
flush_tables().

There is no test case as it hard to repeat the scheduling that causes
the error. I did run the test case in MDEV-28897 to verify
that the bug is fixed.

Author: Julius Goryavsky
Author Date: 2022-06-28 11:55:42 UTC

MDEV-27524 addendum: additional tests

Author: Julius Goryavsky
Author Date: 2022-06-28 09:27:49 UTC

MDEV-28656: post-merge fixes

This commit fixes some inaccuracies introduced by
the automatic merge process when migrating changes
from previous versions to 10.9.

Author: Julius Goryavsky
Author Date: 2022-06-28 03:06:30 UTC

MDEV-28968: mariabackup SST doesn't properly handle --innodb-buffer-pool-filename

The mariabackup SST script cannot properly handle SST from
the joiner side if a non-standard filename is specified in
the "--innodb-buffer-pool-filename" option. Log analysis
shows that the received file is copied to the destination
directory under the original name "ib_buffer_pool", and not
under the name specified using the option on the command
line or in the configuration file.

This commit fixes this bug and adds an updated mtr tests
to check for correct handling of the non-standard filenames
specified using the "--innodb-buffer-pool-filename" option.

Author: Nayuta Yanagisawa
Author Date: 2022-06-21 15:56:16 UTC

MDEV-21310 AUTO_INCREMENT column throws range error on INSERT in partitioned table | Assertion `part_share->auto_inc_initialized || !can_use_for_auto_inc_init()' failed.

This bug is caused by a similar mechanism as MDEV-21027.

The function, check_insert_or_replace_autoincrement, failed to open
all the partitions on INSERT SELECT statements and it results in the
assertion error.

Author: Alexander Barkov
Author Date: 2022-06-08 11:12:03 UTC

MDEV-23287 The INET4 data type

Author: Brandon Nesterenko
Author Date: 2022-06-27 13:39:19 UTC

MDEV-27655 Fix UBSAN's Findings in Replication

Tagged mysys/my_compare.c::ha_key_cmp() with attribute to ignore
UBSAN runtime checks on shift operations. In particular, it is
expected that this function can perform shift of greater length
than the number of bits in the underlying field variables.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-06-27 12:23:41 UTC

MDEV-27691: post probe to disable service connection

Author: Vladislav Vaintroub
Author Date: 2022-06-01 11:46:33 UTC

MDEV-11026 Make InnoDB number of IO write/read threads dynamic

Fix concurrency error - avoid accessing deleted memory, when io_slots is
resized. the deleted memory in this case was vftable pointer in
aiocb::m_internal_task

The fix avoids calling dummy release function, via a flag in task_group.

Author: Julius Goryavsky
Author Date: 2022-06-27 03:03:21 UTC

MDEV-28758: Mariabackup copies binary logs to backup directory

This commit changes the default settings so that, by default,
mariabackup will not copy binary logs files to the backup directory
and will transfer only one (most recet) binary log file between nodes
during SST. If the user wants to make a backup containing all binary
logs or wants to transfer them all between nodes, then user must add
the sst_max_binlogs=-1 option to the [sst] section, to one of the
server configuration sections, or must use the equivalent command
line option (when running mariabackup).

Author: Nayuta Yanagisawa
Author Date: 2022-04-25 09:29:01 UTC

MDEV-27233 Server hangs when using --init-file which loads Spider and creates a Spider table

Spider waits until the server initialization is complete
(i.e., mysqld_server_started == 1) before creating the Spider system
tables like mysql.spider_XX. Spider also wait until the tables are
created before instantiating ha_spider.

This results in dead lock when Spider is loaded and a non-system
Spider table is created by --in-file because queries passed by
--in-file are executed during the server initialization
(i.e., mysqld_server_started == 0).

In the first place, there seems to be no need for Spider to wait until
the server initialization is completed before creating the system
tables. Thus, we simply remove that synchronization.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-06-24 09:55:42 UTC

MDEV-22133 handle_fatal_signal (sig=11) on optimized builds in handle_grant_table instead of ERROR | Buffer overflow (on optimized builds)

Return an error if we can not read a table required for ACL.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2022-06-24 12:49:57 UTC

- Ignore the corrupted index during row_create_prebuilt()

Author: Alexey Botchkov
Author Date: 2022-02-15 09:37:59 UTC

MDEV-27832 disable binary logging for SQL SERVICE.

Binary logging is now disabled for the queries run by SQL SERVICE.
The binlogging can be turned on with the 'SET SQL_LOG_BIN=On' query.

Conflicts:
 sql/sql_prepare.cc

Author: Nikita Malyavin
Author Date: 2022-06-23 22:45:44 UTC

sampling analysis

file=open(...)
for l in file:
     if l.startswith('SAMPLE') and not l.startswith('SAMPLE root'):
         print(l)
         n = dict((p.split('=')
           for p in l.replace('SAMPLE ', '').replace(' REJECT', '').split()
           ))
         if 'REJECT' in l:
             n['rej']=True
         nodes.append(n)

max(int(n['size']) for n in nodes)
max((float(n['p']), serialize(n)) for n in nodes)
max((float(n['size']), serialize(n)) for n in nodes if n['height'] == '0')

Author: sachin
Author Date: 2020-09-14 14:27:02 UTC

MDEV-23264 Unique blobs allow duplicate values upon UPDATE

Problem:-
  We are able to insert duplicate value in table because cmp_binary_offset
  is not able to differentiate between NULL and empty string. So
  check_duplicate_long_entry_key is never called and we don't check for
  duplicate.
Solution
  Added a if condition with is_null() on field which can differentiate
  between NULL and empty string.

Author: Nayuta Yanagisawa
Author Date: 2022-06-22 17:17:24 UTC

MDEV-19190 Assertion `part_share->auto_inc_initialized' failed in ha_partition::get_auto_increment

Author: Vladislav Vaintroub
Author Date: 2022-06-22 09:58:00 UTC

x

Author: Marko Mäkelä
Author Date: 2022-06-22 07:08:24 UTC

MDEV-28800 WIP: Avoid crashes on memory allocation failure

FIXME: heap-use-after-free in ./mtr --suite=innodb_gis
and failures of
innodb.full_crc32_import
innodb.innodb_defrag_concurrent
innodb.instant_alter

FIXME: Allocate locks upfront for page reorganize or split,
so that the operation can gracefully fail before any irreversible
persistent changes are performed.

FIXME: Remove buf_pool_t::running_out(), or defer the invocation
to when actually attempting to allocate lock objects.

Author: Julius Goryavsky
Author Date: 2022-06-21 11:29:45 UTC

MDEV-26562: galera-sst-mariabackup is failing due to missing xtrabackup_checkpoints

This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).

Also fixed broken logic of automatic generation of the Diffie-Hellman parameters
for socat version less than 1.7.3 (which defaults to 512-bit values instead of
2048-bit ones).

Author: Julius Goryavsky
Author Date: 2022-06-16 01:45:45 UTC

MDEV-28656: Inability to roll upgrade without stopping the Galera cluster

This commit fixes a divergence in the IST/SST script code between
the 10.3+ and 10.9+ branches, due to which the cluster cannot contain
nodes with different versions of the server (since in this case SST/IST
scripts may fail).

This commit also includes test fixes specific to 10.4+ branches and
a few manual fixes after automatic "git cherry-pick", as well as server
code change specific to 10.9+.

Author: Marko Mäkelä
Author Date: 2022-06-21 16:20:15 UTC

Merge 10.10

Author: Nikita Malyavin
Author Date: 2022-02-23 19:38:51 UTC

Add tests and code instrumentations

Author: Julius Goryavsky
Author Date: 2022-06-17 11:56:03 UTC

MDEV-28583 postfix: fixing .result files after automatic merge

Author: Igor Babaev
Author Date: 2022-06-20 06:43:17 UTC

MDEV-7487 Semi-join optimization for single-table update/delete statements

This patch allows to use semi-join optimization at the top level of
single-table update and delete statements.
The problem of supporting such optimization became easy to resolve after
processing a single-table update/delete statement started using JOIN
structure. This allowed to use JOIN::prepare() not only for multi-table
updates/deletes but for single-table ones as well. This was done in the
patch for mdev-28883:
 Re-design the upper level of handling UPDATE and DELETE statements.

Note that JOIN::prepare() detects all subqueries that can be considered
as candidates for semi-join optimization. The code added by this patch
looks for such candidates at the top level and if such candidates are found
in the processed single-table update/delete the statement is handled in
the same way as a multi-table update/delete.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Author: Nayuta Yanagisawa
Author Date: 2022-06-20 09:02:30 UTC

MDEV-28739 Trying to lock uninitialized mutex or hang upon shutdown after using Spider with query_cache

Author: Nayuta Yanagisawa
Author Date: 2022-06-20 09:02:30 UTC

MDEV-28739 Trying to lock uninitialized mutex or hang upon shutdown after using Spider with query_cache

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-10-20 09:37:14 UTC

MDEV-26647 (simple_password_check) Include password validation plugin information in the error message if the SQL statement is not satisfied password policy

Make the plugin reporting cause of the error.

Author: Nayuta Yanagisawa
Author Date: 2022-06-20 11:05:32 UTC

MDEV-27575 SIGSEGV in intern_plugin_lock on SHUTDOWN when setting Spider as default storage engine (temporary or global)

When Spider is set as the default storage engine, it is unloaded later
compared with other plugins. That is because Spider is referenced by
@@global.default_storage_engine and cannot be unloaded before the
variable is set to NULL.

spider_db_done() creates a new THD but, in the above mentioned case,
the new THD needs to lock its default_storage_engine and it does not
expect that the server is almost shut down and the variable is NULL.

Luckily, the newly created THD is used for nothing. Thus, we can just
remove the THD creation.

Author: Nikita Malyavin
Author Date: 2022-02-23 19:38:51 UTC

Add tests and code instrumentations

Author: Sergey Petrunia
Author Date: 2022-06-18 16:52:14 UTC

Cleanups for: 0762dd928 Improve pruning in greedy_search by sorting tables during search

Author: Jan Lindström
Author Date: 2022-06-12 07:39:15 UTC

Fixed regression on SST tests.

We incorrectly used my_thread_end(), which
sets mysys_var pointer to NULL, so the next THD::THD will crash.
Removed my_thread_init()/end() pairs and because
Wsrep_allowlist_service::allowlist_cb is not always called from a
new thread added a thread to do so.

Fix co-authored by Sergei Golubchik <serg@mariadb.org>
and mkaruza <mario.karuza@galeracluster.com>

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Alexander Barkov
Author Date: 2022-06-15 08:39:00 UTC

Switching libmariadb to "Libmariadb changes for MDEV-27009 Add UCA-14.0.0 collations"

Author: Sujatha Sivakumar
Author Date: 2021-08-24 11:45:04 UTC

MDEV-20122 Deprecate MASTER_USE_GTID=Current_Pos to favor new MASTER_DEMOTE_TO_SLAVE option

Deprecate the `Current_Pos` option of MASTER_USE_GTID to suggest
the safe alternative option MASTER_DEMOTE_TO_SLAVE=TRUE. See
the previous commit for the implementation and specifics of this
new option.

Reviewed By:
=============
Andrei Elkin <andrei.elkin@mariadb.com>

Author: Igor Babaev
Author Date: 2022-06-18 07:49:15 UTC

MDEV-27159 Re-design the upper level of handling DML commands

This patch allows to execute only single-table and multi-table
UPDATE and DELETE statements using the method Sql_cmd_dml::execute().
The code that handles INSERT statements has not been touched.

Author: Rucha Deodhar
Author Date: 2022-05-02 09:07:37 UTC

MDEV-27128: Implement JSON Schema Validation FUNCTION

Todo: Add comments, tests for object validation and commit message

Author: Julius Goryavsky
Author Date: 2022-06-17 02:59:29 UTC

MDEV-26562: galera-sst-mariabackup is failing due to missing xtrabackup_checkpoints

This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).

Author: Julius Goryavsky
Author Date: 2022-06-17 11:56:03 UTC

MDEV-28583 postfix: fixing .result files after automatic merge

Author: Igor Babaev
Author Date: 2022-06-17 06:45:38 UTC

MDEV-27159 Re-design the upper level of handling DML commands

This patch allows to execute only single-table and multi-table
UPDATE and DELETE statements using the method Sql_cmd_dml::execute().
The code that handles INSERT statements has not been touched.

Author: Sergei Golubchik
Author Date: 2022-06-16 20:39:35 UTC

fix spider.variable_deprecation test

Author: Vicențiu Ciorbaru
Author Date: 2022-06-16 13:35:55 UTC

INFORMATION_SCHEMA.views denies with show view and select

This commit fixes show view and select privilege denies.

Other cleanup work:
- TODO(cvicentiu) describe this when rebasing.
- get_table_grant -> Take in table strings, not a TABLE_LIST
- get_column_grant -> Take in sctx.
- check_some_grants_remain -> take in role denies.
- check_column_access_denied -> Utility function for show view.

Author: Monty
Author Date: 2022-06-10 13:37:12 UTC

Temporary push to fix a crash in mariabackup

Author: Sergei Golubchik
Author Date: 2022-06-15 20:13:13 UTC

fix sporadic versioning.rpl_row failures

Author: Julius Goryavsky
Author Date: 2022-06-15 23:12:15 UTC

MDEV-28656: Inability to roll upgrade without stopping the Galera cluster

This commit fixes a divergence in the IST/SST script code between
the 10.3+ and 10.9+ branches, due to which the cluster cannot contain
nodes with different versions of the server (since in this case SST/IST
scripts may fail).

This commit also includes test fixes specific to 10.4+ branches.

Author: midenok
Author Date: 2021-12-05 19:20:20 UTC

MDEV-27180 Fully atomic partitioning DDL operations

Atomic DDL for partitioning originally covers crash-safety but it does
not recover fully from failures. F.ex. if error happened during the
rename or drop of partitions the ALTER operation does not return the
table to its original state.

The patch solves the above problem similarly to MDEV-25292 (atomic
CREATE OR REPLACE): new partitions are created as temporary, old
partitions are backed up, everything is guarded by two DDL log chains:
rollback and cleanup. Rollback chain contains the actions that bring
back the original table. Cleanup chain deletes the backup files.

The generic operation of alter partition is as follows:

  1. Create new partitions as tmp partitions;
  2. Fill TMP partitions with data;
  3. Rename old and not needed partitions to backup partitions;
  4. Rename tmp partitions to original partititions;
  5. Do any required logging (mariabackup, binary);
  6. If everything is ok, drop the backup partitions;
  7. In the case any of pp.1-5 fails drop new partitions, rename back
     backup partitions.

Each rename operation is executed right after the corresponding DDL
logging. Originally they were done in different loops:
write_log_changed_partitions(), mysql_change_partitions(), etc. The
actual table operations were done in ha_partition handler:
ha_change_partitions(), ha_drop_partitions(),
ha_rename_partitions(). Now these calls are deprecated and removed.

Instead the above deprecated interfaces the following classes now
handle DDL logging and table operations:

      Alter_partition_logger
      Alter_partition_add
      Alter_partition_change

Alter_partition_logger handles the basic operation, DDL logging and
table renames. Alter_partition_add does the stuff for adding the
partitions. Alter_partition_change combines Alter_partition_logger and
Alter_partition_add for complex operations such as REORGANIZE.

ha_partition::change_partitions() call was not fully removed. Stages
1-4 from that call was renamed to allocate_partitions() and that is
used by Alter_partition_add for initializing file handler arrays
(m_added_file, m_reorged_file). ha_partition::prepare_new_partition()
is renamed to create_partition() because it actually does ha_create()
and the new name better describes what it does.

DDL_LOG_DELETE/DDL_LOG_RENAME/DDL_LOG_REPLACE are now pure file
operations. To process both .par and .frm files we now issue two
actions instead of one. That makes interface more simple and that will
be used by MDEV-16417 in future. All the table operations are done via
other actions, such as DDL_LOG_RENAME_TABLE, DDL_LOG_DROP_TABLE, etc.

DDL_LOG_RENAME_TABLE has new alter_partition option. With this option
it does simple ha_rename_table() which is needed for renaming the
partitions. The action for renaming the partitions must start from
phase DDL_RENAME_PHASE_TABLE.

The testing was refactored a bit. Many new fail/crash points was
added, the test was powered by binary and mariabackup logging, it
tests more ALTER commands with variations of partitions-only and
subpartitions, without and under LOCK TABLES.

EXCHANGE PARTITION was fixed: after failure under LOCK TABLES the
table became unlocked. Fixed by correcting place of
reopen_tables(). Tested by partition_debug.

Author: Monty
Author Date: 2022-06-15 20:49:09 UTC

MDEV-28858 Wrong result with table elimination combined with not_null_range_scan

The bug was that build_notnull_conds_for_range_scans() did not take into
account the join_tab is not yet sorted with constant tables first.
Fixed the bug by testing explicitely if a table is a const table.

Author: Shunpoco
Author Date: 2022-05-20 03:23:20 UTC

MDEV-21027: Assertion `part_share->auto_inc_initialized || !can_use_for_auto_inc_init()' failed in ha_partition::set_auto_increment_if_higher

ha_partition::set_auto_increment_if_higher expects
part_share->auto_inc_initialized is true or can_use_for_auto_inc_init()
is false (but as the comment of this method saids, it returns false
only if we use Spider engine with DROP TABLE or ALTER TABLE query).
However, part_share->auto_inc_initialized becomes true only after all
partitions are opened (since 6dce6aecebe6ef78a14cb5c5c5daa8a355551e40).

Therefore, I added a conditional expression in order to read all
partitions when we REPLACE a table that has an AUTO_INCREMENT column.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-12-13 15:15:21 UTC

MDEV-5215 Granted to PUBLIC

Author: Dmitry Shulga
Author Date: 2022-06-15 11:01:19 UTC

MDEV-5816: Stored programs: validation of stored program statements

Fixed the test main.sp

Author: Jan Lindström
Author Date: 2022-06-12 07:39:15 UTC

Fixed regression on SST tests. We incorrectly used my_thread_end(), which
sets mysys_var pointer to NULL, so the next THD::THD will crash.
Removed my_thread_init()/end() pairs and because
Wsrep_allowlist_service::allowlist_cb is not always called from a
new thread added a thread to do so.

Fix co-authored by Sergei Golubchik <serg@mariadb.org>
and mkaruza <mario.karuza@galeracluster.com>

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Brandon Nesterenko
Author Date: 2022-06-14 11:57:48 UTC

MDEV-20122

Change rpl_global_gtid_slave_state->load in_statement parameter
to true because the insert into gtid_slave_pos is still a part
of the current CM statement

Author: Oleksandr "Sanja" Byelkin
Author Date: 2022-06-14 11:35:15 UTC

more test fixes