lp:maria

Author: Rucha Deodhar
Author Date: 2021-09-01 13:39:01 UTC

MDEV-26238: Remove inconsistent behaviour of --default-* options
in my_print_defaults

Analysis: --defaults* option is recognized anywhere in the commandline
instead of only at the beginning because handle_options() recognizes
options in any order.
Fix: use get_defaults_options() which recognizes --defaults* options only at
the beginning. After this is done, we only want to recognize other options
given in any order which can be done using handle_options(). So only skip
--defaults* options and pass rest of them to handle_options().
Also, removed -e, -g and -c because only my_print_defaults supports them.

Author: Eugene Kosov
Author Date: 2020-06-18 14:26:01 UTC

MDEV-22441 implement a generic way to change a value of a variable in a scope

Example:
{
  auto _= make_scope_value(var, tmp_value);
}

make_scope_value(): a function which returns RAII object which temporary
changes a value of a variable

detail::Scope_value: actual implementation of such RAII class.
It shouldn't be used directly! That's why it's inside a namespace detail.

Author: Lukáš Javorský
Author Date: 2019-11-22 14:03:20 UTC

MDEV-21108 Add option for setting install paths of groonga

Include gronnga and groonga-normalizer-mysql install path

Author: Vladislav Vaintroub
Author Date: 2021-12-02 12:42:06 UTC

Fix compiler errors, MSVC

1. static const my_wc_t some_name = whatever;

As far as compiler is concerned, some_name is not a constant,
and can't be used where constant expression is expected, bringing
error C2099: initializer is not a constant
So, either this needs to be replaced with #defines, in C (
in C++, constexpr would be possible)

2. Fix some truncation bugs.

Author: Nayuta Yanagisawa
Author Date: 2021-11-05 07:39:28 UTC

MDEV-5271 Support engine-defined attributes per partition

Make it possible to specify engine-defined attributes on partitions
as well as tables.

If an engine-defined attribute is only specified at the table level,
it applies to all the partitions in the table.
This is a backward-compatible behavior.

If the same attribute is specified both at the table level and the
partition level, the per-partition one takes precedence.
So, we can consider per-table attributes as default values.

One cannot specify engine-defined attributes on subpartitions.

Implementation details:

* We store per-partition attributes in the partition_element class
  because we already have the part_comment field, which is for
  per-partition comments.

* In the case of ALTER TABLE statements, the partition_elements in
  table->part_info is set up by mysql_unpack_partition().
  So, we parse per-partition attributes after the call of the function.

Author: Sergey Petrunia
Author Date: 2021-11-19 09:17:14 UTC

Make the Optimizer Trace of reqular query and PS EXECUTE be identical

Print this piece when we've just made the choice to convert to semi-join.
Also, print it when we've already made that choice before:

  transformation": {
     "select_id": 2,
     "from": "IN (SELECT)",
     "to": "semijoin",
     "chosen": true
   }

Author: Sergey Petrunia
Author Date: 2021-11-19 09:17:14 UTC

Make the Optimizer Trace of reqular query and PS EXECUTE be identical

Print this piece when we've just made the choice to convert to semi-join.
Also, print it when we've already made that choice before:

  transformation": {
     "select_id": 2,
     "from": "IN (SELECT)",
     "to": "semijoin",
     "chosen": true
   }

Author: Alexey Bychko
Author Date: 2021-11-27 05:30:51 UTC

MDEV-27109 mysql_config mariadb_config lists non existant -lmariadb

server part of MDEV-27109.
added dependency devel->shared and conflict with previous versions

Author: Sergey Petrunia
Author Date: 2021-11-23 14:55:08 UTC

MDEV-26249: Crash in Explain_node::print_explain_for_children with slow query log

The problem was caused by this:
- The select in derived table uses two-phase optimization (due to a
  possible LATERAL DERIVED).
- The primary select has "Impossible where" and so it short-cuts its
  optimization.
- The optimization for the SELECT in the derived table is never finished,
  and EXPLAIN data structure has a dangling pointer to select #2.

Fixed with this: if the select has "Impossible where", do not add links
to derived table subselects into the EXPLAIN data structure. We are not
going to execute those anyway.

Author: Robert Bindar
Author Date: 2021-11-24 15:51:34 UTC

Fix build failure on mac due to invalid access on private member from rw_lock

Author: Eric Herman
Author Date: 2021-11-05 18:03:38 UTC

Remove DYNAMIC_ARRAY get_index_dynamic()

The DYNAMIC_ARRAY copies values in and copies values out. Without a
comparitor function, get_index_dynamic() does not make sense.

This function is not used. If we have a need for a function like it
in the future, I propose we write a new one with unit tests showing
how it is used and demostrating that it behaves as expected.

Author: ryancaicse
Author Date: 2021-11-10 10:35:17 UTC

MDEV-26558 Fix a deadlock due to cyclic dependence

Fix a potential deadlock bug between locks ctrl_mutex and entry->mutex

Author: Julius Goryavsky
Author Date: 2021-11-16 06:32:14 UTC

MDEV-26915: SST scripts do not take log_bin_index setting into account

Currently, SST scripts assume that the filename specified in
the --log-bin-index argument either does not contain an extension
or uses the standard ".index" extension. Similar assumptions are
used for the log_bin_index parameter read from the configuration
file. This commit adds support for arbitrary extensions for the
index file paths.

Author: Julius Goryavsky
Author Date: 2021-11-16 04:21:18 UTC

MDEV-26064: mariabackup SST fails when starting with --innodb-force-recovery

If the server is started with the --innodb-force-recovery argument
on the command line, then during SST this argument can be passed to
mariabackup only at the --prepare stage, and accordingly it must be
removed from the --mysqld-args list (and it is not should be passed
to mariabackup otherwise).

This commit fixes a flaw in the SST scripts and add a test that
checks the ability to run the joiner node in a configuration that
uses --innodb-force-recovery=1.

Author: Elena Stepanova
Author Date: 2021-11-22 21:55:29 UTC

MDEV-26891 Server crashes upon INSERT DELAYED with virtual columns (test)

Author: Sergei Krivonos
Author Date: 2021-11-18 01:04:20 UTC

use c++14 standard

Author: Sergey Petrunia
Author Date: 2021-11-22 07:55:38 UTC

MDEV-26337: subquery with groupby and ROLLUP returns incorrect results on LEFT JOIN on INDEXED values

Make const_expression_in_where() ignore the equalities injected by
LATERAL DERIVED optimization, unless these equalities are used for
ref access.

This follows what make_join_select()/make_cond_for_table() does:
these functions will discard such equalities too.

Author: Sergey Petrunia
Author Date: 2021-11-22 07:55:38 UTC

MDEV-26337: subquery with groupby and ROLLUP returns incorrect results on LEFT JOIN on INDEXED values

Make const_expression_in_where() ignore the equalities injected by
LATERAL DERIVED optimization, unless these equalities are used for
ref access.

This follows what make_join_select()/make_cond_for_table() does:
these functions will discard such equalities too.

Author: Sergei Krivonos
Author Date: 2021-11-17 07:41:13 UTC

Improve LibFMT detection

Signed-off-by: Sergei Krivonos <sergei.krivonos@mariadb.com>

Author: Sergei Krivonos
Author Date: 2021-11-09 18:06:22 UTC

MDEV-23766: fix by assert (Windows)

Author: Sergei Krivonos
Author Date: 2021-11-17 07:41:13 UTC

Improve LibFMT detection

Signed-off-by: Sergei Krivonos <sergei.krivonos@mariadb.com>

Author: Sergei Krivonos
Author Date: 2021-11-17 07:41:13 UTC

Improve LibFMT detection

Signed-off-by: Sergei Krivonos <sergei.krivonos@mariadb.com>

Author: Alexey Bychko
Author Date: 2021-10-21 08:51:35 UTC

MDEV-14938 make buildbot to include galera into bintars

this commit adds cmake module to process external directory with
galera binaries to include galera library and binaries into server
bintar.
it introduces new cmake variable EXT_GALERA_PATH representing relative
or absolute path to galera which can be either just files in directory,
compiled tree or unpacked tarball.
if EXT_GALERA_PATH is used, bintar name will include glibc version, for
example mariadb-10.2.42-linux-glibc_224-x86_64.tar.gz

Author: sachin
Author Date: 2021-01-29 11:59:14 UTC

MDEV-662 Lag Free Alter On Slave

This commit implements two phase binloggable ALTER.
When a new

  @@session.binlog_alter_two_phase = YES

ALTER query gets logged in two parts, the START ALTER
and the COMMIT or ROLLBACK ALTER. START Alter is written
in binlog as soon as at its handling necessary locks are
acquired for the table. The timing is such that any
concurrent DML:s that update the same table are either
committed, thus logged into binary log having done work
on the old version of the table, or will be queued for
execution on its new version.

The "COMPLETE" COMMIT or ROLLBACK ALTER are written after
the most of ALTER work is done. When its result is positive
COMMIT ALTER is written, otherwise when there were errors
since START ALTER, ROLLBACK ALTER is written. Replication
of two-phase binloggable ALTER is cross-version safe.
Specifically the OLD slave merely does not recognized the
start alter part, still memorizing its gtid.

Two phase logged ALTER is read from binlog by mysqlbinlog
to produce BINLOG 'string', where 'string' contains base64
encoded Query_log_event containing either the start part of
ALTER, or a completion part. The Query details can be
displayed with `-v` flag, similarly to ROW format events.
Notice, mysqlbinlog output containing parts of two-phase
binloggable ALTER is processable correctly only by
binlog_alter_two_phase server.

Thanks to all people involved into early discussion of the
feature including Kristian Nielsen, those who helped to
design, implement and test: Sergei Golubchik, Andrei Elkin,
Sujatha Sivakumar, Brandon Nesterenko, Alice Sherepa.

Author: Julius Goryavsky
Author Date: 2021-11-16 04:21:18 UTC

MDEV-26064: mariabackup SST fails when starting with --innodb-force-recovery

If the server is started with the --innodb-force-recovery argument
on the command line, then during SST this argument can be passed to
mariabackup only at the --prepare stage, and accordingly it must be
removed from the --mysqld-args list (and it is not should be passed
to mariabackup otherwise).

This commit fixes a flaw in the SST scripts and add a test that
checks the ability to run the joiner node in a configuration that
uses --innodb-force-recovery=1.

Author: Sergei Krivonos
Author Date: 2021-11-14 04:11:12 UTC

MDEV-27036: allow Json_writer_[array|object] from Json_writer

Author: Sergei Krivonos
Author Date: 2021-11-12 01:36:10 UTC

MDEV-16462: add assert to detect duplicated JSON keys

Author: Monty
Author Date: 2021-11-09 12:45:53 UTC

Another fixup, to be combined with the previous commits

Fixed a lot of inconsistencies in optimizer cost calculation. The main
objective was get cost calculation as similar (and accurate) as
possible to make different plans more comparable.

- Replaced constant 2.0 with new define TABLE_SCAN_SETUP_COST.
- Added RECORD_COPY_COST, the cost of finding the next row and copying
  it to record for table scans.
- Added INDEX_COPY_COST, the cost of finding the next key and copying it
  to record for index scans.
- Added INDEX_NEXT_FIND_COST, the cost of finding the next index entry and
  checking it against filter.
- Some scan cost estimates did not take into account
  TIME_FOR_COMPARE. Now all scan costs takes this into
  account. (main.show_explain)
- Fixed that we don't calculate TIME_FOR_COMPARE twice for some plans,
  like in optimize_straight_join() and greedy_search()
- JOIN_TAB::scan_time() did not take into account index only scans,
  which produced a wrong cost when index scan was used. Fixed by
  adding support for covering keys. Cached also the calculated values
  to avoid future calls during optimization phase.
- Fixed that most index cost calculation are done the same way and
  more close to 'range' calculations. The effects of this change are:
  - Cost of index scan is now lower than before, which causes some
   tests result to change. (innodb.innodb, main.show_explain)
  - Fixed the EQ_REF takes into account clustered and covered keys.
  - Ensured that index_scan_cost() ==
    range(scan_of_all_rows_in_table_using_one_range) +
    MULTI_RANGE_READ_INFO_CONST. One effect of this is that if there is
    choice of doing a full index scan and a range-index scan over almost
    the whole table then index scan will be preferred (no range-read
    setup cost).
  - Rowid filter setup cost and filter compare cost now takes into account
    fetching and checking the rowid (INDEX_NEXT_FIND_COST).
   (main.partition_pruning heap.heap_btree main.log_state)
- Introduced ha_scan_time() that takes into account the CPU cost of
  finding the next row and copying the row from the engine to
  'record'. This causes costs of table scan to slightly increase and
  some test to changed their plan from ALL to RANGE or ALL to ref.
  (innodb.innodb_mysql, main.select_pkeycache)
- Introduced ha_scan_and_compare_time() to is like ha_scan_time() but also
  adds the cost of checking the where clause (TIME_FOR_COMPARE).
- Cost of index scan was too low before compared to anything else.
  - Introduced ha_keyread_time(rows) that takes into account finding the
    next row and copying the key value to 'record' (INDEX_COPY_COST).
  - Introduced ha_key_scan_time() for calculating an index scan over all
     rows.
  - Added IDX_LOOKUP_COST to keyread_time() as a startup cost.
  - Added index_only_fetch_cost() as a convenience function to
    OPT_RANGE.
  - keyread_time() cost is slightly reduced to prefer shorter keys.
  - All of the above caused some index_merge combinations to be
    rejected because of cost (main.index_intersect)
- Added checking of the WHERE clause of the accepted rows to ROR costs in
  get_best_ror_intersect()
- Fixed bug in get_best_ror_intersect() where 'min_cost' was not updated,
  and the cost we compared with was not the one that was used.
- Removed '- 0.001' from 'join->best_read' and optimize_straight_join()
  to ensure that the 'Last_query_cost' status variable contains the same
  value as the one that was calculated by the optimizer.
- Extend get_range_limit_read_cost() to take into considering
  cost_for_index_read() if there where no quick keys. This will reduce
  the computed cost for ORDER BY with LIMIT in some cases.
  (main.innodb_ext_key)
- Added INDEX_NEXT_FIND_COST to Range_rowid_filter_cost_info::lookup_cost
  to account of the time to find and check the next key value against the
  container
- Changed 'JOIN_TAB:::scan_time() to take into consideration clustered and
  covered keys. The values are now cached and we only have to call this
  function once. Other calls are changed to use the cached values.
  Function renamed to JOIN_TAB::estimate_scan_time().

Other things:
- Added some 'if (thd->trace_started())' to speed up code
- Removed not used function Cost_estimate::is_zero()
- Simplified testing of HA_POS_ERROR in get_best_ror_intersect().
  (No cost changes)

Author: Tzachi Zidenberg
Author Date: 2020-07-05 12:21:43 UTC

MDEV-27008: aarch64: use compiler flag outline-atomics if available

outline-atomics compilation flag changes behaviour of builtin_atomics,
by adding runtime detection of LSE atomics. If these are supported, they
will be used. This gains LSE atomics use without hurting compatibility
with older aarch64 machines.

Backport of f843e215f3f640de377d8fd60e85bb372fb138fd to fix f502ccbcb5dfce29067434885a23db8d1bd5f134

Author: Sergei Krivonos
Author Date: 2021-11-04 15:32:05 UTC

MDEV-19308: bloom filter

Author: Alexey Yurchenko
Author Date: 2021-09-05 14:07:05 UTC

MENT-1626: JSON file interface to wsrep node state.

Integration with status reporter in wsrep-lib.

Status reporter reports changes in wsrep state and logged errors/
warnings to a json file which then can be read and interpreted by
an external monitoring tool.

Rationale: until the server is fully initialized it is unaccessible
by client and the only source of information is an error log which
is not machine-friendly. Since wsrep node can spend a very long time
in initialization phase (state transfer), it may be a very long time
that automatic tools can't easily monitor its liveness and progression.

New variable: wsrep_status_file specifies the output file name.
If not set, no file is created and no reporting is done.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Vladislav Vaintroub
Author Date: 2021-11-03 13:35:13 UTC

appveyor - do not use cygwin's broken bison

Author: Sergei Krivonos
Author Date: 2021-11-03 12:06:55 UTC

MDEV-18477: add generic_hash_fnv1a

Author: Sergei Krivonos
Author Date: 2021-11-03 12:06:55 UTC

MDEV-18477: add generic_hash_fnv1a

Author: Eugene
Author Date: 2021-10-11 08:19:57 UTC

MDEV-26668 CHECK TABLE fails to report schema mismatch

check_index_consistency(): always check clustered index against all
fields defined in MariaDB table. Also check for a possible FTS index
at the end.

innobase_match_index_columns(): the change is not a refactoring only.
Notice a while loop which was removed. Once it encounters a DATA_SYS
field it skips all the left fields! This is fixed now.

ha_innobase::open(): now returns an error when MariaDB and InnoDB table
definitions differ

innodb_strict_mode: affects the above check in a sandard way:
makes it an error or a warning.

MDEV-18186 test case was moved to another file because previous
if (have_debug) code in the test case makes SHOW CREATE TABLE behave
differently in release and debug modes.

Author: Sergei Krivonos
Author Date: 2021-10-29 18:01:28 UTC

MDEV-19129: Xcode compatibility update: deprecated vfork -> fork

Author: Vicențiu Ciorbaru
Author Date: 2021-07-26 20:43:16 UTC

MDEV-26242: Assertion `i >= 0' failed on setting default_tmp_storage_engine to 'DEFAULT' in 10.7

Locking / unlocking plugins is already handled by Sys_var_plugin::check
method. No need to do that in the specialized checking code, use
var->save_value instead to get the plugin handle.

Author: Nayuta Yanagisawa
Author Date: 2021-10-29 10:04:53 UTC

MDEV-26127 Assertion `err != DB_DUPLICATE_KEY' failed or InnoDB: Failing assertion: id != 0 on ALTER ... REBUILD PARTITION

Author: Nayuta Yanagisawa
Author Date: 2021-10-29 10:04:53 UTC

MDEV-26127 Assertion `err != DB_DUPLICATE_KEY' failed or InnoDB: Failing assertion: id != 0 on ALTER ... REBUILD PARTITION

Author: Daniel Black
Author Date: 2021-10-29 05:42:45 UTC

plugin query_reponse_time - link to atomic

Notable by:

main.plugin_vars w7 [ fail ]
        Test ended at 2021-10-28 17:43:48
CURRENT_TEST: main.plugin_vars
mysqltest: At line 70: query 'install plugin query_response_time soname 'query_response_time'' failed: 1126: Can't open shared library '/buildbot/aarch64-rhel-8/build/mysql-test/var/plugins/query_response_time.so' (errno: 11, undefined symbol: __aarch64_ldadd4_relax)

Fixes: f502ccbcb5dfce29067434885a23db8d1bd5f134

Author: Vladislav Vaintroub
Author Date: 2021-10-28 21:39:29 UTC

Fix broken clang-cl header files (declared xbegin instead of _xbegin)

clang-cl won't have lock elision then

Author: Sergei Krivonos
Author Date: 2021-10-28 15:35:33 UTC

MDEV-26929: fixed opt_trace test for --mysqld=--optimizer_trace=enabled=on

Author: Marko Mäkelä
Author Date: 2021-10-28 14:27:51 UTC

fixup! 758f7e579d01388262ed254559d180570cd446a4

Author: sjaakola
Author Date: 2021-10-21 11:49:51 UTC

MDEV-23328 Server hang due to Galera lock conflict resolution

Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-10-21 11:49:51 UTC

MDEV-23328 Server hang due to Galera lock conflict resolution

Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-10-21 11:49:51 UTC

MDEV-23328 Server hang due to Galera lock conflict resolution

Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-10-21 11:49:51 UTC

MDEV-23328 Server hang due to Galera lock conflict resolution

Mutex order violation when wsrep bf thread kills a conflicting trx,
the stack is

          wsrep_thd_LOCK()
          wsrep_kill_victim()
          lock_rec_other_has_conflicting()
          lock_clust_rec_read_check_and_lock()
          row_search_mvcc()
          ha_innobase::index_read()
          ha_innobase::rnd_pos()
          handler::ha_rnd_pos()
          handler::rnd_pos_by_record()
          handler::ha_rnd_pos_by_record()
          Rows_log_event::find_row()
          Update_rows_log_event::do_exec_row()
          Rows_log_event::do_apply_event()
          Log_event::apply_event()
          wsrep_apply_events()

and mutexes are taken in the order

          lock_sys->mutex -> victim_trx->mutex -> victim_thread->LOCK_thd_data

When a normal KILL statement is executed, the stack is

          innobase_kill_query()
          kill_handlerton()
          plugin_foreach_with_mask()
          ha_kill_query()
          THD::awake()
          kill_one_thread()

        and mutexes are

          victim_thread->LOCK_thd_data -> lock_sys->mutex -> victim_trx->mutex

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This also fixed unprotected calls to wsrep_thd_abort
that will use wsrep_abort_transaction. This is fixed
by holding THD::LOCK_thd_data while we abort transaction.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-09-15 06:16:44 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This patch also fixes mutex locking order and unprotected
THD member accesses on bf aborting case. We try to hold
THD::LOCK_thd_data during bf aborting. Only case where it
is not possible is at wsrep_abort_transaction before
call wsrep_innobase_kill_one_trx where we take InnoDB
mutexes first and then THD::LOCK_thd_data.

This will also fix possible race condition during
close_connection and while wsrep is disconnecting
connections.

Added wsrep_bf_kill_debug test case

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

force thread change and sleep on some critical points

Add more debug_sync points to wsrep_abort_transaction and
add two new test cases

(1) galera_to_error : After TOI error server does not accept new
statements
(2) galera_disconnect_debug: Disconnect victim connection while we
are debug_sync point on wsrep_abort_transaction

Fix regression on wsrep_close_connections by informing
wsrep_abort_transaction do we hold LOCK_thread_lock
or not and by introducing find_thread_by_id that
will not lock LOCK_thread_lock.

Remove unnecessary assigment

Add shutdown test with forced unireg_abort

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-10-20 09:37:14 UTC

MDEV-26647 (simple_password_check) Include password validation plugin information in the error message if the SQL statement is not satisfied password policy

Make the plugin reporting cause of the error.

Author: Jan Lindström
Author Date: 2021-10-21 08:33:16 UTC

WRONG

Author: Vicențiu Ciorbaru
Author Date: 2021-10-19 17:59:41 UTC

Update minimum cmake version to 3.1.0

Our cmake files actually depend on cmake 3.1.0 policies. See MDEV-26319,
and PR #1897, where policy CMP0054 NEW is needed to avoid dereferencing
strings that may be variable names.

Most of the code assumes this policy to be in place and additionally,
all our builders use cmake version >= 3.1.

Author: Sergei Krivonos
Author Date: 2021-10-19 13:07:35 UTC

MDEV-23766: moving JSON writer consistency check into Json_writer

Author: Nayuta Yanagisawa
Author Date: 2021-10-18 15:40:01 UTC

Add #error after ifdef HAVE_HANDLERSOCKET

Author: Vicențiu Ciorbaru
Author Date: 2021-10-13 10:13:27 UTC

MDEV-17964: Assertion `status == 0' failed in add_role_user_mapping_action

This happens upon CREATE USER and DROP ROLE.

The underlying problem is that our HASH implementation shuffles elements
around when performing an update or delete. This means that when doing a
scan through the HASH table by index, in search of elements to delete or
update one must restart the scan to make sure nothing is missed if at least
one delete / update happened.

More specifically, what happened in this case:
The hash has 131 element, DROP ROLE removes the element
[119]. Its [119]->next was element [129], so [129] is moved to [119].
Now we need to compact the hash, removing the last element [130]. It
gets one bit off its hash value and becomes element [2]. The existing
element [2] is moved to [129], and old [130] is moved to [2].

We cannot simply move [130] to [129] and make [2]->next=130, it won't
work if [2] is itself in the collision list and doesn't belong in [2].

The handle_grant_struct code assumed that it is safe to continue by only
reexamining the currently modified / deleted element index, but that is
not true.

Missing to delete an element in the hash triggered the assertion in
the test case. DROP ROLE would not clear all necessary role->role or
role->user mappings.

To fix the problem we ensure that the scan is restarted, only if an
element was deleted / updated, similar to how bubble-sort keeps sorting
until it finds no more elements to swap.

Author: Eric Herman
Author Date: 2021-09-03 04:38:54 UTC

MDEV-26221: DYNAMIC_ARRAY use size_t for sizes

https://jira.mariadb.org/browse/MDEV-26221
my_sys DYNAMIC_ARRAY and DYNAMIC_STRING inconsistancy

The DYNAMIC_STRING uses size_t for sizes, but DYNAMIC_ARRAY used uint.
This patch adjusts DYNAMIC_ARRAY to use size_t like DYNAMIC_STRING.

As the MY_DIR member number_of_files is copied from a DYNAMIC_ARRAY,
this is changed to be size_t.

As MY_TMPDIR members 'cur' and 'max' are copied from a DYNAMIC_ARRAY,
these are also changed to be size_t.

The lists of plugins and stored procedures use DYNAMIC_ARRAY,
but their APIs assume a size of 'uint'; these are unchanged.

Author: Marko Mäkelä
Author Date: 2021-10-14 09:06:50 UTC

MDEV-26826 The guess in buf_page_get_gen() may not be helpful

buf_page_get_low(), buf_page_get_gen(): Remove the parameter "guess".
We will have to access the buf_pool.page_hash anyway, in order to
acquire the page_hash_latch. So, knowing the block upfront might only
help us in case the block is behind a deep hash bucket.

We will also remove the build option WITH_INNODB_ROOT_GUESS,
any references to the preprocessor symbol BTR_CUR_ADAPT,
as well as the data fields
trx_undo_t::guess_block and btr_search_t::root_guess.

Author: Jan Lindström
Author Date: 2021-10-13 10:35:49 UTC

MDEV-24062: : Galera test failure on galera_var_replicate_myisam_on

10.6 version

Author: Daniel Black
Author Date: 2021-10-13 02:34:01 UTC

rocksdb: feature info incorrect when invalid zstd exists

It could be possible that an invalid rocksdb version exists like
zstd < 1.1.2. So don't just report the feature enabled because
it wants to be, only if its valid.

Author: Eugene
Author Date: 2021-10-12 17:00:09 UTC

fix C++20 !!! build failure: iterator concept was not fully implemented

Author: Sergei Krivonos
Author Date: 2021-10-06 08:31:08 UTC

Xcode compatibility update

Author: Alexey Botchkov
Author Date: 2021-10-05 07:58:54 UTC

MDEV-26540 Assertion `inited==RND' failed in handler::ha_rnd_end on DELETE.

We supposed to run the ha_rnd_end() when rnd_seq is on.
Also the ha_spider::info() should not raise the error in the
mysql_delete.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-10-01 09:00:29 UTC

MDEV-26459 Assertion `block_size <= 0xFFFFFFFFL' failed in calculate_block_sizes for 10.7 only

Limit default allocation block in tree of Unique class

Author: Alexey Botchkov
Author Date: 2021-09-30 11:57:32 UTC

MDEV-24383 SIGSEGV in heap_info from make_join_statistics on 2nd SP exec.

Since some optimizer switches lead to the changes in the
parser-generated structures, we have to reload cached SP-s after
the change.

Author: midenok
Author Date: 2021-05-27 17:54:19 UTC

MDEV-24176 Server crashes after insert in the table with virtual
column generated using date_format() and if()

When table is reopened from cache vcol_info contains stale
expression. We refresh expression via TABLE::vcol_fix_exprs() but
first we must prepare a proper context (Vcol_expr_context) and do some
preparations which meet some requirements:

1. fields must not be fixed, we unfix them with cleanup() via
cleanup_processor.

2. Item_ident must have cached_table cleared. Again, this is stale
data which was already freed. cached_table interferes with
find_field_in_tables().

We cannot clear cached_table directly in Item_ident::cleanup()
method. Lots of spaghetti logic depends on cached_table existence even
after cleanup() done.

3. If Item_ident has table_name non-NULL we trigger expr update. That
is done via Item_ident::check_vcol_func_processor() and
VCOL_TABLE_ALIAS flag.

(4-7) The below conditions are prepared by Vcol_expr_context and used in
both fix_session_expr_for_read() and TABLE::vcol_fix_exprs():

4. Any fix_expr() must be done on expr_arena as there may be new items
created. It was a bug in fix_session_expr_for_read(). It was just not
reproduced because there were no second refix. Now refix is done for
more cases so it does reproduce. Tests affected: vcol.binlog

5. Also name resolution context must be narrowed to the single table
with all crutches in place. Tests affected: vcol.update

6. sql_mode must be clean and not fail expr update.

sql_mode such as MODE_NO_BACKSLASH_ESCAPES, MODE_NO_ZERO_IN_DATE, etc
must not affect vcol expression update. If the table was created
successfully any further evaluation must not fail. Tests affected:
main.func_like

7. Warnings are disabled during expr update. It is assumed that these
warnings were printed before during initialization phase or later
during execution phase. Tests affected: vcol.wrong_arena

Dictionary: refresh, update and refix are the synonyms for
vcol_fix_exprs() or fix_session_expr_for_read() calls.

8. cleanup_excluding_fields_processor() removed. It was just a quick
hack to exclude wrongly working Item_field from processing. Now it
works due to correct execution environment (see next commit). Related
to MDEV-10355.

Author: Eugene
Author Date: 2021-08-09 18:54:12 UTC

MDEV-18543 IMPORT TABLESPACE fails after instant DROP COLUMN

ALTER TABLE IMPORT doesn't properly handle instant alter metadata.
This patch makes IMPORT read, parse and apply instant alter metadata at the
very beginning of operation. So, cases when source table has some metadata
and destination table doesn't have it now works fine.

DISCARD already removes instant metadata so importing normal table into
instant table worked fine before this patch.

decrypt_decompress(): decrypts and decompresses page if needed

handle_instant_metadata(): this should be the first thing to read source
table. Basically, it applies instant metadata to a destination
dict_table_t object. This is the first thing to read FSP flags so
all possible checks of it were moved to this function.

PageConverter::update_index_page(): it doesn't now read instant metadata.
This logic were moved into handle_instant_metadata()

row_import::match_flags(): this is a first part row_import::match_schema().
As a separate function it's used by handle_instant_metadata().

ANONYMOUS_VAR: this is a handy thing to use along with make_scope_exit()

full_crc32_import.test shows different results, because no
dict_table_close() and dict_table_open_on_id() happens.
Thus, SHOW CREATE TABLE shows a little bit older table definition.

Author: Vicențiu Ciorbaru
Author Date: 2021-09-28 15:14:11 UTC

MTR: loops should divide milliseconds by milliseconds

timeout is in seconds, sleeptime is in miliseconds. Otherwise we sleep
for 10 times longer than the timeout.

Author: midenok
Author Date: 2021-09-28 09:40:06 UTC

MDEV-22165 CONVERT TABLE: move in partition from existing table

Syntax for CONVERT TABLE

ALTER TABLE tbl_name CONVERT TABLE tbl_name TO PARTITION partition_name partition_spec

Examples:

    ALTER TABLE t1 CONVERT TABLE tp2 TO PARTITION p2 VALUES LESS THAN MAX_VALUE();

New ALTER_PARTITION_CONVERT_IN command for
fast_alter_partition_table() is done in alter_partition_convert_in()
function which basically does ha_rename_table().

Table structure and data check is basically the same as in EXCHANGE
PARTITION command. And these are done by
compare_table_with_partition() and check_table_data().

Atomic DDL is done by the scheme from MDEV-22166 (see the
corresponding commit message). The only differnce is that it also has
to drop source table frm and that is done by WFRM_DROP_CONVERTED_FROM.

Initial patch was done by Dmitry Shulga <dmitry.shulga@mariadb.com>

Author: sjaakola
Author Date: 2021-09-15 06:16:44 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This patch also fixes mutex locking order and unprotected
THD member accesses on bf aborting case. We try to hold
THD::LOCK_thd_data during bf aborting. Only case where it
is not possible is at wsrep_abort_transaction before
call wsrep_innobase_kill_one_trx where we take InnoDB
mutexes first and then THD::LOCK_thd_data.

This will also fix possible race condition during
close_connection and while wsrep is disconnecting
connections.

Added wsrep_bf_kill_debug test case

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-09-15 06:16:44 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This patch also fixes mutex locking order and unprotected
THD member accesses on bf aborting case. We try to hold
THD::LOCK_thd_data during bf aborting. Only case where it
is not possible is at wsrep_abort_transaction before
call wsrep_innobase_kill_one_trx where we take InnoDB
mutexes first and then THD::LOCK_thd_data.

This will also fix possible race condition during
close_connection and while wsrep is disconnecting
connections.

Added wsrep_bf_kill_debug test case

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-09-15 06:16:44 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This patch also fixes mutex locking order and unprotected
THD member accesses on bf aborting case. We try to hold
THD::LOCK_thd_data during bf aborting. Only case where it
is not possible is at wsrep_abort_transaction before
call wsrep_innobase_kill_one_trx where we take InnoDB
mutexes first and then THD::LOCK_thd_data.

This will also fix possible race condition during
close_connection and while wsrep is disconnecting
connections.

Added wsrep_bf_kill_debug test case

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: sjaakola
Author Date: 2021-09-15 06:16:44 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This patch also fixes mutex locking order and unprotected
THD member accesses on bf aborting case. We try to hold
THD::LOCK_thd_data during bf aborting. Only case where it
is not possible is at wsrep_abort_transaction before
call wsrep_innobase_kill_one_trx where we take InnoDB
mutexes first and then THD::LOCK_thd_data.

This will also fix possible race condition during
close_connection and while wsrep is disconnecting
connections.

Added wsrep_bf_kill_debug test case

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Marko Mäkelä
Author Date: 2021-09-23 08:21:51 UTC

MDEV-26445 fixup: Truncate while only holding the tablespace latch

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-09-23 08:00:24 UTC

Myke select number in explain uint (as it is in SELECT_LEX)

Author: rmfalves
Author Date: 2021-08-06 22:18:30 UTC

MDEV-25152 - Insert linebreaks in mysqldump --extended-insert (with tests update)

Closes: #1865

Author: Marko Mäkelä
Author Date: 2021-09-22 13:40:47 UTC

MDEV-26626 fixup: Do not advance checkpoint during startup

While the redo log is being resized in srv_start(),
we must not write checkpoint information to the old log.

Thanks to Matthias Leich for noticing this.

Author: Jan Lindström
Author Date: 2021-09-22 05:30:05 UTC

MDEV-26571 : galera_sr.GCF-627 MTR failed: Result length mismatch

Test changes only: do not output mysql.wsrep_streaming_log
contents.

Author: Eugene
Author Date: 2021-09-21 17:49:08 UTC

fix gcc and clang build

after upgrading to bison (GNU Bison) 3.8.1 build is broken like this:

build_gcc/sql/sql_yacc.cc:44772:1: error: label ‘yyexhaustedlab’ defined but not used [-Werror=unused-label]
44772 | yyexhaustedlab:
      | ^~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors

and:

build/sql/sql_yacc.cc:44772:1: error: unused label 'yyexhaustedlab' [-Werror,-Wunused-label]
yyexhaustedlab:
^~~~~~~~~~~~~~~
1 error generated.

fixing by disabling the diagnostics for sql_yacc.yy

Author: Alexey Bychko
Author Date: 2021-09-21 07:44:39 UTC

MDEV-23506 mariadb-connector-c-devel package from standard RHEL 8 repo conflicts with MariaDB's packages

added alternative name for MariaDB-devel package to replace
mariadb-connector-c-devel from RHEL 8 distribution.
this patch is for 10.3+ on RHEL/Centos 8

Author: Vladislav Vaintroub
Author Date: 2021-09-20 15:11:12 UTC

MDEV-26603 asynchronous redo log write

- Split log_write_up_to into 2 functions - the one initiating the write
and the write completion. In case of async write, the completion is called
by another thread.
- Allow asyncronous writes to log, if caller does not want to wait.
- Change srv_sync_log_buffer_in_background() so it does not wait for write
completion.

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-09-19 17:23:49 UTC

MDEV-22883: Assertion `!parse_error || lex.sphead == 0' failed in Table_triggers_list::check_n_load | SIGSEGV in strxmov

Abort execution of SHOW in case of fatal error.

Author: Alexey Bychko
Author Date: 2021-09-17 07:35:15 UTC

MDEV-26265 bintars depend on dynamic libraries

added macros to find static libraries.
for most libraries name is the same for static and dynamic versions.
if static lib name is different (like jemalloc with position independent
code) - name of static library may be specified directly without calling
macro

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-09-16 18:51:22 UTC

MDEV-22118: handler::update_auto_increment(): Assertion `nr != 0' failed. | SIGABRT in handler::update_auto_increment

Changed insert_id default and range to exclude 0 (because it is special value)

Author: sjaakola
Author Date: 2021-09-15 06:16:44 UTC

MDEV-25114 Crash: WSREP: invalid state ROLLED_BACK (FATAL)

This patch is the plan D variant for fixing potetial mutex locking
order exercised by BF aborting and KILL command execution.

In this approach, KILL command is replicated as TOI operation.
This guarantees total isolation for the KILL command execution
in the first node: there is no concurrent replication applying
and no concurrent DDL executing. Therefore there is no risk of
BF aborting to happen in parallel with KILL command execution
either. Potential mutex deadlocks between the different mutex
access paths with KILL command execution and BF aborting cannot
therefore happen.

TOI replication is used, in this approach, purely as means
to provide isolated KILL command execution in the first node.
KILL command should not (and must not) be applied in secondary
nodes. In this patch, we make this sure by skipping KILL
execution in secondary nodes, in applying phase, where we
bail out if applier thread is trying to execute KILL command.
This is effective, but skipping the applying of KILL command
could happen much earlier as well.

This patch also fixes mutex locking order and unprotected
THD member accesses on bf aborting case. We try to hold
THD::LOCK_thd_data during bf aborting. Only case where it
is not possible is at wsrep_abort_transaction before
call wsrep_innobase_kill_one_trx where we take InnoDB
mutexes first and then THD::LOCK_thd_data.

This will also fix possible race condition during
close_connection and while wsrep is disconnecting
connections.

Added wsrep_bf_kill_debug test case

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Author: Oleksandr "Sanja" Byelkin
Author Date: 2021-09-16 12:12:48 UTC

MDEV-22447: SIGSEGV in Item::val_uint on EXPLAIN

take into account possible NULL

Author: Eugene
Author Date: 2021-09-16 10:52:20 UTC

MDEV-26621 assertion failue "index->table->persistent_autoinc" in /storage/innobase/btr/btr0btr.cc during IMPORT

dict_index_t::clear_instant_alter(): when searhing for an AUTO_INCREMENT column
don't skip the beginning of the list because the field can be at the beginning of the list

Author: Sujatha Sivakumar
Author Date: 2021-08-24 11:45:04 UTC

MDEV-20122: With MASTER_USE_GTID=current_pos, slave's I/O thread only checks gtid_current_pos when thread is first started

Deprecate master_use_gtid=current_pos

Author: Sergey Petrunia
Author Date: 2021-09-14 17:18:07 UTC

Use JSON_NAME, not the "histogram_hb_v2" constant

Author: Nikita Malyavin
Author Date: 2021-09-14 09:11:37 UTC

set timeout=30

Author: Sergei Golubchik
Author Date: 2021-09-09 18:59:14 UTC

wip

Author: Alexey Botchkov
Author Date: 2021-09-08 06:33:17 UTC

MDEV-19275

STDCALL fixed.

Author: Daniel Black
Author Date: 2020-04-26 03:35:57 UTC

MDEV-25424: my_multi_malloc_large to use large pages

Change my_multi_malloc_large to use my_large_malloc.

The my_multi_malloc_large interface now takes a
final argument of a pointer to the size actually allocated.
This is needed because my_large_free(munmap) requires the
size.

Use size_t as its the native type for sizes of memory.

Return allocated amount with my_large_free, however first
adjusted with my_psi_key_free to ensure the right pointer
address is used.

Closes: #1513

Author: Sergey Petrunia
Author Date: 2021-09-07 07:38:36 UTC

Fix compilation on windows

Author: Jan Lindström
Author Date: 2021-09-06 12:34:54 UTC

Merge branch '10.6' into bb-10.6-ramesh

Author: Dmitry Shulga
Author Date: 2021-09-05 05:27:40 UTC

MDEV-22165: reverted changed in prep_alter_part_table

Author: Brandon Nesterenko
Author Date: 2021-09-03 16:39:52 UTC

MDEV-23746: Warnings about memory lost after using --flashback server option with binlog_row_image=minimal

Problem:
========

Where the combination of --flashback with binlog_row_image=’MINIMAL’
is invalid, and mariadb-binlog will exit early with error if reading
a binary log written to in MINIMAL binlog_row_image using
--flashback, memory leak errors are inappropriately printed as well.

Solution:
========
Suppress the memory leak errors upon the early termination of
mariadb-binlog.

Reviewed By:
============
<TODO>

Author: Vladislav Vaintroub
Author Date: 2021-09-03 01:17:55 UTC

tweaks

Author: Vladislav Vaintroub
Author Date: 2021-09-01 15:44:24 UTC

MDEV-4742 - remove leading zero handling, and cleanups.

Leading zeros added a single byte overhead per numeric string,
even when they were. Sorting leading zeros offers only for little value
(except determinism in sort). I decided to drop it for now, we can be
like ICU, which drops leading zeros, in numeric sorting,
even with IDENTICAL collation strength.

Also, disabled virtual stored columns (thus also indexes), on Serg's request
Hopefully it is temporarily, and will be reenabled soon, when everyone is
as happy with key generation algorithm as I am.

Author: Daniel Black
Author Date: 2021-08-31 04:31:18 UTC

MDEV-12933: compression chaos - make info static

Prevent compile error:

libmariadbd.a(plugin.c.o):/buildbot/amd64-ubuntu-1804-valgrind/build/plugin/provider_lz4/plugin.c:38: multiple definition of `info'
libmariadbd.a(plugin.c.o):/buildbot/amd64-ubuntu-1804-valgrind/build/plugin/provider_snappy/plugin.c:40: first defined here
collect2: error: ld returned 1 exit status
libmysqld/CMakeFiles/libmysqld.dir/build.make:102: recipe for target 'libmysqld/libmariadbd.so.19' failed
CMakeFiles/Makefile2:14235: recipe for target 'libmysqld/CMakeFiles/libmysqld.dir/all' failed
make[2]: *** [libmysqld/libmariadbd.so.19] Error 1
make[1]: *** [libmysqld/CMakeFiles/libmysqld.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 96%] Building CXX object sql/CMakeFiles/sql.dir/rpl_utility.cc.o
[ 96%] Building CXX object sql/CMakeFiles/sql.dir/rpl_utility_server.cc.o
[ 96%] Building CXX object sql/CMakeFiles/sql.dir/rpl_injector.cc.o
../libmariadbd.a(plugin.c.o):/buildbot/amd64-ubuntu-1804-valgrind/build/plugin/provider_lz4/plugin.c:38: multiple definition of `info'
../libmariadbd.a(plugin.c.o):/buildbot/amd64-ubuntu-1804-valgrind/build/plugin/provider_snappy/plugin.c:40: first defined here
collect2: error: ld returned 1 exit status

Author: midenok
Author Date: 2021-08-30 13:10:52 UTC

Vanilla cleanup: DBUG_EVALUATE_IF() macro replaced by shorter form DBUG_IF()

Author: Nayuta Yanagisawa
Author Date: 2021-08-26 02:17:34 UTC

MDEV-26247 Spider: Valid LEFT JOIN results in ERROR 1064