Author: Sujatha Sivakumar
Author Date: 2020-02-24 07:04:28 UTC

MDEV - 21469: Cleanup

Author: Alexander Barkov
Author Date: 2020-02-20 08:41:44 UTC

MDEV-21743 Split up SUPER privilege to smaller privileges

Author: seppo
Author Date: 2020-02-23 08:29:42 UTC

MDEV-21723 Async slave thread BF abort and replaying fixes (#1448)

If async replication slave thread conflicts with cluster replication,
then the async slave transaction should be BF aborted, and depending on the
state of async slave transaction execution, potentially also replayed.
There were problems in such BF abort implementation and the replaying was not
started.
This pull request contains fixes which make sure that if async slave thread is
marked to abort and replay, it will complete carry out the rollback and
release all locks and resources before starting the replaying. After replaying,
async slave transactions is treated as successful, so the slave thread will
continue as usual, handling next replication event.

There is also new mtr test: galera.galera_slave_replay, which stresses both a
certification failure for async slave thread and a successful BF abort
followed by replaying.

Author: Anel Husakovic
Author Date: 2019-12-31 17:02:54 UTC

MDEV-21374: When "--help --verbose" prints out configuration file paths, the --defaults-file option is not considered

* `--defaults-file` option is showed only in `--help --verbose` if
applied
* `--default-extra-file` is showing correctly now in `--help --verbose`,
previously it was treated as a directory with appended `my.cnf`

Author: varun
Author Date: 2020-02-22 20:04:58 UTC

Disable the packing and use memcmp() comparison when all fields are fixed-size fields

Author: varun
Author Date: 2020-02-22 19:42:03 UTC

Fix for MDEV-21791

Author: Marko Mäkelä
Author Date: 2020-02-22 15:32:45 UTC

MDEV-12353: Reduce log volume of page_cur_delete_rec()

mrec_ext_t: Introduce DELETE_ROW_FORMAT_REDUNDANT,
DELETE_ROW_FORMAT_DYNAMIC.

mtr_t::page_delete(): Write DELETE_ROW_FORMAT_REDUNDANT or
DELETE_ROW_FORMAT_DYNAMIC log records. We log the byte offset
of the preceding record, so that on recovery we can easily
find everything to update. For DELETE_ROW_FORMAT_DYNAMIC,
we must also write the header and data size of the record.

We will retain the physical logging for ROW_FORMAT=COMPRESSED pages.

page_zip_dir_balance_slot(): Renamed from page_dir_balance_slot(),
and specialized for ROW_FORMAT=COMPRESSED only.

page_rec_set_n_owned(), page_dir_slot_set_n_owned(),
page_dir_balance_slot(): New variants that do not write any log.

page_mem_free(): Take data_size, extra_size as parameters.
Always zerofill the record payload.

page_cur_delete_rec(): For other than ROW_FORMAT=COMPRESSED,
only write log by mtr_t::page_delete().

Author: Andrei Elkin
Author Date: 2020-02-21 20:31:12 UTC

MDEV-742 review notes address

Cleaned ha_prepare() from replication specifics introduction which is moved
into specialized method, being called later;
--skip-log-bin server start does not mess xa recovery with a new binlogged flag;
binlogging won`t happen thanks to proper binlog_hton method initialization.

Author: Jan Lindström
Author Date: 2020-02-22 11:07:31 UTC

Merge branch 'codership-10.2-slave_replay' into 10.2

Author: Sujatha Sivakumar
Author Date: 2020-02-21 17:04:01 UTC

MDEV-21469: Implement crash-safe logging of the user XA

Fixing build bot failures.

Addresses: Build issue and ASAN issues inside XA_Prepare_log_event
constructor.

Author: Michael Widenius
Author Date: 2020-02-12 16:31:09 UTC

Give a better error message if we can't load mysql.plugin table

Author: varun
Author Date: 2020-02-20 23:52:24 UTC

Adding DBUG_ASSERTS to the code

Author: Jan Lindström
Author Date: 2020-02-21 09:57:29 UTC

Disable galera_as_slave_gtid_replicate_do_db_cc because it crashes.

Author: Rasmus Johansson
Author Date: 2020-02-21 05:31:47 UTC

MDEV-21303 Make executables MariaDB named

Author: Nikita Malyavin
Author Date: 2020-02-21 03:49:14 UTC

invert period columns order in key

Author: midenok
Author Date: 2020-02-19 10:51:17 UTC

MDEV-16290 ALTER TABLE ... RENAME COLUMN syntax

The existing syntax for renaming a column uses "ALTER TABLE ... CHANGE"
command. This requires full column specification to rename the column.
This patch adds new syntax "ALTER TABLE ... RENAME COLUMN", which do not
expect users to provide full column specification. It means that the new
syntax would pick in-place or copy algorithm in the same way as that of
existing "ALTER TABLE ... CHANGE" command. The existing syntax
"ALTER TABLE ... CHANGE" will continue to work.

Syntax changes
==============

ALTER TABLE tbl_name
    [alter_specification [, alter_specification] ...]
    [partition_options]

Following is a new <alter_specification> added:

 | RENAME COLUMN <oldname> TO <newname>

Where <oldname> and <newname> are identifiers for old name and new
name of the column.

Related to: WL#10761

Author: Sergei Golubchik
Author Date: 2020-02-20 19:01:55 UTC

fixup! init to 0

Author: Oleksandr Byelkin
Author Date: 2020-02-19 16:50:30 UTC

MDEV-19650: Privilege bug on MariaDB 10.4

Also fixes:
MDEV-21487: Implement option for mysql_upgrade that allows root@localhost to be replaced
MDEV-21486: Implement option for mysql_install_db that allows root@localhost to be replaced

Add user mariadb.sys to be definer of user view
(and has right on underlying table global_priv for
required operation over global_priv
(SELECT,UPDATE,DELETE,INSERT,FILE))

Also changed definer of gis functions in case of creation,
but they work with any definer so upgrade script do not try
to push this change.

Author: Sergey Petrunia
Author Date: 2020-02-20 10:35:19 UTC

MDEV-21610 Different query results from 10.4.11 to 10.4.12

Part#2: cleanup:

In the part 1 of the fix, DS-MRR implementation would peek into
the JOIN_TAB to get the rowid filter from

  table->reginfo.join_tab->rowid_filter

This doesn't look good from code isolation standpoint (why should a
storage engine assume it is used through a JOIN_TAB?).

Fixed this by storing the 'un-pushed' rowid_filter in the DsMrr_impl
structure. The filter survives across multi_range_read_init() calls.

It is discarded when somebody calls index_end() or rnd_end() and cleans
up the DsMrr_impl.

Author: THIRUNARAYANAN BALATHANDAYUTHAPANI
Author Date: 2020-02-20 09:42:45 UTC

MDEV-15528 Punch holes when pages are freed

The following parameters are deprecated:

innodb-background-scrub-data-uncompressed
innodb-background-scrub-data-compressed
innodb-background-scrub-data-interval
innodb-background-scrub-data-check-interval

Removed scrubbing code completely and also removed
information_schema.innodb_tablespaces_scrubbing tables

Author: Nikita Malyavin
Author Date: 2020-02-20 08:42:27 UTC

MDEV-20494 ER_NOT_FORM_FILE or Assertion `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())' failed upon adding partition to a versioned table

Problem: fast_alter_partition_table tries to write the shadow frm with zero-length period constraint name. In case of explicit write-lock table reopens and shadow frm is read. Period constraint name is saved to frm in format "zero means two bytes more" (see extra2_write_str/extra2_read_len), so zero value here corrupts frm.

Solution: generate constraint names before fast_alter_partition_table is issued

Author: varun
Author Date: 2019-06-17 05:47:55 UTC

Addressing the performance regression because of the optimizer trace

Author: varun
Author Date: 2019-06-17 05:47:55 UTC

Addressing the performance regression because of the optimizer trace

Author: Sachin Setiya
Author Date: 2020-02-20 06:33:45 UTC

Cleanup

Author: Eugene
Author Date: 2020-02-19 12:44:33 UTC

MDEV-21774 Innodb, Windows : restore file sharing logic in Innodb

recv_sys_t opened redo log files along with log_sys_t. That's why I
removed file sharing logic from InnoDB
in 9ef2d29ff44de2013c95666a011b993e5c2e5674
But it was actually used to ensure that only one MariaDB instance
will touch the same InnoDB files.

os0file.cc: revert some changes done previously

mapped_file_t::map(): now has arguments read_only, nvme

file_io::open(): now has argument read_only

class file_os_io: make final

log_file_t::open(): now has argument read_only

Author: Vladislav Vaintroub
Author Date: 2020-02-07 21:12:35 UTC

MDEV-21534 - Improve innodb redo log group commit performance

Introduce special synchronization primitive group_commit_lock
for more efficient synchronization of redo log writing and flushing.

The goal is to reduce CPU consumption on log_write_up_to, to reduce
the spurious wakeups, and improve the throughput in write-intensive
benchmarks.

Author: varun
Author Date: 2020-02-19 20:24:07 UTC

Addressing initial review

Author: Sergei Golubchik
Author Date: 2020-02-19 19:46:29 UTC

Merge tag 'mariadb-5.5.67' into 10.0

Author: Sergei Golubchik
Author Date: 2020-02-19 19:46:29 UTC

Merge tag 'mariadb-5.5.67' into 10.0

Author: Sergey Petrunia
Author Date: 2020-02-19 18:27:38 UTC

MDEV-21628: Index condition pushdown condition ... not used with BKA

(Backport to 10.3)
Partitioning storage now supports MRR but doesn't support Index Condition
Pushdown (aka ICP). This causes counter-intuitive query plans for queries
that use BKA and conditions that depend on index fields:
- If the condition refers to other tables, BKA's variant of ICP is used
   to handle it.
- If the condition depends on this table only, the optimizer will try to
  use regular ICP for it, which will fail because the storage engine
  doesn't support ICP.

Make the optimizer be smarter in the second case: if we were not able to
use regular ICP, use BKA's variant of ICP..

Author: Eugene
Author Date: 2020-01-11 19:05:28 UTC

MDEV-14425 deprecate and ignore innodb_log_files_in_group

Now there can be only one log file instead of several which
logically work as a single file.

Possible names of redo log files: ib_logfile0,
ib_logfile101 (for just created one)

innodb_log_fiels_in_group: value of this variable is not used
by InnoDB. Possible values are still 1..100, to not break upgrade

LOG_FILE_NAME: add constant of value "ib_logfile0"
LOG_FILE_NAME_PREFIX: add constant of value "ib_logfile"

get_log_file_path(): convenience function that returns full
path of a redo log file

SRV_N_LOG_FILES_MAX: removed

srv_n_log_files: we can't remove this for compatibility reasons,
but now server doesn't use this variable

log_sys_t::file::fd: now just one, not std::vector

log_sys_t::log_capacity: removed word 'group'

find_and_check_log_file(): part of logic from huge srv_start()
moved here

recv_sys_t::files: file descriptors of redo log files.
There can be several of those in case we're upgrading
from older MariaDB version.

recv_sys_t::remove_extra_log_files: whether to remove
ib_logfile{1,2,3...} after successfull upgrade.

recv_sys_t::read(): open if needed and read from one
of several log files

recv_sys_t::files_size(): open if needed and return files count

redo_file_sizes_are_correct(): check that redo log files
sizes are equal. Just to log an error for a user.
Corresponding check was moved from srv0start.cc

namespace deprecated: put all deprecated variables here to
prevent usage of it by us, developers

Author: Daniel Black
Author Date: 2019-10-15 04:03:53 UTC

MDEV-16662: cmake remove empty INSTALL_DEBUG_TARGET

No adverse effects since this was made a null function in
6b53f9d781cc19cbec96c3eb048e6407021685a2.

This function had the last remaining cmake CMP0026 violation.

Author: Nikita Malyavin
Author Date: 2019-03-04 09:09:30 UTC

Assertion `old_part_id == m_last_part' failed in ha_partition::update_row or `part_id == m_last_part' in ha_partition::delete_row upon UPDATE/DELETE after dropping versioning

PRIMARY KEY change hadn't been treated as partition reorganization in case of partitioning by KEY() (without parameters).

* set `*partition_changed= true` in the described case.
* since add/drop system versioning does not affect alter_info->key_list, it required separate attention

Author: varun
Author Date: 2020-02-17 19:37:50 UTC

Fixing buildbot failures

Author: Vladislav Vaintroub
Author Date: 2020-02-17 18:56:50 UTC

MDEV-15053 : make buf_buddy_stat_t use atomics

Author: Daniel Black
Author Date: 2019-10-15 04:03:53 UTC

MDEV-16662: cmake remove empty INSTALL_DEBUG_TARGET

No adverse effects since this was made a null function in
6b53f9d781cc19cbec96c3eb048e6407021685a2.

This function had the last remaining cmake CMP0026 violation.

Author: midenok
Author Date: 2020-02-16 19:05:00 UTC

MDEV-17554 style fixes

Author: Vlad Lesin
Author Date: 2019-11-27 22:16:22 UTC

Add some extra logging to catch the case when page is written with wrong offset.

Author: Vlad Lesin
Author Date: 2019-11-27 22:16:22 UTC

Add some extra logging to catch the case when page is written with wrong offset.

Author: midenok
Author Date: 2020-02-15 12:18:54 UTC

Compilation fix

Author: Daniel Bartholomew
Author Date: 2020-02-14 15:46:08 UTC

bump the VERSION

Author: Jan Lindström
Author Date: 2020-02-14 06:20:52 UTC

Fix Galera test galera_as_slave_ctas.

Author: Sergei Golubchik
Author Date: 2020-02-13 16:09:02 UTC

alpha -> beta

Author: Vicențiu Ciorbaru
Author Date: 2020-02-13 11:23:03 UTC

iMore rdiff upd

Author: Eugene
Author Date: 2020-02-07 19:43:52 UTC

MDEV-21669 InnoDB: Table ... contains <n> indexes inside InnoDB, which is different from the number of indexes <n> defined in the MariaDB

compare_keys_but_name(): do not use KEY_PART_INFO::field for Field::is_equal().
Following the logic of that code we need to compare fields of a table. But
KEY_PART_INFO::field sometimes (when key part is shorter than table field) is a
different field. In that case Field::is_equal() returns incorrect result and
problems occur.

KEY_PART_INFO::field may become some strange field in
open_frm_error open_table_from_share(). I think this is an incorrect logic,
some tecnhical debt. I'm not fixing it right now, because I don't have time.
But I'm making Field::field_length a const class member. Then, the only fishy
code which changed that field requires now a const_cast<>. I'm bringing
attention to that code with it. This change should not affect logic of the
program in any way.

Author: Oleksandr Byelkin
Author Date: 2020-02-12 13:30:31 UTC

MDEV-21646: Failure to compile my_addr_resolve.c with binutils-2.34

Emulate older API

Author: Jan Lindström
Author Date: 2020-02-12 08:32:30 UTC

MDEV-21556 : galera.lp1376747-4 MTR failed: Result length mismatch

Add proper wait condition instead of sleeps.

Author: Jan Lindström
Author Date: 2020-02-11 08:25:07 UTC

Fix typo.

Author: Robert Bindar
Author Date: 2020-02-10 09:16:12 UTC

mdev-20647

Author: varun
Author Date: 2020-02-09 15:02:03 UTC

Fixing optimizer trace tests failing

Author: Jan Lindström
Author Date: 2020-02-07 12:18:17 UTC

MDEV-21667 : Galera test failure on MW-336

Problem seems to be the fact that we did not enforce correct
applier thread numbers after every command that effects them.
Test changes only.

Author: Jan Lindström
Author Date: 2020-02-07 12:18:17 UTC

MDEV-21667 : Galera test failure on MW-336

Problem seems to be the fact that we did not enforce correct
applier thread numbers after every command that effects them.
Test changes only.

Author: Rasmus Johansson
Author Date: 2020-02-05 14:41:35 UTC

MDEV-21303 Make executables MariaDB named

Author: Elena Stepanova
Author Date: 2020-02-06 19:46:25 UTC

MDEV-21664 Add opt files for have_innodb_Xk.inc

Author: Andrei Elkin
Author Date: 2020-02-04 20:08:45 UTC

MDEV-21037 assert cache_mngr->trx_cache.empty() in binlog_close_connection

was caused by missed proper cleanup of connection`s binlog state under
union of conditions: user LOCK on table with trigger + MSTA + SAVEPOINT.

The issue is fixed with refined decision to reset
THD::binlog_table_maps at statement end is true regardless of the
status of `pending` event in binlog cache.

Author: Sergey Vojtovich
Author Date: 2020-01-21 13:21:20 UTC

InnoDB redo log IO methods

normal - conventional IO using read()/write()/fdatasync()
mmap - memory mapped IO, expected to be faster but less secure

Author: Alexey Botchkov
Author Date: 2020-02-04 13:12:55 UTC

MDEV-17399 Add support for JSON_TABLE.

Syntax for the JSON_TABLE added.

Author: Jan Lindström
Author Date: 2020-02-04 07:00:36 UTC

MDEV-20625 : MariaDB asserting when enabling wsrep_on

We need to release global system variables mutex before
doing wsrep_init to avoid race with next show status and
we need to save wsrep_on value as it is changed on wsrep_init.
Added test case.

Author: Anel Husakovic
Author Date: 2020-02-03 18:54:01 UTC

Cleanup after review

Author: Jan Lindström
Author Date: 2020-02-03 12:34:31 UTC

Merge branch 'codership-10.5-galera_sync_wait_upto' into 10.5

Author: Andy Fiddaman
Author Date: 2020-01-14 11:44:39 UTC

MDEV-21476 auth_socket: add support for illumos with getpeerucred()

Fixes #1434

Author: Sachin Setiya
Author Date: 2019-10-09 15:46:31 UTC

MDEV-20001 Potential dangerous regression: INSERT INTO >=100 rows fail for myisam table with HASH indexes

Problem:-

So the issue is when we do bulk insert with rows
> MI_MIN_ROWS_TO_DISABLE_INDEXES(100) , We try to disable the indexes to
speedup insert. But current logic also disables the long unique indexes.

Solution:- In ha_myisam::start_bulk_insert if we find long hash index
(HA_KEY_ALG_LONG_HASH) we will not disable the index.

This commit also refactors the mi_disable_indexes_for_rebuild function,
Since this is function is called at only one place, it is inlined into
start_bulk_insert

mi_clear_key_active is added into myisamdef.h because now it is also used
in ha_myisam.cc file.

(Same is done for Aria Storage engine)

Author: midenok
Author Date: 2020-01-31 21:05:32 UTC

MDEV-20494 ER_NOT_FORM_FILE or assertion upon adding partition to period table

Clone of Virtual_column_info includes name on mysql_prepare_alter_table().

Author: Eugene
Author Date: 2020-01-31 16:54:39 UTC

how to check errors from object construction

Author: Eugene
Author Date: 2020-01-30 05:11:06 UTC

introduce HASH_REPLACE() for hash_table_t

HASH_REPLACE(): allows to not travel through linked list twice
when HASH_INSERT() happens right after HASH_DELETE()

Author: midenok
Author Date: 2020-01-31 10:32:24 UTC

MDEV-21195 INSERT chooses wrong partition for RANGE partitioning by DECIMAL column

Use FLOOR rounding for DECIMAL_RESULT item_expr in partition function.

Author: Vladislav Vaintroub
Author Date: 2020-01-30 20:11:24 UTC

MDEV-16308 : protocol messed up sporadically

Context involves semicolon batching, and the error starts 10.2
No reproducible examples were made yet, but TCP trace suggests
multiple packets that are "squeezed" together (e.g overlong OK packet
that has a trailer which is belongs to another packet)

Remove thd->get_stmt_da()->set_skip_flush() when processing a batch.
skip_flush stems from the COM_MULTI code, which was checked in during
10.2 (and is never used)

The preliminary "fix" is by suspicion, to be evaluated by volunteers.

Author: Jan Lindström
Author Date: 2020-01-30 10:20:54 UTC

Fix compile error on Windows.

Author: Monty
Author Date: 2020-01-29 14:41:04 UTC

Fixed compiler warnings from gcc 7.4.1

- Fixed possible error in rocksdb/rdb_datadic.cc

Author: Sujatha Sivakumar
Author Date: 2020-01-23 10:47:55 UTC

MDEV-20923:UBSAN: member access within address … which does not point to an object of type 'xid_count_per_binlog'

Problem:
-------
Accessing a member within 'xid_count_per_binlog' structure results in
following error when 'UBSAN' is enabled.

member access within address 0xXXX which does not point to an object of type
'xid_count_per_binlog'

Analysis:
---------
The problem appears to be that no constructor for 'xid_count_per_binlog' is
being called, and thus the vtable will not be initialized.

Fix:
---
Defined a parameterized constructor for 'xid_count_per_binlog' class.

Author: Marko Mäkelä
Author Date: 2020-01-29 10:42:45 UTC

Cleanup: More logical names and member functions

recv_sys_t::free_corrupted_page(): Renamed from recv_recover_corrupt_page().

recv_sys_t::is_memory_exhausted(): Renamed from recv_sys_heap_check().

recv_sys_t::max_blocks(): Accessor for max_log_blocks.

recv_sys_t::blocks: Buffer pool blocks reserved for redo log records.

For the debug functions related to recv_sys_t::alloc(), use the
logical name free().

Rename some variables 'store_to_hash' to 'store', because recv_sys.pages
is not a hash table.

Author: Vicențiu Ciorbaru
Author Date: 2020-01-28 11:52:47 UTC

MDEV-14330: move tokudb manpages to right packages

Move tokuftdump and tokuft_logprint man pages to storage/tokudb.

The man pages are now part of tokudb-engine cmake component. This change
is mostly for RPM & DEB based packaging generated through CMake & CPack.

Debian upstream already handles this change via the custom scripts in debian/

Author: Vicențiu Ciorbaru
Author Date: 2020-01-28 16:03:13 UTC

Move mysqltest.1 man page to appropriate package

Author: Vicentiu Ciorbaru
Author Date: 2020-01-28 15:59:47 UTC

Move mysqltest.1 man page to test package

Author: Oleksandr Byelkin
Author Date: 2020-01-27 20:50:16 UTC

MDEV-20632: Recursive CTE cycle detection using CYCLE clause

Added CYCLE clause to recursive CTE.

Author: Alexey Botchkov
Author Date: 2019-03-30 21:47:28 UTC

MDEV-7974 XA PREPARE-d transactions survive disconnect/server restart.

trans_detach() implemented to survive disconnect.
XA_prepare_log_event added to reflect XA PREPARE in the binlog.

MDEV-7974 review/contribution:
gtid record of XA-prepare should no be merged into the replicated
trx. This patch redirects it into the default mysql.gtid_slave_pos table. A test is added.

MDEV-7974 review/contribution:
part2 (sources code) gtid record of XA-prepare should no be merged into the replicated
trx...

MDEV-7974 XA trnasactions.

XID_STATE::registered_for_binlog is replaced with rw_trans flag
in ha_prepare().

MDEV-7974 XA transactions.

FLUSH TABLES WITH READ LOCK now blocks the XA COMMIT/ROLLBACK/PREPARE
statements as they write to the binlog/Innodb redo.

MDEV-7974 XA transactions.

tokudb_maria.xa test fixed.

MDEV-7974: record_gtid for XA prepared is fixed to exec the gtid statement after the xa trx got prepared and logged.

MDEV-7974

Almost ultimate merge with the XA refactoring (to cover is_binlogged).
todo: test out the master side; etc.

MDEV-7974

Cleanup/refactoring for is_binlogged and applier_reset_xa_trans.
todo: fix the slave side allocation and test.

MDEV-7974

FTWRL && XA ROLLBACK, xa.test is refined/corrected.

MDEV-7974: slave side groupping with XA_prepare, mixed engines and few mtr tests update with 5.7 are added.

MDEV-7974: more to STANDLONE groupping: recovery and io thread retransmit rel.

MDEV-7974: hton ref_count and more tests.

ref_count may be found 1 at shutdown caused by "native" trx detach
which makes THD::ha_data[].ha_ptr zero to hook away the problematic counter decr.
The test part includes rpl_xa_gaps (todo: turn RC on slave whenever ROW format).

MDEV-7974: slave temp failure retry related.

XA state cleanup is completed;
todo: revise/refactor/catch-up-with-upstream/ reattachment mechanism.

MDEV-742 (intermediate commit)

typ == XID_EVENT || typ == XA_PREPARE_LOG_EVENT augmentation.

MDEV-742

A number of cleanup, testing, optimistic fixes and xa prepare logging fixing (still to test out)

MDEV-742

Waiting for prior-to-commit at XA-*prepare*; tested against rpl_optimistic_xa part I; log-bin=ON,OFF tried to OK;

TODO: where is signal-to-following-commit from XA-prepare?

MDEV-742

Fixed thr assignment to XA-terminate parts; proved wakeup consequent commit OK; sketched the 2nd part of rpl_optimistic_xa

MDEV-742

Completed the 2nd part of rpl_optimistic_xa;
added its log-slave-updates=0 version.

Added tc->log,unlog for XA; tested with Binlog checkpoint.

rocksdb xa detaches; pseudo slave tested.

refinement in trans_cant_save_rb; todo added for applier_reset_xa_trans().

binlog checkpoint test is added

manual merge after rebase to compile

hton close connection fixed; tested manually.

More post rebase corrections invoked by rpl_xa_survive_disconnect.

few binlog suite and related tests cosmetic fixes.

server xid corrected

MDEV-742

Removal from execution de-,at-tach_native_trx(); THD_TRANS reset enforce in ha_prepare() to follow commit,rollback patterns.

MDEV-742 - removed already relocated test

MDEV-742 - cleaned off ha_ptr_backup completely.

MDEV-742 - finally removed unused replace_trx_in_thd:s.

MDEV-742 - more cleanup is sql.

Gtid_log_event::xid formatID size fixed.
+std::atomic<int32_t> XID_cache_element_para::cnt

MDEV-742 - Part X: parallel xid hash refinement.

A sequence of xid-namesake XA transactions can be logged on master as
a serious of binlog event groups of

XA_prepare(xid)
XA_compete(xid)
...
XA_compete(xid)
XA_prepare(xid)

where `_complete` stands for COMMIT or ROLLBACK.
Optimistic parallel slave scheduler may assign all of the sequence to
one Worker which ensures no concurrent (optimistic) execution of two
namasake XA:s.

To faciliate that there is a parallel slave local xid hash, one per
replication source. A xid-keyed record is placed there when XA-start
is seen by the Driver thread. Every XA-start will increment the record
usage counter, initially 1. The counter drops by 1 per each
XA-complete by the assigned Worker. The record is evicted when the
counter reaches zero to indicate there are no more XA-start in its
queue for this xid.

A test file is added.

MDEV-742: few test result file update. Interrupted at unlog_xid.

MDEV-742: fixed missed unlog by xa-rollback on unprepared xa.

MDEV-742: cosmetics in 2 tests.

Author: mkaruza
Author Date: 2019-04-01 11:23:05 UTC

Galera GTID support

Support for galera GTID consistency thru cluster. All nodes in cluster
should have same GTID for replicated events which are originating from cluster.
Cluster originating commands need to contain sequential WSREP GTID seqno
Ignore manual setting of gtid_seq_no=X.

In master-slave scenario where master is non galera node replicated GTID is
replicated and is preserved in all nodes.

To have this - domain_id, server_id and seqnos should be same on all nodes.
Node which bootstraps the cluster, to achieve this, sends domain_id and
server_id to other nodes and this combination is used to write GTID for events
that are replicated inside cluster.

Cluster nodes that are executing non replicated events are going to have different
GTID than replicated ones, difference will be visible in domain part of gtid.

With wsrep_gtid_domain_id you can set domain_id for WSREP cluster.

Functions WSREP_LAST_WRITTEN_GTID, WSREP_LAST_SEEN_GTID and
WSREP_SYNC_WAIT_UPTO_GTID now works with "native" GTID format.

Fixed galera tests to reflect this chances.

Add variable to manually update WSREP GTID seqno in cluster

Add variable to manipulate and change WSREP GTID seqno. Next command
originating from cluster and on same thread will have set seqno and
cluster should change their internal counter to it's value.
Behavior is same as using @@gtid_seq_no for non WSREP transaction.

Author: Nikita Malyavin
Author Date: 2020-01-21 09:48:47 UTC

Period: add versioning combination

Author: Nikita Malyavin
Author Date: 2020-01-13 07:53:40 UTC

add frm corruption check for extra2.application_period

Author: Sergey Petrunia
Author Date: 2020-01-19 21:06:51 UTC

MDEV-21383: Possible range plan is not used under certain conditions

[Variant 2 of the fix: collect the attached conditions]

Problem:
make_join_select() has a section of code which starts with
 "We plan to scan all rows. Check again if we should use an index."

the code in that section will [unnecessarily] re-run the range
optimizer using this condition:

  condition_attached_to_current_table AND current_table's_ON_expr

Note that the original invocation of range optimizer in
make_join_statistics was done using the whole select's WHERE condition.
Taking the whole select's WHERE condition and using multiple-equalities
allowed the range optimizer to infer more range restrictions.

The fix:
- Do range optimization using a condition that is an AND of this table's
condition and all of the previous tables' conditions.
- Also, fix the range optimizer to prefer SEL_ARGs with type=KEY_RANGE
over SEL_ARGS with type=MAYBE_KEY, regardless of the key part.
Computing
key_and(
  SEL_ARG(type=MAYBE_KEY key_part=1),
  SEL_ARG(type=KEY_RANGE, key_part=2)
)
will now produce the SEL_ARG with type=KEY_RANGE.

Author: Sergey Vojtovich
Author Date: 2020-01-23 21:49:42 UTC

discover_check_version() -> rebind()

Author: Alexey Botchkov
Author Date: 2020-01-23 13:34:24 UTC

MENT-464 ASAN MTR quick test - some failures to be investigated.

The ASAN retuns very small frame size when ran with the ASAN,
so we have to rely on it's smallest possible value.

Author: Sujatha Sivakumar
Author Date: 2020-01-22 10:50:05 UTC

MDEV-21490: binlog tests fail with valgrind: Conditional jump or move depends on uninitialised value in sql_ex_info::init

Problem:
=======
P1)
==19140== Conditional jump or move depends on uninitialised value(s)
==19140== at 0xE2BC02: sql_ex_info::init(char const*, char const*, bool)
(log_event.cc:3083)
code:
----
    return ((cached_new_format != -1) ? cached_new_format :
      (cached_new_format=(field_term_len > 1 ||
        enclosed_len > 1 ||
        line_term_len > 1 || line_start_len > 1 ||
        escaped_len > 1)));

All the above variables are not initialized.

P2)
==21819== Conditional jump or move depends on uninitialised value(s)
==21819== at 0x9509FB: Rows_log_event::Rows_log_event(char const*, unsigned
               int, Format_description_log_event const*) (log_event.cc:9571)

Code: Uninitialized values is reported for 'var_header_len' variable.
----
    if (var_header_len < 2 ||
        event_len < static_cast<unsigned int>(var_header_len +
          (post_start - buf)))

P3) ==28175== Conditional jump or move depends on uninitialised value(s)
==28175== at 0x4C3317C: strnlen (vg_replace_strmem.c:428)
...
==28175== by 0xF9894F: my_snprintf (my_vsnprintf.c:716)
==28175== by 0x955905: Table_map_log_event::pack_info(Protocol*) (log_event.cc:11553)
code:'m_table_id' is uninitialized.
----
void Table_map_log_event::pack_info(Protocol *protocol)
{
    char buf[256];
    size_t bytes= my_snprintf(buf, sizeof(buf),
                                 "table_id: %lu (%s.%s)",
                              m_table_id, m_dbnam, m_tblnam);

Fix:
===
P1 - Fix)

Initialize cached_new_format,field_term_len, enclosed_len, line_term_len,
line_start_len, escaped_len members in default constructor.

P2 - Fix)
"var_header_len" is initialized by reading the event buffer. In case of an
invalid event the buffer will contain invalid data. Hence added a check to
validate the event data. If event_len is smaller than valid header length
return immediately.

P3 - Fix)
'm_table_id' within Table_map_log_event is initialized by reading data from
the event buffer. Use 'VALIDATE_BYTES_READ' macro to validate the current
state of the buffer. If it is invalid return immediately.

Author: Sujatha Sivakumar
Author Date: 2020-01-22 10:50:05 UTC

MDEV-21490: binlog tests fail with valgrind: Conditional jump or move depends on uninitialised value in sql_ex_info::init

Problem:
=======
P1)
==19140== Conditional jump or move depends on uninitialised value(s)
==19140== at 0xE2BC02: sql_ex_info::init(char const*, char const*, bool)
(log_event.cc:3083)
code:
----
    return ((cached_new_format != -1) ? cached_new_format :
      (cached_new_format=(field_term_len > 1 ||
        enclosed_len > 1 ||
        line_term_len > 1 || line_start_len > 1 ||
        escaped_len > 1)));

All the above variables are not initialized.

P2)
==21819== Conditional jump or move depends on uninitialised value(s)
==21819== at 0x9509FB: Rows_log_event::Rows_log_event(char const*, unsigned
               int, Format_description_log_event const*) (log_event.cc:9571)

Code: Uninitialized values is reported for 'var_header_len' variable.
----
    if (var_header_len < 2 ||
        event_len < static_cast<unsigned int>(var_header_len +
          (post_start - buf)))

P3) ==28175== Conditional jump or move depends on uninitialised value(s)
==28175== at 0x4C3317C: strnlen (vg_replace_strmem.c:428)
...
==28175== by 0xF9894F: my_snprintf (my_vsnprintf.c:716)
==28175== by 0x955905: Table_map_log_event::pack_info(Protocol*) (log_event.cc:11553)
code:'m_table_id' is uninitialized.
----
void Table_map_log_event::pack_info(Protocol *protocol)
{
    char buf[256];
    size_t bytes= my_snprintf(buf, sizeof(buf),
                                 "table_id: %lu (%s.%s)",
                              m_table_id, m_dbnam, m_tblnam);

Fix:
===
P1 - Fix)

Initialize cached_new_format,field_term_len, enclosed_len, line_term_len,
line_start_len, escaped_len members in default constructor.

P2 - Fix)
"var_header_len" is initialized by reading the event buffer. In case of an
invalid event the buffer will contain invalid data. Hence added a check to
validate the event data. If event_len is smaller than valid header length
return immediately.

P3 - Fix)
'm_table_id' within Table_map_log_event is initialized by reading data from
the event buffer. Use 'VALIDATE_BYTES_READ' macro to validate the current
state of the buffer. If it is invalid return immediately.

Author: Sergei Golubchik
Author Date: 2020-01-22 17:18:22 UTC

C/C ssl unit test fixes

* enable ssl tests (by specifying CERT_PATH)
* generate combined and password-protected client certs
* generate server-cert fingerprint
* generate cacert ${hash}.0 as openssl expects from a system CA dir
* add system CA test to ssl_7937.test too
* add --ssl-verify-server-cert for a wrong hostname

Author: Vlad Lesin
Author Date: 2020-01-22 18:13:14 UTC

MDEV-14183: aria_pack segfaults in compress_maria_file

Post-push fix. aria_pack_mdev14183 test is unstable.

The fix is the following:
1. Disable the test for embedded server.
2. Create non-"transactional" Aria table in the test, as aria_pack does not
support "transactional" Aria tables.

Author: Sujatha Sivakumar
Author Date: 2020-01-22 10:50:05 UTC

MDEV-21490: binlog tests fail with valgrind: Conditional jump or move depends on uninitialised value in sql_ex_info::init

Problem:
=======
P1)
==19140== Conditional jump or move depends on uninitialised value(s)
==19140== at 0xE2BC02: sql_ex_info::init(char const*, char const*, bool)
(log_event.cc:3083)
code:
----
    return ((cached_new_format != -1) ? cached_new_format :
      (cached_new_format=(field_term_len > 1 ||
        enclosed_len > 1 ||
        line_term_len > 1 || line_start_len > 1 ||
        escaped_len > 1)));

All the above variables are not initialized.

P2)
==21819== Conditional jump or move depends on uninitialised value(s)
==21819== at 0x9509FB: Rows_log_event::Rows_log_event(char const*, unsigned
               int, Format_description_log_event const*) (log_event.cc:9571)

Code: Uninitialized values is reported for 'var_header_len' variable.
----
    if (var_header_len < 2 ||
        event_len < static_cast<unsigned int>(var_header_len +
          (post_start - buf)))

P3) ==28175== Conditional jump or move depends on uninitialised value(s)
==28175== at 0x4C3317C: strnlen (vg_replace_strmem.c:428)
...
==28175== by 0xF9894F: my_snprintf (my_vsnprintf.c:716)
==28175== by 0x955905: Table_map_log_event::pack_info(Protocol*) (log_event.cc:11553)
code:'m_table_id' is uninitialized.
----
void Table_map_log_event::pack_info(Protocol *protocol)
{
    char buf[256];
    size_t bytes= my_snprintf(buf, sizeof(buf),
                                 "table_id: %lu (%s.%s)",
                              m_table_id, m_dbnam, m_tblnam);

Fix:
===
P1 - Fix)

Initialize cached_new_format,field_term_len, enclosed_len, line_term_len,
line_start_len, escaped_len members in default constructor.

P2 - Fix)
"var_header_len" is initialized by reading the event buffer. In case of an
invalid event the buffer will contain invalid data. Hence added a check to
validate the event data. If event_len is smaller than valid header length
return immediately.

P3 - Fix)
'm_table_id' within Table_map_log_event is initialized by reading data from
the event buffer. Use 'VALIDATE_BYTES_READ' macro to validate the current
state of the buffer. If it is invalid return immediately.

Author: Jan Lindström
Author Date: 2020-01-22 09:58:50 UTC

MDEV-17571 : Make systemd timeout behavior more compatible with long Galera SSTs

This is 10.4 version.

Idea is to create monitor thread for both donor and joiner that will
periodically if needed extend systemd timeout while SST is being
processed. In 10.4 actual SST is executed by running SST script
and exchanging messages on pipe using blocking fgets. This fix
starts monitoring thread before SST script is started and
we stop monitoring thread when SST has been completed.

Author: Nikita Malyavin
Author Date: 2020-01-21 09:48:47 UTC

Period: add versioning combination

Author: Sergei Golubchik
Author Date: 2020-01-17 14:08:11 UTC

Bug#29630767 - USE OF UNINITIALIZED VALUE IN LIBMYSQL (CLIENT.CC FUNCTION RUN_PLUGIN_AUTH)

Author: Anel Husakovic
Author Date: 2020-01-17 15:23:05 UTC

Add uninstall target

Author: Jan Lindström
Author Date: 2020-01-16 11:18:44 UTC

MDEV-17062 : Test failure on galera.MW-336

Add mutex protection while we calculate required slave
thread change and create them. Add error handling.

Author: Vicențiu Ciorbaru
Author Date: 2020-01-16 11:37:21 UTC

MDEV-21210: main.uniques_crash-7912 tries to allocate 1TB of memory

Update uniques_crash-7912 to only cover 5GB of RAM and also work on
unix.

Author: Jan Lindström
Author Date: 2020-01-15 07:06:32 UTC

MDEV-18464 : Killing thread can cause mutex deadlock if done concurrently with Galera/replication victim kill

Following issues here:

Whenever Galera BF (brute force) transaction decides to abort conflicting transaction it will kill that thread using thd::awake()
Whenever replication selects a thread as a victim it will call thd::awake()
User KILL [QUERY|CONNECTION] ... for a thread it will also call thd::awake()
Whenever one of these actions is executed we will hold number of InnoDB internal mutexes and thd mutexes.
Sometimes these mutexes are taken in different order causing mutex deadlock.

In this patch we will fix Galera BF and user kill cases so that we enqueue
victim thread to a list while we hold InnoDB mutexes and we then release them.
A new background thread will pick victim thread from this new list and uses
thd::awake() with no InnoDB mutexes. Idea is similar to replication background
kill. This fix enforces that we take LOCK_thd_data -> lock sys mutex -> trx mutex
always in this order.

wsrep_mysqld.cc
 Here we introduce a list where victim threads are stored,
 condition variable to be used to wake up background thread
 and mutex to protect list.

wsrep_thd.cc
 Create a new background thread to handle victim thread
 abort. We may take wsrep_thd_LOCK mutex here but not any
 InnoDB mutexes.

wsrep_innobase_kill_one_trx
 Remove all the wsrep code that was moved to wsrep_thd.cc
 We just enqueue required information to background kill
 list and cancel victim trx lock wait if there is such.
 Here we have InnoDB lock sys mutex and trx mutex so here
 we can't take wsrep_thd_LOCK mutex.

wsrep_abort_transaction
 Cleanup only.

Author: Gagan Goel
Author Date: 2019-12-30 20:11:01 UTC

MDEV-20732 Correctly set the length of the FORMAT() result for float data type as argument.

Author: midenok
Author Date: 2019-10-22 16:47:10 UTC

MDEV-20874 Wrong handling of 'table was dropped' error in purge thread

Author: Julius Goryavsky
Author Date: 2020-01-14 13:23:15 UTC

MDEV-19457: sys_vars.wsrep_provider_basic failed in buildbot

If the initialization of the wsrep provider failed, in some
cases the internal variable wrep_inited indicating that the
initialization has already been completed is still set to
"1", which then leads to confusion in the initialization
status. To solve the problem, we should set this variable
to "1" only if the wsrep provider initialization really
completed successfully.

An earlier issue has already been fixed for branch 10.4,
and this patch contains a fix for earlier versions (where
Galera 3.x is used).

Author: Julius Goryavsky
Author Date: 2020-01-14 12:57:14 UTC

MDEV-17601: MariaDB Galera does not expect 'mbstream' as streamfmt

Setting "streamfmt=mbstream" in the "[sst]" section causes SST to fail
because the format automatically switches to 'tar' by default (insead
of mbstream).

To fix this, we need to add mbstream to the list of valid values for
the format, making it synonymous with xbstream. This must be done both
in the SST script and when parsing the options of the corresponding
utilities.

Author: Jan Lindström
Author Date: 2020-01-14 06:05:05 UTC

Merge branch 'codership-10.4-MDEV-19803' into 10.4

Author: Eugene
Author Date: 2020-01-08 16:59:40 UTC

MDEV-20377 fix build for CRC32