Author: Julius Goryavsky
Author Date: 2019-07-16 10:48:04 UTC

MDEV-18565: Galera mtr-suite fails if galera library is not installed

Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.

As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.

In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.

To eliminate all these issues, this patch makes the following changes:

1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.

2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.

3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).

4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.

5. Added garbd presence check and garbd path variable.

https://jira.mariadb.org/browse/MDEV-18565

Author: Oleksandr Byelkin
Author Date: 2019-07-12 14:44:39 UTC

switch to CC 3.1

Author: varun
Author Date: 2019-07-16 09:49:07 UTC

Added a basic test case for order by limit execution with a sort nest

Author: Julius Goryavsky
Author Date: 2019-07-16 09:33:11 UTC

MDEV-18565: Galera mtr-suite fails if galera library is not installed

Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.

As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.

In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.

To eliminate all these issues, this patch makes the following changes:

1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.

2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.

3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).

4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.

5. Added garbd presence check and garbd path variable.

https://jira.mariadb.org/browse/MDEV-18565

Author: Sergey Petrunia
Author Date: 2019-07-16 09:23:52 UTC

rocksdb.unique_check: attempt to remove race condtitions from the test

Author: Sujatha
Author Date: 2019-07-16 08:30:27 UTC

MDEV-19925: Column ... cannot be converted from type 'varchar(20)' to type 'varchar(20)'

Description:
============
In row based replication, when replicating from a table with a field with
character set set to UTF8mb3 to the same table with the same field set to
character set UTF8mb4 I get a confusing error message:

For VARCHAR: VARCHAR(1) 'utf8mb3' to VARCHAR(1) 'utf8mb4'
"Column 0 of table 'test.t1' cannot be converted from type 'varchar(3)' to
type 'varchar(1)'"

Similar issue with CHAR type as well.

Issue with respect to BLOB types:

For BLOB: LONGBLOB to TINYBLOB - Error message displays incorrect blob type.
"Column 0 of table 'test.t1' cannot be converted from type 'tinyblob' to type
'tinyblob'"

Analysis:
=========
In Row based replication charset information is not sent as part of metadata
from master to slave.

For VARCHAR field its character length is converted into equivalent
octets/bytes and stored internally. At the time of displaying the data to user
it is converted back to original character length.

For example:
VARCHAR(2)- utf8mb3 is stored as:2*3 = VARCHAR(6)
At the time of displaying it to user
VARCHAR(6)- charset utf8mb3:6/3= VARCHAR(2).

At present the internally converted octect length is sent from master to slave
with out providing the charset information. On slave side if the type
conversion fails 'show_sql_type' function is used to get the type specific
information from metadata. Since there is no charset information is available
the filed type is displayed as VARCHAR(6).

This results in confused error message.

For CHAR fields
CHAR(1)- utf8mb3 - CHAR(3)
CHAR(1)- utf8mb4 - CHAR(4)

'show_sql_type' function which retrieves type information from metadata uses
(bytes/local charset length) to get actual character length. If slave's chaset
is 'utf8mb4' then

CHAR(3/4)-->CHAR(0)
CHAR(4/4)-->CHAR(1).

This results in confused error message.

Analysis for BLOB type issue:

BLOB's length is represented in two forms.
1. Actual length
i.e
  (length < 256) type= MYSQL_TYPE_TINY_BLOB;
  (length < 65536) type= MYSQL_TYPE_BLOB; ...

2. packlength - The number of bytes used to represent the length of the blob
  1- tinyblob
  2- blob ...

In row based replication only the packlength is written in the binary log. On
the slave side this packlength is interpreted as actual length of the blob.
Hence the length is always < 256 and the type is displayed as tiny blob.

Fix:
===
For CHAR and VARCHAR fields display their length in bytes for both source and
target fields. For target field display the charset information if it is
relevant.

For blob type changed the code to use the packlength and display appropriate
blob type in error message.

Author: Jan Lindström
Author Date: 2019-07-16 06:21:26 UTC

MDEV-19970 Galera test failure on galera_sr.galera-features#56

Move test to big_test as it takes significant amount of time and
add wait to get slave threads start and setup to replicate.

Author: Jan Lindström
Author Date: 2019-07-16 06:21:26 UTC

MDEV-19970 Galera test failure on galera_sr.galera-features#56

Move test to big_test as it takes significant amount of time and
add wait to get slave threads start and setup to replicate.

Author: Julius Goryavsky
Author Date: 2019-07-09 14:54:08 UTC

Improved error messages and added another path to the version check utility

Author: Ian Gilfillan
Author Date: 2019-07-01 15:21:57 UTC

Update man pages for 10.5

Author: Nikita Malyavin
Author Date: 2019-07-15 10:30:45 UTC

MDEV-17005 ASAN heap-use-after-free in innobase_get_computed_value

This is the race between DELETE and INSERT (or other any two operations accessing to the table).
What should happen in good case:
1. ALTER TABLE is issued. vc_templ->default_rec is initialized with temporary share's default_fields
2. temporary share is freed, but datadict is still there, with garbage in vc_templ->default_rec
3. DELETE is issued. It is first after ALTER TABLE finished.
4. ha_innobase::open() is called, ib_table->get_ref_count() should be one
5. we reinitialize vc_templ, so no garbage anymore

What actually happens:
3. DELETE is issued.
4. ha_innobase::open() is called and ib_table->get_ref_count() is 1
5. INSERT (or SELECT etc.) is issued in parallel
6. ha_innobase::open() is called and ib_table->get_ref_count() is 1
7. we check ib_table->get_ref_count() and it is 2 in both threads when we want reinitialize vc_templ
8. garbage is there

Fix:
* Do not store pointers to SHARE memory in table dict, copy it instead.
* But then we don't need to refresh it each time when refcount=1.

Author: Oleksandr Byelkin
Author Date: 2019-07-15 10:46:54 UTC

fix plugin test as it is in 10.4 with new client

Author: Sujatha
Author Date: 2019-07-15 08:00:10 UTC

MDEV-11154: Write_on_release_cache(log_event.cc) function will not write "COMMIT", if use "mysqlbinlog ... | mysql ..."

Problem:
=======
Executing command, "mysqlbinlog --read-from-remote-server --host='xx.xx.xx.xx'
--port=3306 --user=xxx --password=xxx --database=mysql --to-last-log
mysql-bin.000001 --start-position=1098699 --stop-never |mysql -uxxx -pxxx", we
found that last data read from remote couldn't commit.

Analysis:
========
The purpose of 'Write_on_release_cache' is that the contents of the Cache will
automatically be written to a dedicated result file on destruction. Flush
operation on the result file is controlled by a flag 'FLUSH_F'. Events which
require force flush upon their destruction will have to enable this
'Write_on_release_cache::FLUSH_F'. At present the 'FLUSH_F' flag is defined as
an enum as shown below.

enum flag
{
  FLUSH_F
};

Since 'FLUSH_F' is the first member without initialization it get the default
value '0'. Because of this the following flush condition never succeeds.

if (m_flags & FLUSH_F)
  fflush(m_file);

At present the file gets flushed only during my_fclose(result_file) operation.
When continuous streaming is enabled through --stop-never option it never gets
flushed and hence events are not replicated.

Fix:
===
Initialize the enum value to non zero value.

Author: Jan Lindström
Author Date: 2019-07-15 04:02:18 UTC

Try #2

Author: Vladislav Vaintroub
Author Date: 2019-07-14 19:44:40 UTC

wip

Author: Sergei Golubchik
Author Date: 2019-07-09 18:08:46 UTC

don't push: CONNECT postin.sql test

Author: Sergei Golubchik
Author Date: 2019-07-14 10:17:32 UTC

really make CPACK_RPM_DEBUGINFO_PACKAGE configurable

Author: Galina Shalygina
Author Date: 2019-05-14 20:44:04 UTC

MDEV-11588 Support for ONLY_FULL_GROUP_BY functional dependency

Implement a case when INNER join is used.
Support cases:
1. Primary key or unique keys fields are used in GROUP BY.
   Then all fields of this table can be used in SELECT list or HAVING.
2. Virtual column is used.
   If it is defined with GROUP BY fields then it can be used in SELECT list
   and HAVING.
3. Materialized derived table or view is used.
   If at least one field of this table is used in GROUP BY then all fields
   of this table can be used in SELECT list and HAVING.
4. Subquery is used.
   Control outer references in subqueries.
5. Field is equal to some constant.
   Then it can be used in SELECT list and HAVING.

Author: Sergei Golubchik
Author Date: 2019-07-12 17:31:57 UTC

compilation error with gcc 8.3.0

error: type qualifiers ignored on cast result type [-Werror=ignored-qualifiers]

Author: Oleksandr Byelkin
Author Date: 2019-07-12 08:03:33 UTC

MDEV-17042: prepared statement does not return error with SQL_MODE STRICT_TRANS_TABLES.

Use for parameters value conversion functions which issue warnings.

Author: Jan Lindström
Author Date: 2019-07-12 05:23:34 UTC

Fix typo.

Author: Alexander Barkov
Author Date: 2019-07-10 13:15:52 UTC

MDEV-20016 Add MariaDB_DATA_TYPE_PLUGIN

Author: Alexander Barkov
Author Date: 2019-07-10 13:15:52 UTC

MDEV-20016 Add MariaDB_DATA_TYPE_PLUGIN

Author: Oleksandr Byelkin
Author Date: 2019-07-11 12:50:50 UTC

MDEV-15572: view.test, server crash with --big-tables=1

Check that table is really opened before cleanup using handler.

Author: Sujatha
Author Date: 2019-07-11 12:07:43 UTC

MDEV-11154- Upmerge Test

Author: Oleksandr Byelkin
Author Date: 2019-05-14 12:01:15 UTC

MDEV-16932: ASAN heap-use-after-free in my_charlen_utf8 / my_well_formed_char_length_utf8 on 2nd execution of SP with ALTER trying to add bad CHECK

Make automatic name generation during execution (not prepare).

Check result of memory allocation operation.

Author: Alexander Barkov
Author Date: 2019-07-11 10:50:39 UTC

MDEV-20042 Implement EXTRA2_FIELD_DATA_TYPE_INFO in FRM

Author: Sujatha
Author Date: 2019-07-11 10:21:24 UTC

MDEV-11154 - upmerge testing

Author: Anel Husakovic
Author Date: 2019-07-10 09:27:35 UTC

After review fixes

- Indentation fixed
- Logic of empty string fixed
- Added read_variable_length() to opaque data type
- Added logic and test case for MYSQL_TYPE_NEWDECIMAL
- Added new utf8 test
- Added support encoding for other opaque data types
(MYSQL_TYPE_{LONG/MEDIUM/TINY}BLOB, MYSQL_TYPE_VARCHAR, MYSQL_TYPE_YEAR) found in json
suite of mysql and test cases
- Added big array test (--do-test=anel/mysql_j)
- alter force @todo

Author: Alexander Barkov
Author Date: 2019-04-04 12:36:08 UTC

MDEV-274 The data type for IPv6/IPv4 addresses in MariaDB

Author: Thirunarayanan Balathandayuthapani
Author Date: 2019-07-10 11:48:56 UTC

MDEV-19978 Page read from tablespace is corrupted

Problem:
=======
  Checksum fields can have value as zero. In that case, InnoDB falsely
consider that page should be all zeroes. It leads to wrong detection of page
corruption.

Solution:
========
 Remove the condition that checks if checksum fields are zero then
page should be all zeroes.

Author: Oleksandr Byelkin
Author Date: 2019-07-10 11:40:54 UTC

MDEV-18027: Running out of file descriptors and eventual crash

For automatic number of opened files limit take into account number of table instances for table cache

Author: midenok
Author Date: 2019-07-10 07:07:33 UTC

Unused code [separate commit]

Author: Julius Goryavsky
Author Date: 2019-07-09 14:54:08 UTC

Improved error messages and added another path to the version check utility

Author: Nikita Malyavin
Author Date: 2019-07-09 11:20:00 UTC

Add const qualifiers to Field::cmp, Field::cmp_max and Field::cmp_binary

Author: Alexander Barkov
Author Date: 2019-07-09 11:10:45 UTC

MDEV-19996 Bison grammar: turn singe-character operators into <kwd>

Author: varun
Author Date: 2019-07-09 09:44:19 UTC

Correcting the value of top_join_tab_count, to not include inner tables of semi-join nest for the tables at the top level

Author: Sergei Golubchik
Author Date: 2019-06-26 09:46:59 UTC

bugfix: RPM installation complains about policy files, mariadb.service is not installed

add a versioned libsepol dependency to the server rpm

Author: midenok
Author Date: 2019-06-25 07:53:33 UTC

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Cause
Stale thd->m_stmt_da->m_sql_errno which is from different invocation.

Fix
Reset error state before attempt to open table.

Author: midenok
Author Date: 2019-06-25 07:53:33 UTC

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Cause
Stale thd->m_stmt_da->m_sql_errno which is from different invocation.

Fix
Reset error state before attempt to open table.

Author: midenok
Author Date: 2019-07-08 11:58:18 UTC

Fix test case for MDEV-16222

* Mitigate race candition when got_no_such_table remains uncleared.
* Remove warnings about deprecated SELECT .. FROM .. INTO ...

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Author: Sujatha
Author Date: 2019-07-08 11:56:19 UTC

MDEV-11154: Write_on_release_cache(log_event.cc) function will not write "COMMIT", if use "mysqlbinlog ... | mysql ..."

Problem:
=======
Executing command, "mysqlbinlog --read-from-remote-server --host='xx.xx.xx.xx'
--port=3306 --user=xxx --password=xxx --database=mysql --to-last-log
mysql-bin.000001 --start-position=1098699 --stop-never |mysql -uxxx -pxxx", we
found that last data read from remote couldn't commit.

Analysis:
========
The purpose of 'Write_on_release_cache' is that the contents of the Cache will
automatically be written to a dedicated result file on destruction. Flush
operation on the result file is controlled by a flag 'FLUSH_F'. Events which
require force flush upon their destruction will have to enable this
'Write_on_release_cache::FLUSH_F'. At present the 'FLUSH_F' flag is defined as
an enum as shown below.

enum flag
{
  FLUSH_F
};

Since 'FLUSH_F' is the first member without initialization it get the default
value '0'. Because of this the following flush condition never succeeds.

if (m_flags & FLUSH_F)
  fflush(m_file);

At present the file gets flushed only during my_fclose(result_file) operation.
When continuous streaming is enabled through --stop-never option it never gets
flushed and hence events are not replicated.

Fix:
===
Initialize the enum value to non zero value.

Author: midenok
Author Date: 2019-06-25 07:53:33 UTC

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Cause
Stale thd->m_stmt_da->m_sql_errno which is from different invocation.

Fix
Reset error state before attempt to open table.

Author: varun
Author Date: 2019-07-08 06:07:34 UTC

MDEV-17963: Assertion `field_pos < field_count' failed in Protocol_text::store, Assertion `field_handlers == 0 || field_pos < field_count'

The problem was that sp_head::MULTI_RESULTS was not set correctly for ANALYZE statement
with SELECT ... INTO variable.
This is a follow up fix for MDEV-7023

Author: Anel Husakovic
Author Date: 2019-07-05 05:38:47 UTC

MDEV-19948 `SHOW GRANTS FOR user` return privileges individually

Author: Sergey Petrunia
Author Date: 2019-07-07 15:03:52 UTC

Update RocksDB the revision that fixes the compile error on power8

Author: Sergei Golubchik
Author Date: 2019-07-04 14:18:59 UTC

force-enable debug output in the pam plugin

Author: Kentoku SHIBA
Author Date: 2019-07-06 07:30:55 UTC

MDEV-16248 Row based replication to spider with float column fails on delete/update

Author: HF
Author Date: 2019-07-06 07:29:28 UTC

Make Win compiler happy.

Author: Sujatha
Author Date: 2019-07-05 15:22:25 UTC

MDEV-11154: Write_on_release_cache(log_event.cc) function will not write "COMMIT", if use "mysqlbinlog ... | mysql ..."

Problem:
=======
Executing command, "mysqlbinlog --read-from-remote-server --host='xx.xx.xx.xx'
--port=3306 --user=xxx --password=xxx --database=mysql --to-last-log
mysql-bin.000001 --start-position=1098699 --stop-never |mysql -uxxx -pxxx", we
found that last data read from remote couldn't commit.

Analysis:
========
The purpose of 'Write_on_release_cache' is that the contents of the Cache will
automatically be written to a dedicated result file on destruction. Flush
operation on the result file is controlled by a flag 'FLUSH_F'. Events which
require force flush upon their destruction will have to enable this
'Write_on_release_cache::FLUSH_F'. At present the 'FLUSH_F' flag is defined as
an enum as shown below.

enum flag
{
  FLUSH_F
};

Since 'FLUSH_F' is the first member without initialization it get the default
value '0'. Because of this the following flush condition never succeeds.

if (m_flags & FLUSH_F)
  fflush(m_file);

At present the file gets flushed only during my_fclose(result_file) operation.
When continuous streaming is enabled through --stop-never option it never gets
flushed and hence events are not replicated.

Fix:
===
Initialize the enum value to non zero value.

Author: Jan Lindström
Author Date: 2019-07-05 05:44:47 UTC

Merge branch 'tempesta-tech-sysprg/10.3-MDEV-18565' into 10.3

Author: Sergei Golubchik
Author Date: 2019-06-30 09:16:33 UTC

MDEV-19726 MariaDB server or backup RPM install assumes mysql user exists

mysqld_safe shouldn't try to mkdir or chown if --dry-run

Author: varun
Author Date: 2019-06-17 05:48:07 UTC

Addressing performance impact of the optimizer trace

Author: Jan Lindström
Author Date: 2019-07-04 11:25:07 UTC

Remove wsrep_replicated as it is not really testing anything here.

Author: Marko Mäkelä
Author Date: 2019-07-03 14:31:20 UTC

MDEV-18582: Extend SHOW STATUS LIKE 'Innodb_%'

Many InnoDB internal variables and counters were only exposed
in an unstructured fashion via SHOW ENGINE INNODB STATUS.

Expose more variables via SHOW STATUS. Many of these were
exported in XtraDB.

Also, introduce SHOW_SIZE_T and use the proper size for
exporting the InnoDB variables.

Remove some unnecessary indirection via export_vars, and
bind some variables directly.

dict_sys_t::rough_size(): Replaces dict_sys_get_size()
and includes the hash table sizes.

This is based on a contribution by Tony Liu from ServiceNow.

Author: Mostafa Hussein
Author Date: 2018-09-29 09:30:27 UTC

use -f with pgrep

pgrep will not be able to get th pid using the full path which is $libexec/mysqld unless -f is being used

Author: Anel Husakovic
Author Date: 2019-07-02 12:29:11 UTC

Add test case with JSONB_TYPE_LARGE_OBJECT

- Have to change length to support 2 bytes only (large=false)
- Have to use `read_variable_length()` since length of 8th bit of each
of 5 maximum allowed byte needs to be validated - this should be done on
OPAQUE type also @todo

Author: Thirunarayanan Balathandayuthapani
Author Date: 2019-07-02 06:35:19 UTC

- Changed the debug assert of checking page id mismatch in fil_io()
to release assert.
- Crash the server if page id doesn't match while reading the
page in buf_page_io_complete().

Author: Robert Bindar
Author Date: 2019-07-01 10:21:07 UTC

MDEV-19696 - Cleanup gcc sync builtins

Author: Sachin Setiya
Author Date: 2019-07-01 08:56:17 UTC

MDEV-8874 Replication filters configured in my.cnf are ignored if slave reset and reconfigured

Remove IF condiotion , it was causing galera tests to fails

Author: Andrei Elkin
Author Date: 2019-06-28 14:46:43 UTC

MDEV-7140 poradic buildbot failures in rpl.rpl_semi_sync_uninstall_plugin

The test allowed non-deterministic execution thanks to unresetable status
var of Slave_connections.

Fixed with expecting a correct value for Slaves_connected.

Author: Marko Mäkelä
Author Date: 2019-06-27 18:54:27 UTC

MDEV-19845: Define my_timer_cycles() inline

We remove support for the proprietary Sun Microsystems compiler,
and rely on the __GNUC__ syntax instead.

Also, we remove support for clock_gettime(CLOCK_SGI_CYCLE),
because IRIX was supported by Silicon Graphics until December 2013.

For now, we retain support for IA-64, 32-bit POWER and 32-bit SPARC,
even though those platforms are likely no longer widely used.

Author: Anel Husakovic
Author Date: 2019-06-18 13:58:41 UTC

MDEV-19679 - CREATE SERVER needs tweaks for compatibility with CONNECT engine

Author: Monty
Author Date: 2019-06-26 22:58:48 UTC

Added type conversion to fix compilation error on windows

Author: Monty
Author Date: 2019-06-26 18:00:51 UTC

Added s3_protocol_version=auto

Author: Marko Mäkelä
Author Date: 2019-06-26 18:57:42 UTC

MDEV-19845: Adaptive spin loops

Starting with the Intel Skylake microarchitecture, the PAUSE
instruction latency is about 140 clock cycles instead of earlier 10.
On AMD processors, the latency could be 10 or 50 clock cycles,
depending on microarchitecture.

Because of this big range of latency, let us scale the loops around
the PAUSE instruction based on timing results at server startup.

my_cpu_relax_multiplier: New variable: How many times to invoke PAUSE
in a loop. Only defined for IA-32 and AMD64.

my_cpu_init(): Determine with RDTSC the time to run 16 PAUSE instructions
in two unrolled loops according, and based on the quicker of the two
runs, initialize my_cpu_relax_multiplier. This form of calibration was
suggested by Mikhail Sinyavin from Intel.

LF_BACKOFF(), ut_delay(): Use my_cpu_relax_multiplier when available.

ut_delay(): Define inline in my_cpu.h.

UT_COMPILER_BARRIER(): Remove. This does not seem to have any effect,
because in our ut_delay() implementation, no computations are being
performed inside the loop. The purpose of UT_COMPILER_BARRIER() was to
prohibit the compiler from reordering computations. It was not
emitting any code.

Author: Monty
Author Date: 2019-06-26 16:47:56 UTC

Limit minium aria_block_size to 4096

MDEV-19585 Assertion with S3 table and flush_tables

The limit has to be increased so that MariaDB can create system tables.
It should not have any notable impact on performance.

There should not be any notable performance differences between 1K and 4K,
especially for temporary tables. In most cases using bigger blocks is also
faster (with the possible exception of doing key reads of not fixed length
keys).

Author: Anel Husakovic
Author Date: 2019-06-25 07:05:41 UTC

MDEV-19761 - Before Trigger not processed for Not Null Column

Author: Sergey Vojtovich
Author Date: 2018-10-19 19:13:47 UTC

MDEV-16686 - Assertion `(*tables)->reginfo.lock_type >= TL_READ' failed
             in lock_external upon using a mix of trigger, view, SP

Prevent TL_IGNORE tables from being added to the SP's m_sptabs hash.

Test cases merged from MySQL rev 41c5f4a0235ac9375080c4c79d207fe429e94a2c
BUG#19988193: ASSERTION `(*TABLES)->REGINFO.LOCK_TYPE >= TL_READ'
              FAILED IN LOCK_EXTERNAL
BUG#21198646: ASSERTION FAILED: (*TABLES)->REGINFO.LOCK_TYPE >=
              TL_READ FILE LOCK.CC, LINE 356

Author: Eugene
Author Date: 2019-06-23 09:21:06 UTC

WIP

Author: Eugene
Author Date: 2019-06-17 13:54:47 UTC

NFC: refactor Field::is_equal() and related stuff

Make Field::is_equal() const and return bool as it's a naturally fitting
type for it. Also it's agrument was narrowed to Column_definition.

InnoDB can change type of some columns by itself. InnoDB-specific code used to
reside in Field_xxx:is_equal() methods. Now engine-specific stuff was
moved to a virtual methods of handler::can_convert{string,varstring,blob,geom}.
These methods are called by Field::can_be_converted_by_engine() which is a
double dispatch pattern.

Some InnoDB-specific code still resides in compare_keys_but_name(). It should
be moved from here someday to handler::compare_key_parts(...) or similar.

IS_EQUAL_WITH_REINTERPRET_COMPATIBLE_CHARSET
IS_EQUAL_WITH_REINTERPRET_COMPATIBLE_CHARSET_BUT_COLLATE: both was removed

IS_EQUAL_NO, IS_EQUAL_YES are not needed now and should be removed
along with deprecated handler::check_if_incompatible_data().

HA_EXTENDED_TYPES_CONVERSION: was removed as such logic is not needed now by
server code.

ALTER_COLUMN_EQUAL_PACK_LENGTH: was renamed to a more generic
ALTER_COLUMN_TYPE_CHANGE_BY_ENGINE

Author: Roman Nozdrin
Author Date: 2019-06-19 21:31:51 UTC

MDEV-19831 find_select_handler() now tries its best to find a handlerton that
    is able to processes the whole query. For that it traverses tables from
    subqueries.

    Select_handler now cleans up temporary table structures on dctor call.

    Enable outer to inner JOIN rewrites for smart engine Select handler.

Author: Vladislav Vaintroub
Author Date: 2019-06-21 16:51:36 UTC

remove workaround from MDEV-9409

Author: Sergey Vojtovich
Author Date: 2019-06-20 14:02:40 UTC

MDEV-16470 - Session user variables tracker

Based on contribution by Dapeng Huang.

Author: Sergey Petrunia
Author Date: 2019-06-20 18:34:15 UTC

Fix tests: some build hosts have ramdisk in /run/shm instead of /dev/shm

Author: Roman Nozdrin
Author Date: 2019-06-19 21:31:51 UTC

MDEV-XXX find_select_handler now tries its best to find a handlerton that
    is able to processes the whole query. find_select_handler traverses
    all global tables for the purpose.

    Select_handler now cleans up temporary table structures on dctor call.

Author: Marko Mäkelä
Author Date: 2019-06-18 15:01:44 UTC

Remove page_copy_rec_list_end_to_created_page()

Also remove the log records MLOG_LIST_END_COPY_CREATED,
MLOG_COMP_LIST_END_COPY_CREATED and
page_parse_copy_rec_list_to_created_page().

FIXME: Do not disable tests

FIXME: Remove remaining references to the function and deduplicate code.

FIXME: Implement new reusable function (template?) for insert+reorganize.

Author: Sujatha
Author Date: 2019-06-19 09:18:18 UTC

MDEV-15010: Wrong Seconds_Behind_Master when only starting the SQL_Thread

Problem:
=======
When I run replication normally, I get a value in "Seconds_Behind_Master" from
"SHOW SLAVE STATUS". However, when I run only the SQL Thread (on local relay
logs that have been downloaded previously), I have "NULL" in
"Seconds_Behind_Master". I would expect to have a numeric value.

Fix:
===
Implemented following changes.
case 1: "Seconds_Behind_Master" shows 0, when SQL thread is in sync with IO
        thread and IO thread is running.
case 2: "Seconds_Behind_Master" reports NULL, when SQL thread is in sync with
        IO thread and IO thread is stopped.
case 3: "Seconds_Behind_Master" reports NULL, when SQL thread is stopped while
        IO thread is up and running.
case 4: "Seconds_Behind_Master" reports a valid numerical value when IO thread
        is stopped and SQL thread is consuming existing relay log.

Author: Sachin Setiya
Author Date: 2019-06-19 05:05:39 UTC

MDEV-19049 Server crashes in check_duplicate_long_entry_key, ASAN stack-buffer-overflow in Field_blob::get_key_image

Long Unique keys should always be last unique key.

Author: Jan Lindström
Author Date: 2019-06-04 10:19:40 UTC

MDEV-19660: wsrep_rec_get_foreign_key() is dereferencing a stale pointer to a page that was previously latched

In row_ins_foreign_check_on_constraint(), clustered index record is being passed to wsrep_append_foreign_key() after releasing the latch. If a record has been changed by other thread in the meantime then it could lead to a crash when
wsrep_rec_get_foreign_key () tries to access the record.

wsrep_append_foreign_key
 Added call to error reporting function and debug error injection.

row_ins_foreign_report_err_low
 Moved actual error reporting to a file to this low level function.

row_ins_foreign_report_err
 Use new row_ins_foreign_report_err_low function for error reporting
 to a file.

wsrep_report_foreign_key_error
 New function to report foreign key errors when appending a key for
 Galera processing.

row_ins_foreign_check_on_constraint
 Use cascade->pcur->old_rec instead of clust_rec.

Author: Eugene
Author Date: 2019-05-27 10:18:24 UTC

MDEV-19531 Add colors to mtr

Colors possibility auto detected. [ such ] stuff is colored.

Patch by Sergei Golubchik

Author: Alexander Barkov
Author Date: 2019-06-18 00:58:15 UTC

MDEV-17363 - Compressed columns cannot be restored from dump

In collaboration with Sergey Vojtovich <svoj@mariadb.org>

The COMPRESSED clause is now a part of the data type and goes immediately
after the data type and length, but before the CHARACTER SET clause,
and before column attributes such as DEFAULT, COLLATE, ON UPDATE,
SYSTEM VERSIONING, engine specific column attributes.

In the old reduction, the COMPRESSED clause was a column attribute.

New syntax:
  <varchar or text data type> <length> <compression> <character set> <column attributes>
  <varbinary or blob data type> <length> <compression> <column attributes>

New syntax examples:
  VARCHAR(1000) COMPRESSED CHARACTER SET latin1 DEFAULT ''
  BLOB COMPRESSED DEFAULT ''

Deprecate syntax examples:
  VARCHAR(1000) CHARACTER SET latin1 COMPRESSED DEFAULT ''
  TEXT CHARACTER SET latin1 DEFAULT '' COMPRESSED
  VARBINARY(1000) DEFAULT '' COMPRESSED

As a side effect:
- COMPRESSED is not a valid SP name in SQL/PSM routines any more
  (but it's still valid SP name in sql_mode=ORACLE)

- COMPRESSED is now allowed in combination with GENERATED ALWAYS AS:

  TEXT COMPRESSED GENERATED ALWAYS AS REPEAT('a',1000)

Author: Kentoku SHIBA
Author Date: 2019-06-17 16:43:07 UTC

MDEV-18737 Spider "Out of memory" on armv7hl

This is an issue of memory alignment of variable argument when calling my_multi_malloc().
The fix is strictly casting allocating size to "uint".

Author: varun
Author Date: 2019-06-17 11:57:39 UTC

Suggestion-3

Author: Michael Widenius
Author Date: 2019-06-13 17:42:13 UTC

Fixed Aria recovery progress printing

- When recovery failed, errors would not be printed on
  new lines.
- Print more information if file lengths are changed
- Added logging of table name for entries INCOMPLETE_LOG and
  REDO_REPAIR_TABLE

Author: Vladislav Vaintroub
Author Date: 2019-06-17 08:56:00 UTC

MDEV-19750 mysql command wrong encoding

Restore the detection of default charset in command line utilities.
It worked up to 10.1, but was broken by Connector/C.

Moved code for detection of default charset from sql-common/client.c
to mysys, and make command line utilities to use this code if charset
was not specified on the command line.

Author: Anel Husakovic
Author Date: 2019-06-10 16:07:49 UTC

MDEV-19636 Usage message for plugin activation option doesn't mention FORCE_PLUS_PERMANENT

In addition same message is applied for mtr

Author: Rasmus Johansson
Author Date: 2019-06-17 07:43:01 UTC

MDEV-19781 Create MariaDB named commands on Windows

Author: Marko Mäkelä
Author Date: 2019-06-17 07:07:54 UTC

MDEV-19780 Remove the TokuDB storage engine

The TokuDB storage engine has been deprecated by upstream
Percona Server 8.0 in favor of MyRocks and will not be available
in subsequent major upstream releases.

Let us remove it from MariaDB Server as well.
MyRocks is actively maintained, and it can be used instead.

Author: Sergey Petrunia
Author Date: 2019-06-15 16:55:57 UTC

MDEV-17045: MyRocks tables cannot be updated when binlog_format=MIXED.

Author: Rasmus Johansson
Author Date: 2019-05-18 06:11:01 UTC

MDEV-17592 Create MariaDB named commands/symlinks

Author: Oleksandr Byelkin
Author Date: 2019-06-14 13:03:03 UTC

fix for WSREP call

Author: midenok
Author Date: 2019-06-13 09:13:08 UTC

mysql-test-run: imply --force when --max-test-fail is specified from command line

Author: Alexey Botchkov
Author Date: 2019-06-14 07:32:30 UTC

MDEV-18661 loading the audit plugin causes performance regression.

Plugin fixed to not lock the LOCK_operations when not active.
Server fixed to lock the LOCK_plugin less - do it once per thread
and then only if a plugin was installed/uninstalled.

Author: Georg Richter
Author Date: 2019-06-14 04:39:27 UTC

Fix for embedded test

Author: Anel Husakovic
Author Date: 2019-06-13 14:55:58 UTC

Do empty job for package/package body - refactor store_schema_params()

Author: Nikita Malyavin
Author Date: 2019-06-13 13:28:08 UTC

mtr: add --continue-on-error; clarify --max-test-run; deprecate --force

* add --continue-on-error in favor of `--force --force`. Works same as
same-named mysqltest option. Actually, just bypasses it to mysqltest.

* --max-test-run does not require --force anymore.

* --force behaves as before, but marked as deprecated,
e. g. sets `--max-test-run` option to 10, if not set up otherwise

Author: Julius Goryavsky
Author Date: 2019-06-13 11:59:54 UTC

MDEV-18863: Galera SST scripts can't read [mysqldN] option groups

Some users and some scripts (for example, mysqld_multi.sh) use special
option groups with names like [mysqld1], [mysqld2], ..., [mysqldN].

But SST scripts can't currently fully support these option groups.
The only option group-related value it gets from the server is
--defaults-group-suffix from the server, if that option was set
for mysqld when the server was started.

However, the SST script does not get told by the server to read
these option groups, so this means that the SST script will fail to
read options like innodb-data-home-dir when it is in a option group
like [mysqld1]...[mysqldN].

Moreover, SST scripts ignore many parameters that can be passed
to them explicitly and cannot transfer them further, for example,
to the input of mariabackup utility. Ideally, we want to transfer
all the parameters of the original mysqld call to utilities such
as mariabackup, however the SST script does not receive these
parameters and therefore cannot transfer them to mariabackup.

To correct these shortcomings, we need to add a transfer to the
script of all the parameters of the original mysqld call, and in
the SST scripts themselves provide for the transfer of these
parameters to utilities such as mariabackup. To prevent these
parameters from mixing with the script's own parameters, they
should be transferred to SST script after the special option
"--mysqld-args", followed by the line of the original parameters,
as received by mysqld call at the time of launch (further all
these parameters will be passed to mariabackup, for example).

In addition, the SST scripts themselves must be refined so that
they can read the parameters from the user-selected group, not just
from the global mysqld configuration group. And also so that they
can receive the parameters (which important for their work) as
command-line arguments.

Author: Nikita Malyavin
Author Date: 2019-03-04 09:09:30 UTC

Assertion `old_part_id == m_last_part' failed in ha_partition::update_row or `part_id == m_last_part' in ha_partition::delete_row upon UPDATE/DELETE after dropping versioning

PRIMARY KEY change hadn't been treated as partition reorganization in case of partitioning by KEY() (without parameters).

* set `*partition_changed= true` in the described case.
* since add/drop system versioning does not affect alter_info->key_list, it required separate attention

Author: Nikita Malyavin
Author Date: 2019-06-13 06:15:52 UTC

mysql-test-run: mention --force in --max-test-fail description