lp:maria

Author: Nikita Malyavin
Author Date: 2019-07-15 10:30:45 UTC

MDEV-17005 ASAN heap-use-after-free in innobase_get_computed_value

This is the race between DELETE and INSERT (or other any two operations accessing to the table).
What should happen in good case:
1. ALTER TABLE is issued. vc_templ->default_rec is initialized with temporary share's default_fields
2. temporary share is freed, but datadict is still there, with garbage in vc_templ->default_rec
3. DELETE is issued. It is first after ALTER TABLE finished.
4. ha_innobase::open() is called, ib_table->get_ref_count() should be one
5. we reinitialize vc_templ, so no garbage anymore

What actually happens:
3. DELETE is issued.
4. ha_innobase::open() is called and ib_table->get_ref_count() is 1
5. INSERT (or SELECT etc.) is issued in parallel
6. ha_innobase::open() is called and ib_table->get_ref_count() is 1
7. we check ib_table->get_ref_count() and it is 2 in both threads when we want reinitialize vc_templ
8. garbage is there

Fix:
* Do not store pointers to SHARE memory in table dict, copy it instead.
* But then we don't need to refresh it each time when refcount=1.

Author: Julius Goryavsky
Author Date: 2019-07-22 10:24:01 UTC

The test for the wsrep_info plugin needs the same flexible wsrep version checking as the tests for Galera (continuation of MDEV-18565 task)

Author: Julius Goryavsky
Author Date: 2019-07-22 10:20:49 UTC

The test for the wsrep_info plugin needs the same flexible wsrep version checking as the tests for Galera (continuation of MDEV-18565 task)

Author: Anel Husakovic
Author Date: 2019-06-25 07:05:41 UTC

MDEV-19761 - Before Trigger not processed for Not Null Column

Author: midenok
Author Date: 2019-07-21 20:30:49 UTC

MDEV-17554 Auto-create new partition for system versioned tables with history partitioned by INTERVAL/LIMIT

When there are E empty partitions left, auto-create N new empty
partitions.

This scheme must not allow partition overflow. I.e. E-fill time must
not exceed N-creation time. This means that low values for INTERVAL
and LIMIT must not be allowed for auto-creation. In case when overflow
is detected there is no need to do anything special: a warning will be
issued and the user will run manual rebuild to redistribute records
correctly. This is important because automatic ADD must be done fast,
without forced rebuild, by the obvious reason.

Initial version implements hard-coded values of 1 for E and N. As well
as auto-creation threshold MinInterval = 1 hour, MinLimit = 1000.

The name for newly added partition will be first chosen as "pX", where
X is partition number and "p" is hard-coded name prefix. If this name
is already occupied, the X will be incremented until the resulting
name will be free to use.

Auto-creation mechanism is applied to every table having LIMIT or
INTERVAL clause. Note that there is no much sense in specifying
explicit partition list in this case and this is covered by
MDEV-19903. The syntax to explicitly turn it on/off as well as
user-defined values for E, N and name prefix is subject for further
discussion and feature requests.

ALTER TABLE ADD PARTITION is now always fast. If there some history
partition overflow occurs manual ALTER TABLE REBUILD PARTITION is
needed.

=== Dependency hints (auto-detected by git-deps) ===
70709fd23f2 MDEV-17553 Enable setting start datetime for interval partitioned history of system versioned tables
b538d4a077c Code cleanups [MDEV-17554]
72904149a09 Tests: moved partition_rotation.test to partition.test [MDEV-17554]
8c2411c2f77 MDEV-16370 row-based binlog events (updates by primary key) can not be applied multiple times to system versioned tables
ae37105e465 MDEV-16546 System versioning setting to allow history modification

Author: Vladislav Vaintroub
Author Date: 2019-07-22 06:49:09 UTC

wip

Author: Sergei Golubchik
Author Date: 2019-07-21 10:09:17 UTC

MDEV-20110 don't try to load client plugins with invalid names

reported by lixtelnis

Author: Sergei Golubchik
Author Date: 2019-07-21 10:09:17 UTC

MDEV-20110 don't try to load client plugins with invalid names

reported by lixtelnis

Author: Andrei Elkin
Author Date: 2019-07-20 19:02:55 UTC

MDEV-20091 DROP TEMPORARY table is logged despite no CREATE was logged

MDEV-5589 commit set up a policy to skip DROP TEMPORARY TABLE
binary logging in case the target table has not been "CREATEed" in
binlog (no CREATE Query-log-event was logged into the binary log).

It turns out that

1. the rule did not cover non-existing table DROPped with IF-EXISTS
   clause. The logged-create knowledge for the non-existing one does
   not even need MDEV-5589 patch, and

2. connection close disobeys it to trigger automatic DROP-IF-EXISTS
   binlogging.

Either 1 or 2 or even both is/are also responsible for unexpected
binlog records observed in MDEV-17863, actually rendering
a referred @@global.read_only irrelevant as far as the described stored
procedure definition *and* the ROW binlog-format are concerned.

Author: Sachin Setiya
Author Date: 2019-07-21 11:19:34 UTC

Finally able to compile and server does not crash

Author: varun
Author Date: 2019-07-20 11:57:16 UTC

Making sure that we don't create a sort nest when there is only one
table, sorting by first non-const table is already handled

Author: Eugene
Author Date: 2019-07-19 15:23:10 UTC

Cleanup: this is how to use span

Author: Sachin Setiya
Author Date: 2018-05-17 09:10:55 UTC

MDEV-16239 Many test in rpl suite fails

Fix rpl_skip_error test.
  We cant reset Slave_skipped_errors(even with FLUSH STATUS), So instead
of absolute slave_skipped_errors we look for delta of slave_skipped_errors
Fix rpl.rpl_binlog_errors and binlog_encryption.rpl_binlog_errors
  We create the $load_file and $load_file2 but we never remove them.
Fix rpl_000011.test
  Instead of real value use delta value , Since flush status wont flush
LONGLONG variable.
Fix rpl_row_find_row_debug
  Instead of searching whole log_error_ file we will use search_pattern_in_file
which runs pattern search only on latest test run , instead of full file.
Fix rpl_ip_mix rpl_ip_mix2
  We should call reset slave all because we also want to reset master_host
otherwise show slave status wont be empty and making repeat N a failure.
Fix rpl_rotate_logs
  First we have to remove master.info file (cleanup) and second we have to
call reset slave all because if we do not call reset slave all then we wont
read master.info file beacuse we already have master config in memory.
And this makes start slave to pass , which shoud fail becuase its permision
is 000
Fix circular_serverid0 test
  The reason is that ++dbug_rows_event_count == 2 in queue_event does
not take --repeat into account. So I have reseted the dbug_rows_event_count
in if body.

Author: Pali
Author Date: 2019-06-07 14:30:27 UTC

Switch Perl DBI scripts from DBD::mysql to DBD::MariaDB driver

Perl DBD::MariaDB driver is available CPAN and is already used in
production environment.

Author: Pali
Author Date: 2019-06-07 14:30:27 UTC

Switch Perl DBI scripts from DBD::mysql to DBD::MariaDB driver

Perl DBD::MariaDB driver is available CPAN and is already used in
production environment.

Author: Eugene
Author Date: 2019-07-19 10:25:46 UTC

MDEV-19471 Add ASAN-poisoned redzones for mem_heap_t

Store REDZONE_SIZE poined bytes before every allocated chunk of memory

Author: Sachin Setiya
Author Date: 2019-07-17 13:23:09 UTC

MDEV-7409 On RBR, extend the PROCESSLIST info to include at least the name of the recently used table

When RBR is used, add the db name to db Field and table name to Status
Field of the "SHOW FULL PROCESSLIST" command for SQL thread.

Author: Marko Mäkelä
Author Date: 2019-07-19 06:48:17 UTC

Fix innodb-system-table-view for --embedded

Author: Nikita Malyavin
Author Date: 2018-06-22 13:26:43 UTC

MDEV-16490: It's possible to make a system versioned table without any versioning field

* do not allow versioned table to be without versioned (non-system) fields
* prohibit changing field versioning, when removing table versioning

Author: Oleksandr Byelkin
Author Date: 2019-07-18 11:58:59 UTC

new 3.1

Author: Pali
Author Date: 2019-06-07 14:30:27 UTC

Switch Perl DBI scripts from DBD::mysql to DBD::MariaDB driver

Perl DBD::MariaDB driver is available CPAN and is already used in
production environment.

Author: Julius Goryavsky
Author Date: 2019-07-17 15:08:40 UTC

MDEV-18863: Galera SST scripts can't read [mysqldN] option groups

Some users and some scripts (for example, mysqld_multi.sh) use special
option groups with names like [mysqld1], [mysqld2], ..., [mysqldN].

But SST scripts can't currently fully support these option groups.
The only option group-related value it gets from the server is
--defaults-group-suffix from the server, if that option was set
for mysqld when the server was started.

However, the SST script does not get told by the server to read
these option groups, so this means that the SST script will fail to
read options like innodb-data-home-dir when it is in a option group
like [mysqld1]...[mysqldN].

Moreover, SST scripts ignore many parameters that can be passed
to them explicitly and cannot transfer them further, for example,
to the input of mariabackup utility. Ideally, we want to transfer
all the parameters of the original mysqld call to utilities such
as mariabackup, however the SST script does not receive these
parameters and therefore cannot transfer them to mariabackup.

To correct these shortcomings, we need to add a transfer to the
script of all the parameters of the original mysqld call, and in
the SST scripts themselves provide for the transfer of these
parameters to utilities such as mariabackup. To prevent these
parameters from mixing with the script's own parameters, they
should be transferred to SST script after the special option
"--mysqld-args", followed by the line of the original parameters,
as received by mysqld call at the time of launch (further all
these parameters will be passed to mariabackup, for example).

In addition, the SST scripts themselves must be refined so that
they can read the parameters from the user-selected group, not
just from the global mysqld configuration group. And also so
that they can receive the parameters (which important for their
work) as command-line arguments.

Author: Sachin Setiya
Author Date: 2019-07-17 10:26:29 UTC

MDEV-17614 INSERT on dup key update is replication unsafe

Make INSERT...ON DUPLICATE KEY UPDATE unsafe while using stmt or mixed format
When there is more then one unique key.
Although there is two exception.
  1. Auto Increment key is not counted because Innodb will get gap lock for
    failed Insert and concurrent insert will get a next increment value. But if
    user supplies auto inc value it can be unsafe.
  2. Count only unique keys for which insertion is performed.

So this patch also addresses the bug id #72921

Author: Julius Goryavsky
Author Date: 2019-07-16 09:33:11 UTC

MDEV-18565: Galera mtr-suite fails if galera library is not installed

Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.

As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.

In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.

To eliminate all these issues, this patch makes the following changes:

1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.

2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.

3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).

4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.

5. Added garbd presence check and garbd path variable.

https://jira.mariadb.org/browse/MDEV-18565

Author: Julius Goryavsky
Author Date: 2019-07-16 10:48:04 UTC

MDEV-18565: Galera mtr-suite fails if galera library is not installed

Currently, running mtr with an incorrect (for example, new or
obsolete) version of wsrep_provider (for example, with the 26
version of libgalera_smm.so) leads to the failure of tests in
several suites with vague error diagnostics.

As for the galera_3nodes suite, the mtr also does not effectively
check all the prerequisites after merge with MDEV-18426 fixes.
For example, tests that using mariabackup do not check for presence
of ss and socat/nc. This is due to improper handling of relative
paths in mtr scripts.

In addition, some tests in different suites can be run without
setting the environment variables such as MTR_GALERA_TFMT, XBSTREAM,
and so on.

To eliminate all these issues, this patch makes the following changes:

1. Added auxiliary wsrep_mtr_check utility (which located in the
mysql-test/lib/My/SafeProcess subdirectory), which compares the
versions of the wsrep API that used by the server and by the wsrep
provider library, and it does this comparison safely, without
accessing the API if the versions do not match.

2. All checks related to the presence of mariabackup and utilities
that necessary for its operation transferred from the local directories
of different mtr suites (from the suite.pm files) to the main suite.pm
file. This not only reduces the amount of code and eliminates duplication
of identical code fragments, but also avoids problems due to the inability
of mtr to consider relative paths to include files when checking skip
combinations.

3. Setting the values of auxiliary environment variables that
are necessary for Galera, SST scripts and mariabackup (to work
properly) is moved to the main mysql-test-run.pl script, so as
not to duplicate this code in different suites, and to avoid
partial corrections of the same errors for different suites
(while other suites remain uncorrected).

4. Fixed duplication of the have_file_key_management.inc and
have_filekeymanagement.inc files between different suites,
these checks are also transferred to the top level.

5. Added garbd presence check and garbd path variable.

https://jira.mariadb.org/browse/MDEV-18565

Author: Sujatha
Author Date: 2019-07-17 06:37:42 UTC

MDEV-19925: Column ... cannot be converted from type 'varchar(20)' to type 'varchar(20)'

Description:
============
In row based replication, when replicating from a table with a field with
character set set to UTF8mb3 to the same table with the same field set to
character set UTF8mb4 I get a confusing error message:

For VARCHAR: VARCHAR(1) 'utf8mb3' to VARCHAR(1) 'utf8mb4'
"Column 0 of table 'test.t1' cannot be converted from type 'varchar(3)' to
type 'varchar(1)'"

Similar issue with CHAR type as well.

Issue with respect to BLOB types:

For BLOB: LONGBLOB to TINYBLOB - Error message displays incorrect blob type.
"Column 0 of table 'test.t1' cannot be converted from type 'tinyblob' to type
'tinyblob'"

Analysis:
=========
In Row based replication charset information is not sent as part of metadata
from master to slave.

For VARCHAR field its character length is converted into equivalent
octets/bytes and stored internally. At the time of displaying the data to user
it is converted back to original character length.

For example:
VARCHAR(2)- utf8mb3 is stored as:2*3 = VARCHAR(6)
At the time of displaying it to user
VARCHAR(6)- charset utf8mb3:6/3= VARCHAR(2).

At present the internally converted octect length is sent from master to slave
with out providing the charset information. On slave side if the type
conversion fails 'show_sql_type' function is used to get the type specific
information from metadata. Since there is no charset information is available
the filed type is displayed as VARCHAR(6).

This results in confused error message.

For CHAR fields
CHAR(1)- utf8mb3 - CHAR(3)
CHAR(1)- utf8mb4 - CHAR(4)

'show_sql_type' function which retrieves type information from metadata uses
(bytes/local charset length) to get actual character length. If slave's chaset
is 'utf8mb4' then

CHAR(3/4)-->CHAR(0)
CHAR(4/4)-->CHAR(1).

This results in confused error message.

Analysis for BLOB type issue:

BLOB's length is represented in two forms.
1. Actual length
i.e
  (length < 256) type= MYSQL_TYPE_TINY_BLOB;
  (length < 65536) type= MYSQL_TYPE_BLOB; ...

2. packlength - The number of bytes used to represent the length of the blob
  1- tinyblob
  2- blob ...

In row based replication only the packlength is written in the binary log. On
the slave side this packlength is interpreted as actual length of the blob.
Hence the length is always < 256 and the type is displayed as tiny blob.

Fix:
===
For CHAR and VARCHAR fields display their length in bytes for both source and
target fields. For target field display the charset information if it is
relevant.

For blob type changed the code to use the packlength and display appropriate
blob type in error message.

Author: Jan Lindström
Author Date: 2019-07-16 06:21:26 UTC

MDEV-19970 Galera test failure on galera_sr.galera-features#56

Move test to big_test as it takes significant amount of time and
add wait to get slave threads start and setup to replicate.

Author: Jan Lindström
Author Date: 2019-07-15 04:02:18 UTC

Try #2

Author: Sergei Golubchik
Author Date: 2019-07-09 18:08:46 UTC

don't push: CONNECT postin.sql test

Author: Sergei Golubchik
Author Date: 2019-07-14 10:17:32 UTC

really make CPACK_RPM_DEBUGINFO_PACKAGE configurable

Author: Galina Shalygina
Author Date: 2019-05-14 20:44:04 UTC

MDEV-11588 Support for ONLY_FULL_GROUP_BY functional dependency

Implement a case when INNER join is used.
Support cases:
1. Primary key or unique keys fields are used in GROUP BY.
   Then all fields of this table can be used in SELECT list or HAVING.
2. Virtual column is used.
   If it is defined with GROUP BY fields then it can be used in SELECT list
   and HAVING.
3. Materialized derived table or view is used.
   If at least one field of this table is used in GROUP BY then all fields
   of this table can be used in SELECT list and HAVING.
4. Subquery is used.
   Control outer references in subqueries.
5. Field is equal to some constant.
   Then it can be used in SELECT list and HAVING.

Author: Sergei Golubchik
Author Date: 2019-07-12 17:31:57 UTC

compilation error with gcc 8.3.0

error: type qualifiers ignored on cast result type [-Werror=ignored-qualifiers]

Author: Jan Lindström
Author Date: 2019-07-12 05:23:34 UTC

Fix typo.

Author: Alexander Barkov
Author Date: 2019-07-10 13:15:52 UTC

MDEV-20016 Add MariaDB_DATA_TYPE_PLUGIN

Author: Alexander Barkov
Author Date: 2019-07-10 13:15:52 UTC

MDEV-20016 Add MariaDB_DATA_TYPE_PLUGIN

Author: Sujatha
Author Date: 2019-07-11 12:07:43 UTC

MDEV-11154- Upmerge Test

Author: Oleksandr Byelkin
Author Date: 2019-05-14 12:01:15 UTC

MDEV-16932: ASAN heap-use-after-free in my_charlen_utf8 / my_well_formed_char_length_utf8 on 2nd execution of SP with ALTER trying to add bad CHECK

Make automatic name generation during execution (not prepare).

Check result of memory allocation operation.

Author: Alexander Barkov
Author Date: 2019-07-11 10:50:39 UTC

MDEV-20042 Implement EXTRA2_FIELD_DATA_TYPE_INFO in FRM

Author: Sujatha
Author Date: 2019-07-11 10:21:24 UTC

MDEV-11154 - upmerge testing

Author: Anel Husakovic
Author Date: 2019-07-10 09:27:35 UTC

After review fixes

- Indentation fixed
- Logic of empty string fixed
- Added read_variable_length() to opaque data type
- Added logic and test case for MYSQL_TYPE_NEWDECIMAL
- Added new utf8 test
- Added support encoding for other opaque data types
(MYSQL_TYPE_{LONG/MEDIUM/TINY}BLOB, MYSQL_TYPE_VARCHAR, MYSQL_TYPE_YEAR) found in json
suite of mysql and test cases
- Added big array test (--do-test=anel/mysql_j)
- alter force @todo

Author: Alexander Barkov
Author Date: 2019-04-04 12:36:08 UTC

MDEV-274 The data type for IPv6/IPv4 addresses in MariaDB

Author: Thirunarayanan Balathandayuthapani
Author Date: 2019-07-10 11:48:56 UTC

MDEV-19978 Page read from tablespace is corrupted

Problem:
=======
  Checksum fields can have value as zero. In that case, InnoDB falsely
consider that page should be all zeroes. It leads to wrong detection of page
corruption.

Solution:
========
 Remove the condition that checks if checksum fields are zero then
page should be all zeroes.

Author: Oleksandr Byelkin
Author Date: 2019-07-10 11:40:54 UTC

MDEV-18027: Running out of file descriptors and eventual crash

For automatic number of opened files limit take into account number of table instances for table cache

Author: Julius Goryavsky
Author Date: 2019-07-09 14:54:08 UTC

Improved error messages and added another path to the version check utility

Author: Nikita Malyavin
Author Date: 2019-07-09 11:20:00 UTC

Add const qualifiers to Field::cmp, Field::cmp_max and Field::cmp_binary

Author: Alexander Barkov
Author Date: 2019-07-09 11:10:45 UTC

MDEV-19996 Bison grammar: turn singe-character operators into <kwd>

Author: varun
Author Date: 2019-07-09 09:44:19 UTC

Correcting the value of top_join_tab_count, to not include inner tables of semi-join nest for the tables at the top level

Author: Sergei Golubchik
Author Date: 2019-06-26 09:46:59 UTC

bugfix: RPM installation complains about policy files, mariadb.service is not installed

add a versioned libsepol dependency to the server rpm

Author: midenok
Author Date: 2019-06-25 07:53:33 UTC

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Cause
Stale thd->m_stmt_da->m_sql_errno which is from different invocation.

Fix
Reset error state before attempt to open table.

Author: midenok
Author Date: 2019-06-25 07:53:33 UTC

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Cause
Stale thd->m_stmt_da->m_sql_errno which is from different invocation.

Fix
Reset error state before attempt to open table.

Author: midenok
Author Date: 2019-07-08 11:58:18 UTC

Fix test case for MDEV-16222

* Mitigate race candition when got_no_such_table remains uncleared.
* Remove warnings about deprecated SELECT .. FROM .. INTO ...

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Author: Sujatha
Author Date: 2019-07-08 11:56:19 UTC

MDEV-11154: Write_on_release_cache(log_event.cc) function will not write "COMMIT", if use "mysqlbinlog ... | mysql ..."

Problem:
=======
Executing command, "mysqlbinlog --read-from-remote-server --host='xx.xx.xx.xx'
--port=3306 --user=xxx --password=xxx --database=mysql --to-last-log
mysql-bin.000001 --start-position=1098699 --stop-never |mysql -uxxx -pxxx", we
found that last data read from remote couldn't commit.

Analysis:
========
The purpose of 'Write_on_release_cache' is that the contents of the Cache will
automatically be written to a dedicated result file on destruction. Flush
operation on the result file is controlled by a flag 'FLUSH_F'. Events which
require force flush upon their destruction will have to enable this
'Write_on_release_cache::FLUSH_F'. At present the 'FLUSH_F' flag is defined as
an enum as shown below.

enum flag
{
  FLUSH_F
};

Since 'FLUSH_F' is the first member without initialization it get the default
value '0'. Because of this the following flush condition never succeeds.

if (m_flags & FLUSH_F)
  fflush(m_file);

At present the file gets flushed only during my_fclose(result_file) operation.
When continuous streaming is enabled through --stop-never option it never gets
flushed and hence events are not replicated.

Fix:
===
Initialize the enum value to non zero value.

Author: midenok
Author Date: 2019-06-25 07:53:33 UTC

MDEV-16222 Assertion `0' failed in row_purge_remove_sec_if_poss_leaf on table with virtual columns and indexes

Cause
Stale thd->m_stmt_da->m_sql_errno which is from different invocation.

Fix
Reset error state before attempt to open table.

Author: varun
Author Date: 2019-07-08 06:07:34 UTC

MDEV-17963: Assertion `field_pos < field_count' failed in Protocol_text::store, Assertion `field_handlers == 0 || field_pos < field_count'

The problem was that sp_head::MULTI_RESULTS was not set correctly for ANALYZE statement
with SELECT ... INTO variable.
This is a follow up fix for MDEV-7023

Author: Anel Husakovic
Author Date: 2019-07-05 05:38:47 UTC

MDEV-19948 `SHOW GRANTS FOR user` return privileges individually

Author: Sergey Petrunia
Author Date: 2019-07-07 15:03:52 UTC

Update RocksDB the revision that fixes the compile error on power8

Author: Sergei Golubchik
Author Date: 2019-07-04 14:18:59 UTC

force-enable debug output in the pam plugin

Author: Kentoku SHIBA
Author Date: 2019-07-06 07:30:55 UTC

MDEV-16248 Row based replication to spider with float column fails on delete/update

Author: HF
Author Date: 2019-07-06 07:29:28 UTC

Make Win compiler happy.

Author: Sujatha
Author Date: 2019-07-05 15:22:25 UTC

MDEV-11154: Write_on_release_cache(log_event.cc) function will not write "COMMIT", if use "mysqlbinlog ... | mysql ..."

Problem:
=======
Executing command, "mysqlbinlog --read-from-remote-server --host='xx.xx.xx.xx'
--port=3306 --user=xxx --password=xxx --database=mysql --to-last-log
mysql-bin.000001 --start-position=1098699 --stop-never |mysql -uxxx -pxxx", we
found that last data read from remote couldn't commit.

Analysis:
========
The purpose of 'Write_on_release_cache' is that the contents of the Cache will
automatically be written to a dedicated result file on destruction. Flush
operation on the result file is controlled by a flag 'FLUSH_F'. Events which
require force flush upon their destruction will have to enable this
'Write_on_release_cache::FLUSH_F'. At present the 'FLUSH_F' flag is defined as
an enum as shown below.

enum flag
{
  FLUSH_F
};

Since 'FLUSH_F' is the first member without initialization it get the default
value '0'. Because of this the following flush condition never succeeds.

if (m_flags & FLUSH_F)
  fflush(m_file);

At present the file gets flushed only during my_fclose(result_file) operation.
When continuous streaming is enabled through --stop-never option it never gets
flushed and hence events are not replicated.

Fix:
===
Initialize the enum value to non zero value.

Author: Jan Lindström
Author Date: 2019-07-05 05:44:47 UTC

Merge branch 'tempesta-tech-sysprg/10.3-MDEV-18565' into 10.3

Author: Sergei Golubchik
Author Date: 2019-06-30 09:16:33 UTC

MDEV-19726 MariaDB server or backup RPM install assumes mysql user exists

mysqld_safe shouldn't try to mkdir or chown if --dry-run

Author: varun
Author Date: 2019-06-17 05:48:07 UTC

Addressing performance impact of the optimizer trace

Author: Jan Lindström
Author Date: 2019-07-04 11:25:07 UTC

Remove wsrep_replicated as it is not really testing anything here.

Author: Marko Mäkelä
Author Date: 2019-07-03 14:31:20 UTC

MDEV-18582: Extend SHOW STATUS LIKE 'Innodb_%'

Many InnoDB internal variables and counters were only exposed
in an unstructured fashion via SHOW ENGINE INNODB STATUS.

Expose more variables via SHOW STATUS. Many of these were
exported in XtraDB.

Also, introduce SHOW_SIZE_T and use the proper size for
exporting the InnoDB variables.

Remove some unnecessary indirection via export_vars, and
bind some variables directly.

dict_sys_t::rough_size(): Replaces dict_sys_get_size()
and includes the hash table sizes.

This is based on a contribution by Tony Liu from ServiceNow.

Author: Mostafa Hussein
Author Date: 2018-09-29 09:30:27 UTC

use -f with pgrep

pgrep will not be able to get th pid using the full path which is $libexec/mysqld unless -f is being used

Author: Anel Husakovic
Author Date: 2019-07-02 12:29:11 UTC

Add test case with JSONB_TYPE_LARGE_OBJECT

- Have to change length to support 2 bytes only (large=false)
- Have to use `read_variable_length()` since length of 8th bit of each
of 5 maximum allowed byte needs to be validated - this should be done on
OPAQUE type also @todo

Author: Thirunarayanan Balathandayuthapani
Author Date: 2019-07-02 06:35:19 UTC

- Changed the debug assert of checking page id mismatch in fil_io()
to release assert.
- Crash the server if page id doesn't match while reading the
page in buf_page_io_complete().

Author: Robert Bindar
Author Date: 2019-07-01 10:21:07 UTC

MDEV-19696 - Cleanup gcc sync builtins

Author: Sachin Setiya
Author Date: 2019-07-01 08:56:17 UTC

MDEV-8874 Replication filters configured in my.cnf are ignored if slave reset and reconfigured

Remove IF condiotion , it was causing galera tests to fails

Author: Marko Mäkelä
Author Date: 2019-06-27 18:54:27 UTC

MDEV-19845: Define my_timer_cycles() inline

We remove support for the proprietary Sun Microsystems compiler,
and rely on the __GNUC__ syntax instead.

Also, we remove support for clock_gettime(CLOCK_SGI_CYCLE),
because IRIX was supported by Silicon Graphics until December 2013.

For now, we retain support for IA-64, 32-bit POWER and 32-bit SPARC,
even though those platforms are likely no longer widely used.

Author: Anel Husakovic
Author Date: 2019-06-18 13:58:41 UTC

MDEV-19679 - CREATE SERVER needs tweaks for compatibility with CONNECT engine

Author: Monty
Author Date: 2019-06-26 22:58:48 UTC

Added type conversion to fix compilation error on windows

Author: Monty
Author Date: 2019-06-26 18:00:51 UTC

Added s3_protocol_version=auto

Author: Marko Mäkelä
Author Date: 2019-06-26 18:57:42 UTC

MDEV-19845: Adaptive spin loops

Starting with the Intel Skylake microarchitecture, the PAUSE
instruction latency is about 140 clock cycles instead of earlier 10.
On AMD processors, the latency could be 10 or 50 clock cycles,
depending on microarchitecture.

Because of this big range of latency, let us scale the loops around
the PAUSE instruction based on timing results at server startup.

my_cpu_relax_multiplier: New variable: How many times to invoke PAUSE
in a loop. Only defined for IA-32 and AMD64.

my_cpu_init(): Determine with RDTSC the time to run 16 PAUSE instructions
in two unrolled loops according, and based on the quicker of the two
runs, initialize my_cpu_relax_multiplier. This form of calibration was
suggested by Mikhail Sinyavin from Intel.

LF_BACKOFF(), ut_delay(): Use my_cpu_relax_multiplier when available.

ut_delay(): Define inline in my_cpu.h.

UT_COMPILER_BARRIER(): Remove. This does not seem to have any effect,
because in our ut_delay() implementation, no computations are being
performed inside the loop. The purpose of UT_COMPILER_BARRIER() was to
prohibit the compiler from reordering computations. It was not
emitting any code.

Author: Monty
Author Date: 2019-06-26 16:47:56 UTC

Limit minium aria_block_size to 4096

MDEV-19585 Assertion with S3 table and flush_tables

The limit has to be increased so that MariaDB can create system tables.
It should not have any notable impact on performance.

There should not be any notable performance differences between 1K and 4K,
especially for temporary tables. In most cases using bigger blocks is also
faster (with the possible exception of doing key reads of not fixed length
keys).

Author: Sergey Vojtovich
Author Date: 2018-10-19 19:13:47 UTC

MDEV-16686 - Assertion `(*tables)->reginfo.lock_type >= TL_READ' failed
             in lock_external upon using a mix of trigger, view, SP

Prevent TL_IGNORE tables from being added to the SP's m_sptabs hash.

Test cases merged from MySQL rev 41c5f4a0235ac9375080c4c79d207fe429e94a2c
BUG#19988193: ASSERTION `(*TABLES)->REGINFO.LOCK_TYPE >= TL_READ'
              FAILED IN LOCK_EXTERNAL
BUG#21198646: ASSERTION FAILED: (*TABLES)->REGINFO.LOCK_TYPE >=
              TL_READ FILE LOCK.CC, LINE 356

Author: Eugene
Author Date: 2019-06-23 09:21:06 UTC

WIP

Author: Eugene
Author Date: 2019-06-17 13:54:47 UTC

NFC: refactor Field::is_equal() and related stuff

Make Field::is_equal() const and return bool as it's a naturally fitting
type for it. Also it's agrument was narrowed to Column_definition.

InnoDB can change type of some columns by itself. InnoDB-specific code used to
reside in Field_xxx:is_equal() methods. Now engine-specific stuff was
moved to a virtual methods of handler::can_convert{string,varstring,blob,geom}.
These methods are called by Field::can_be_converted_by_engine() which is a
double dispatch pattern.

Some InnoDB-specific code still resides in compare_keys_but_name(). It should
be moved from here someday to handler::compare_key_parts(...) or similar.

IS_EQUAL_WITH_REINTERPRET_COMPATIBLE_CHARSET
IS_EQUAL_WITH_REINTERPRET_COMPATIBLE_CHARSET_BUT_COLLATE: both was removed

IS_EQUAL_NO, IS_EQUAL_YES are not needed now and should be removed
along with deprecated handler::check_if_incompatible_data().

HA_EXTENDED_TYPES_CONVERSION: was removed as such logic is not needed now by
server code.

ALTER_COLUMN_EQUAL_PACK_LENGTH: was renamed to a more generic
ALTER_COLUMN_TYPE_CHANGE_BY_ENGINE

Author: Roman Nozdrin
Author Date: 2019-06-19 21:31:51 UTC

MDEV-19831 find_select_handler() now tries its best to find a handlerton that
    is able to processes the whole query. For that it traverses tables from
    subqueries.

    Select_handler now cleans up temporary table structures on dctor call.

    Enable outer to inner JOIN rewrites for smart engine Select handler.

Author: Vladislav Vaintroub
Author Date: 2019-06-21 16:51:36 UTC

remove workaround from MDEV-9409

Author: Sergey Vojtovich
Author Date: 2019-06-20 14:02:40 UTC

MDEV-16470 - Session user variables tracker

Based on contribution by Dapeng Huang.

Author: Sergey Petrunia
Author Date: 2019-06-20 18:34:15 UTC

Fix tests: some build hosts have ramdisk in /run/shm instead of /dev/shm

Author: Roman Nozdrin
Author Date: 2019-06-19 21:31:51 UTC

MDEV-XXX find_select_handler now tries its best to find a handlerton that
    is able to processes the whole query. find_select_handler traverses
    all global tables for the purpose.

    Select_handler now cleans up temporary table structures on dctor call.

Author: Marko Mäkelä
Author Date: 2019-06-18 15:01:44 UTC

Remove page_copy_rec_list_end_to_created_page()

Also remove the log records MLOG_LIST_END_COPY_CREATED,
MLOG_COMP_LIST_END_COPY_CREATED and
page_parse_copy_rec_list_to_created_page().

FIXME: Do not disable tests

FIXME: Remove remaining references to the function and deduplicate code.

FIXME: Implement new reusable function (template?) for insert+reorganize.

Author: Sujatha
Author Date: 2019-06-19 09:18:18 UTC

MDEV-15010: Wrong Seconds_Behind_Master when only starting the SQL_Thread

Problem:
=======
When I run replication normally, I get a value in "Seconds_Behind_Master" from
"SHOW SLAVE STATUS". However, when I run only the SQL Thread (on local relay
logs that have been downloaded previously), I have "NULL" in
"Seconds_Behind_Master". I would expect to have a numeric value.

Fix:
===
Implemented following changes.
case 1: "Seconds_Behind_Master" shows 0, when SQL thread is in sync with IO
        thread and IO thread is running.
case 2: "Seconds_Behind_Master" reports NULL, when SQL thread is in sync with
        IO thread and IO thread is stopped.
case 3: "Seconds_Behind_Master" reports NULL, when SQL thread is stopped while
        IO thread is up and running.
case 4: "Seconds_Behind_Master" reports a valid numerical value when IO thread
        is stopped and SQL thread is consuming existing relay log.

Author: Sachin Setiya
Author Date: 2019-06-19 05:05:39 UTC

MDEV-19049 Server crashes in check_duplicate_long_entry_key, ASAN stack-buffer-overflow in Field_blob::get_key_image

Long Unique keys should always be last unique key.

Author: Jan Lindström
Author Date: 2019-06-04 10:19:40 UTC

MDEV-19660: wsrep_rec_get_foreign_key() is dereferencing a stale pointer to a page that was previously latched

In row_ins_foreign_check_on_constraint(), clustered index record is being passed to wsrep_append_foreign_key() after releasing the latch. If a record has been changed by other thread in the meantime then it could lead to a crash when
wsrep_rec_get_foreign_key () tries to access the record.

wsrep_append_foreign_key
 Added call to error reporting function and debug error injection.

row_ins_foreign_report_err_low
 Moved actual error reporting to a file to this low level function.

row_ins_foreign_report_err
 Use new row_ins_foreign_report_err_low function for error reporting
 to a file.

wsrep_report_foreign_key_error
 New function to report foreign key errors when appending a key for
 Galera processing.

row_ins_foreign_check_on_constraint
 Use cascade->pcur->old_rec instead of clust_rec.

Author: Eugene
Author Date: 2019-05-27 10:18:24 UTC

MDEV-19531 Add colors to mtr

Colors possibility auto detected. [ such ] stuff is colored.

Patch by Sergei Golubchik

Author: Alexander Barkov
Author Date: 2019-06-18 00:58:15 UTC

MDEV-17363 - Compressed columns cannot be restored from dump

In collaboration with Sergey Vojtovich <svoj@mariadb.org>

The COMPRESSED clause is now a part of the data type and goes immediately
after the data type and length, but before the CHARACTER SET clause,
and before column attributes such as DEFAULT, COLLATE, ON UPDATE,
SYSTEM VERSIONING, engine specific column attributes.

In the old reduction, the COMPRESSED clause was a column attribute.

New syntax:
  <varchar or text data type> <length> <compression> <character set> <column attributes>
  <varbinary or blob data type> <length> <compression> <column attributes>

New syntax examples:
  VARCHAR(1000) COMPRESSED CHARACTER SET latin1 DEFAULT ''
  BLOB COMPRESSED DEFAULT ''

Deprecate syntax examples:
  VARCHAR(1000) CHARACTER SET latin1 COMPRESSED DEFAULT ''
  TEXT CHARACTER SET latin1 DEFAULT '' COMPRESSED
  VARBINARY(1000) DEFAULT '' COMPRESSED

As a side effect:
- COMPRESSED is not a valid SP name in SQL/PSM routines any more
  (but it's still valid SP name in sql_mode=ORACLE)

- COMPRESSED is now allowed in combination with GENERATED ALWAYS AS:

  TEXT COMPRESSED GENERATED ALWAYS AS REPEAT('a',1000)

Author: Kentoku SHIBA
Author Date: 2019-06-17 16:43:07 UTC

MDEV-18737 Spider "Out of memory" on armv7hl

This is an issue of memory alignment of variable argument when calling my_multi_malloc().
The fix is strictly casting allocating size to "uint".

Author: varun
Author Date: 2019-06-17 11:57:39 UTC

Suggestion-3

Author: Michael Widenius
Author Date: 2019-06-13 17:42:13 UTC

Fixed Aria recovery progress printing

- When recovery failed, errors would not be printed on
  new lines.
- Print more information if file lengths are changed
- Added logging of table name for entries INCOMPLETE_LOG and
  REDO_REPAIR_TABLE

Author: Vladislav Vaintroub
Author Date: 2019-06-17 08:56:00 UTC

MDEV-19750 mysql command wrong encoding

Restore the detection of default charset in command line utilities.
It worked up to 10.1, but was broken by Connector/C.

Moved code for detection of default charset from sql-common/client.c
to mysys, and make command line utilities to use this code if charset
was not specified on the command line.

Author: Anel Husakovic
Author Date: 2019-06-10 16:07:49 UTC

MDEV-19636 Usage message for plugin activation option doesn't mention FORCE_PLUS_PERMANENT

In addition same message is applied for mtr

Author: Rasmus Johansson
Author Date: 2019-06-17 07:43:01 UTC

MDEV-19781 Create MariaDB named commands on Windows

Author: Marko Mäkelä
Author Date: 2019-06-17 07:07:54 UTC

MDEV-19780 Remove the TokuDB storage engine

The TokuDB storage engine has been deprecated by upstream
Percona Server 8.0 in favor of MyRocks and will not be available
in subsequent major upstream releases.

Let us remove it from MariaDB Server as well.
MyRocks is actively maintained, and it can be used instead.

Author: Sergey Petrunia
Author Date: 2019-06-15 16:55:57 UTC

MDEV-17045: MyRocks tables cannot be updated when binlog_format=MIXED.

Author: Rasmus Johansson
Author Date: 2019-05-18 06:11:01 UTC

MDEV-17592 Create MariaDB named commands/symlinks

Author: Oleksandr Byelkin
Author Date: 2019-06-14 13:03:03 UTC

fix for WSREP call