maria:bb-10.6-MDEV-16708-3

Last commit made on 2021-06-15
Get this branch:
git clone -b bb-10.6-MDEV-16708-3 https://git.launchpad.net/maria

Branch merges

Branch information

Name:
bb-10.6-MDEV-16708-3
Repository:
lp:maria

Recent commits

d907bc9... by Dmitry Shulga <email address hidden>

MDEV-16708: Unsupported commands for prepared statements

Within this task the following changes were made:
  - Added sending of metadata info in prepare phase for the admin related
    command (check table, checksum table, repair, optimize, analyze).

  - Refactored implementation of HELP command to support its execution in
    PS mode

  - Added support for execution of LOAD INTO and XA- related statements
    in PS mode

  - Modified mysqltest to enable any statements except PREPARE FROM,
    EXECUTE, EXECUTE IMMEDIATE, DEALLOCATE to be run in PS mode.

  - Fixed the following issues:
      The statement
        explain select (select 2)
      executed in regular and PS mode produces different results:

      MariaDB [test]> prepare stmt from "explain select (select 2)";
      Query OK, 0 rows affected (0,000 sec)
      Statement prepared
      MariaDB [test]> execute stmt;
      +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
      | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
      | 1 | PRIMARY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
      | 2 | SUBQUERY | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
      2 rows in set (0,000 sec)
      MariaDB [test]> explain select (select 2);
      +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
      | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
      | 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
      +------+-------------+-------+------+---------------+------+---------+------+------+----------------+
      1 row in set, 1 warning (0,000 sec)

      In case the statement
          CREATE TABLE t1 SELECT * FROM (SELECT 1 AS a, (SELECT a+0)) a
      is run in PS mode it fails with the error
          ERROR 1054 (42S22): Unknown column 'a' in 'field list'.

  - Uniform handling of read-only variables both in case the SET var=val
      statement is executed as regular or prepared statement.

  - Fixed assertion firing on handling LOAD DATA statement for temporary tables

  - Relaxed assert condition in the function lex_end_stage1() by adding
    the commands SQLCOM_ALTER_EVENT, SQLCOM_CREATE_PACKAGE,
    SQLCOM_CREATE_PACKAGE_BODY to a list of supported command

  - Removed raising of the error ER_UNSUPPORTED_PS in the function
    check_prepared_statement() for the ALTER VIEW command

  - Added initialisation of the data member st_select_lex_unit::last_procedure
    (assign NULL value) in the constructor

    Without this change the test case main.ctype_utf8 fails with the following
    report in case it is run with the option --ps-protocol.
      mysqltest: At line 2278: query 'VALUES (_latin1 0xDF) UNION VALUES(_utf8'a' COLLATE utf8_bin)' failed: 2013: Lost connection

  - The following bug reports were fixed:
        MDEV-24460: Multiple rows result set returned from stored
                    routine over prepared statement binary protocol is
                    handled incorrectly
        CONC-519: mariadb client library doesn't handle server_status and
                  warning_count fields received in the packet
                  COM_STMT_EXECUTE_RESPONSE.

    Reasons for these bug reports have the same nature and caused by
    missing loop iteration on results sent by server in response to
    COM_STMT_EXECUTE packet.

    Enclosing of statements for processing of COM_STMT_EXECUTE response
    in the construct like
      do
      {
        ...
      } while (!mysql_stmt_next_result());
    fixes the above mentioned bug reports.

 - Some tests has been disabled by the reason that they contains
   multi-statements that not supported in PS mode. Unfortunately,
   some of these multi-statements can't be converted to a sequence of
   single-statements by the reason that multi-statementness is
   a requirement for tests. Therefore the whole test file will be
   skipped in case it is run with --ps-protocol

6fbf978... by Marko Mäkelä

MDEV-25750: Assertion wait_lock->is_waiting() in lock_wait_rpl_report

lock_wait_rpl_report(): Tolerate the loss of a lock wait.
commit c68007d958aaab0953a01c96eb02326f19d3c9d7 (MDEV-24738)
had introduced this rare failure.

1bd681c... by Marko Mäkelä

MDEV-25506 (3 of 3): Do not delete .ibd files before commit

This is a complete rewrite of DROP TABLE, also as part of other DDL,
such as ALTER TABLE, CREATE TABLE...SELECT, TRUNCATE TABLE.

The background DROP TABLE queue hack is removed.
If a transaction needs to drop and create a table by the same name
(like TRUNCATE TABLE does), it must first rename the table to an
internal #sql-ib name. No committed version of the data dictionary
will include any #sql-ib tables, because whenever a transaction
renames a table to a #sql-ib name, it will also drop that table.
Either the rename will be rolled back, or the drop will be committed.

Data files will be unlinked after the transaction has been committed
and a FILE_RENAME record has been durably written. The file will
actually be deleted when the detached file handle returned by
fil_delete_tablespace() will be closed, after the latches have been
released. It is possible that a purge of the delete of the SYS_INDEXES
record for the clustered index will execute fil_delete_tablespace()
concurrently with the DDL transaction. In that case, the thread that
arrives later will wait for the other thread to finish.

HTON_TRUNCATE_REQUIRES_EXCLUSIVE_USE: A new handler flag.
ha_innobase::truncate() now requires that all other references to
the table be released in advance. This was implemented by Monty.

ha_innobase::delete_table(): If CREATE TABLE..SELECT is detected,
we will "hijack" the current transaction, drop the table in
the current transaction and commit the current transaction.
This essentially fixes MDEV-21602. There is a FIXME comment about
making the check less failure-prone.

ha_innobase::truncate(), ha_innobase::delete_table():
Implement a fast path for temporary tables. We will no longer allow
temporary tables to use the adaptive hash index.

dict_table_t::mdl_name: The original table name for the purpose of
acquiring MDL in purge, to prevent a race condition between a
DDL transaction that is dropping a table, and purge processing
undo log records of DML that had executed before the DDL operation.
For #sql-backup- tables during ALTER TABLE...ALGORITHM=COPY, the
dict_table_t::mdl_name will differ from dict_table_t::name.

dict_table_t::parse_name(): Use mdl_name instead of name.

dict_table_rename_in_cache(): Update mdl_name.

For the internal FTS_ tables of FULLTEXT INDEX, purge would
acquire MDL on the FTS_ table name, but not on the main table,
and therefore it would be able to run concurrently with a
DDL transaction that is dropping the table. Previously, the
DROP TABLE queue hack prevented a race between purge and DDL.
For now, we introduce purge_sys.stop_FTS() to prevent purge from
opening any table, while a DDL transaction that may drop FTS_
tables is in progress. The function fts_lock_table(), which will
be invoked before the dictionary is locked, will wait for
purge to release any table handles.

trx_t::drop_table_statistics(): Drop statistics for the table.
This replaces dict_stats_drop_index(). We will drop or rename
persistent statistics atomically as part of DDL transactions.
On lock conflict for dropping statistics, we will fail instantly
with DB_LOCK_WAIT_TIMEOUT, because we will be holding the
exclusive data dictionary latch.

trx_t::commit_cleanup(): Separated from trx_t::commit_in_memory().
Relax an assertion around fts_commit() and allow DB_LOCK_WAIT_TIMEOUT
in addition to DB_DUPLICATE_KEY. The call to fts_commit() is
entirely misplaced here and may obviously break the consistency
of transactions that affect FULLTEXT INDEX. It needs to be fixed
separately.

dict_table_t::n_foreign_key_checks_running: Remove (MDEV-21175).
The counter was a work-around for missing meta-data locking (MDL)
on the SQL layer, and not really needed in MariaDB.

ER_TABLE_IN_FK_CHECK: Replaced with ER_UNUSED_28.

HA_ERR_TABLE_IN_FK_CHECK: Remove.

row_ins_check_foreign_constraints(): Do not acquire
dict_sys.latch either. The SQL-layer MDL will protect us.

This was reviewed by Thirunarayanan Balathandayuthapani
and tested by Matthias Leich.

3f78fbc... by Marko Mäkelä

MDEV-25180 fixup: Assertion table->def_trx_id == trx->id... failed in row_merge_drop_indexes()

We must invoke online_retry_drop_indexes() before
ha_innobase_inplace_ctx::trx->id is assigned.

6a4e5bf... by Marko Mäkelä

MDEV-25852: Orphan #sql*.ibd files are left behind

The implementation of MDEV-24626 was not entirely correct.
We could occasionally fail to remove some *.ibd files on recovery.

deferred_spaces: Keep track of FILE_DELETE records.

deferred_spaces.add(): Do not allow duplicate file names.

recv_rename_files(): Preserve some of renamed_spaces entries for
deferred_spaces.reinit_all().

Thanks to Thirunarayanan Balathandayuthapani for noticing that
deferred_spaces.add() must filter out duplicate file names,
as well as some debugging help.

65f1a42... by Marko Mäkelä

Merge 10.5 into 10.6

b25d2a4... by Marko Mäkelä

Merge 10.4 into 10.5

c7ee039... by Marko Mäkelä

Merge 10.3 to 10.4

75a65d3... by Marko Mäkelä

MDEV-25886 CHECK TABLE crash with DB_MISSING_HISTORY if innodb_read_only

Occasionally, the test innodb.alter_copy would fail in MariaDB 10.6.1,
reporting DB_MISSING_HISTORY during CHECK TABLE. It started to occur during
the development of MDEV-25180, which introduced purge_sys.stop_SYS().
If we delay purge more during DDL operations, then the test would
almost always fail. The reason is that during startup we will restore
a purge view, and CHECK TABLE would still use REPEATABLE READ
even though innodb_read_only is set and other isolation levels
than READ UNCOMMITTED are not guaranteed to work.

ha_innobase::check(): Use READ UNCOMMITTED isolation level if
innodb_read_only is set or innodb_force_recovery exceeds 3.

dict_set_corrupted(): Do not update the persistent data dictionary
if innodb_force_recovery exceeds 3.

29e8c15... by Anel Husakovic <email address hidden>

MDEV-25857: MTR should report at least last test that was executed in case of shutdown and not-completed

- Report the test name in case not all tests are completed and server
closed the connection
- Rerport the failure of the last test with the server log in case of
server shutdown.
- Ignore stackdump files (obtained on Windows).

Reviewed by: <email address hidden>