maria:bb-10.4-MDEV-19238

Last commit made on 2019-04-17
Get this branch:
git clone -b bb-10.4-MDEV-19238 https://git.launchpad.net/maria

Branch merges

Branch information

Name:
bb-10.4-MDEV-19238
Repository:
lp:maria

Recent commits

5dc4d9b... by Kentoku SHIBA on 2019-04-17

MDEV-19238 Mariadb spider - crashes on where null

e7029e8... by Marko Mäkelä on 2019-04-17

Merge 10.3 into 10.4

250799f... by Marko Mäkelä on 2019-04-17

Merge 10.2 into 10.3

169c009... by Marko Mäkelä on 2019-04-17

MDEV-12699 Improve crash recovery of corrupted data pages

InnoDB crash recovery used to read every data page for which
redo log exists. This is unnecessary for those pages that are
initialized by the redo log. If a newly created page is corrupted,
recovery could unnecessarily fail. It would suffice to reinitialize
the page based on the redo log records.

To add insult to injury, InnoDB crash recovery could hang if it
encountered a corrupted page. We will fix also that problem.
InnoDB would normally refuse to start up if it encounters a
corrupted page on recovery, but that can be overridden by
setting innodb_force_recovery=1.

Data pages are completely initialized by the records
MLOG_INIT_FILE_PAGE2 and MLOG_ZIP_PAGE_COMPRESS.
MariaDB 10.4 additionally recognizes MLOG_INIT_FREE_PAGE,
which notifies that a page has been freed and its contents
can be discarded (filled with zeroes).

The record MLOG_INDEX_LOAD notifies that redo logging has
been re-enabled after being disabled. We can avoid loading
the page if all buffered redo log records predate the
MLOG_INDEX_LOAD record.

For the internal tables of FULLTEXT INDEX, no MLOG_INDEX_LOAD
records were written before commit aa3f7a107ce3a9a7f80daf3cadd442a61c5493ab.
Hence, we will skip these optimizations for tables whose
name starts with FTS_.

This is joint work with Thirunarayanan Balathandayuthapani.

fil_space_t::enable_lsn, file_name_t::enable_lsn: The LSN of the
latest recovered MLOG_INDEX_LOAD record for a tablespace.

mlog_init: Page initialization operations discovered during
redo log scanning. FIXME: This really belongs in recv_sys->addr_hash,
and should be removed in MDEV-19176.

recv_addr_state: Add the new state RECV_WILL_NOT_READ to
indicate that according to mlog_init, the page will be
initialized based on redo log record contents.

recv_add_to_hash_table(): Set the RECV_WILL_NOT_READ state
if appropriate. For now, we do not treat MLOG_ZIP_PAGE_COMPRESS
as page initialization. This works around bugs in the crash
recovery of ROW_FORMAT=COMPRESSED tables.

recv_mark_log_index_load(): Process a MLOG_INDEX_LOAD record
by resetting the state to RECV_NOT_PROCESSED and by updating
the fil_name_t::enable_lsn.

recv_init_crash_recovery_spaces(): Copy fil_name_t::enable_lsn
to fil_space_t::enable_lsn.

recv_recover_page(): Add the parameter init_lsn, to ignore
any log records that precede the page initialization.
Add DBUG output about skipped operations.

buf_page_create(): Initialize FIL_PAGE_LSN, so that
recv_recover_page() will not wrongly skip applying
the page-initialization record due to the field containing
some newer LSN as a leftover from a different page.
Do not invoke ibuf_merge_or_delete_for_page() during
crash recovery.

recv_apply_hashed_log_recs(): Remove some unnecessary lookups.
Note if a corrupted page was found during recovery.
After invoking buf_page_create(), do invoke
ibuf_merge_or_delete_for_page() via mlog_init.ibuf_merge()
in the last recovery batch.

ibuf_merge_or_delete_for_page(): Relax a debug assertion.

innobase_start_or_create_for_mysql(): Abort startup if
a corrupted page was found during recovery. Corrupted pages
will not be flagged if innodb_force_recovery is set.
However, the recv_sys->found_corrupt_fs flag can be set
regardless of innodb_force_recovery if file names are found
to be incorrect (for example, multiple files with the same
tablespace ID).

376bf4e... by Marko Mäkelä on 2019-04-17

MDEV-19241 InnoDB fails to write MLOG_INDEX_LOAD upon completing ALTER TABLE

Similar to what was done in commit aa3f7a107ce3a9a7f80daf3cadd442a61c5493ab
for FULLTEXT INDEX, we must ensure that MLOG_INDEX_LOAD records will always
be written if redo logging was disabled.

row_merge_build_indexes(): Invoke row_merge_write_redo() also when
online operation is not being executed or an error occurs.
In case of an error, invoke flush_observer->interrupted() so that
the pages will not be flushed but merely evicted from the buffer pool.
Before resuming redo logging, it is crucial for the correctness of
mariabackup and InnoDB crash recovery to flush or evict all affected pages
and to write MLOG_INDEX_LOAD records.

ee4a2fe... by Alexander Barkov <email address hidden> on 2019-04-17

Adding missing ';' at the end of a rule

f202f3d... by Oleksandr Byelkin <email address hidden> on 2019-04-16

MDEV-19263: Server crashes in mysql_handle_single_derived upon 2nd execution of PS

Fix case of prelocking placeholder

653a56f... by Kentoku SHIBA on 2019-04-16

MDEV-17508 Fix bug for spider when using "not like" (#1282)

645f77a... by Igor Babaev on 2019-04-08

MDEV-19195 Active Record unit test fails with MariaDB 10.4.3

Currently usage of range rowid filters can be combined only with
ref access and single index range access. So if the optimizer has
chosen some other quick select method to access a joined table
then no range rowid filter can be used for this table.

d9d79e4... by Marko Mäkelä on 2019-04-16

MDEV-17494 Refuse ALGORITHM=INSTANT when the row size is too large

With the MDEV-15562 instant DROP COLUMN, clustered index records
will contain traces of dropped columns, as follows:

In ROW_FORMAT=REDUNDANT, dropped columns will be stored as 0 bytes,
but they will consume 1 or 2 bytes per column in the record header.

In ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC, dropped columns will
be stored as NULL if allowed. This will consume 1 bit per nullable
column.

In ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC, dropped NOT NULL columns
will be stored as 0 bytes if allowed. This will consume 1 byte per
NOT NULL variable-length column. Fixed-length columns will be stored
using the fixed number of bytes.

The metadata record will be 20 bytes larger than user records, because
it will contain a metadata BLOB pointer.

We must refuse ALGORITHM=INSTANT (and require a table rebuild) if
the metadata record would grow too big to fit in the index page.

If SQL_MODE includes STRICT_TRANS_TABLES or STRICT_ALL_TABLES, we should
refuse ALGORITHM=INSTANT if the maximum length of user records would
exceed the maximum size of an index page, similar to what
row_create_index_for_mysql() does during CREATE TABLE. This limit
would kick in when the default values for any instantly added columns
in the metadata record are NULL or short, but the allowed maximum values
are long.

instant_alter_column_possible(): Add the parameter "bool strict" to
enable checks for the user record size, and always check the metadata
record size.