MDEV-24845 : Oddities around innodb_fatal_semaphore_wait_threshold and global.innodb_disallow_writes
We will remove variable innodb_disallow_writes because it is badly
designed and implemented. Parameter will be marked as removed.
Instead we will be using
* Galera provider is paused i.e. all commits will wait
* FLUSH TABLES WITH READ LOCK (FTWRL) to avoid any DDL during SST, this
global read lock i.e MDL_BACKUP_FTWRL1 will conflict MDL_BACKUP_DML
taken by InnoDB background threads doing writes. Similarly, it will conflict
with MBL_BACKUP_START used by mariabackup.
* We force flushing all dirty pages from buffer pool and force InnoDB checkpoint
* Encryption, purge, background statistics and FTS-optimize threads will
acquire MDL_BACKUP_DML before continuing. This will conflict with lock
acquired in FTWRL. Note that we will not use waiting. If MDL-lock can't be
acquired we will skip the operation.
handler.cc
handler.h
Add new API function ha_force_checkpoint to force checkpoint
inside InnoDB.
xtrabackup.cc
Remove INNODB_DISALLOW_WRITES code
mdl.cc
New functions to trylock and release global MDL lock.
wsrep_sst.cc
Force flushing all dirty pages from buffer pool and force InnoDB checkpoint
mysqld.cc
Mark innodb-disallow-writes variable as removed.
dict0stats.cc
dict_stats_func
Acquire backup lock and release it after we have done
fil0crypt.cc
fil_crypt_start_encrypting_space()
fil_crypt_thread()
Acquire backup lock and release it after we have done
fts0opt.cc
fts_optimize_sync_table()
fts_optimize_callback()
Acquire backup lock and release it after we have done
ha_innodb.cc
Remove all WITH_INNODB_DISALLOW_WRITES code
wsrep_force_checkpoint()
New API function to flush dirty pages from buffer pool and
force full checkpoint.
trx_purge,cc
trx_purge()
Acquire and release global MDL_BACKUP_DML lock
try
r
MDEV-27234: Data dictionary recovery was not READ COMMITTED
This also fixes MDEV-20198: Instant ALTER TABLE is not crash safe
InnoDB dictionary recovery wrongly used the READ UNCOMMITTED isolation
level, causing some mismatch. For example, if a table was renamed or
replaced in a transaction, according to READ UNCOMMITTED the table might
not exist at all.
We implement READ COMMITTED isolation level for accessing the dictionary
tables SYS_TABLES, SYS_COLUMNS, SYS_INDEXES, SYS_FIELDS, SYS_VIRTUAL,
SYS_FOREIGN, SYS_FOREIGN_COLS. For most of these tables, no secondary
index exists. For the secondary indexes (on SYS_TABLES.ID,
SYS_FOREIGN.FOR_NAME, SYS_FOREIGN.REF_NAME), we will always look up
the primary key in the clustered index and check if the record actually
is a committed version.
dict_check_sys_tables(): Recover tablespaces also from delete-marked
committed records, so that if a matching .ibd file exists, it will
be removed by fil_delete_tablespace() when the committed delete-marked
SYS_INDEXES record of the clustered index is purged
in row_purge_remove_clust_if_poss_low().
fil_ibd_open(): Change the Boolean parameter "validate" to a ternary
one, to suppress error messages when the file might not exist.
It is possible that a .ibd file was deleted and the server shut down
before the SYS_INDEXES and SYS_TABLES records were purged. Hence, if
dict_check_sys_tables() finds a committed delete-marked record,
we must not complain if the tablespace file is not found.
On Windows, we msut treat ERROR_PATH_NOT_FOUND (directory not found)
in the same way as ERROR_FILE_NOT_FOUND. This fixes a few failures where
a previous test successfully executed DROP DATABASE (and deleted all
files and the directory), but a committed delete-marked SYS_TABLES
record had not been purged before server restart.
dict_getnext_system_low(): Do not filter out delete-marked records.
dict_startscan_system(), dict_getnext_system(): Do filter out
delete-marked records, for accessing the INFORMATION_SCHEMA tables.
dict_sys_tables_rec_read(): Return the DB_TRX_ID of the committed
version of the record. This is needed in dict_load_table_low().
dict_load_foreign_cols(), dict_load_foreign(): Add a parameter for
the current transaction identifier. In some DDL operations, the
FOREIGN KEY constraints are being loaded from the data dictionary
before the DDL transaction has been committed. For SYS_FOREIGN
and SYS_FOREIGN_COLS, we must implement the special case of
READ COMMITTED that the changes of the uncommitted current transaction
are visible.
dict_load_foreign(): Validate the table name. We could find a
SYS_FOREIGN.ID via a committed delete-marked secondary index record
that does not match the REF_NAME or FOR_NAME of the secondary index record.
dict_load_index_low(): Optionally take the table as a parameter,
so that table->def_trx_id can be updated in case of a
committed delete-marked SYS_INDEXES record corresponding
to DROP INDEX, but not corresponding to an index stub of ADD INDEX.
dict_load_indexes(): Do not update table->def_trx_id
in case of delete-marked records.
rec_is_metadata(), rec_offs_make_valid(), rec_get_offsets_func(),
row_build_low(): Relax some assertions. We may now have
!index->is_instant() even if a metadata record is present in the index.
Previously, the recovery of instant ADD/DROP COLUMN assumed
that READ UNCOMMITTED of the data dictionary will be performed.
Now, we will have a READ COMMITTED copy of the data dictionary
cache, and a READ UNCOMMITTED copy of the metadata record.
btr_page_reorganize_low(): Correctly update the FIL_PAGE_TYPE
when rolling back an instant ADD/DROP COLUMN operation.
row_rec_to_index_entry_impl(): Relax some assertions,
and disallow accessing "extra" fields. This fixes the recovery
of a crash during an instant ADD COLUMN after a successful
instant DROP COLUMN, in the test innodb.instant_alter_crash.
Tested by: Matthias Leich
