In the exists2in transformation, if 1) we are in a ps/sp execution and
2) the equalities picked out for the transformation intersect with the
temporary query arena free list, skip the transformation.
Further analysis:
Consider the following example:
--8<---------------cut here---------------start------------->8---
set optimizer_switch=default;
CREATE TABLE t1 (a INT);
CREATE TABLE t2 (b INT);
PREPARE st FROM "
SELECT * FROM t2
HAVING 0 IN (
SELECT a FROM t1
WHERE EXISTS (
SELECT a FROM t1
WHERE b = a
)
)
";
EXECUTE st;
EXECUTE st;
drop table t1, t2;
--8<---------------cut here---------------end--------------->8---
1. During the JOIN::prepare() of the first execution, the Item_field referring to t2.b is replaced by an Item_ref
2. The item_ref is moved around in the exists2in transformation and subsequent optimizations
3. At the end of the 1st ps execution, the Item_ref is freed due to its creation in the ps execution as part of the cleanup
4. During the JOIN::prepare() of the second execution, it tries to restore the optimized version of the statement, in doing so it does a sanity check and causes a sigsegv when trying to access the freed Item_ref.
Why the problem does not occur with the transformation disabled:
- during step 1 above, the replacement also adds an item to the change_list
- during step 2 above, the optimizations cause the equality {{b = a}} to be moved around as a whole, thereby moving its {{args}} which is an {{Item **}} instead of the {{item *}}'s as a whole
- during step 3 above, the change is rolled back, so the {{Item **}} that is the args of the {{Item_func_eq}} goes from {{b (Item_ref) = a}} becomes {{b (Item_field) = a}}
- during step 4 above, there's no use-after-free in the restored optimized statment because it automatically contains the {{b (Item_field) = a}}.
Why we can't do the same with the transformation enabled:
- during step 2, we break the {{b = a}} into the outer_expr (b) and local_field (a), add the outer_expr to left_expr and local_field to the inner select item list. we cannot move the equality as a whole during this transformation.
9f909e5...
by
=?utf-8?q?Jan_Lindstr=C3=B6m?= <email address hidden>
MDEV-30197 : Missing DBUG_RETURN or DBUG_VOID_RETURN macro in function "Wsrep_schema::restore_view()"
Here user is starting server with unsupported client charset.
We need to create wsrep_schema tables using explicit latin1
charset to avoid errors in restoring view.
1ac00c5...
by
Daniele Sciascia <email address hidden>
MDEV-30855 Remove test galera.galera_bf_abort_group_commit
This test was re-enabled in commit 0174a9ff3d412ada22205edc19066, and
has been failing since then.
The test is configured such that Galera runs with commit ordering
disabled, a configuration which is which was meant for testing the
performance penalty of commit ordering (not meant to be used in
practice).
Moreover, we have test galera_sr.galera_sr_bf_abort, which is
identical, but runs with commit ordering enabled.
No reasons to keep the failing test around.
6966d7f...
by
Teemu Ollakka <email address hidden>
MDEV-29293 MariaDB stuck on starting commit state
This is a backport from 10.5.
The problem seems to be a deadlock between KILL command execution
and BF abort issued by an applier, where:
* KILL has locked victim's LOCK_thd_kill and LOCK_thd_data.
* Applier has innodb side global lock mutex and victim trx mutex.
* KILL is calling innobase_kill_query, and is blocked by innodb
global lock mutex.
* Applier is in wsrep_innobase_kill_one_trx and is blocked by
victim's LOCK_thd_kill.
The fix in this commit removes the TOI replication of KILL command
and makes KILL execution less intrusive operation. Aborting the
victim happens now by using awake_no_mutex() and ha_abort_transaction().
If the KILL happens when the transaction is committing, the
KILL operation is postponed to happen after the statement
has completed in order to avoid KILL to interrupt commit
processing.
Notable changes in this commit:
* wsrep client connections's error state may remain sticky after
client connection is closed. This error message will then pop
up for the next client session issuing first SQL statement.
This problem raised with test galera.galera_bf_kill.
The fix is to reset wsrep client error state, before a THD is
reused for next connetion.
* Release THD locks in wsrep_abort_transaction when locking
innodb mutexes. This guarantees same locking order as with applier
BF aborting.
* BF abort from MDL was changed to do BF abort on server/wsrep-lib
side first, and only then do the BF abort on InnoDB side. This
removes the need to call back from InnoDB for BF aborts which originate
from MDL and simplifies the locking.
* Removed wsrep_thd_set_wsrep_aborter() from service_wsrep.h.
The manipulation of the wsrep_aborter can be done solely on
server side. Moreover, it is now debug only variable and
could be excluded from optimized builds.
* Remove LOCK_thd_kill from wsrep_thd_LOCK/UNLOCK to allow more
fine grained locking for SR BF abort which may require locking
of victim LOCK_thd_kill. Added explicit call for
wsrep_thd_kill_LOCK/UNLOCK where appropriate.
* Wsrep-lib was updated to version which allows external
locking for BF abort calls.
Changes to MTR tests:
* Disable galera_bf_abort_group_commit. This test is going to
be removed (MDEV-30855).
* Record galera_gcache_recover_manytrx as result file was incomplete.
Trivial change.
* Make galera_create_table_as_select more deterministic:
Wait until CTAS execution has reached MDL wait for multi-master
conflict case. Expected error from multi-master conflict is
ER_QUERY_INTERRUPTED. This is because CTAS does not yet have open
wsrep transaction when it is waiting for MDL, query gets interrupted
instead of BF aborted. This should be addressed in separate task.
* A new test galera_kill_group_commit to verify correct behavior
when KILL is executed while the transaction is committing.
Co-authored-by: Seppo Jaakola <email address hidden>
Co-authored-by: Jan Lindström <email address hidden>
Signed-off-by: Julius Goryavsky <email address hidden>
MDEV-30143 Segfault on select query using index for group-by and filesort
The problem was trying to access JOIN_TAB::select which is set to NULL
when using the filesort. The correct way is accessing either
JOIN_TAB::select or JOIN_TAB::filesort->select depending on whether
the filesort is used.
This commit introduces member function JOIN_TAB::get_sql_select()
encapsulating that check so the code duplication is eliminated.
The new condition (s->table->quick_keys.is_set(best_key->key))
was added to best_access_path() to eliminate a Valgrind error.
The cause of that error was using TRASH_ALLOC(quick_key_parts)
instead of bzero(quick_key_parts); hence, accessing
s->table->quick_key_parts[best_key->key]) without prior checking
for quick_keys.is_set() might have caused reading "dirty" memory
MDEV-31185 rw_trx_hash_t::find() unpins pins too early
rw_trx_hash_t::find() acquires element->mutex, then unpins pins, used for
lf_hash element search. After that the "element" can be deallocated and
reused by some other thread.
If we take a look rw_trx_hash_t::insert()->lf_hash_insert()->lf_alloc_new()
calls, we will not find any element->mutex acquisition, as it was not
initialized yet before it's allocation. rw_trx_hash_t::insert() can reuse
the chunk, unpinned in rw_trx_hash_t::find().
The scenario is the following:
1. Thread 1 have just executed lf_hash_search() in
rw_trx_hash_t::find(), but have not acquired element->mutex yet.
2. Thread 2 have removed the element from hash table with
rw_trx_hash_t::erase() call.
3. Thread 1 acquired element->mutex and unpinned pin 2 pin with
lf_hash_search_unpin(pins) call.
4. Some thread purged memory of the element.
5. Thread 3 reused the memory for the element, filled element->id,
element->trx.
6. Thread 1 crashes with failed "DBUG_ASSERT(trx_id == trx->id)"
assertion.
Note that trx_t objects are also reused, see the code around trx_pools
for details.
The fix is to invoke "lf_hash_search_unpin(pins);" after element->trx is
stored in local variable in rw_trx_hash_t::find().
Reviewed by: Nikita Malyavin, Marko Mäkelä.
f4ce1e4...
by
Robin Newhouse <email address hidden>
All-green GitLab CI in 10.4 branch
Note to mergers: Do not merge this commit to 10.5+. An additional PR
will be created for the 10.5 branch which is compatible with later
branches.
Include cppcheck and FlawFinder for SAST scanning.
From 10.6, cherry-picked 12bf5c46 (Remove unused French translations in
Connect engine) and c6072ed9 (Ensure that source files contain only
valid UTF8 encodings). Necessary for FlawFinder to execute and useful
anyway.
Removing MSAN build and test as it was not introduced until 10.5 and
does not successfully build.
Remove failing upgrade test since Fedora installs MariaDB 10.5 and the
10.5->10.4 upgrade rightfully complains
Add to skiplist failing test: main.func_math (MDEV-20966)
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Ensure that source files contain only valid UTF8 encodings (#2188)
Modern software (including text editors, static analysis software,
and web-based code review interfaces) often requires source code files
to be interpretable via a consistent character encoding, with UTF-8 or
ASCII (a strict subset of UTF-8) as the default. Several of the MariaDB
source files contain bytes that are not valid in either the UTF-8 or
ASCII encodings, but instead represent strings encoded in the
ISO-8859-1/Latin-1 or ISO-8859-2/Latin-2 encodings.
These inconsistent encodings may prevent software from correctly
presenting or processing such files. Converting all source files to
valid UTF8 characters will ensure correct handling.
Comments written in Czech were replaced with lightly-corrected
translations from Google Translate. Additionally, comments describing
the proper handling of special characters were changed so that the
comments are now purely UTF8.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Co-authored-by: Andrew Hutchings <email address hidden>
Remove unused French translations in Connect engine (#2252)
These files are currently not being used nor compiled in MariaDB. The
use of large lists of 'case' statements in these source files are also
not a great way to represent translated strings. This git history can
be referred to when a better translation interface can be implemented
in the future.
Therefore, these files can be removed to cleanup the MariaDB codebase.
All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
