glibc:nsz/mtag

Last commit made on 2021-03-19
Get this branch:
git clone -b nsz/mtag https://git.launchpad.net/glibc

Branch merges

Branch information

Name:
nsz/mtag
Repository:
lp:glibc

Recent commits

601c47c... by Szabolcs Nagy <email address hidden> on 2021-02-09

aarch64: Optimize __libc_mtag_tag_zero_region

This is a target hook for memory tagging, the original was a naive
implementation. Uses the same algorithm as __libc_mtag_tag_region,
but with instructions that also zero the memory. This was not
benchmarked on real cpu, but expected to be faster than the naive
implementation.

138ff9a... by Szabolcs Nagy <email address hidden> on 2021-02-09

aarch64: Optimize __libc_mtag_tag_region

This is a target hook for memory tagging, the original was a naive
implementation. The optimized version relies on "dc gva" to tag 64
bytes at a time for large allocations and optimizes small cases without
adding too many branches. This was not benchmarked on real cpu, but
expected to be faster than the naive implementation.

5def14a... by Szabolcs Nagy <email address hidden> on 2021-02-04

aarch64: inline __libc_mtag_new_tag

This is a common operation when heap tagging is enabled, so inline the
instructions instead of using an extern call.

8ab2764... by Szabolcs Nagy <email address hidden> on 2021-02-04

aarch64: inline __libc_mtag_address_get_tag

This is a common operation when heap tagging is enabled, so inline the
instruction instead of using an extern call.

The .inst directive is used instead of the name of the instruction (or
acle intrinsics) because malloc.c is not compiled for armv8.5-a+memtag
architecture, runtime cpu support detection is used.

Prototypes are removed from the comments as they were not always
correct.

5ed1f23... by Szabolcs Nagy <email address hidden> on 2021-02-17

malloc: Use mtag_enabled instead of USE_MTAG

Use the runtime check where possible: it should not cause slow down in
the !USE_MTAG case since then mtag_enabled is constant false, but it
allows compiling the tagging logic so it's less likely to break or
diverge when developers only test the !USE_MTAG case.

Reviewed-by: DJ Delorie <email address hidden>

db0edd3... by Szabolcs Nagy <email address hidden> on 2021-02-08

malloc: Use branches instead of mtag_granule_mask

The branches may be better optimized since mtag_enabled is widely used.

Granule size larger than a chunk header is not supported since then we
cannot have both the chunk header and user area granule aligned. To
fix that for targets with large granule, the chunk layout has to change.

So code that attempted to handle the granule mask generally was changed.
This simplified CHUNK_AVAILABLE_SIZE and the logic in malloc_usable_size.

Reviewed-by: DJ Delorie <email address hidden>

15d1d07... by Szabolcs Nagy <email address hidden> on 2021-02-16

malloc: Change calloc when tagging is disabled

When glibc is built with memory tagging support (USE_MTAG) but it is not
enabled at runtime (mtag_enabled) then unconditional memset was used
even though that can be often avoided.

This is for performance when tagging is supported but not enabled.
The extra check should have no overhead: tag_new_zero_region already
had a runtime check which the compiler can now optimize away.

Reviewed-by: DJ Delorie <email address hidden>

35218b8... by Szabolcs Nagy <email address hidden> on 2021-01-29

malloc: Only support zeroing and not arbitrary memset with mtag

The memset api is suboptimal and does not provide much benefit. Memory
tagging only needs a zeroing memset (and only for memory that's sized
and aligned to multiples of the tag granule), so change the internal
api and the target hooks accordingly. This is to simplify the
implementation of the target hook.

Reviewed-by: DJ Delorie <email address hidden>

d4f0f40... by Szabolcs Nagy <email address hidden> on 2021-01-27

malloc: Use global flag instead of function pointer dispatch for mtag

A flag check can be faster than function pointers because of how
branch prediction and speculation works and it can also remove a layer
of indirection when there is a mismatch between the malloc internal
tag_* api and __libc_mtag_* target hooks.

Memory tagging wrapper functions are moved to malloc.c from arena.c and
the logic now checks mmap_enabled. The definition of tag_new_usable is
moved after chunk related definitions.

This refactoring also allows using mtag_enabled checks instead of
USE_MTAG ifdefs when memory tagging support only changes code logic
when memory tagging is enabled at runtime. Note: an "if (false)" code
block is optimized away even at -O0 by gcc.

Reviewed-by: DJ Delorie <email address hidden>

2689b5c... by Szabolcs Nagy <email address hidden> on 2021-02-16

malloc: Refactor TAG_ macros to avoid indirection

This does not change behaviour, just removes one layer of indirection
in the internal memory tagging logic.

Use tag_ and mtag_ prefixes instead of __tag_ and __mtag_ since these
are all symbols with internal linkage, private to malloc.c, so there
is no user namespace pollution issue.

Reviewed-by: DJ Delorie <email address hidden>