lp:lttng-ust

Author: Mathieu Desnoyers
Author Date: 2024-03-20 20:47:39 UTC

Introduce LTTNG_UST_MAP_POPULATE_POLICY environment variable

Problem Statement
-----------------

commit 4d4838bad480 ("Use MAP_POPULATE to reduce pagefault when available")
was first introduced in tag v2.11.0 and never backported to stable
branches. Its purpose was to reduce the tracer fast-path latency caused
by handling minor page faults the first time a given application writes
to each page of the ring buffer after mapping them. The discussion
thread leading to this commit can be found here [1]. When using
LTTng-UST for diagnosing real-time applications with very strict
constraints, this added latency is unwanted.

That commit introduced the MAP_POPULATE flag when mapping the ring
buffer pages, which causes the kernel to pre-populate the page table
entries (PTE).

This has, however, unintended consequences for the following scenarios:

* Short-lived applications which write very little to the ring buffer end
  up taking more time to start, because of the time it takes to
  pre-populate all the ring buffer pages, even though they typically won't
  be used by the application.

* Containerized workloads using cpusets will also end up having longer
  application startup time than strictly required, and will populate
  PTE for ring buffers of CPUs which are not present in the cpuset.

There are, therefore, two sets of irreconcilable requirements:
short-lived and containerized workloads benefit from lazily populating
the PTE, whereas real-time workloads benefit from pre-populating them.

This will therefore require a tunable environment variable that will let
the end-user choose the behavior for each application.

Solution
--------

Allow users to specify whether they want to pre-populate
shared memory pages within the application with an environment
variable.

LTTNG_UST_MAP_POPULATE_POLICY
    If set, override the policy used to populate shared memory pages within the
    application. The expected values are:

    none
        Do not pre-populate any pages, take minor faults on first access while
        tracing.

    cpu_possible
        Pre-populate pages for all possible CPUs in the system, as listed by
        /sys/devices/system/cpu/possible.

    Default: none. If the policy is unknown, use the default.

Choice of the default
---------------------

Given that users with strict real-time constraints already have to setup
their tracing with specific options (see the "--read-timer"
lttng-enable-channel(3) option [2]), it makes sense that the default
is to lazily populate the ring buffer PTE, and require users with
real-time constraints to explicitly enable the pre-populate through an
environment variable.

Effect on default behavior
--------------------------

The default behavior for ring buffer PTE mapping will be changing across
LTTng-UST versions in the following way:

- 2.10 and earlier: lazily populate PTE,
- 2.11-2.13: pre-populate PTE,
- 2.14: lazily populate PTE.

LTTng-UST 2.14 will revert back to the 2.10 lazy populate scheme by
default.

[1] https://lists.lttng.org/pipermail/lttng-dev/2019-July/thread.html#29094
[2] https://lttng.org/docs/v2.13/#doc-channel-timers

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6743b08cd1fe0d956caaf6aad63005555bb9640e

Author: Mathieu Desnoyers
Author Date: 2024-04-19 18:09:36 UTC

Version 2.12.10

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia7eb272d7a0bb0a3d56fcfb713566314de588628

Author: Mathieu Desnoyers
Author Date: 2024-04-19 18:05:22 UTC

Version 2.13.8

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6bc014b4933ac37b161bb36d82f6df65b5fbe618

Author: Michael Jeanson
Author Date: 2022-07-29 14:43:15 UTC

fix: Unify possible CPU number fallback

The MUSL specific fallback to get the number of possible CPUs in the
system has the same issue with hot-unplugged CPUs as the Glibc
implementation we worked around by using the possible CPU mask from
sysfs.

To address this, unify our fallback code across all C libraries to get
the maximum CPU id from the directories in "/sys/devices/system/cpu".

Change-Id: I5541742dc1de8e011a942880825fa88c656f0905
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Michael Jeanson
Author Date: 2022-07-29 14:43:15 UTC

fix: Unify possible CPU number fallback

The MUSL specific fallback to get the number of possible CPUs in the
system has the same issue with hot-unplugged CPUs as the Glibc
implementation we worked around by using the possible CPU mask from
sysfs.

To address this, unify our fallback code across all C libraries to get
the maximum CPU id from the directories in "/sys/devices/system/cpu".

Change-Id: I5541742dc1de8e011a942880825fa88c656f0905
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2019-10-17 19:26:22 UTC

Version 2.9.7

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Jonathan Rajotte
Author Date: 2017-11-06 21:10:44 UTC

ABI: refuse non-matching ABI minor version on event registration

In scenarios where a lttng-tools 2.8 (lttng-ust 2.8) stack is running
and an application linked against a lttng-ust 2.7, event registration
will fail on fields size validation [1]. This is not expected based on
the ABI versioning exposed by lttng-ust 2.7 (6.0) and lttng-ust 2.8
(6.1).

The same happen if the scenario is reversed.

This is the result of a change in _ustctl_basic_type.

2.8 introduced enumeration to _ustctl_basic_type. The defined padding is
of 296 while the new union member is 312 (310 of real data + 2 for
alignment) pushing the structure size to 312 instead of the previous
296. This should have been an major ABI break but until now the problem
did not surface.

To prevent this, refuse non matching minor version. No need to check for
particular major,minor version since only 6.0 (ust 2.7) and 6.1 (ust
2.8) exist until a major ABI break.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Jonathan Rajotte
Author Date: 2017-11-06 21:10:44 UTC

ABI: refuse non-matching ABI minor version on event registration

In scenarios where a lttng-tools 2.8 (lttng-ust 2.8) stack is running
and an application linked against a lttng-ust 2.7, event registration
will fail on fields size validation [1]. This is not expected based on
the ABI versioning exposed by lttng-ust 2.7 (6.0) and lttng-ust 2.8
(6.1).

The same happen if the scenario is reversed.

This is the result of a change in _ustctl_basic_type.

2.8 introduced enumeration to _ustctl_basic_type. The defined padding is
of 296 while the new union member is 312 (310 of real data + 2 for
alignment) pushing the structure size to 312 instead of the previous
296. This should have been an major ABI break but until now the problem
did not surface.

To prevent this, refuse non matching minor version. No need to check for
particular major,minor version since only 6.0 (ust 2.7) and 6.1 (ust
2.8) exist until a major ABI break.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2016-05-20 21:49:06 UTC

Version 2.6.7

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2015-09-24 20:24:08 UTC

Version 2.5.7

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2015-01-26 20:21:13 UTC

Version 2.4.4

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2014-03-10 15:16:35 UTC

Fix: incorrect urcu git URL in README

Reported-by: Ruslan Uvashev <ruslan.uvashev@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2014-02-28 17:08:31 UTC

Version 2.2.3

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2014-02-05 00:38:19 UTC

Fix: liblttng-ust-fork Makefile flags mismatch

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2013-07-11 20:14:39 UTC

Version 2.0.8

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Christian Babeux
Author Date: 2012-12-21 18:38:22 UTC

Add a README.cygwin detailing Cygwin specific build/install instructions

Signed-off-by: Christian Babeux <christian.babeux@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Christian Babeux
Author Date: 2012-10-23 19:57:55 UTC

Fix: Fix self-assign warning on struct ustfork_clone_info init

Use the proper field designator syntax (C99) to initialize the
ustfork_clone_info struct.

Signed-off-by: Christian Babeux <christian.babeux@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

Author: Mathieu Desnoyers
Author Date: 2012-03-06 23:10:59 UTC

Merge branch 'master' into dev