memprof_rtl.cpp calls InitializeShadowMemory() - which
dynamically/"randomly" chooses a base address for the shadow mapping -
prior to InitializeAllocator(). If we are unlucky, the shadow memory may
be mapped in the same region where the allocator wants to be.
This patch fixes the issue by changing the allocator to dynamically
choosing a base address, as suggested by Vitaly. For comparison, HWASan
already dynamically chooses the base addresses for the shadow mapping
and allocator.
The "unlucky" failure was observed on a new buildbot:
https://lab.llvm.org/buildbot/#/builders/66/builds/1361/steps/17/logs/stdio
---------
Co-authored-by: Vitaly Buka <vitalybuka@gmail.com>
memprof often fails when ASLR entropy is too high ('sudo sysctl
vm.mmap_rnd_bits=32; ninja check-memprof'), which is the default setting
for newer versions of Ubuntu
(https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/jammy/commit/?h=hwe-6.5-next--2024.03.04-1--auto&id=6b522637c6a7dabd8530026ae933fb5ff17e877f).
This patch fixes the issue by moving the allocator base, analogously to
ASan (https://reviews.llvm.org/D148280).
Explanation from the ASan patch: when CONFIG_ARCH_MMAP_RND_BITS == 32,
it will frequently conflict with memprof's allocator, because the PIE
program segment base address of 0x555555555554 plus an ASLR shift of up
to ((2**32) * 4K == 0x100000000000) will sometimes exceed memprof's
hardcoded base address of 0x600000000000. We fix this by simply moving
the allocator base to 0x500000000000, which is below the PIE program
segment base address. This is cleaner than trying to move it to another
location that is sandwiched between the PIE program and library
segments, because if either of those grow too large, it will collide
with the allocator region.
Note that we will never need to change this base address again (unless
we want to increase the size of the allocator), because ASLR cannot be
set above 32-bits for x86-64 Linux (the PIE program segment and library
segments would collide with each other; see also ARCH_MMAP_RND_BITS_MAX
in https://github.com/torvalds/linux/blob/master/arch/x86/Kconfig).
According comments on D44404, something like that was the goal.
Reviewed By: morehouse, kstoimenov
Differential Revision: https://reviews.llvm.org/D114991
To see how to extract a shared allocator interface for D101204,
found some unused code. Tests passed. Are they safe to remove?
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D101559
See RFC for background:
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html
Follow on companion to the clang/llvm instrumentation support in D85948
and committed earlier.
This patch adds the compiler-rt runtime support for the memory
profiling.
Note that much of this support was cloned from asan (and then greatly
simplified and renamed). For example the interactions with the
sanitizer_common allocators, error handling, interception, etc.
The bulk of the memory profiling specific code can be found in the
MemInfoBlock, MemInfoBlockCache, and related classes defined and used
in memprof_allocator.cpp.
For now, the memory profile is dumped to text (stderr by default, but
honors the sanitizer_common log_path flag). It is dumped in either a
default verbose format, or an optional terse format.
This patch also adds a set of tests for the core functionality.
Differential Revision: https://reviews.llvm.org/D87120
