This reverts commit 5e3d48a68096a0017a0fa4bb89f2d48767c8a7e4.
This relands commit 9886788a8a500a1b429a6db64397c849b112251c.
This was originally causing build failures on more esoteric platforms
that have different definitions of getcpu. This is only intended to be
supported on x86-64 currently, so just use preprocessor definitions to
special case the function.
This patch adds in support for pinning a benchmarking process to a
specific CPU (in the subprocess benchmarking mode on Linux). This is
intended to be used in environments where a certain set of CPUs is
isolated from the scheduler using something like cgroups and thus should
present less potential for noise than normal. This also opens up the
door for doing multithreaded benchmarking as we can now pin benchmarking
processes to specific CPUs that we know won't interfere with each other.
This patch switches most of the uses of intptr_t to uintptr_t within
llvm-exegesis for the subprocess memory support. In the vast majority of
cases we do not want a signed component of the address, hence making
intptr_t undesirable. intptr_t is left for error handling, for example
when making syscalls and we need to see if the syscall returned -1.
Glibc v2.40 changes the definition of __rseq_size to the usable area of
the struct rather than the actual size of the struct to accommodate
users trying to figure out what features can be used. This change breaks
llvm-exegesis trying to disable rseq as the size registered in the
kernel is no longer equal to __rseq_size. This patch adds a check to see
if __rseq_size is less than 32 bytes and uses 32 as a value if it is
given alignment requirements.
Fixes#100791.
There are several insstances in the subprocess executor in llvm-exegesis
where we fail to close file descriptors after using them. This leaves
them open, which can cause issues later on if a third-party binary is
using the exegesis libraries and executing many blocks before exiting.
Leaving the descriptors open until process exit is also bad practice.
This patch fixes that by explicitly calling close() when we are done
with a specific file descriptor.
This patch fixes#86583.
This reverts commit 1fe9c417a0bf143f9bb9f9e1fbf7b20f44196883.
This relands commit 6bbe8a296ee91754d423c59c35727eaa624f7140.
This was causing build failures on one of the ARMv8 builders. Still not
completely sure why, but relanding it to see if the failure pops up
again. If it does, the plan is to fix forward by disabling tests on ARM
temporarily as llvm-exegesis does not currently use SubprocessMemory
on ARM.
This reverts commit c3a41aac5f32475b9a0499e6e888e713763566dc.
This relands commit bd493756fa51e538575fc320aae50d75394f0567.
Apparently I forgot to update a couple values, so this change failed on
every builder that builds those sections (should be every Linux
platform) rather than something architecture specific like originally
thought. I swore I updated the values and ran check-llvm before merging.
Wondering If I forgot to push those changes...
Before this patch, llvm-exegesis would leave processes lingering that
experienced signals like segmentation faults. They would up in a
signal-delivery-stop state under the ptrace and never exit. This does
not cause problems (or at least many) in llvm-exegesis as they are
cleaned up after the main process exits, which usually happens quickly.
However, in downstream use, when many blocks are being executed (many of
which run into signals) within a single process, these processes stay
around and can easily exhaust the process limit on some systems.
This patch cleans them up by sending SIGKILL after information about the
signal that was sent has been gathered.
This reverts commit 8003f553a01a9a2a7eb09fe07e88f1ba9ee7d3a7.
This (and/or a related commit) was causing build failures on one of the
buildbots that needs more investigation. More information is available
at https://lab.llvm.org/buildbot/#/builders/178/builds/7015.
This reverts commit 1c3b15e9f5bc671e40bcf5d3475f5425466754ce.
This (and/or) a related patch was causing build failures on one of the
buildbots. More information is available at
https://lab.llvm.org/buildbot/#/builders/178/builds/7015.
This patch switches from manually using the Linux syscall to get the
current thread ID to using the relevant LLVM Support libraries that
abstract over the low level system details.
This reverts commit aefad27096bba513f06162fac2763089578f3de4.
This relands commit 6bbe8a296ee91754d423c59c35727eaa624f7140.
This patch was casuing build failures on non-Linux platforms due to the
default implementations for the functions not being updated. This ended
up causing out-of-line definition errors. Fixed for the relanding.
This reverts commit 6bbe8a296ee91754d423c59c35727eaa624f7140.
This breaks building LLVM on macOS, failing with
llvm/tools/llvm-exegesis/lib/SubprocessMemory.cpp:146:33: error: out-of-line definition of 'setupAuxiliaryMemoryInSubprocess' does not match any declaration in 'llvm::exegesis::SubprocessMemory'
Expected<int> SubprocessMemory::setupAuxiliaryMemoryInSubprocess(
This patch adds the thread ID to the subprocess memory shared memory
names. This avoids conflicts for downstream consumers that might want to
consume llvm-exegesis across multiple threads, which would otherwise run
into conflicts due to the same PID running multiple instances.
There were a couple things in the comments of BenchmarkRunner.cpp (and
maybe other files, I can't really remember) that were bugging me. This
patch fixes a couple of minor issues specifically in BenchmarkRunner
like typos and makes a couple instances more clear.
This patch replaces --num-repetitions with --min-instructions to make it
more clear that the value refers to the minimum number of instructions
in the final assembled snippet rather than the number of repetitions of
the snippet. This patch also refactors some llvm-exegesis internal
variable names to reflect the name change.
Fixes#76890.
This patch removes the llvm:: prefix within llvm-exegesis where it is
not necessary. This is most occurrences of the prefix within exegesis as
exegesis is within the llvm namespace. This patch makes things more
consistent as the vast majority of the code did not use the llvm::
prefix for anything.
This patch adds support for validation counters. Validation counters can
be used to measure events that occur during snippet execution like cache
misses to ensure that certain assumed invariants about the benchmark
actually hold. Validation counters are setup within a perf event group,
so are turned on and off at exactly the same time as the "group leader"
counter that measures the desired value.
Currently, BenchmarkRunner scales the per snippet counters by
multiplying the raw counter values by the number of instructions (casted
to a double) divided by the minimum number of instructions. This is
incorrect for the loop repetition mode for snippets that don't fit a
whole number of times into the minimum instruction count. For example,
with 3 instructions in the snippet and the minimum number of
instructions set to 4, the loop repetitor will execute a total of six
instructions, but BenchmarkRunner will scale the raw count by 3/4
instead of 3/6=1/2. This will also be incorrect for the duplicate
snippet repetitor after #77224.
This patch fixes this behavior by dividing the raw count by the ceiling
of the number of repetitions divided by the instruction count.
This refactoring gets things ready for validation counters where the
plan is to reuse the existing Counter infrastructure to contain event
groups that consist of a single event that is being measured along with
validation counters.
This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
This patch removes the redundant RegisterInitialValues parameter from
assembleToStream and friends as it is included within the BenchmarkKey
struct that is also passed to all the functions that need this
information.
This patch refactors InstrBenchmark to BenchmarkResult. Most of the
renaming away from things prefixed with Instr was performed in a
previous commit, but this specific instance was missed.
This patch adds in validation at two different levels that address
annotations are page aligned. This is necessary as otherwise the mmap
calls will fail as MAP_FIXED/MAP_FIXED_NOREPLACE require page aligned
addresses. This happens silently in the subprocess. This patch adds
validation at snippet parsing time to give feedback to the user and also
adds asserts at code generation/address usage time to ensure that other
users of the Exegesis APIs conform to the same requirements.
This patch refactors the MMAP platform-specific preprocessor directives
in llvm-exegesis to a single file instead of having duplicate code split
across multiple files. These originally got introduced to get buildbots
green again due to platform specific failures.
This patch switches to using explicit snippet crashes that contain more
information about the specific type of error (like the address for a
segmentation fault) that occurred. All these new error classes inherit
from SnippetExecutionFailure to allow for easily grabbing all of them in
addition to filtering for specific types using the standard LLVM error
primitives.
…ndling.
Returns an error *and* a benchmark rather than an error *or* a
benchmark. This allows users to have custom error handling while still
being able to inspect the benchmark.
Apart from this small API change, this is an NFC.
This is an alternative to #74211.
This reverts commit 30d700117b772d94d8474ec56bd6f9cc423fc613.
This relands commit 3ab41f912a6c219a93b87c257139822ea07c8863.
When I was updating the patch to use llvm::to_integer, I only ran the
lit tests and didn't run the unit tests, one of which started to fail.
This patch fixes the broken unit test.
Core dumps are currently enabled within the llvm-exegesis subprocess
executor. This can create a lot of core dumps when going through
different snippets that might segfault when experimenting with memory
annotations. These core dumps are not really needed as the information
about the segfault is reported directly to the user.
This patch refactors ExecutableFunction to use a named constructor
pattern, namely adding the create function, so that errors occurring
during the creation of an ExecutableFunction can be propogated back up
rather than having to deal with them in report_fatal_error.
If there were some scheduler effects where something like the parent
process got interrupted while the child process continued to run, there
would be nothing blocking it from exiting before the parent process
issued a PTRACE_ATTACH call. This would cause transient failures as this
occurred pretty rarely. This patch removes the possibility of a
transient failure by ensuring that the parent process attaches to the
child process before sending the counter file descriptor through the
socket, ensuring that the child process has at most progressed to being
blocked in the read call for the counter file descriptor.
There are still some transient failures on the clang-avx512 builder on
the new subprocess memory tests. Some of them seem to be related to an
inability to fork, but it's hard to debug currently as there is no
explicit error handling for a failed fork call, and nice error reporting
for a failed fork is something that we should have regardless.
Some error logging in llvm-exegesis under the subprocess executor just
prints a generic failure information rather than any details about the
error as we omit printing the string version of errno. This patch adds
in printing errno at all relevant points in the subprocess executor that
were previously missed.
Reviewed By: courbet
Differential Revision: https://reviews.llvm.org/D157682
Currently BenchmarkRunner.cpp stores the return code of recvmsg as
size_t. Not only is this incorrect (as recvmsg returns ssize_t), but it
also makes the error code check after the statement completely irrelvant
as it checks if the number of bytes read is greater than zero (which
will always be true for an unsigned type).
When compiling against recent glibc (>= 2.35) but old kernel headers (< 4.18), `SYS_rseq` is not defined and thus llvm-exegesis fails to build. So also check that `SYS_rseq` is defined before trying to use it.
Fixes https://github.com/llvm/llvm-project/issues/64456
Reviewed By: MaskRay, gchatelet
Differential Revision: https://reviews.llvm.org/D157189
Due to arguably a bug in GCC[0], using `__has_builtin` is not sufficient to check whether `__builtin_thread_pointer` can actually be compiled by GCC. This makes it impossible to compile LLVM with `llvm-exegesis` enabled with e.g. GCC 10 as it does have the builtin, but no implementation for architectures such as x86.
This patch works around this issue by making it a cmake configure check whether the builtin can be compiled and used, rather than relying on the broken preprocessor macro.
[0] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96952, demonstration: https://godbolt.org/z/9z5nWM6Ef
Differential Revision: https://reviews.llvm.org/D155828
This patch switches from moving the performance counter file descriptor
to the child process to socket calls rather than using the pidfd_getfd
system call which was introduced in kernel 5.6. This significantly
expands the range of kernel versions that are supported.
Reviewed By: courbet
Differential Revision: https://reviews.llvm.org/D154275
While Clang targets have supported __builtin_thread_pointer for a very
long time (e.g., 2007 for AArch32, 2015 for AArch64), for some GCC
ports, the support is very new (11.0 for x86[1], while we need to
support GCC 7), and many ports haven't implemented
__builtin_thread_pointer yet (m68k, powerpc, etc).
[1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96955
