Summary:
This was previously a Trieber stack, which is a perfectly fine generic
and lock-free data structure. However, this used some expensive CAS
operations and had issues with ABA. Because the only user of this was
the slab cache mechanism, we can pretty safely specialize it. Instead,
we simply search a fixed size buffer for some sentinal values and CAS
into it.
For allocations that only ever hit the cache, this improves performance
from ~9000 cycles to ~6000 cycles and similar improvements for workloads
that feel the pain of small thread counts hitting the cache.
The lit-based test runner introduced in c776a52f only discovered
libc.test.src tests with a strict __unit__.__build__ or
__hermetic__.__build__ suffix. This missed four categories of tests:
1. libc.test.include.* tests (e.g. isnan_test, signbit_test)
2. libc.test.integration.* tests (e.g. pthread, unistd, startup)
3. libc.test.src.* tests that have no __unit__/__hermetic__ marker (e.g.
errno_test, dirent_test, htonl)
4. libc.test.src.* tests with extra option suffixes between the type
marker and .__build__ (e.g. __unit__.__NO_FMA_OPT.__build__)
Wire up the two missing build dependencies so that check-libc-lit builds
include and integration tests before running them, and update
_isTestExecutable() to recognise all four patterns.
The pattern documentation was consolidated into the _isTestExecutable()
docstring, where it is next to the code it describes, to avoid the two
diverging in future.
Tested:
Compared the test count from a full `ninja check-libc` run (2765 tests)
against `llvm-lit --show-tests libc/test` after this change and
confirmed the counts match exactly.
The lit-based test execution added in c776a52fa263 did not propagate the
GPU loader executable to the lit site configuration. When building for
GPU targets, the loader (amdhsa-loader, nvptx-loader, or a
user-specified LIBC_GPU_LOADER_EXECUTABLE) was resolved into the
libc.utils.gpu.loader target but never passed through to the generated
lit.site.cfg.py.
Fix this by resolving the executable path from the libc.utils.gpu.loader
target property and including it in configure_lit_site_cfg's PATHS list.
In lit.site.cfg.py.in, if no explicit LIBC_TEST_CMD is set but a GPU
loader is present, construct libc_test_cmd from the loader path,
mirroring the logic in add_libc_hermetic() in LLVMLibCTestRules.cmake.
Tested:
* GPU build with LIBC_GPU_LOADER_EXECUTABLE set to /bin/echo used to
fail (tried to run the binary), now it works.
* Native compiler check-libc-lit continues to work after this change.
Implementing `asinpi` for single-precision. it continues what is done in
#152690 that implemented `asinpif16` with header-only approach that is
followed since #147386
Add implementation for `atan2l` that falls back to `atan2f128` if
float128 support is available.
We do this for now in lieu of 80-bit-specific implementation. Going
forward, we should be choosing 64-bit, 80-bit, or 128-bit specific
implementation based on the specific "long double"
implementation. Also, once llvm-libc will have its own software
implementation of float128, depending on host type presence
would not be needed.
`atan2l` is one of the remaining dependencies needed for building libc++
against llvm-libc (it's used in `<complex>` header).
Older code may use `st_atime` which recorded timestamps with one-second
precision, instead of `struct timespec st_atim` that is available in
later POSIX versions.
Add `#define st_atime` (& friends) to type declaration as suggested in
https://man7.org/linux/man-pages/man3/stat.3type.html
Resolves ifunc targets before `main()` runs in static libc
This enables static binaries to use ifunc-based dispatch during early
process startup, so optimized implementations can be selected based on
CPU features. Without this relocation step in startup, those targets are
not ready when program code begins executing.
This change:
- adds IRELATIVE relocation handling for x86_64, AArch64, ARMv7 and RISC-V,
- reads `AT_HWCAP` / `AT_HWCAP2` from auxv and passes them to resolvers
where required (notably AArch64),
- runs IRELATIVE processing after base-address discovery and before TLS
setup,
- adds integration tests for both the ifunc path and the no-ifunc path,
- Changed the load bias type for ptrdiff_t to intptr_t to align with
IRELATIVE handling, which uses intptr_t for load bias calculations.
Refactors the payload_functions math family to be header-only.
part of: https://github.com/llvm/llvm-project/issues/181823
Target Functions:
- getpayload
- getpayloadbf16
- getpayloadf
- getpayloadf128
- getpayloadf16
Summary:
The NVIDIA ITS protocol allows lanes to diverge inside of a warp. We
previously had contingencies around this, but there were cases where
issues would still show up under highly stressed usage.
The rules state that as long as the PC is the same, threads can
reconverge. This means that we can see a 'convergent' warp even when
they took completely divergent paths to get there. This resulted in the
'index' value in the RPC port lookup loop thinking we were in a
convergent group while all the indices were different. Fix this with a
broadcast to force the expected behavior
Additionally, we did not force that the threads were actually done with
their 'work_fn'. If the work included something that caused divergence
the other threads could continue and toggle the mailbox, resulting in
the server seeing unfinished work. Fix this with an explicit sync and
have one thread do it.
Add a test to make sure this actually works.
Created mathvec directories and unittest framework for vector math
functions, as well as an initial implementation of vector expf, which is
presently CR for round-to-nearest.
---------
Co-authored-by: Pierre Blanchard <pierre.blanchard@arm.com>
