A CMake change included in CMake 4.0 makes `AIX` into a variable
(similar to `APPLE`, etc.)
ff03db6657
However, `${CMAKE_SYSTEM_NAME}` unfortunately also expands exactly to
`AIX` and `if` auto-expands variable names in CMake. That means you get
a double expansion if you write:
`if (${CMAKE_SYSTEM_NAME} MATCHES "AIX")`
which becomes:
`if (AIX MATCHES "AIX")`
which is as if you wrote:
`if (ON MATCHES "AIX")`
You can prevent this by quoting the expansion of "${CMAKE_SYSTEM_NAME}",
due to policy
[CMP0054](https://cmake.org/cmake/help/latest/policy/CMP0054.html#policy:CMP0054)
which is on by default in 4.0+. Most of the LLVM CMake already does
this, but this PR fixes the remaining cases where we do not.
Some code paths normalize ".." and thus don't create the directory. But some execute in a
shell thus requiring the directory to exist to be able to take the parent directory.
This patch normalizes all the `TARGET_SUBDIR` variables to avoid this issue.
The comment seems to be wrong by now. The only difference this makes for
GCC is that there are fewer private symbols exported from the dylib,
which can't ever be accessed by user code anyways.
This was needed before https://github.com/llvm/llvm-project/pull/115077
since the compiler-rt test build made assumptions about the build
layout of libc++ and libc++abi, but now they link against a local
installation of these libraries so we no longer need this workaround.
This moves the configuration of the CMake features to turn off RTTI,
exceptions and friends to the beginning of the CMake file, where we
configure other optional parts of the library.
This fixes an important bug where we would disable the benchmarks
because these options were not defined yet, leading to the build
thinking they were defined to OFF.
Instead of building the benchmarks separately via CMake and running them
separately from the test suite, this patch merges the benchmarks into
the test suite and handles both uniformly.
As a result:
- It is now possible to run individual benchmarks like we run tests
(e.g. using libcxx-lit), which is a huge quality-of-life improvement.
- The benchmarks will be run under exactly the same configuration as
the rest of the tests, which is a nice simplification. This does
mean that one has to be careful to enable the desired optimization
flags when running benchmarks, but that is easy with e.g.
`libcxx-lit <...> --param optimization=speed`.
- Benchmarks can use the same annotations as the rest of the test
suite, such as `// UNSUPPORTED` & friends.
When running the tests via `check-cxx`, we only compile the benchmarks
because running them would be too time consuming. This introduces a bit
of complexity in the testing setup, and instead it would be better to
allow passing a --dry-run flag to GoogleBenchmark executables, which is
the topic of https://github.com/google/benchmark/issues/1827.
I am not really satisfied with the layering violation of adding the
%{benchmark_flags} substitution to cmake-bridge, however I believe
this can be improved in the future.
Instead of placing artifacts for testing the runtimes at <build>/test,
place those artifacts at <build>/<project>/test. This prevents
cluttering the build directory with the runtimes' test artifacts for
everyone else.
As a drive-by, remove LIBCXX_BINARY_INCLUDE_DIR which wasn't used
anymore.
These were split in 0e8208eca1c316b7302de7803ab0d85a1dd77076, with the
only functional difference between them at the time being `--prepend_env
PATH=%{lib-dir}` in the static config and `--prepend_env
PATH=%{install-prefix}/bin` in the shared library config.
However this difference is unnecessary - the static library config
doesn't need any `--prepend_env` argument at all. Before
0e8208eca1c316b7302de7803ab0d85a1dd77076, both configurations used the
same config file, where the `--prepend_env` argument was unnecessary but
benign in the static case.
Reduce the unnecessary config duplication in this case, and return these
configs to using one single config file for both setups.
We always strive to test libc++ as close as possible to the way we are
actually shipping it. This was approximated reasonably well by setting
up the minimal driver flags when running the test suite, however we were
running the test suite against the library located in the build
directory.
This patch improves the situation by installing the library (the
headers, the built library, modules, etc) into a fake location and then
running the test suite against that fake "installation root".
This should open the door to getting rid of the temporary copy of the
headers we make during the build process, however this is left for a
future improvement.
Note that this adds quite a bit of verbosity whenever running the test
suite because we install the headers beforehand every time. We should be
able to override this to silence it, however CMake doesn't currently
give us a way to do that, see https://gitlab.kitware.com/cmake/cmake/-/issues/26085.
- Landing page: add link to the libc++ Discord channel
- Landing page: reorder "Getting Involved" above "Design documents"
- Landing page: remove "Notes and Known Issues" which was completely outdated
- Rename "Using Libc++" to "User Documentation" and update contents
- Rename "Building Libc++" to "Vendor Documentation" and update contents
The "BuildingLibcxx" and "UsingLibcxx" pages have basically been used for
vendor and user documentation respectively. However, they were named in
a way that doesn't really make that clear. Renaming the pages now gives
us a location to clearly document what we target at vendors and what we
target at users, and to do that separately.
This is an intermediate and fairly mechanical step towards unifying the
benchmarks with the rest of the test suite. Moving this around requires
a few changes, notably making sure we don't throw a wrench into the
discovery process of the normal test suite. This won't be a problem
anymore once benchmarks are taken into account by the test setup out of
the box.
This patch removes support for generating code coverage information of
libc++, which is unmaintained and I've never been able to utilize. It
would be great to have support for generating code coverage in the test
suite, however this incarnation of it doesn't seem to be properly hooked
up into the test suite. Since it gets in the way of making the test
suite more standalone, remove it.
The premise of LIBCXX_CONFIGURE_IDE is nice, however in practice this
setting has not been maintained and as a result it basically doesn't
work properly. For example, it doesn't take into account the headers we
generate, and its handling of the tests is too naive for it to be really
helpful.
This patch removes the setting in order to simplify the CMake setup a
bit, but most importantly to remove unnecessary interactions between our
main CMake build of the library and the test suite.
There were some ad-hoc settings that allowed running the benchmarks
against the native C++ Standard Library. While this ability is very
useful, it was done before the test suite was quite independent of
libc++ itself. Instead, it is better to streamline running the
benchmarks on the native standard library by using a custom Lit
configuration like we do with the test suite.
A follow-up patch will rework the integration of benchmarks with the Lit
configuration used for the test suite so that we can reuse the same
mechanism for both, making it easy to benchmark the native standard
library.
It will also make benchmarks way more user-friendly to run since we will
be able to run them like we run individual tests, which is a pain point
right now.
This patch makes libc++ build with -fsized-deallocation. That flag is
enabled by default in recent versions of Clang, so this patch will make
libc++ forward-compatible with ToT Clang.
We weren't applying the libdir subdir to header directories but this is
necessary for correctness when building e.g. ASan variant. This change
also updates path construction logic accross all runtimes and ensures
they're consistent.
Instead of using FOO_TEST_DEPS global variables that don't get updated
properly from subdirectories, use targets to propagate the dependencies
across directories.
In essence, this header has always been related to configuration of
the library but we didn't want to put it inside <__config> due to
complexity reasons. Now that we have sub-headers in <__config>, we
can move <__availability> to it and stop including it everywhere since
we already obtain the required macros via <__config>.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
This patch removes the two-level backend dispatching mechanism we had in
the PSTL. Instead of selecting both a PSTL backend and a PSTL CPU
backend, we now only select a top-level PSTL backend. This greatly
simplifies the PSTL configuration layer.
While this patch technically removes some flexibility from the PSTL
configuration mechanism because CPU backends are not considered
separately, it opens the door to a much more powerful configuration
mechanism based on chained backends in a follow-up patch.
This is a step towards overhauling the PSTL dispatching mechanism.
This reverts commit 4109b18ee5de1346c2b89a5c89b86bae5c8631d3.
It looks like the automatic detection has false positives. This broke
the following build https://github.com/llvm/llvm-project/pull/85262
The libc++ CMake build aims to provide a general-purpose configuration
that matches the canonical LLVM setup (libc++ / libc++abi / libunwind),
not the configuration used for compatibility on any given system. For
these "compatibility" configurations, we use CMake caches like
Apple.cmake and FreeBSD.cmake.
Defaulting to libcxxrt on FreeBSD makes it look as though we're trying
to build the system compatible configuration on FreeBSD, which isn't
really correct and causes confusion more than anything else.
Fixes#84476
Installs the source files of the experimental libc++ modules. These
source files (.cppm) are used by the Clang to build the std and
std.compat modules.
The design of this patch is based on a discussing in SG-15 on
12.12.2023. (SG-15 is the ISO C++ Tooling study group):
- The modules are installed at a location, that is not known to build
systems and compilers.
- Next to the library there will be a module manifest json file.
This json file contains the information to build the module from the
libraries sources. This information includes the location where the
sources are installed. @ruoso supplied the specification of this json
file.
- If possible, the compiler has an option to give the location of the
module manifest file
(https://github.com/llvm/llvm-project/pull/76451).
Currently there is no build system support, but it expected to be added
in the future.
Fixes: https://github.com/llvm/llvm-project/issues/73089
Revert "Revert #76246 and #76083"
This reverts commit 5c150e7eeba9db13cc65b329b3c3537b613ae61d.
Adds a small fix that should properly disable the tests on Windows.
Unfortunately the original poster has not provided feedback and the
original patch did not fail in the LLVM CI infrastructure.
Modules are known to fail on Windows due to non compliance of the
C library. Currently not having this patch prevents testing on other
platforms.
These cause test build failures on Windows.
This reverts the following commits:
57ca74843586c9a93c425036c5538aae0a2cfa60
d06ae33ec32122bb526fb35025c1f0cf979f1090
Previously there were two ways to override the verbose abort function
which gets called when a hardening assertion is triggered:
- compile-time: define the `_LIBCPP_VERBOSE_ABORT` macro;
- link-time: provide a definition of `__libcpp_verbose_abort` function.
This patch adds a new configure-time approach: the vendor can provide
a path to a custom header file which will get copied into the build by
CMake and included by the library. The header must provide a definition
of the
`_LIBCPP_ASSERTION_HANDLER` macro which is what will get called should
a hardening assertion fail. As of this patch, overriding
`_LIBCPP_VERBOSE_ABORT` will still work, but the previous mechanisms
will be effectively removed in a follow-up patch, making the
configure-time mechanism the sole way of overriding the default handler.
Note that `_LIBCPP_ASSERTION_HANDLER` only gets invoked when a hardening
assertion fails. It does not affect other cases where
`_LIBCPP_VERBOSE_ABORT` is currently used (e.g. when an exception is
thrown in the `-fno-exceptions` mode).
The library provides a default version of the custom header file that
will get used if it's not overridden by the vendor. That allows us to
always test the override mechanism and reduces the difference in
configuration between the pristine version of the library and
a platform-specific version.
This removes the entire modules testing infrastructure.
The current infrastructure uses CMake to generate the std and std.compat
module. This requires quite a bit of plumbing and uses CMake. Since
CMake introduced module support in CMake 3.26, modules have a higher
CMake requirement than the rest of the LLVM project. (The LLVM project
requires 3.20.) The main motivation for this approach was how libc++
generated its modules. Every header had its own module partition. This
was changed to improve performance and now only two modules remain. The
code to build these can be manually crafted.
A followup patch will reenable testing modules, using a different
approach.
I recently came across LIBCXXABI_USE_LLVM_UNWINDER and was surprised to
notice it was disabled by default. Since we build libunwind by default
and ship it in the LLVM toolchain, it would seem to make sense that
libc++ and libc++abi rely on libunwind for unwinding instead of using
the system-provided unwinding library (if any).
Most importantly, using the system unwinder implies that libc++abi is
ABI compatible with that system unwinder, which is not necessarily the
case. Hence, it makes a lot more sense to instead default to using the
known-to-be-compatible LLVM unwinder, and let vendors manually select a
different unwinder if desired.
As a follow-up change, we should probably apply the same default to
compiler-rt.
Differential Revision: https://reviews.llvm.org/D150897Fixes#77662
rdar://120801778
When we use the -nostdlib++ flag, we don't need to explicitly link
against compiler-rt, since the compiler already links against it by
default. This simplifies the flags that we need to use when building
with Clang and GCC, and opens the door to further simplifications since
most platforms won't need to detect whether libgcc and libgcc_s are
supported anymore.
Furthermore, on platforms where -nostdlib++ is used, this patch prevents
manually linking compiler-rt *before* other system libraries. For
example, Apple platforms have several compiler-rt symbols defined in
libSystem.dylib. If we manually link against compiler-rt, we end up
overriding the default link order preferred by the compiler and
potentially using the symbols from the clang-provided libclang_rt.a
library instead of the system provided one.
Note that we don't touch how libunwind links against compiler-rt when it
builds the .so/.a because libunwind currently doesn't use -nodefaultlibs
and we want to avoid rocking the boat too much.
rdar://119506163
This commit introduces basic annotations for `std::basic_string`,
mirroring the approach used in `std::vector` and `std::deque`.
Initially, only long strings with the default allocator will be
annotated. Short strings (_SSO - short string optimization_) and strings
with non-default allocators will be annotated in the near future, with
separate commits dedicated to enabling them. The process will be similar
to the workflow employed for enabling annotations in `std::deque`.
**Please note**: these annotations function effectively only when libc++
and libc++abi dylibs are instrumented (with ASan). This aligns with the
prevailing behavior of Memory Sanitizer.
To avoid breaking everything, this commit also appends
`_LIBCPP_INSTRUMENTED_WITH_ASAN` to `__config_site` whenever libc++ is
compiled with ASan. If this macro is not defined, string annotations are
not enabled. However, linking a binary that does **not** annotate
strings with a dynamic library that annotates strings, is not permitted.
Originally proposed here: https://reviews.llvm.org/D132769
Related patches on Phabricator:
- Turning on annotations for short strings:
https://reviews.llvm.org/D147680
- Turning on annotations for all allocators:
https://reviews.llvm.org/D146214
This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.
The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.
This Pull Request introduces basic annotations for `std::basic_string`.
Long strings exhibit structural similarities to `std::vector` and will
be annotated accordingly. Short strings are already implemented, but
will be turned on separately in a forthcoming commit. Look at [a
comment](https://github.com/llvm/llvm-project/pull/72677#issuecomment-1850554465)
below to read about SSO issues at current moment.
Due to the functionality introduced in
[D132522](dd1b7b797a),
the `__sanitizer_annotate_contiguous_container` function now offers
compatibility with all allocators. However, enabling this support will
be done in a subsequent commit. For the time being, only strings with
the default allocator will be annotated.
If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com
There are a few drive-by fixes:
- Since the combination RTTI disabled and exceptions enabled do not
work, this combination is prohibited.
- A small NFC in any fixing clang-tidy.
The code in the Buildkite configuration is prepared for using the std
module. There are more fixes needed for that configuration which will be
done in a separate commit.
Patch 4b1fe097f9a3882f437bc3b829ef02331e28a8d6 introduced a bug when
building libc++ for Fuchsia, it disabled the libc++.so linker script by
default. This patch restores its original behavior.
We were detecting which sanitizer flags to use when building
libc++.dylib but we were never actually adding those flags to the
targets, which means that our sanitized builds would basically build the
dylib without any sanitizers enabled.
We really shouldn't be depending on far away configuration options like
LLVM_HAVE_LINK_VERSION_SCRIPT here. This patch simplifies the enablement
of the linker scripts and as a result gets rid of an undesirable
dependency on HandleLLVMOptions.cmake.
As a drive-by, the patch also stops taking into account whether Python3
is available. This should have no bearing on whether we generate a
linker script or not, which is required for correctness. If someone
tries to build libc++ and generate a linker script but Python3 is not
available, they should get an error instead of silently getting an
incorrect installation of the library.
1. Instead of using individual "boolean" macros, have an "enum" macro
`_LIBCPP_HARDENING_MODE`. This avoids issues with macros being
mutually exclusive and makes overriding the hardening mode within a TU
more straightforward.
2. Rename the safe mode to debug-lite.
This brings the code in line with the RFC:
https://discourse.llvm.org/t/rfc-hardening-in-libc/73925Fixes#65101
This is necessary in order to implement some papers like P2467R1, which
require using C++23 declarations in the dylib. It is a good habit to
keep building the dylib with a recent standard version regardless.
With this patch, we also stop strictly enforcing that the targets are
built with C++23. Concretely, C++23 will soon be required in order to
build the dylib, but not enforcing it strictly works around some issues
like the documentation bots using an old and unsupported compiler. Since
these bots do not actually build the library, not strictly enforcing the
C++ Standard makes our CMake build more resilient to these kinds of
situation. This is just a workaround though, the better way of going
about would be to update the compiler on the documentation bot but we
don't seem to have control over that.
