This patch implements the forwarding to frozen C++03 headers as
discussed in
https://discourse.llvm.org/t/rfc-freezing-c-03-headers-in-libc. In the
RFC, we initially proposed selecting the right headers from the Clang
driver, however consensus seemed to steer towards handling this in the
library itself. This patch implements that direction.
At a high level, the changes basically amount to making each public
header look like this:
```
// inside <vector>
#ifdef _LIBCPP_CXX03_LANG
# include <__cxx03/vector>
#else
// normal <vector> content
#endif
```
In most cases, public headers are simple umbrella headers so there isn't
much code in the #else branch. In other cases, the #else branch contains
the actual implementation of the header.
This will allow using the $<LINK_LIBRARY> generator expression in some
of our configurations. We should separately pursue officially bumping
the minimum CMake version across all LLVM so we can use this feature
more widely.
It makes more sense to start testing libc++ with the latest compiler and
only then to run the LLDB data formatter tests, since that provides more
signal than starting with the data formatter tests.
Instead of building the benchmarks separately via CMake and running them
separately from the test suite, this patch merges the benchmarks into
the test suite and handles both uniformly.
As a result:
- It is now possible to run individual benchmarks like we run tests
(e.g. using libcxx-lit), which is a huge quality-of-life improvement.
- The benchmarks will be run under exactly the same configuration as
the rest of the tests, which is a nice simplification. This does
mean that one has to be careful to enable the desired optimization
flags when running benchmarks, but that is easy with e.g.
`libcxx-lit <...> --param optimization=speed`.
- Benchmarks can use the same annotations as the rest of the test
suite, such as `// UNSUPPORTED` & friends.
When running the tests via `check-cxx`, we only compile the benchmarks
because running them would be too time consuming. This introduces a bit
of complexity in the testing setup, and instead it would be better to
allow passing a --dry-run flag to GoogleBenchmark executables, which is
the topic of https://github.com/google/benchmark/issues/1827.
I am not really satisfied with the layering violation of adding the
%{benchmark_flags} substitution to cmake-bridge, however I believe
this can be improved in the future.
This improves the CI output by providing collapsable sections for
sub-parts of our build.
This was originally opened as #75233.
Co-authored-by: eric <eric@efcs.ca>
We've been increasing the coverage of libc++ LLDB tests in the pre-merge
CI (see https://github.com/llvm/llvm-project/pull/110570). Unfortunately
the tests are spread across different targets. It would be great if we
had a single target that libc++ maintainers could run.
We do this by passing the `libc++` test-category as a parameter to
LLDB's [`dotest` testing
framework](https://lldb.llvm.org/resources/test.html). This will only
run LLDB tests that have been marked as belonging to the `libc++`
category.
This removes the need for macOS nodes in Buildkite. It also moves to the
proper way of testing backdeployment, which is to actually run on the
target OS itself, instead of using packaged dylibs from previous OS
versions and trying to emulate backdeployment with DYLD_LIBRARY_PATH.
As a drive-by change, also fix a few back-deployment annotations that
were incorrect and add support for minor versions in the Lit feature
determining availability from the target triple.
Before changing the compiler version in
https://github.com/llvm/llvm-project/pull/108761, we first of all need
to upgrade the HEAD version to `Clang-20` in the Docker files and push
new builder images to the CI.
When bringing up a new cross compiler from scratch, we build
libunwind/libcxx in a setup where the toolchain is incomplete and unable
to perform the normal linker checks; this requires a few special cases
in the CMake files.
We simulate that scenario by removing the libc++ headers, libunwind and
libc++ libraries from the installed toolchain.
We need to set CMAKE_CXX_COMPILER_WORKS since CMake fails to probe the
compiler. We need to set CMAKE_CXX_COMPILER_TARGET, since LLVM's
heuristics fail when CMake hasn't been able to probe the environment
properly. (This is normal; one has to set those options when setting up
such a toolchain from scratch.)
This adds CI coverage for these build scenarios, which otherwise seldom
are tested by some build flow (but are essential when setting up a cross
compiler from scratch).
We always strive to test libc++ as close as possible to the way we are
actually shipping it. This was approximated reasonably well by setting
up the minimal driver flags when running the test suite, however we were
running the test suite against the library located in the build
directory.
This patch improves the situation by installing the library (the
headers, the built library, modules, etc) into a fake location and then
running the test suite against that fake "installation root".
This should open the door to getting rid of the temporary copy of the
headers we make during the build process, however this is left for a
future improvement.
Note that this adds quite a bit of verbosity whenever running the test
suite because we install the headers beforehand every time. We should be
able to override this to silence it, however CMake doesn't currently
give us a way to do that, see https://gitlab.kitware.com/cmake/cmake/-/issues/26085.
This patch decouples macOS CI testing from BuildKite, which makes the
maintenance of macOS CI easier and more accessible to all contributors.
Right now, the macOS CI is running entirely on machines owned by the
LLVM Foundation with only a small set of contributors having direct
access to them. In particular, updating these machines is currently
a very time-consuming manual process that requires taking the machines
offline, and using Github-provided instances makes that an order of
magnitude easier.
The story for performing back-deployment testing still needs to be
figured out, so for now we are retaining some jobs under BuildKite.
The base of android-buildkite-builder is buildkite-builder, not
android-build-base. android-build-base is only used for its /opt/android
directory, so move the Docker installation step into
android-buildkite-builder.
Install bzip2 for extracting ndk_platform.tar.bz2.
Add "set -e" to RUN heredocs to catch failing commands.
The Android Emulator has started printing this message, so pass the
`-no-metrics` option:
```
##############################################################################
## WARNING - ACTION REQUIRED ##
## Consider using the '-metrics-collection' flag to help improve the ##
## emulator by sending anonymized usage data. Or use the '-no-metrics' ##
## flag to bypass this warning and turn off the metrics collection. ##
## In a future release this warning will turn into a one-time blocking ##
## prompt to ask for explicit user input regarding metrics collection. ##
## ##
## Please see '-help-metrics-collection' for more details. You can use ##
## '-metrics-to-file' or '-metrics-to-console' flags to see what type of ##
## data is being collected by emulator as part of usage statistics. ##
##############################################################################
```
In order to test libc++ under the "Apple System Library" configuration,
we need to run the tests using DYLD_LIBRARY_PATH. This is required
because libc++ gets an install_name of /usr/lib when built as a system
library, which means that we must override the copy of libc++ used by
the whole process. This effectively reverts 2cf2f1b, which was the wrong
solution for the problem I was having.
Of course, this assumes that the just-built libc++ is sufficient to
replace the system library, which is not actually the case
out-of-the-box. Indeed, the system library contains a few symbols that
are not provided by the upstream library, leading to undefined symbols
when replacing the system library by the just-built one.
To solve this problem, we separately build shims that provide those
missing symbols and we manually link against them when we build
executables in the tests. While this is somewhat brittle, it provides a
localized and unintrusive way to allow testing the Apple system
configuration in an upstream environment, which has been a frequent
request.
We were not making any distinction between e.g. the "Apple-flavored"
libc++ built from trunk and the system-provided standard library on
Apple platforms. For example, any test that would be XFAILed on a
back-deployment target would unexpectedly pass when run on that
deployment target against the tip of trunk Apple-flavored libc++. In
reality, that test would be expected to pass because we're running
against the latest libc++, even if it is Apple-flavored.
To solve this issue, we introduce a new feature that describes whether
the Standard Library in use is the one provided by the system by
default, and that notion is different from the underlying standard
library flavor. We also refactor the existing Lit features to make a
distinction between availability markup and the library we're running
against at runtime, which otherwise limit the flexibility of what we can
express in the test suite. Finally, we refactor some of the
back-deployment versions that were incorrect (such as thinking that LLVM
10 was introduced in macOS 11, when in reality macOS 11 was synced with
LLVM 11).
Fixes#82107
This patch makes a few adjustments to the way we run the tests in the
Apple configuration on macOS:
First, we stop using DYLD_LIBRARY_PATH. Using that environment variable
leads to libc++.dylib being replaced by the just-built one for the whole
process, and that assumes compatibility between the system-provided
dylib and the just-built one. Unfortunately, that is not the case
anymore due to typed allocation, which is only available in the system
one. Instead, we want to layer the just-built libc++ on top of the
system-provided one, which seems to be what happens when we set a rpath
instead.
Second, add a missing XFAIL for a std::print test that didn't work as
expected when building with availability annotations enabled. When we
enable these annotations, std::print falls back to a non-unicode and
non-terminal output, which breaks the test.
Since switching the Windows CI environment over to GitHub Actions
runners, the mingw tests run in a setup where the default compiler
binary is a mingw-targeting compiler, so we don't need to specify a
custom executable name.
For the i686 tests, we still specify a custom compiler name, in order to
target i686 instead of x86_64.
This is needed for a workaround to make sure the link later succeeds. I
don't know the reason for that but it is definitely needed.
https://github.com/llvm/llvm-project/pull/89234 will/wants to correct
the triple normalisation for -none- and this means that clang prior to
19, and clang 19 and above will have different answers and therefore
different library paths.
I don't want to bootstrap a clang just for libcxx CI, or require that
anyone building for Arm do the same, so ask the compiler what the triple
should be.
This will be compatible with 17 and 19 when we do update to that
version.
I'm assuming $CC is what anyone locally would set to override the
compiler, and `cc` is the binary name in our CI containers. It's not
perfect but it should cover most use cases.
If LLVM is configured with -DLLVM_DEFAULT_TARGET_TRIPLE, or compiler_rt
is configured with -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE, while the
argument is not normalized, such as Debian-style vendor-less triple,
clang will try to find libclang_rt in lib/<normalized_triple>, while
libclang_rt is placed into lib/<triple_arg>.
Let's also place libclang_rt into lib/<normalized_triple>.
`libcxx/utils/ci/run-buildbot` is also updated to use
`armv7m-none-unknown-eabi` as normalized triple instead of
current `armv7m-none-eabi`.
ChangeLog of this PR:
This patch has been applied and revert twice:
1. The first try is https://github.com/llvm/llvm-project/pull/88407, and
then it is found that it causes some CI failures.
https://lab.llvm.org/buildbot/#/builders/98/builds/36366
It is then resolved by another commit:
1693009679https://lab.llvm.org/buildbot/#/builders/77/builds/36372
It is caused that `COMPILER_RT_DEFAULT_TARGET_TRIPLE` is overwrite
without taking care about `CACHE`.
2. The second try https://github.com/llvm/llvm-project/pull/88835,
resolves https://lab.llvm.org/buildbot/#/builders/77/builds/36372
and in fact only one `execute_process` is needed.
Then we find some other CI failures.
https://github.com/mstorsjo/llvm-mingw/actions/runs/8730621159https://buildkite.com/llvm-project/libcxx-ci/builds/34897#018eec06-612c-47f1-9931-d3bd88bf7ced
It is due to misunderstanding `-print-effective-triple`: which will
output `thumbv7-w64-windows-gnu` for `armv7-w64-windows-gnu` or some
other thumb-enabled arm triples.
In fact we should use `-print-target-triple`.
For armv7m-picolibc, `armv7m-none-eabi` is hardcoded in
libcxx/utils/ci/run-buildbot, while in fact `armv7m-none-unknown-eabi`
is the real normalized triple.
This enables testing of the LLDB libc++ specific data formatters.
This is enabled in the bootstrap build since building LLDB requires
Clang and this
is quite expensive. Adding this test changes the build time from 31 to
34 minutes.
In order to test the LLDB data formatters make is required and SWIG
needs to be updated to version 4.
As drive-by, this patch sorts the entries and removes some duplicates.
The OSS fuzz build has been broken for a while because of changes
to the cmake configuration. This patch fixes the issues by defaulting to
libunwind as the default runtime
