~~NB: This PR depends on #78876. Ignore the first commit when reviewing,
and don't merge it until #78876 is resolved. When/if #78876 lands, I'll
clean this up.~~
This partially restores parity with the old, since removed debug build.
We now can re-enable a bunch of the disabled tests. Some things of note:
- `bounded_iter`'s converting constructor has never worked. It needs a
friend declaration to access the other `bound_iter` instantiation's
private fields.
- The old debug iterators also checked that callers did not try to
compare iterators from different objects. `bounded_iter` does not
currently do this, so I've left those disabled. However, I think we
probably should add those. See
https://github.com/llvm/llvm-project/issues/78771#issuecomment-1902999181
- The `std::vector` iterators are bounded up to capacity, not size. This
makes for a weaker safety check. This is because the STL promises not to
invalidate iterators when appending up to the capacity. Since we cannot
retroactively update all the iterators on `push_back()`, I've instead
sized it to the capacity. This is not as good, but at least will stop
the iterator from going off the end of the buffer.
There was also no test for this, so I've added one in the `std`
directory.
- `std::string` has two ambiguities to deal with. First, I opted not to
size it against the capacity. https://eel.is/c++draft/string.require#4
says iterators are invalidated on an non-const operation. Second,
whether the iterator can reach the NUL terminator. The previous debug
tests and the special-case in https://eel.is/c++draft/string.access#2
suggest no. If either of these causes widespread problems, I figure we
can revisit.
- `resize_and_overwrite.pass.cpp` assumed `std::string`'s iterator
supported `s.begin().base()`, but I see no promise of this in the
standard. GCC also doesn't support this. I fixed the test to use
`std::to_address`.
- `alignof.compile.pass.cpp`'s pointer isn't enough of a real pointer.
(It needs to satisfy `NullablePointer`, `LegacyRandomAccessIterator`,
and `LegacyContiguousIterator`.) `__bounded_iter` seems to instantiate
enough to notice. I've added a few more bits to satisfy it.
Fixes#78805
This patch removes many annotations that are not relevant anymore since
we don't support or test back-deploying to macOS < 10.13. It also cleans
up raw usage of target triples to identify versions of dylibs shipped on
prior versions of macOS, and uses the target-agnostic Lit features
instead. Finally, it reorders both the Lit backdeployment features and
the corresponding availability macros in the library in a way that makes
more sense, and reformulates the Lit backdeployment features in terms of
when a version of LLVM was introduced instead of encoding the system
versions on which it hasn't been introduced yet. Although one can be
derived from the other, encoding the negative form is extremely
error-prone.
Fixes#80901
We were not making any distinction between e.g. the "Apple-flavored"
libc++ built from trunk and the system-provided standard library on
Apple platforms. For example, any test that would be XFAILed on a
back-deployment target would unexpectedly pass when run on that
deployment target against the tip of trunk Apple-flavored libc++. In
reality, that test would be expected to pass because we're running
against the latest libc++, even if it is Apple-flavored.
To solve this issue, we introduce a new feature that describes whether
the Standard Library in use is the one provided by the system by
default, and that notion is different from the underlying standard
library flavor. We also refactor the existing Lit features to make a
distinction between availability markup and the library we're running
against at runtime, which otherwise limit the flexibility of what we can
express in the test suite. Finally, we refactor some of the
back-deployment versions that were incorrect (such as thinking that LLVM
10 was introduced in macOS 11, when in reality macOS 11 was synced with
LLVM 11).
Fixes#82107
Unconditionally change std::string's alignment to 8.
This change saves memory by providing the allocator more freedom to
allocate the most
efficient size class by dropping the alignment requirements for
std::string's
pointer from 16 to 8. This changes the output of std::string::max_size,
which makes it ABI breaking.
That said, the discussion concluded that we don't care about this ABI
break. and would like this change enabled universally.
The ABI break isn't one of layout or "class size", but rather the value
of "max_size()" changes, which in turn changes whether `std::bad_alloc`
or `std::length_error` is thrown for large allocations.
This change is the child of PR #68807, which enabled the change behind
an ABI flag.
This commit introduces basic annotations for `std::basic_string`,
mirroring the approach used in `std::vector` and `std::deque`.
Initially, only long strings with the default allocator will be
annotated. Short strings (_SSO - short string optimization_) and strings
with non-default allocators will be annotated in the near future, with
separate commits dedicated to enabling them. The process will be similar
to the workflow employed for enabling annotations in `std::deque`.
**Please note**: these annotations function effectively only when libc++
and libc++abi dylibs are instrumented (with ASan). This aligns with the
prevailing behavior of Memory Sanitizer.
To avoid breaking everything, this commit also appends
`_LIBCPP_INSTRUMENTED_WITH_ASAN` to `__config_site` whenever libc++ is
compiled with ASan. If this macro is not defined, string annotations are
not enabled. However, linking a binary that does **not** annotate
strings with a dynamic library that annotates strings, is not permitted.
Originally proposed here: https://reviews.llvm.org/D132769
Related patches on Phabricator:
- Turning on annotations for short strings:
https://reviews.llvm.org/D147680
- Turning on annotations for all allocators:
https://reviews.llvm.org/D146214
This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.
The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.
This Pull Request introduces basic annotations for `std::basic_string`.
Long strings exhibit structural similarities to `std::vector` and will
be annotated accordingly. Short strings are already implemented, but
will be turned on separately in a forthcoming commit. Look at [a
comment](https://github.com/llvm/llvm-project/pull/72677#issuecomment-1850554465)
below to read about SSO issues at current moment.
Due to the functionality introduced in
[D132522](dd1b7b797a),
the `__sanitizer_annotate_contiguous_container` function now offers
compatibility with all allocators. However, enabling this support will
be done in a subsequent commit. For the time being, only strings with
the default allocator will be annotated.
If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com
Extend `std::basic_string` tests to cover more buffer situations and
length in general, particularly non-SSO cases after SSO test cases
(changing buffers). This commit is a side effect of working on tests for
ASan annotations.
Related PR: https://github.com/llvm/llvm-project/pull/72677
While doing this, I also found a few tests that were either clearly
incorrect (e.g. testing the wrong function) or that lacked basic test
coverage like testing std::string itself (e.g. the test was only checking
std::basic_string with a custom allocator). In these cases, I did a few
conservative drive-by changes.
Differential Revision: https://reviews.llvm.org/D140550
Co-authored-by: Brendan Emery <brendan.emery@esrlabs.com>
The use_system_cxx_lib Lit feature was only used for back-deployment
testing. However, one immense hole in that setup was that we didn't
have a proper way to test Apple's own libc++ outside of back-deployment,
which was embodied by the fact that we needed to define _LIBCPP_DISABLE_AVAILABILITY
when testing (see change in libcxx/utils/libcxx/test/params.py).
This led to the apple-system testing configuration not checking for
availability markup, which is obviously quite bad since the library
we ship actually has availability markup.
Using stdlib=<VENDOR>-libc++ instead to encode back-deployment restrictions
on tests is simpler and it makes it possible to naturally support tests
such as availability markup checking even in the tip-of-trunk Apple-libc++
configuration.
Differential Revision: https://reviews.llvm.org/D146366
This has been done using the following command
find libcxx/test -type f -exec perl -pi -e 's|^([^/]+?)((?<!::)size_t)|\1std::\2|' \{} \;
And manually removed some false positives in std/depr/depr.c.headers.
The `std` module doesn't export `::size_t`, this is a preparation for that module.
Reviewed By: ldionne, #libc, EricWF, philnik
Differential Revision: https://reviews.llvm.org/D146088
We pretty consistently don't define those cause they are not needed,
and it removes the potential pitfall to think that these tests are
being run. This doesn't touch .compile.fail.cpp tests since those
should be replaced by .verify.cpp tests anyway, and there would be
a lot to fix up.
As a fly-by, I also fixed a bit of formatting, removed a few unused
includes and made some very minor, clearly NFC refactorings such as
in allocator.traits/allocator.traits.members/allocate.verify.cpp where
the old test basically made no sense the way it was written.
Differential Revision: https://reviews.llvm.org/D146236
This patch switches the build compiler for AIX from ibm-clang to clang. ibm-clang++_r has `-pthread` by default, but clang for AIX doesn't, so `-pthread` had to be added to the test config. A bunch of tests now pass, so the `XFAIL` was removed. This patch also switch the build to use the visibility support available in clang-15 to control symbols exported by the shared library (AIX traditionally uses explicit export lists for this purpose).
Reviewed By: #libc, #libc_abi, daltenty, #libunwind, ldionne
Differential Revision: https://reviews.llvm.org/D127470
Since those features are general properties of the environment, it makes
sense to use them from libc++abi too, and so the name libcpp-has-no-xxx
doesn't make sense.
Differential Revision: https://reviews.llvm.org/D126482
This makes the code a bit simpler and (I think) removes the undefined behaviour from the normal string layout.
Reviewed By: ldionne, Mordante, #libc
Spies: labath, dblaikie, JDevlieghere, krytarowski, jgorbe, jingham, saugustine, arichardson, libcxx-commits
Differential Revision: https://reviews.llvm.org/D123580
These are the last™ changes to the tests for constexpr preparation.
Reviewed By: Quuxplusone, #libc, Mordante
Spies: Mordante, EricWF, libcxx-commits
Differential Revision: https://reviews.llvm.org/D120951
... from testing with MSVC's STL. Mostly truncation warnings and variables that are only used in `LIBCPP_ASSERT`.
Differential Revision: https://reviews.llvm.org/D116878
As explained in https://stackoverflow.com/a/70339311/627587, the fact
that shrink_to_fit wasn't defined as inline lead to issues when explicitly
instantiating basic_string. While explicit instantiations are always
somewhat brittle, this one was clearly a bug on our end.
Differential Revision: https://reviews.llvm.org/D115656
Make test_allocator etc. constexpr-friendly so they can be used to test constexpr string and possibly constexpr vector
Reviewed By: Quuxplusone, #libc, ldionne
Differential Revision: https://reviews.llvm.org/D110994
Even if these comments have a benefit in .h files (for editors that
care about language but can't be configured to treat .h as C++ code),
they certainly have no benefit for files with the .cpp extension.
Discussed in D110794.
Now that Lit supports regular expressions inside XFAIL & friends, it is
much easier to write Lit annotations based on the triple.
Differential Revision: https://reviews.llvm.org/D104747
This fixes a long standing issue where the triple is not always set
consistently in all configurations. This change also moves the
back-deployment Lit features to using the proper target triple
instead of using something ad-hoc.
This will be necessary for using from scratch Lit configuration files
in both normal testing and back-deployment testing.
Differential Revision: https://reviews.llvm.org/D102012
C++98 and C++03 are effectively aliases as far as Clang is concerned.
As such, allowing both std=c++98 and std=c++03 as Lit parameters is
just slightly confusing, but provides no value. It's similar to allowing
both std=c++17 and std=c++1z, which we don't do.
This was discovered because we had an internal bot that ran the test
suite under both c++98 AND c++03 -- one of which is redundant.
Differential Revision: https://reviews.llvm.org/D80926
Tests that require support for Clang-verify are already marked as such
explicitly by their extension, which is .verify.cpp. Requiring the use
of an explicit Lit feature is, after thought, not really helpful.
This is a change in design: we have been bitten in the past by tests not
being enabled when we thought they were. However, the issue was mostly
with file extensions being ignored. The fix for that is not to blindly
require explicit features all the time, but instead to report all files
that are in the suite but that don't match any known test format. This
can be implemented in a follow-up patch.
Instead of having different names for the same Lit feature accross code
bases, use the same name everywhere. This NFC commit is in preparation
for a refactor where all three projects will be using the same Lit
feature detection logic, and hence it won't be convenient to use
different names for the feature.
Differential Revision: https://reviews.llvm.org/D78370
The libc++ test suite has a lot of old Lit features used to XFAIL tests
and mark them as UNSUPPORTED. Many of them are to workaround problems on
old compilers or old platforms. As time goes by, it is good to go and
clean those up to simplify the configuration of the test suite, and also
to reflect the testing reality. It's not useful to have markup that gives
the impression that e.g. clang-3.3 is supported, when we don't really
test on it anymore (and hence several new tests probably don't have the
necessary markup on them).
Forcing -Werror and other warnings means that the test suite isn't
actually testing what most people are seeing in their code -- it seems
better and less arbitrary to compile these tests as close as possible
to the compiler default instead.
Removing -Werror also means that we get to differentiate between
diagnostics that are errors and those that are warnings, which makes
the test suite more precise.
Differential Revision: https://reviews.llvm.org/D76311
Some tests do not fail at all when -verify is not supported, unless some
arbitrary warning flag is added to make them fail. We currently used
-Werror=unused-result to make them fail, but doing so makes the test
suite a lot more inscrutable. It seems better to just disable those
tests when -verify is not supported.
Differential Revision: https://reviews.llvm.org/D76256
Those tests fail when linking against a new dylib but running against
macosx10.7. I believe this is caused by a duplicate definition of the
RTTI for exception classes in libc++.dylib and libc++abi.dylib, but
this matter still needs some investigation.
This issue was not caught previously because all the tests always linked
against the same dylib used for running (because LIT made it impossible
to do otherwise before r349171).
rdar://problem/46809586
llvm-svn: 354940
Summary:
Freestanding is *weird*. The standard allows it to differ in a bunch of odd
manners from regular C++, and the committee would like to improve that
situation. I'd like to make libc++ behave better with what freestanding should
be, so that it can be a tool we use in improving the standard. To do that we
need to try stuff out, both with "freestanding the language mode" and
"freestanding the library subset".
Let's start with the super basic: run the libc++ tests in freestanding, using
clang as the compiler, and see what works. The easiest hack to do this:
In utils/libcxx/test/config.py add:
self.cxx.compile_flags += ['-ffreestanding']
Run the tests and they all fail.
Why? Because in freestanding `main` isn't special. This "not special" property
has two effects: main doesn't get mangled, and main isn't allowed to omit its
`return` statement. The first means main gets mangled and the linker can't
create a valid executable for us to test. The second means we spew out warnings
(ew) and the compiler doesn't insert the `return` we omitted, and main just
falls of the end and does whatever undefined behavior (if you're luck, ud2
leading to non-zero return code).
Let's start my work with the basics. This patch changes all libc++ tests to
declare `main` as `int main(int, char**` so it mangles consistently (enabling us
to declare another `extern "C"` main for freestanding which calls the mangled
one), and adds `return 0;` to all places where it was missing. This touches 6124
files, and I apologize.
The former was done with The Magic Of Sed.
The later was done with a (not quite correct but decent) clang tool:
https://gist.github.com/jfbastien/793819ff360baa845483dde81170feed
This works for most tests, though I did have to adjust a few places when e.g.
the test runs with `-x c`, macros are used for main (such as for the filesystem
tests), etc.
Once this is in we can create a freestanding bot which will prevent further
regressions. After that, we can start the real work of supporting C++
freestanding fairly well in libc++.
<rdar://problem/47754795>
Reviewers: ldionne, mclow.lists, EricWF
Subscribers: christof, jkorous, dexonsmith, arphaman, miyuki, libcxx-commits
Differential Revision: https://reviews.llvm.org/D57624
llvm-svn: 353086
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351648
Libc++ is used as a system library on macOS and iOS (amongst others). In order
for users to be able to compile a binary that is intended to be deployed to an
older version of the platform, clang provides the
availability attribute <https://clang.llvm.org/docs/AttributeReference.html#availability>_
that can be placed on declarations to describe the lifecycle of a symbol in the
library.
See docs/DesignDocs/AvailabilityMarkup.rst for more information.
Differential Revision: https://reviews.llvm.org/D31739
llvm-svn: 302172
