There are currently lots of `_LIBCPP_HAS_ASAN` and
`__libcpp_is_constant_evaluated()` checks which aren't needed, since it
is centrally checked inside `__debug_utils/sanitizers.h`.
Without this patch `basic_string` cannot be properly resized to be
`max_size()` elements in size, even if an allocation is successful.
`__grow_by` allocates one less element than required, resulting in an
out-of-bounds access. At the same time, `max_size()` has an off-by-one
error, since there has to be space to store the null terminator, which
is currently ignored.
This commit introduces basic annotations for `std::basic_string`,
mirroring the approach used in `std::vector` and `std::deque`.
Initially, only long strings with the default allocator will be
annotated. Short strings (_SSO - short string optimization_) and strings
with non-default allocators will be annotated in the near future, with
separate commits dedicated to enabling them. The process will be similar
to the workflow employed for enabling annotations in `std::deque`.
**Please note**: these annotations function effectively only when libc++
and libc++abi dylibs are instrumented (with ASan). This aligns with the
prevailing behavior of Memory Sanitizer.
To avoid breaking everything, this commit also appends
`_LIBCPP_INSTRUMENTED_WITH_ASAN` to `__config_site` whenever libc++ is
compiled with ASan. If this macro is not defined, string annotations are
not enabled. However, linking a binary that does **not** annotate
strings with a dynamic library that annotates strings, is not permitted.
Originally proposed here: https://reviews.llvm.org/D132769
Related patches on Phabricator:
- Turning on annotations for short strings:
https://reviews.llvm.org/D147680
- Turning on annotations for all allocators:
https://reviews.llvm.org/D146214
This PR is a part of a series of patches extending AddressSanitizer C++
container overflow detection capabilities by adding annotations, similar
to those existing in `std::vector` and `std::deque` collections. These
enhancements empower ASan to effectively detect instances where the
instrumented program attempts to access memory within a collection's
internal allocation that remains unused. This includes cases where
access occurs before or after the stored elements in `std::deque`, or
between the `std::basic_string`'s size (including the null terminator)
and capacity bounds.
The introduction of these annotations was spurred by a real-world
software bug discovered by Trail of Bits, involving an out-of-bounds
memory access during the comparison of two strings using the
`std::equals` function. This function was taking iterators
(`iter1_begin`, `iter1_end`, `iter2_begin`) to perform the comparison,
using a custom comparison function. When the `iter1` object exceeded the
length of `iter2`, an out-of-bounds read could occur on the `iter2`
object. Container sanitization, upon enabling these annotations, would
effectively identify and flag this potential vulnerability.
This Pull Request introduces basic annotations for `std::basic_string`.
Long strings exhibit structural similarities to `std::vector` and will
be annotated accordingly. Short strings are already implemented, but
will be turned on separately in a forthcoming commit. Look at [a
comment](https://github.com/llvm/llvm-project/pull/72677#issuecomment-1850554465)
below to read about SSO issues at current moment.
Due to the functionality introduced in
[D132522](dd1b7b797a),
the `__sanitizer_annotate_contiguous_container` function now offers
compatibility with all allocators. However, enabling this support will
be done in a subsequent commit. For the time being, only strings with
the default allocator will be annotated.
If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com
Extend `std::basic_string` tests to cover more buffer situations and
length in general, particularly non-SSO cases after SSO test cases
(changing buffers). This commit is a side effect of working on tests for
ASan annotations.
Related PR: https://github.com/llvm/llvm-project/pull/72677
While doing this, I also found a few tests that were either clearly
incorrect (e.g. testing the wrong function) or that lacked basic test
coverage like testing std::string itself (e.g. the test was only checking
std::basic_string with a custom allocator). In these cases, I did a few
conservative drive-by changes.
Differential Revision: https://reviews.llvm.org/D140550
Co-authored-by: Brendan Emery <brendan.emery@esrlabs.com>
This is an ongoing series of commits that are reformatting our
Python code.
Reformatting is done with `black`.
If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.
If you run into any problems, post to discourse about it and
we will try to help.
RFC Thread below:
https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Reviewed By: #libc, kwk, Mordante
Differential Revision: https://reviews.llvm.org/D150763
This has been done using the following command
find libcxx/test -type f -exec perl -pi -e 's|^([^/]+?)((?<!::)size_t)|\1std::\2|' \{} \;
And manually removed some false positives in std/depr/depr.c.headers.
The `std` module doesn't export `::size_t`, this is a preparation for that module.
Reviewed By: ldionne, #libc, EricWF, philnik
Differential Revision: https://reviews.llvm.org/D146088
The converting constructor is ill-formed, and `==` is missing. (I didn't implement `!=` since the test is C++20-and-later only; I'll let the compiler do it for us.)
Drive-by: change 4-space indent on line 27 to 2-space indent to be consistent with the rest of the test.
Differential Revision: https://reviews.llvm.org/D131079
This patch switches the build compiler for AIX from ibm-clang to clang. ibm-clang++_r has `-pthread` by default, but clang for AIX doesn't, so `-pthread` had to be added to the test config. A bunch of tests now pass, so the `XFAIL` was removed. This patch also switch the build to use the visibility support available in clang-15 to control symbols exported by the shared library (AIX traditionally uses explicit export lists for this purpose).
Reviewed By: #libc, #libc_abi, daltenty, #libunwind, ldionne
Differential Revision: https://reviews.llvm.org/D127470
Since those features are general properties of the environment, it makes
sense to use them from libc++abi too, and so the name libcpp-has-no-xxx
doesn't make sense.
Differential Revision: https://reviews.llvm.org/D126482
We shouldn't be calling `move` via ADL -- and neither should anybody
in the wild be calling it via ADL, so it's not like we need to test
this ADL ability of `move` in particular.
Reviewed as part of D119860.
Some embedded platforms do not wish to support the C library functionality
for handling wchar_t because they have no use for it. It makes sense for
libc++ to work properly on those platforms, so this commit adds a carve-out
of functionality for wchar_t.
Unfortunately, unlike some other carve-outs (e.g. random device), this
patch touches several parts of the library. However, despite the wide
impact of this patch, I still think it is important to support this
configuration since it makes it much simpler to port libc++ to some
embedded platforms.
Differential Revision: https://reviews.llvm.org/D111265
With the STL containers, I didn't enable move operations in C++03 mode
because that would change the overload resolution for things that today
are copy operations. With iostreams, though, the copy operations aren't
present at all, and so I see no problem with enabling move operations
even in (Clang's greatly extended) C++03 mode.
Clang's C++03 mode does not support delegating constructors.
Differential Revision: https://reviews.llvm.org/D104310
When porting libc++ to embedded systems, it can be useful to drop support
for localization, which these systems don't implement or care about.
Differential Revision: https://reviews.llvm.org/D90072
C++98 and C++03 are effectively aliases as far as Clang is concerned.
As such, allowing both std=c++98 and std=c++03 as Lit parameters is
just slightly confusing, but provides no value. It's similar to allowing
both std=c++17 and std=c++1z, which we don't do.
This was discovered because we had an internal bot that ran the test
suite under both c++98 AND c++03 -- one of which is redundant.
Differential Revision: https://reviews.llvm.org/D80926
The testing script used to test libc++ historically did not like directories
without any testing files, so these tests had been added. Since this is
not necessary anymore, we can now remove these files. This has the benefit
that the total number of tests reflects the real number of tests more
closely, and we also skip some unnecessary work (especially relevant when
running tests over SSH).
However, some nothing_to_do.pass.cpp tests actually serve the purpose of
documenting that an area of the Standard doesn't need to be tested, or is
tested elsewhere. These files are not removed by this commit.
Removal done with:
import os
import itertools
for (dirpath, dirnames, filenames) in itertools.chain(os.walk('./libcxx/test'),
os.walk('./libcxxabi/test')):
if len(filenames + dirnames) > 1 and \
any(p == 'nothing_to_do.pass.cpp' for p in filenames):
os.remove(os.path.join(dirpath, 'nothing_to_do.pass.cpp'))
Summary:
This is a re-application of r357533 and r357531. They had been reverted
because we thought the commits broke the LLDB data formatters, but it
turns out this was because only r357531 had been included in the CI
run.
Before this patch, we would only ever throw an exception if the badbit
was set on the stream. The Standard is currently very unclear on how
exceptions should be propagated and what error flags should be set by
the input stream operations. This commit changes libc++ to behave under
a different (but valid) interpretation of the Standard. This interpretation
of the Standard matches what other implementations are doing.
This effectively implements the wording in p1264r0. It hasn't been voted
into the Standard yet, however there is wide agreement that the fix is
correct and it's just a matter of time before the fix is standardized.
PR21586
PR15949
rdar://problem/15347558
Reviewers: mclow.lists, EricWF
Subscribers: christof, dexonsmith, cfe-commits
Differential Revision: https://reviews.llvm.org/D49863
llvm-svn: 357775
This reverts commits r357533 and r357531, which broke the LLDB
data formatters. I'll hold off until we know how to fix the data
formatters accordingly.
llvm-svn: 357536
Summary:
Before this patch, we would only ever throw an exception if the badbit
was set on the stream. The Standard is currently very unclear on how
exceptions should be propagated and what error flags should be set by
the input stream operations. This commit changes libc++ to behave under
a different (but valid) interpretation of the Standard. This interpretation
of the Standard matches what other implementations are doing.
I will submit a paper in San Diego to clarify the Standard such that the
interpretation used in this commit (and other implementations) is the only
possible one.
PR21586
PR15949
rdar://problem/15347558
Reviewers: mclow.lists, EricWF
Subscribers: christof, dexonsmith, cfe-commits
Differential Revision: https://reviews.llvm.org/D49863
llvm-svn: 357531
Summary:
Freestanding is *weird*. The standard allows it to differ in a bunch of odd
manners from regular C++, and the committee would like to improve that
situation. I'd like to make libc++ behave better with what freestanding should
be, so that it can be a tool we use in improving the standard. To do that we
need to try stuff out, both with "freestanding the language mode" and
"freestanding the library subset".
Let's start with the super basic: run the libc++ tests in freestanding, using
clang as the compiler, and see what works. The easiest hack to do this:
In utils/libcxx/test/config.py add:
self.cxx.compile_flags += ['-ffreestanding']
Run the tests and they all fail.
Why? Because in freestanding `main` isn't special. This "not special" property
has two effects: main doesn't get mangled, and main isn't allowed to omit its
`return` statement. The first means main gets mangled and the linker can't
create a valid executable for us to test. The second means we spew out warnings
(ew) and the compiler doesn't insert the `return` we omitted, and main just
falls of the end and does whatever undefined behavior (if you're luck, ud2
leading to non-zero return code).
Let's start my work with the basics. This patch changes all libc++ tests to
declare `main` as `int main(int, char**` so it mangles consistently (enabling us
to declare another `extern "C"` main for freestanding which calls the mangled
one), and adds `return 0;` to all places where it was missing. This touches 6124
files, and I apologize.
The former was done with The Magic Of Sed.
The later was done with a (not quite correct but decent) clang tool:
https://gist.github.com/jfbastien/793819ff360baa845483dde81170feed
This works for most tests, though I did have to adjust a few places when e.g.
the test runs with `-x c`, macros are used for main (such as for the filesystem
tests), etc.
Once this is in we can create a freestanding bot which will prevent further
regressions. After that, we can start the real work of supporting C++
freestanding fairly well in libc++.
<rdar://problem/47754795>
Reviewers: ldionne, mclow.lists, EricWF
Subscribers: christof, jkorous, dexonsmith, arphaman, miyuki, libcxx-commits
Differential Revision: https://reviews.llvm.org/D57624
llvm-svn: 353086
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351648
This makes them consistent (many comments already used uppercase).
The special REQUIRES, UNSUPPORTED, and XFAIL comments are excluded from this change.
llvm-svn: 309468
Guard typedefs and static_asserts with _LIBCPP_VERSION.
test/std/containers/sequences/vector.bool/move_assign_noexcept.pass.cpp
test/std/containers/sequences/vector.bool/move_noexcept.pass.cpp
test/std/containers/sequences/vector.bool/swap_noexcept.pass.cpp
Additionally deal with conditional compilation.
test/std/containers/associative/map/map.cons/move_noexcept.pass.cpp
test/std/containers/associative/multimap/multimap.cons/move_noexcept.pass.cpp
Additionally deal with typedefs used by other typedefs.
Fixes D29135.
llvm-svn: 294154
