We introduced an optimization to std::sort in 4eddbf9f10a6. However,
that optimization led to issues where users that were passing invalid
comparators to std::sort could start seeing OOB reads. This led to
the revert of the std::sort optimization from the LLVM 16 release
(see https://llvm.org/D146421).
This patch introduces _LIBCPP_ASSERTs to the places in the algorithm
where we make an assumption that the comparator will be consistent and
hence avoid a bounds check based on that. If the comparator happens not
to be consistent with itself, these are the places where we would
incorrectly go out of bounds. This allows users that enable libc++
assertions to catch such misuse at the cost of basically a bounds
check. For users that do not enable libc++ assertions (which is 99.9%
of users since assertions are off by default), this is basically a
no-op, and in fact the assertion will turn into a __builtin_assume,
making it explicit to the compiler that it can rely on the fact that
we're not going out of bounds.
I think this patch strikes the right balance. Folks that want absolute
performance will get what they want, since it is a precondition for the
comparator to be consistent, so the bounds checks are technically not
mandatory. Folks who want more safety *already* need to be enabling
libc++ assertions to catch other types of bugs (like operator[] OOB),
so this solution should also work for them.
I do think we have a lot of work towards popularizing the use of libc++
assertions and integrating it better so that users don't have to know
about the obscure _LIBCPP_ENABLE_ASSERTIONS macro to enable them, but
that's a separate concern.
rdar://106897934
Differential Revision: https://reviews.llvm.org/D147089
This adds an extension to std::pair in C++03 mode where we provide an
assignment operator from a pair<U, V>. Previously, any code trying to
trigger such an assignment operator would have tried using the
`operator=(pair const&)` copy assignment operator, which would then
have tried creating a `pair const&` by using the unconstrained
pair<U, V> constructor.
After this patch, pair will instead go through operator= directly if
its member types are assignable. If they are not assignable, the extension
operator= is disabled with SFINAE and the pair(pair<U, V>) constructor
will be used. Since that constructor is unconstrained, that will either
work or give a hard error.
This should be pretty transparent to users, but this is technically a
behavior change in C++03. Indeed, if a type has both a valid cross-type
assignment operator *and* a valid cross-type constructor, the library
will now prefer the cross-type assignment instead of going through the
cross-type constructor and then using the copy-constructor. Since this
is the mandated behavior in C++11, however, one could argue that any user
broken by that needs to change their code.
The motivation for this change is to allow simplifying the definition
of std::map's value_type, which requires handling assignment to a pair
of references properly. This patch will allow removing complexity from
https://llvm.org/D121485 instead of adding complexity in that patch.
Differential Revision: https://reviews.llvm.org/D150119
- add the `from_range_t` constructors and the related deduction guides;
- add the `insert_range`/`assign_range`/etc. member functions.
(Note: this patch is split from https://reviews.llvm.org/D142335)
Differential Revision: https://reviews.llvm.org/D149826
Several headers are missing includes for things they use.
type_traits.is_enum needs to export type_traits.integral_constant so that clients can access its `value` member without explicitly including __type_traits/integral_constant.h themselves.
Make `subrange_fwd` a peer submodule to `subrange` rather than a submodule of it, and have `subrange` export `subrange_fwd`. That will make it easier to programmatically generate modules for the private detail headers, and it will accomplish the same effect that __ranges/subrange.h will make subrange_kind visible.
Reviewed By: Mordante, #libc
Differential Revision: https://reviews.llvm.org/D150055
Makes sure the formatter for the vector<bool>::reference is enabled
when only the header <vector> is included. Before this change it
required <vector> and <format> to be included. This violated the
requirements in the Standard.
Fixes: https://llvm.org/PR61314
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D149543
This removes the need for a custom libc++ build to have a basic set of PSTL algorithms.
Reviewed By: ldionne, #libc
Spies: miyuki, libcxx-commits, arichardson
Differential Revision: https://reviews.llvm.org/D149624
This revision is a part of a series of patches extending AddressSanitizer C++ container overflow detection capabilities by adding annotations, similar to those existing in std::vector, to std::string and `std::deque` collections. These changes allow ASan to detect cases when the instrumented program accesses memory which is internally allocated by the collection but is still not in-use (accesses before or after the stored elements for `std::deque`, or between the size and capacity bounds for `std::string`).
The motivation for the research and those changes was a bug, found by Trail of Bits, in a real code where an out-of-bounds read could happen as two strings were compared via a std::equals function that took `iter1_begin`, `iter1_end`, `iter2_begin` iterators (with a custom comparison function). When object `iter1` was longer than `iter2`, read out-of-bounds on `iter2` could happen. Container sanitization would detect it.
This revision introduces annotations for `std::deque`. Each chunk of the container can now be annotated using the `__sanitizer_annotate_double_ended_contiguous_container` function, which was added in the rG1c5ad6d2c01294a0decde43a88e9c27d7437d157. Any attempt to access poisoned memory will trigger an ASan error. Although false negatives are rare, they are possible due to limitations in the ASan API, where a few (usually up to 7) bytes before the container may remain unpoisoned. There are no false positives in the same way as with `std::vector` annotations.
This patch only supports objects (deques) that use the standard allocator. However, it can be easily extended to support all allocators, as suggested in the D146815 revision.
Furthermore, the patch includes the addition of the `is_double_ended_contiguous_container_asan_correct` function to libcxx/test/support/asan_testing.h. This function can be used to verify whether a `std::deque` object has been correctly annotated.
Finally, the patch extends the unit tests to verify ASan annotations (added LIBCPP_ASSERTs).
If a program is compiled without ASan, all helper functions will be no-ops. In binaries with ASan, there is a negligible performance impact since the code from the change is only executed when the deque container changes in size and it’s proportional to the change. It is important to note that regardless of whether or not these changes are in use, every access to the container's memory is instrumented.
Reviewed By: #libc, philnik
Spies: mikhail.ramalho, Enna1, #sanitizers, philnik, libcxx-commits
Differential Revision: https://reviews.llvm.org/D132092
This was contributed ~10 years ago, but we don't officially support it
and I am not aware of any bot testing it, so this has likely rotten to
the point where it is unusable.
Differential Revision: https://reviews.llvm.org/D138680
This revision is a part of a series of patches extending
AddressSanitizer C++ container overflow detection
capabilities by adding annotations, similar to those existing
in std::vector, to std::string and std::deque collections.
These changes allow ASan to detect cases when the instrumented
program accesses memory which is internally allocated by
the collection but is still not in-use (accesses before or
after the stored elements for std::deque, or between the size and
capacity bounds for std::string).
The motivation for the research and those changes was a bug,
found by Trail of Bits, in a real code where an out-of-bounds read
could happen as two strings were compared via a std::equals function
that took iter1_begin, iter1_end, iter2_begin iterators
(with a custom comparison function).
When object iter1 was longer than iter2, read out-of-bounds on iter2
could happen. Container sanitization would detect it.
In revision D132522, support for non-aligned memory buffers (sharing
first/last granule with other objects) was added, therefore the
check for standard allocator is not necessary anymore.
This patch removes the check in std::vector annotation member
function (__annotate_contiguous_container) to support
different allocators.
Additionally, this revision fixes unpoisoning in std::vector.
It guarantees that __alloc_traits::deallocate may access returned memory.
Originally suggested in D144155 revision.
If you have any questions, please email:
- advenam.tacet@trailofbits.com
- disconnect3d@trailofbits.com
Reviewed By: #libc, #sanitizers, philnik, vitalybuka, ldionne
Spies: mikhail.ramalho, manojgupta, ldionne, AntonBikineev, ayzhao, hans, EricWF, philnik, #sanitizers, libcxx-commits
Differential Revision: https://reviews.llvm.org/D136765
This revision is part of our efforts to support container annotations with (almost) every allocator.
That patch is necessary to enable support for most annotations (D136765). Without a way to turn off annotations, it's hard to use ASan with area allocators (no calls to destructors).
This is an answer to a request about it. This patch provides a solution to the aforementioned issue by introducing a new template structure `__asan_annotate_container_with_allocator`, which allows the disabling of container annotations for a specific allocator.
This patch also introduces `_LIBCPP_HAS_ASAN_CONTAINER_ANNOTATIONS_FOR_ALL_ALLOCATORS` FTM.
To turn off annotations, it is sufficient to create a template specialization with a false value using a [Unary Type Trait](https://en.cppreference.com/w/cpp/types/integral_constant).
The proposed structure is being used in the code enabling annotations for all allocators in `std::vector`, `std::basic_string`, and `std::deque`. (D136765 D146214 D146815)
Possibility to do it was added to ASan API in rGdd1b7b797a116eed588fd752fbe61d34deeb24e4 commit.
For context on not calling a destructor, look at https://eel.is/c++draft/basic.life#5 and notes there, you may also read a discussion in D136765.
Reviewed By: ldionne, philnik, #libc, hans
Spies: EricWF, mikhail.ramalho, #sanitizers, libcxx-commits, hans, vitalybuka
Differential Revision: https://reviews.llvm.org/D145628
All the used OpenMP pragmas are available with OpenMP 4.0, so they can all share a single conditional block.
Reviewed By: ldionne, #libc
Spies: yaxunl, sstefan1, guansong, jplehr, libcxx-commits, sunshaoce
Differential Revision: https://reviews.llvm.org/D149603
std::bind is supposed to be constexpr-friendly since C++20 and it was
marked as such in our synopsis. However, the tests were not actually
testing any of it and as it happens, std::bind was not really constexpr
friendly. This fixes the issue and makes sure that at least some of the
tests are running in constexpr mode.
Some tests for std::bind check functions that return void, and those
use global variables. These tests haven't been made constexpr-friendly,
however the coverage added by this patch should be sufficient to get
decent confidence.
Differential Revision: https://reviews.llvm.org/D149295
Some configurations flags are always the same. We can just remove them to make the code a bit cleaner.
Reviewed By: ldionne, #libc
Spies: pcwang-thead, libcxx-commits, miyuki
Differential Revision: https://reviews.llvm.org/D149502
This was discovered while working on modules. They can't export
declarations with internal linkage.
Reviewed By: #libc, philnik
Differential Revision: https://reviews.llvm.org/D149593
This is done by removing configurations we don't support.
Reviewed By: ldionne, #libc
Spies: libcxx-commits
Differential Revision: https://reviews.llvm.org/D149499
When cuchar is compiled independently on platforms that don't have <uchar.h>, it doesn't get a declaration of mbstate_t, and so makes an empty declaration for ::mbstate_t. That conflicts with the declarations in __mbstate_t.h and cwchar since both of those headers do get mbstate_t before making ::mbstate_t.
Change `__mbstate_t.h` to just get the underlying declaration for mbstate_t and not make a declaration for ::mbstate_t. Include __mbstate_t.h in uchar.h and wchar.h when their next headers aren't available so that at least mbstate_t gets defined.
Add __std_mbstate_t.h to declare ::mbstate_t for headers that need that when cuchar or cwchar aren't available (because either _LIBCPP_HAS_NO_WIDE_CHARACTERS or _LIBCPP_CXX03_LANG).
Reviewed By: ldionne, Mordante, #libc, philnik
Differential Revision: https://reviews.llvm.org/D148542
During the review of D140653 it was suggested to use vector in
__retarget_buffer instead of manually managing the memory. Due to the
requirements of the Standard it turns out format needs to include vector
leading to a cycle. Therefore switching back to manual memory
management.
This is a preparation to fix https://llvm.org/PR61314
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D148826
__use() can't be marked as constexpr in C++11 because it returns void,
which is not a literal type. But it turns out that we just don't need
to define it at all, since it's never called outside of decltype.
Differential Revision: https://reviews.llvm.org/D149360
This patch makes std::pair's constructors and assignment operators
closer to conforming in C++23. The only missing bit I am aware of
now is `reference_constructs_from_temporary_v` checks, which we
don't have the tools for yet.
This patch also refactors a long-standing non-standard extension where
we'd provide constructors for tuple-like types in all standard modes. The
criteria for being a tuple-like type are different from pair-like types
as introduced recently in the standard, leading to a lot of complexity
when trying to implement recent papers that touch the pair constructors.
After this patch, the pre-C++23 extension is provided in a self-contained
block so that we can easily deprecate and eventually remove the extension
in future releases.
Differential Revision: https://reviews.llvm.org/D143914
Committing on behalf of davidben. Reviewed as D146190
Patch originally by Jan Dörrie in https://reviews.llvm.org/D120064. I've just updated it to include tests, and update documentation that MSVC ABI is not stable.
In the current implementation both `std::optional` and `std::variant` don't perform the EBO on MSVC's ABI. This is because both classes inherit from multiple empty base classes, which breaks the EBO for MSVC. This patch fixes this issue by applying the `empty_bases` declspec attribute, which is already used to fix a similar issue for `std::tuple`.
See https://reviews.llvm.org/D120064 for discussion on MSVC ABI stability. From the discussion, libc++ doesn't have users that expect a stable ABI on MSVC. The fix is thus applied unconditionally to benefit more users. Documentation has been updated to reflect this.
Fixes https://github.com/llvm/llvm-project/issues/61095.
https://github.com/llvm/llvm-project/issues/62396 reports that
GCC 13 barfs on parsing <type_traits> because of the declarations
of `struct __is_convertible`. In GCC 13, `__is_convertible` is a
built-in, but `__is_convertible_to` is not. Clang has both, so
using either should be fine.
Reviewed By: #libc, philnik
Differential Revision: https://reviews.llvm.org/D149313
The module validation script of D144994 validate whether the contents of
an include match its module. An include is the set of files matching the
pattern:
- foo
- foo/*.
- __fwd/foo.h
Several declarations of the stream headers are in the header iosfwd.
This gives issue using the validation script. Adding iosfwd to the set
of matching files gives too many declarations. For example when
validating the fstream header it will pull in declarations of the
istream header. Instead if writing a set of filters the headers are
granularized into smaller headers containing the expected declarations.
Reviewed By: #libc, ldionne
Differential Revision: https://reviews.llvm.org/D148927
In D145589, we made the std::bind placeholders inline constexpr to
satisfy C++17. It turns out that this causes ODR violations since the
shared library provides strong definitions for those placeholders, and
the linker on Windows actually complains about this.
Fortunately, C++17 only encourages implementations to use `inline constexpr`,
it doesn't force them. So instead, we unconditionally define the placeholders
as `extern const`, which avoids the ODR violation and is indistinguishable
from `inline constexpr` for most purposes, since the placeholders are
empty types anyway.
Note that we could also go back to the pre-D145589 state of defining them
as non-inline constexpr variables in C++17, however that is definitely
non-conforming since that means the placeholders have different addresses
in different TUs. This is all a bit pedantic, but all in all I feel that
`extern const` provides the best bang for our buck, and I can't really
find any downsides to that solution.
Differential Revision: https://reviews.llvm.org/D149292
