This reverts commit 8d151f804ff43aaed1edf810bb2a07607b8bba14, which
broke some build bots. I think that is caused by an invalid argument
order when checking __is_comparable in upper_bound.
We were only checking that the comparator was rvalue callable,
when in reality the algorithms always call comparators as lvalues.
This patch also refactors the tests for callable requirements and
expands it to a few missing algorithms.
Fixes#69554
One-sided binary search, aka meta binary search, has been in the public
domain for decades, and has the general advantage of being constant time
in the best case, with the downside of executing at most 2*log(N)
comparisons vs classic binary search's exact log(N). There are two
scenarios in which it really shines: the first one is when operating
over non-random-access iterators, because the classic algorithm requires
knowing the container's size upfront, which adds N iterator increments
to the complexity. The second one is when traversing the container in
order, trying to fast-forward to the next value: in that case the
classic algorithm requires at least O(N*log(N)) comparisons and, for
non-random-access iterators, O(N^2) iterator increments, whereas the
one-sided version will yield O(N) operations on both counts, with a
best-case of O(log(N)) comparisons which is very common in practice.
This patch contains a number of small portability improvements for the
test suite, making it easier to run the test suite with other standard
library implementations.
- Guard checks for _LIBCPP_HARDENING_MODE to avoid -Wundef
- Avoid defining _LIBCPP_HARDENING_MODE even when no hardening mode is
specified -- we should use the default mode of the library in that case.
- Add missing includes and qualify a few function calls.
- Avoid opening namespace std to forward declare stdlib containers. The
test suite should represent user code, and user code isn't allowed to do
that.
The return of malloc is implementation defined when the requested size
is 0. On platforms (such as AIX) that return a null pointer for 0 size,
operator new will throw a bad_alloc exception. operator new should
return a non null pointer for 0 size instead.
There were various places where we incorrectly handled exceptions in the
PSTL. Typical issues were missing `noexcept` and taking iterators by
value instead of by reference.
This patch fixes those inconsistent and incorrect instances, and adds
proper tests for all of those. Note that the previous tests were often
incorrectly turned into no-ops by the compiler due to copy ellision,
which doesn't happen with these new tests.
Previously there were two ways to override the verbose abort function
which gets called when a hardening assertion is triggered:
- compile-time: define the `_LIBCPP_VERBOSE_ABORT` macro;
- link-time: provide a definition of `__libcpp_verbose_abort` function.
This patch adds a new configure-time approach: the vendor can provide
a path to a custom header file which will get copied into the build by
CMake and included by the library. The header must provide a definition
of the
`_LIBCPP_ASSERTION_HANDLER` macro which is what will get called should
a hardening assertion fail. As of this patch, overriding
`_LIBCPP_VERBOSE_ABORT` will still work, but the previous mechanisms
will be effectively removed in a follow-up patch, making the
configure-time mechanism the sole way of overriding the default handler.
Note that `_LIBCPP_ASSERTION_HANDLER` only gets invoked when a hardening
assertion fails. It does not affect other cases where
`_LIBCPP_VERBOSE_ABORT` is currently used (e.g. when an exception is
thrown in the `-fno-exceptions` mode).
The library provides a default version of the custom header file that
will get used if it's not overridden by the vendor. That allows us to
always test the override mechanism and reduces the difference in
configuration between the pristine version of the library and
a platform-specific version.
Found while running libc++'s tests with MSVC's STL.
*
`libcxx/test/std/algorithms/alg.sorting/alg.heap.operations/sort.heap/ranges_sort_heap.pass.cpp`
+ Fix Clang `-Wunused-variable`, because `LIBCPP_ASSERT` expands to
nothing for MSVC's STL.
+ This is the same "always void-cast" change that #73437 applied to the
neighboring `complexity.pass.cpp`. I missed that
`ranges_sort_heap.pass.cpp` was also affected because we had disabled
this test.
*
`libcxx/test/std/input.output/file.streams/fstreams/ifstream.members/buffered_reads.pass.cpp`
*
`libcxx/test/std/input.output/file.streams/fstreams/ofstream.members/buffered_writes.pass.cpp`
+ Fix MSVC "warning C4244: '`=`': conversion from '`__int64`' to
'`_Ty`', possible loss of data".
+ This is a valid warning, possibly the best one that MSVC found in this
entire saga. We're accumulating a `std::vector<std::streamsize>` and
storing the result in `std::streamsize total_size` but we actually have
to start with `std::streamsize{0}` or we'll truncate.
*
`libcxx/test/std/input.output/filesystems/fs.enum/enum.path.format.pass.cpp`
+ Fix Clang `-Wunused-local-typedef` because the following usage is
libc++-only.
+ I'm just expanding it at the point of use, and using the dedicated
`LIBCPP_STATIC_ASSERT` to keep the line length down.
*
`libcxx/test/std/input.output/syncstream/syncbuf/syncstream.syncbuf.assign/swap.pass.cpp`
+ Fix MSVC "warning C4242: 'argument': conversion from '`int`' to
'`const _Elem`', possible loss of data".
+ This is a valid warning (possibly the second-best) as `sputc()`
returns `int_type`. If `sputc()` returns something unexpected, we want
to know, so we should separately say `expected.push_back(CharT('B'))`.
*
`libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_align_nothrow.pass.cpp`
*
`libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size_nothrow.pass.cpp`
+ Fix MSVC "warning C6001: Using uninitialized memory '`x`'."
+ [N4964](https://wg21.link/N4964) \[new.delete.single\]/12:
> *Effects:* The deallocation functions
(\[basic.stc.dynamic.deallocation\]) called by a *delete-expression*
(\[expr.delete\]) to render the value of `ptr` invalid.
+ \[basic.stc.general\]/4:
> When the end of the duration of a region of storage is reached, the
values of all pointers representing the address of any part of that
region of storage become invalid pointer values (\[basic.compound\]).
Indirection through an invalid pointer value and passing an invalid
pointer value to a deallocation function have undefined behavior. Any
other use of an invalid pointer value has implementation-defined
behavior.
+ In certain configurations, after `delete x;` MSVC will consider `x` to
be radioactive (and in other configurations, it'll physically null out
`x` as a safety measure). We can copy it into `old_x` before deletion,
which the implementation finds acceptable.
*
`libcxx/test/std/ranges/range.adaptors/range.elements/general.pass.cpp`
*
`libcxx/test/std/ranges/range.adaptors/range.elements/iterator/deref.pass.cpp`
+ Fix MSVC "warning C4242: 'initializing': conversion from '`_Ty`' to
'`_Ty`', possible loss of data".
+ This was being emitted in `pair` and `tuple`'s perfect forwarding
constructors. Passing `short{1}` allows MSVC to see that no truncation
is happening.
*
`libcxx/test/std/ranges/range.adaptors/range.elements/iterator/member_types.compile.pass.cpp`
+ Fix MSVC "warning C4242: 'initializing': conversion from '`_Ty`' to
'`_Ty2`', possible loss of data".
+ Similarly, this was being emitted in `pair`'s perfect forwarding
constructor. After passing `short{1}`, I reduced repetition by relying
on CTAD. (I can undo that cleanup if it's stylistically undesirable.)
*
`libcxx/test/std/utilities/function.objects/refwrap/refwrap.const/type_conv_ctor.pass.cpp`
+ Fix MSVC "warning C4930: '`std::reference_wrapper<int> purr(void)`':
prototyped function not called (was a variable definition intended?)".
+ There's no reason for `purr()` to be locally declared (aside from
isolating it to a narrow scope, which has minimal benefits); it can be
declared like `meow()` above. 😸
*
`libcxx/test/std/utilities/memory/util.smartptr/util.smartptr.shared/util.smartptr.shared.create/make_shared_for_overwrite.pass.cpp`
*
`libcxx/test/std/utilities/smartptr/unique.ptr/unique.ptr.create/make_unique_for_overwrite.default_init.pass.cpp`
+ Fix MSVC static analysis warnings when replacing `operator new`:
```
warning C28196: The requirement that '(_Param_(1)>0)?(return!=0):(1)' is
not satisfied. (The expression does not evaluate to true.)
warning C6387: 'return' could be '0': this does not adhere to the
specification for the function 'new'.
warning C6011: Dereferencing NULL pointer 'reinterpret_cast<char
*>ptr+i'.
```
+ All we need is a null check, which appears in other `operator new`
replacements:
b85f1f9b18/libcxx/test/std/language.support/support.dynamic/new.delete/new.delete.single/new.size.replace.pass.cpp (L27-L28)
Found while running libc++'s tests with MSVC's STL, where `std::array`
iterators are never pointers.
Most of these changes are reasonably self-explanatory (the `std::array`s
are right there, and the sometimes-slightly-wrapped raw pointer types
are a short distance away). A couple of changes are less obvious:
In `libcxx/test/std/containers/from_range_helpers.h`, `wrap_input()` is
called with `Iter` types that are constructible from raw pointers. It's
also sometimes called with an `array` as the `input`, so the first
overload was implicitly assuming that `array` iterators are pointers. We
can fix this assumption by providing a dedicated overload for `array`,
just like the one for `vector` immediately below. Finally,
`from_range_helpers.h` should explicitly include both `<array>` and
`<vector>`, even though they were apparently being dragged in already.
In `libcxx/test/std/containers/views/views.span/span.cons/iterator_sentinel.pass.cpp`,
fix `throw_operator_minus`. The error was pretty complicated, caused by
the concepts machinery noticing that `value_type` and `element_type`
were inconsistent. In the template instantiation context, you can see
the critical detail that `throw_operator_minus<std::_Array_iterator>` is
being formed.
Fortunately, the fix is extremely simple. To produce `element_type`
(which retains any cv-qualification, unlike `value_type`), we shouldn't
attempt to `remove_pointer` with the iterator type `It`. Instead, we've
already obtained the `reference` type, so we can `remove_reference_t`.
(This is modern code, where we have access to the alias templates, so I
saw no reason to use the older verbose form.)
Found while running libc++'s test suite with MSVC's STL, where we use
both MSVC's compiler and Clang/LLVM.
MSVC's compiler rejects the non-Standard extension of zero-length
arrays. For conformance, I'm changing these occurrences to
`std::array<int, 0>`.
Many of these files already had `#include <array>`; I'm adding it to the
rest.
I wanted to add `-Wzero-length-array` to
`libcxx/utils/libcxx/test/params.py` to prevent future occurrences, but
it complained about product code 😿 :
```
In file included from /home/runner/_work/llvm-project/llvm-project/libcxx/test/std/input.output/iostream.format/input.streams/istream.formatted/istream.formatted.arithmetic/long.pass.cpp:18:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/istream:170:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/ostream:172:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/__system_error/error_code.h:18:
In file included from /home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/__system_error/error_category.h:15:
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/string:811:25: error: zero size arrays are an extension [-Werror,-Wzero-length-array]
811 | char __padding_[sizeof(value_type) - 1];
| ^~~~~~~~~~~~~~~~~~~~~~
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/string:817:19: note: in instantiation of member class 'std::basic_string<char>::__short' requested here
817 | static_assert(sizeof(__short) == (sizeof(value_type) * (__min_cap + 1)), "__short has an unexpected size.");
| ^
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/string:2069:5: note: in instantiation of template class 'std::basic_string<char>' requested here
2069 | _LIBCPP_STRING_V1_EXTERN_TEMPLATE_LIST(_LIBCPP_DECLARE, char)
| ^
/home/runner/_work/llvm-project/llvm-project/build/generic-cxx03/include/c++/v1/__string/extern_template_lists.h:31:60: note: expanded from macro '_LIBCPP_STRING_V1_EXTERN_TEMPLATE_LIST'
31 | _Func(_LIBCPP_EXPORTED_FROM_ABI basic_string<_CharType>& basic_string<_CharType>::replace(size_type, size_type, value_type const*, size_type)) \
| ^
```
I pushed a tiny commit to fix unrelated comment typos, in an attempt to
clear out spurious CI failures.
This patch actually runs the tests for picolibc behind an emulator,
removing a few workarounds and increasing coverage.
Differential Revision: https://reviews.llvm.org/D155521
I've structured this into a series of commits for even easier reviewing,
if that helps. I could easily split this up into separate PRs if
desired, but as this is low-risk with simple edits, I thought one PR
would be easiest.
* Drop unnecessary semicolons after function definitions.
* Cleanup comment typos.
* Cleanup `static_assert` typos.
* Cleanup test code typos.
+ There should be no functional changes, assuming I've changed all
occurrences.
* ~~Fix massive test code typos.~~
+ This was a real problem, but needed more surgery. I reverted those
changes here, and @philnik777 is fixing this properly with #73444.
* clang-formatting as requested by the CI.
Found while running libc++'s test suite with MSVC's STL.
This is structured into a series of commits for easier reviewing; I
could also split this into smaller PRs if desired.
* Add void-casts for `invoke_r` calls to fix MSVC STL `[[nodiscard]]`
warnings.
+ Our rationale is that if someone is calling `invoke_r<NonVoidType>`,
it sure looks like they care about the return value.
* Add `[[maybe_unused]]` to silence `-Wunused-parameter` warnings.
+ This happens because the parameters are used within `LIBCPP_ASSERT`,
which vanishes for MSVC's STL. This also motivates the following
changes.
* Add `[[maybe_unused]]` to fix `-Wunused-variable` warnings.
* Always void-cast `debug_comparisons` to fix `-Wunused-variable`
warnings.
+ As this was already unused with a void-cast in one
`_LIBCPP_HARDENING_MODE` branch, I'm simply lifting it next to the
variable definition.
* Add `[[maybe_unused]]` to fix `-Wunused-local-typedef` warnings.
This is to modify a list of libcxx tests written under the assumption
that iterators for std::array, std::string_view, and std::string are
pointers. The motivation for this PR is to make the tests more universal
and potentially being used to test other C++ standard library
implementations, for example
[microsoft/STL](https://github.com/microsoft/STL).
I can confirm that this patch makes a number of tests compatible with
microsoft STL:
`Failed : 204 (2.12%)` -> `Failed : 136 (1.42%)`
, and does not break any tests on `libcxx`.
This is not a complete list of such incompatibilities, but I am hoping
this will start a discussion about whether we are open to accepting such
changes.
1. Instead of using individual "boolean" macros, have an "enum" macro
`_LIBCPP_HARDENING_MODE`. This avoids issues with macros being
mutually exclusive and makes overriding the hardening mode within a TU
more straightforward.
2. Rename the safe mode to debug-lite.
This brings the code in line with the RFC:
https://discourse.llvm.org/t/rfc-hardening-in-libc/73925Fixes#65101
Before this patch, we would fail to implicitly convert the result of
predicates to bool, which means we'd potentially perform a copy or move
construction of the boolean-testable, which isn't allowed. The same
holds true for comparing iterators against sentinels, which is allowed
to return a boolean-testable type.
We already had tests aiming to ensure correct handling of these types,
but they failed to provide appropriate coverage in several cases due to
guaranteed RVO. This patch fixes the tests, adds tests for missing
algorithms and views, and fixes the actual problems in the code.
Fixes#69074
This makes exception handling a lot simpler, since we don't have to convert any exceptions this way. Is also properly handles all the user-thrown exceptions.
Reviewed By: ldionne, #libc
Spies: arichardson, mstorsjo, libcxx-commits
Differential Revision: https://reviews.llvm.org/D154238
The safe mode is in-between the hardened and the debug modes, extending
the checks contained in the hardened mode with certain checks that are
relatively cheap and prevent common sources of errors but aren't
security-critical. Thus, the safe mode trades off some performance for
a wider set of checks, but unlike the debug mode, it can still be used
in production.
Differential Revision: https://reviews.llvm.org/D158823
Make it a multichoice string to closer mirror the CMake variable. This
allows writing `UNSUPPORTED: libcpp-hardening-mode=unchecked` rather
than `UNSUPPORTED: !libcpp-has-hardened-mode && !libcpp-has-debug-mode`.
Differential Revision: https://reviews.llvm.org/D155906
The hardened mode is intended to only include security-critical,
relatively low-overhead checks that are intended to be usable in
production. By default, assertions are excluded from this mode.
Differential Revision: https://reviews.llvm.org/D155866
`_LIBCPP_ENABLE_ASSERTIONS` was used to enable the "safe" mode in
libc++. Libc++ now provides the hardened mode and the debug mode that
replace the safe mode.
For backward compatibility, enabling `_LIBCPP_ENABLE_ASSERTIONS` now
enables the hardened mode. Note that the hardened mode provides
a narrower set of checks than the previous "safe" mode (only
security-critical checks that are performant enough to be used in
production).
Differential Revision: https://reviews.llvm.org/D154997
This patch only adds new configuration knobs -- the actual assertions
will be added in follow-up patches.
Differential Revision: https://reviews.llvm.org/D153902
I made sure they all had some expected-error output in them. Many of
these tests would be better implemented as a positive test using SFINAE,
but this is beyond the scope of this patch.
Differential Revision: https://reviews.llvm.org/D153980
This makes __debug_three_way_comp consistent with __debug_less and
in particular gets rid of a potential use-after-move caused by the
use of std::forward. In the previous version of the code, we would
call `__do_compare_assert` after forwarding the arguments into the
comparator, which could end up using the arguments after they've been
moved from.
This also simplifies how we call `__do_compare_assert` by using
`if constexpr` and adds a missing test for proxy iterators in
lexicographical_compare_three_way, which could have found this
issue.
Differential Revision: https://reviews.llvm.org/D152753
Some tests in our test suite are unbelievably slow on GCC due to the
use of the always_inline attribute. See [1] for more details.
This patch introduces the GCC-ALWAYS_INLINE-FIXME lit feature to
disable tests that are plagued by that issue. At the same time, it
moves several existing tests from ad-hoc `UNSUPPORTED: gcc-12` markup
to the new GCC-ALWAYS_INLINE-FIXME feature, and marks the slowest tests
reported by the CI as `UNSUPPORTED: GCC-ALWAYS_INLINE-FIXME`.
[1]: https://discourse.llvm.org/t/rfc-stop-supporting-extern-instantiations-with-gcc/71277/1
Differential Revision: https://reviews.llvm.org/D152736
An issue with `operator()` was found during the implementation of https://reviews.llvm.org/D132268.
This patch aims to resolve the issues by updating the operator to use perfect forwarding.
The original motivation for `three_way_comp_ref_type` is given in: https://reviews.llvm.org/D131395
`three_way_comp_ref_type`'s implementation is inspired by `comp_ref_type`, which has two overloads:
```
template <class _Tp, class _Up>
bool operator()(const _Tp& __x, const _Up& __y);
template <class _Tp, class _Up>
bool operator()(_Tp& __x, _Up& __y);
```
`__debug_three_way_comp` is missing the first overload and also declares the typealias`_three_way_comp_ref_type ` incorrectly.
Reviewed By: #libc, philnik
Differential Revision: https://reviews.llvm.org/D150188
Instead of writing something like `XFAIL: use_system_cxx_lib && target=...`
to XFAIL back-deployment tests, introduce named Lit features like
`availability-shared_mutex-missing` to represent those. This makes the
XFAIL annotations leaner, and solves the problem of XFAIL comments
potentially getting out of sync. This would also make it easier for
another vendor to add their own annotations to the test suite by simply
changing how the feature is defined for their OS releases, instead
of having to modify hundreds of tests to add repetitive annotations.
This doesn't touch *all* annotations -- only annotations that were widely
duplicated are given named features (e.g. when filesystem or shared_mutex
were introduced). I still think it probably doesn't make sense to have a
named feature for every single fix we make to the dylib.
This is in essence a revert of 2659663, but since then the test suite
has changed significantly. Back when I did 2659663, the configuration
files we have for the test suite right now were being bootstrapped and
it wasn't clear how to provide these features for back-deployment in
that context. Since then, we have a streamlined way of defining these
features in `features.py` and that doesn't impact the ability for a
configuration file to stay minimal.
The original motivation for this change was that I am about to propose
a change that would touch essentially all XFAIL annotations for back-deployment
in the test suite, and this greatly reduces the number of lines changed
by that upcoming change, in addition to making the test suite generally
better.
Differential Revision: https://reviews.llvm.org/D146359
This has been done using the following command
find libcxx/test -type f -exec perl -pi -e 's|^([^/]+?)((?<!::)size_t)|\1std::\2|' \{} \;
And manually removed some false positives in std/depr/depr.c.headers.
The `std` module doesn't export `::size_t`, this is a preparation for that module.
Reviewed By: ldionne, #libc, EricWF, philnik
Differential Revision: https://reviews.llvm.org/D146088
Don't forward to `min_element` for small types that are trivially copyable, and instead use a naive loop that keeps track of the smallest element (as opposed to an iterator to the smallest element). This allows the compiler to vectorize the loop in some cases.
Reviewed By: #libc, ldionne
Spies: ldionne, libcxx-commits
Differential Revision: https://reviews.llvm.org/D143596
https://wg21.link/LWG2444 updated the comparison complexity of
std:sort_heap to be at most 2N log (N) where N == last - first. In the
current implementation, we invoke __pop_heap exactly N-1 times. In each
call to __pop_heap, we first go down the heap from first to possibly
last in the function __floyd_sift_down. Then, we possibly go back up in
the function __sift_up.
In the function __floyd_sift_down, there is loop in which one comparison
is made in each iteration. The loop runs till __child becomes greater
than (__len - 2) / 2. __child starts at 0 and it is at least set to 2 *
__child + 1 on each iteration. Thus, after k iterations, __child will
be at least 2^k - 1. After log(N) iterations, __child >= 2^(log(N)) -
1 = N - 1 > (__len - 2) / 2. This means that the while loop in the
function __floyd_sift_down would perform at most log(N) comparisons on
each invocation.
In the function __sift_up, there is one comparison made that will almost
always occur. After that there is a do-while loop. The comparison
function is invoked once in each iteration. In the worst case, the loop
will run till __len goes down to zero. It can start from (N-3)/2. In
each iteration, __len goes down to (__len-1) / 2. After k iterations,
__len will be at most (N - 2^(k+1) -1) / 2^(k+1). Thus, __len will
become when (N-2^(k+1)-1) < 2^(k+1) i.e. N < 2^(k+2) + 1. This means
at most log(N) - 1 iterations for the loop. So in total at most log(N)
comparison will be performed in __sift_up.
So overall for each iteration of the loop in __pop_heap, there will at
most 2 log(N) comparisons. So, the total number of comparisons is
at most 2 N log(N).
We also updated the test sort.heap/complexity.pass.cpp to test for the
number of operations.
Differential Revision: https://reviews.llvm.org/D144538
The implementation makes use of the freedom added by LWG 3410. We have
two variants of this algorithm:
* a fast path for random access iterators: This fast path computes the
maximum number of loop iterations up-front and does not compare the
iterators against their limits on every loop iteration.
* A basic implementation for all other iterators: This implementation
compares the iterators against their limits in every loop iteration.
However, it still takes advantage of the freedom added by LWG 3410 to
avoid unnecessary additional iterator comparisons, as originally
specified by P1614R2.
https://godbolt.org/z/7xbMEen5e shows the benefit of the fast path:
The hot loop generated of `lexicographical_compare_three_way3` is
more tight than for `lexicographical_compare_three_way1`. The added
benchmark illustrates how this leads to a 30% - 50% performance
improvement on integer vectors.
Implements part of P1614R2 "The Mothership has Landed"
Fixes LWG 3410 and LWG 3350
Differential Revision: https://reviews.llvm.org/D131395
Per our policy, the latest released AppleClang has been 14 for a while,
so libc++ is removing support for AppleClang 13. Our CI bots have been
moved to AppleClang 14 a few weeks ago.
Differential Revision: https://reviews.llvm.org/D138685
