This does a couple of things:
- code that is only useful for `shrink_to_fit` is moved into that
function
- `shrink_to_fit` is simplified a bit
- `__recommend` is renamed to better reflect what the function actually
does
- `__allocate_long_buffer` asserts that the passed capacity doesn't fit
into the SSO
The Trivial Relocation feature has been removed from the C++26 working
draft. Based on discussions in Kona, it is unlikely that the
"replaceable" type concept will come back in the C++29 time frame.
Since we don't have a use for the type trait in the library at the
moment, remove the code associated to it. If we end up needing something
like it in the future, we can always add it back.
8dae17be2991cd7f0d7fd9aa5aecd064520a14f6 refactors basic_string for more
code reuse. This makes sense in most cases, but has performance overhead
in the case of ~basic_string. The refactoring of ~basic_string to call
__reset_internal_buffer() added a redundant (inside the destructor)
reset of the object, which the optimizer is unable to optimize away in
many cases. This patch prevents a ~1% regression we observed on an
internal workload when applying the original refactoring. This does
slightly pessimize the code readability, but I think this change is
worth it given the performance impact.
I'm hoping to add a benchmark(s) to the upstream libc++ benchmark suite
around string construction/destruction to ensure that this case does not
regress as it seems common in real world applications. I will put up a
separate PR for that when I figure out a reasonable way to write it.
Verify that the operation passed to resize_and_overwrite returns an
integer-like type, matching the behavior of other standard library
implementations like GCC's libstdc++
Fixes#160577
There is a tradition to use U.S. English spellings for APIs. For
example, it's uninitialized_fill and not uninitialised_fill,
specialization not specialisation, etcetera.
This patch adds additional overloads to `map::at` in case its known that
the argument is transparently comparable to the key type. This avoids
actually constructing the key type in some cases, potentially removing
allocations.
```
--------------------------------------------------------
Benchmark old new
--------------------------------------------------------
BM_map_find_string_literal 12.8 ns 2.68 ns
```
This patch introduces a string-internal API to make the allocation and
deallocation the long string simpler. Before this we had a lot of code
duplication, so ensuring that things were actually correct was
non-trivial.
Since `__resize_default_init` is only ever used inside the dylib we can
remove the libc++-internal API and switch to the public one. This patch
inlines a bunch of functions that aren't required anymore and simplifies
the code that way.
This uses the new `__builtin_lt_synthesises_from_spaceship` builtin from
clang to use three way comparison for arbitrary user-defined types that
only provide a spaceship operator.
We've built up quite a few links directly to github within the code
base. We should instead use `llvm.org/PR<issue-number>` to link to bugs,
since that is resilient to the bug tracker changing in the future. This
is especially relevant for tests linking to bugs, since they will
probably be there for decades to come. A nice side effect is that these
links are significantly shorter than the GH links, making them much less
of an eyesore.
This patch also replaces a few links that linked to the old bugzilla
instance on llvm.org.
LLVM 20 contained an ABI break that can result in the size of
`std::unordered_{map,set,multimap,multiset}` and `std::deque` changing
when used with an allocator type that is empty and contains a base class
that is the same across rebound allocator instantiations (e.g.
``Allocator<int>`` and ``Allocator<char>`` are both empty and contain
the same base class).
In addition, the layout of a user-defined type that:
- contains one of the following containers:
`std::unordered_{map,set,multimap,multiset}`, `std::deque`, `std::map`,
`std::set`, `std::multimap`, `std::multiset`, `std::list` or
`std::vector`, and
- passes an empty allocator, comparator or hasher type to that
container, and
- has a member of that same empty allocator, comparator or hasher type
inside the enclosing struct, and
- that member is either marked with `[[no_unique_address]]` or optimized
out via the EBO (empty base optimization) technique
saw its size increase from LLVM 19 to LLVM 20. This was caused by the
usage of `[[no_unique_address]]` within some of libc++'s containers in a
way that allowed subtle interactions with enclosing objects. This is
fixed in LLVM 21 on Clang (returning to the LLVM 19 ABI), however that
implies an ABI break from LLVM 20 to LLVM 21.
Furthermore, fixing this causes a slight regression to constant
evaluation support in `std::unique_ptr`. Specifically, constant
evaluation will now fail when the deleter relies on being
value-initialized for constant-evaluation admissibility. If a
default-initialized deleter can be used during constant evaluation, or
if the default constructor is non-trivial, the `unique_ptr` is not
affected by this regression. In particular, this regression does not
impact any `unique_ptr` using the default deleter.
Note that there is currently no way to realistically fix this ABI break
on GCC, therefore GCC will remain on the ABI introduced in LLVM 19. That
also means that Clang and GCC will have a slightly different ABI for the
small subset of types listed above until we are able to apply the same
fix we did with Clang on GCC.
We fix this regression by surrounding the members of the
`_LIBCPP_COMPRESSED_PAIR` with an anonymous struct. This restricts the
shifting of empty types to the front of the `_LIBCPP_COMPRESSED_PAIR`
instead of throughout the surrounding object. This "frees up" the zero
offset to contain another object of the same type, restoring the ability
to perform EBO or to elide the storage for a type with
`[[no_unique_address]]` in the enclosing (user-defined) struct.
Fixes#154146
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
The use-case for `__is_same_uncvref` seems rather dubious, since not a
single use-cases needed the `remove_cvref_t` to be applied to both of
the arguments. Removing the alias makes it clearer what actually
happens, since we're not using an internal name anymore and it's clear
what the `remove_cvref_t` should apply to.
Currently, when the string shrink into the SSO buffer, the `__rep_.__s`
member isn't activated before the `traits_type::copy` call
yet, so internal `__builtin_memmove` call writing to the buffer causes
constant evaluation failure. The existing test coverage seems a bit
defective and doesn't cover this case - `shrink_to_fit` is called on the
copy of string after erasure, not the original string object.
This PR reorders the `__set_short_size` call, which starts the lifetime
of the SSO buffer, before the copy operation. Test coverage is achieved
by calling `shrink_to_fit` on the original erased string.
This reverts commit c861fe8a71e64f3d2108c58147e7375cd9314521.
Unfortunately, this use of hidden visibility attributes causes
user-defined specializations of standard-library types to also be marked
hidden by default, which is incorrect. See discussion thread on #131156.
...and also reverts the follow-up commits:
Revert "[libc++] Add explicit ABI annotations to functions from the block runtime declared in <__functional/function.h> (#140592)"
This reverts commit 3e4c9dc299c35155934688184319d391b298fff7.
Revert "[libc++] Make ABI annotations explicit for windows-specific code (#140507)"
This reverts commit f73287e623a6c2e4a3485832bc3e10860cd26eb5.
Revert "[libc++][NFC] Replace a few "namespace std" with the correct macro (#140510)"
This reverts commit 1d411f27c769a32cb22ce50b9dc4421e34fd40dd.
This patch introduces `_LIBCPP_{BEGIN,END}_EXPLICIT_ABI_ANNOTATIONS`,
which allow us to have implicit annotations for most functions, and just
where it's not "hide_from_abi everything" we add explicit annotations.
This allows us to drop the `_LIBCPP_HIDE_FROM_ABI` macro from most
functions in libc++.
That type trait represents whether move-assigning an object is
equivalent to destroying it and then move-constructing a new one from
the same argument. This will be useful in a few places where we may want
to destroy + construct instead of doing an assignment, in particular
when implementing some container operations in terms of relocation.
This is effectively adding a library emulation of P2786R12's
is_replaceable trait, similarly to what we do for trivial relocation.
Eventually, we can replace this library emulation by the real
compiler-backed trait.
This is building towards #129328.
There are currently lots of `_LIBCPP_HAS_ASAN` and
`__libcpp_is_constant_evaluated()` checks which aren't needed, since it
is centrally checked inside `__debug_utils/sanitizers.h`.
This patch moves the functions common between our two extern template
lists into a common list and avoids some unnecessary _Uglification. This
makes the code a lot nicer to read and makes the differences between the
two lists obvious.
Previously, the segmented iterator optimization for `std::for_each` was restricted to C++23 and later due to its dependency on `__movable_box`, which is not available in earlier standards. This patch eliminates that restriction, enabling consistent optimizations starting from C++11.
By backporting this enhancement, we improve performance across older standards and create opportunities to extend similar optimizations to other algorithms by forwarding their calls to `std::for_each`.
Requirements on character-like types are updated unconditionally,
because `basic_string` does requires the default-constructibility. It
might be possible to make `basic_string_view` support classes with
non-public trivial default constructor, but this doesn't seem sensible.
libcxxabi's `ItaniumDemangle.h` is also updated to avoid deprecated
features.
This macro isn't required if we define all the functions inline. In
fact, quite a few of the marked functions have already been inlined.
This patch basically only moves code around and adds
`_LIBCPP_HIDE_FROM_ABI` to the places where it's been missing so far.
This also removes inlining hints, since it dropps `inline` in some
places, but that shouldn't make much of a difference. The functions tend
to be either really small, so should be inlined anyways, or are big
enough that they shouldn't be inlined even with an inlinehint.
This PR refactors `basic_string` a bit to simplify its implementation in
the following ways:
- Instead of manually checking whether a string is short or long,
followed by calling the specific functions (e.g., `__get_short_size()`,
`__get_long_size()`), we call the general functions (`size()`) to hide
the conditional checks and make the code more concise.
- Once a string is determined to be short or long, we directly call the
specific functions instead of the general versions to get rid of
unnecessary internal conditional checks. For example, for a long string,
we would directly call `{__set, __get}_long_pointer` instead of `{__set,
__get}_pointer()`.
- Variables that are defined in both the `if` and `else` branches are
now declared in a common scope to reduce redundancy.
- When the string size is calculated multiple times using
`traits_type::length(__s)`, a variable is introduced to store its
length. While modern compilers can optimize this with constant folding,
explicitly storing the length improves code readability and makes the
logic clearer.
- Fixed synopsis with missing default arguments.
`std::erase(_if)` for `basic_string` were made `constexpr` in C++20 by
cplusplus/draft@2c1ab9775c as follow-up
changes of P0980R1.
This patch implements the missed changes that were not tracked in a
specific paper.
This set usage of operator& instead of std::addressof seems not be easy
to "abuse". Some seem easy to misuse, like basic_ostream::operator<<,
trying to do that results in compilation errors since the `widen`
function is not specialized for the hijacking character type. Hence
there are no tests.
Without this patch `basic_string` cannot be properly resized to be
`max_size()` elements in size, even if an allocation is successful.
`__grow_by` allocates one less element than required, resulting in an
out-of-bounds access. At the same time, `max_size()` has an off-by-one
error, since there has to be space to store the null terminator, which
is currently ignored.
The current implementation of the `shrink_to_fit()` function of
`basic_string` swaps to the newly allocated buffer when the new buffer
has the same capacity as the existing one. While this is not incorrect,
it is truly unnecessary to swap to an equally-sized buffer. With equal
capacity, we should keep using the existing buffer and simply deallocate
the new one, avoiding the extra work of copying elements.
The desired behavior was documented in the following comment within the
function:
61ad08792a/libcxx/include/string (L3560-L3566)
However, the existing implementation did not exactly conform to this
guideline, which is a QoI matter.
This PR modifies the `shrink_to_fit()` function to ensure that the
buffer is only swapped when the new allocation is strictly smaller than
the existing one. When the capacities are equal, the new buffer will be
discarded without copying the elements. This is achieved by including
the `==` check in the above conditional logic.
This is technically not necessary in most cases to prevent issues with ADL,
but let's be consistent. This allows us to remove the libcpp-qualify-declval
clang-tidy check, which is now enforced by the robust-against-adl clang-tidy check.
`__init(const value_type*, size_type, size_type)` is part of our ABI,
but we don't actually use the function anymore in the dylib. THis moves
the definition to the `src/` directory to make it clear that the code is
unused. This also allows us to remove it entirely in the unstable ABI.
Having them defined ouf-of-line results in a significant amount of
boilerplate without improving readability, since they're just one or two
lines long anyways.
As a drive-by, add comments between the declarations to make them easier
to distinguish.
The capacity is now passed correctly and a test for this path is added.
Since we changed the implementation of `reserve(size_type)` to only ever
extend,
it doesn't make a ton of sense anymore to have `__shrink_or_extend`,
since the code
paths of `reserve` and `shrink_to_fit` are now almost completely
separate.
This patch splits up `__shrink_or_extend` so that the individual parts
are in `reserve`
and `shrink_to_fit` depending on where they are needed.
This reverts commit 59f57be94f38758616b1339b293b43af845571af.
