shylie/llvm-project
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity
llvm-project/libcxx/include/__memory/compressed_pair.h
Nikolas Klauser f5e687d7bf
[libc++] Fix ABI break introduced by switching to _LIBCPP_COMPRESSED_PAIR (#154686) 
LLVM 20 contained an ABI break that can result in the size of
`std::unordered_{map,set,multimap,multiset}` and `std::deque` changing
when used with an allocator type that is empty and contains a base class
that is the same across rebound allocator instantiations (e.g.
``Allocator<int>`` and ``Allocator<char>`` are both empty and contain
the same base class).

In addition, the layout of a user-defined type that:
- contains one of the following containers:
`std::unordered_{map,set,multimap,multiset}`, `std::deque`, `std::map`,
`std::set`, `std::multimap`, `std::multiset`, `std::list` or
`std::vector`, and
- passes an empty allocator, comparator or hasher type to that
container, and
- has a member of that same empty allocator, comparator or hasher type
inside the enclosing struct, and
- that member is either marked with `[[no_unique_address]]` or optimized
out via the EBO (empty base optimization) technique
saw its size increase from LLVM 19 to LLVM 20. This was caused by the
usage of `[[no_unique_address]]` within some of libc++'s containers in a
way that allowed subtle interactions with enclosing objects. This is
fixed in LLVM 21 on Clang (returning to the LLVM 19 ABI), however that
implies an ABI break from LLVM 20 to LLVM 21.
Furthermore, fixing this causes a slight regression to constant
evaluation support in `std::unique_ptr`. Specifically, constant
evaluation will now fail when the deleter relies on being
value-initialized for constant-evaluation admissibility. If a
default-initialized deleter can be used during constant evaluation, or
if the default constructor is non-trivial, the `unique_ptr` is not
affected by this regression. In particular, this regression does not
impact any `unique_ptr` using the default deleter.

Note that there is currently no way to realistically fix this ABI break
on GCC, therefore GCC will remain on the ABI introduced in LLVM 19. That
also means that Clang and GCC will have a slightly different ABI for the
small subset of types listed above until we are able to apply the same
fix we did with Clang on GCC.

We fix this regression by surrounding the members of the
`_LIBCPP_COMPRESSED_PAIR` with an anonymous struct. This restricts the
shifting of empty types to the front of the `_LIBCPP_COMPRESSED_PAIR`
instead of throughout the surrounding object. This "frees up" the zero
offset to contain another object of the same type, restoring the ability
to perform EBO or to elide the storage for a type with
`[[no_unique_address]]` in the enclosing (user-defined) struct.

Fixes #154146

Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
2025-08-23 07:58:37 +02:00

137 lines
7.7 KiB
C++
Raw Blame History

// -*- C++ -*-
//===----------------------------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#ifndef _LIBCPP___MEMORY_COMPRESSED_PAIR_H
#define _LIBCPP___MEMORY_COMPRESSED_PAIR_H
#include <__config>
#include <__cstddef/size_t.h>
#include <__type_traits/datasizeof.h>
#include <__type_traits/is_empty.h>
#include <__type_traits/is_final.h>
#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
# pragma GCC system_header
#endif
_LIBCPP_BEGIN_NAMESPACE_STD
// ================================================================================================================== //
// The utilites here are for staying ABI compatible with the legacy `__compressed_pair`. They should not be used //
// for new data structures. Use `_LIBCPP_NO_UNIQUE_ADDRESS` for new data structures instead (but make sure you //
// understand how it works). //
// ================================================================================================================== //
// The first member is aligned to the alignment of the second member to force padding in front of the compressed pair
// in case there are members before it.
//
// For example:
// (assuming x86-64 linux)
// class SomeClass {
// uint32_t member1;
// _LIBCPP_COMPRESSED_PAIR(uint32_t, member2, uint64_t, member3);
// }
//
// The layout with __compressed_pair is:
// member1 - offset: 0, size: 4
// padding - offset: 4, size: 4
// member2 - offset: 8, size: 4
// padding - offset: 12, size: 4
// member3 - offset: 16, size: 8
//
// If the [[gnu::aligned]] wasn't there, the layout would instead be:
// member1 - offset: 0, size: 4
// member2 - offset: 4, size: 4
// member3 - offset: 8, size: 8
//
// Furthermore, that alignment must be the same as what was used in the old __compressed_pair layout, so we must
// handle reference types specially since alignof(T&) == alignof(T).
// See https://github.com/llvm/llvm-project/issues/118559.
#ifndef _LIBCPP_ABI_NO_COMPRESSED_PAIR_PADDING
template <class _Tp>
inline const size_t __compressed_pair_alignment = _LIBCPP_ALIGNOF(_Tp);
template <class _Tp>
inline const size_t __compressed_pair_alignment<_Tp&> = _LIBCPP_ALIGNOF(void*);
template <class _ToPad>
inline const bool __is_reference_or_unpadded_object =
(is_empty<_ToPad>::value && !__libcpp_is_final<_ToPad>::value) || sizeof(_ToPad) == __datasizeof_v<_ToPad>;
template <class _Tp>
inline const bool __is_reference_or_unpadded_object<_Tp&> = true;
template <class _Tp>
inline const bool __is_reference_or_unpadded_object<_Tp&&> = true;
template <class _ToPad, bool _Empty = __is_reference_or_unpadded_object<_ToPad> >
class __compressed_pair_padding {
char __padding_[sizeof(_ToPad) - __datasizeof_v<_ToPad>] = {};
};
template <class _ToPad>
class __compressed_pair_padding<_ToPad, true> {};
// TODO: Fix the ABI for GCC as well once https://gcc.gnu.org/bugzilla/show_bug.cgi?id=121637 is fixed
# ifdef _LIBCPP_COMPILER_GCC
# define _LIBCPP_COMPRESSED_PAIR(T1, Initializer1, T2, Initializer2) \
_LIBCPP_NO_UNIQUE_ADDRESS __attribute__((__aligned__(::std::__compressed_pair_alignment<T2>))) T1 Initializer1; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T1> _LIBCPP_CONCAT3(__padding1_, __LINE__, _); \
_LIBCPP_NO_UNIQUE_ADDRESS T2 Initializer2; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T2> _LIBCPP_CONCAT3(__padding2_, __LINE__, _)
# define _LIBCPP_COMPRESSED_TRIPLE(T1, Initializer1, T2, Initializer2, T3, Initializer3) \
_LIBCPP_NO_UNIQUE_ADDRESS \
__attribute__((__aligned__(::std::__compressed_pair_alignment<T2>), \
__aligned__(::std::__compressed_pair_alignment<T3>))) T1 Initializer1; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T1> _LIBCPP_CONCAT3(__padding1_, __LINE__, _); \
_LIBCPP_NO_UNIQUE_ADDRESS T2 Initializer2; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T2> _LIBCPP_CONCAT3(__padding2_, __LINE__, _); \
_LIBCPP_NO_UNIQUE_ADDRESS T3 Initializer3; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T3> _LIBCPP_CONCAT3(__padding3_, __LINE__, _)
# else
# define _LIBCPP_COMPRESSED_PAIR(T1, Initializer1, T2, Initializer2) \
struct { \
_LIBCPP_NO_UNIQUE_ADDRESS \
__attribute__((__aligned__(::std::__compressed_pair_alignment<T2>))) T1 Initializer1; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T1> _LIBCPP_CONCAT3(__padding1_, __LINE__, _); \
_LIBCPP_NO_UNIQUE_ADDRESS T2 Initializer2; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T2> _LIBCPP_CONCAT3(__padding2_, __LINE__, _); \
}
# define _LIBCPP_COMPRESSED_TRIPLE(T1, Initializer1, T2, Initializer2, T3, Initializer3) \
struct { \
_LIBCPP_NO_UNIQUE_ADDRESS \
__attribute__((__aligned__(::std::__compressed_pair_alignment<T2>), \
__aligned__(::std::__compressed_pair_alignment<T3>))) T1 Initializer1; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T1> _LIBCPP_CONCAT3(__padding1_, __LINE__, _); \
_LIBCPP_NO_UNIQUE_ADDRESS T2 Initializer2; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T2> _LIBCPP_CONCAT3(__padding2_, __LINE__, _); \
_LIBCPP_NO_UNIQUE_ADDRESS T3 Initializer3; \
_LIBCPP_NO_UNIQUE_ADDRESS ::std::__compressed_pair_padding<T3> _LIBCPP_CONCAT3(__padding3_, __LINE__, _); \
}
# endif
#else
# define _LIBCPP_COMPRESSED_PAIR(T1, Name1, T2, Name2) \
_LIBCPP_NO_UNIQUE_ADDRESS T1 Name1; \
_LIBCPP_NO_UNIQUE_ADDRESS T2 Name2
# define _LIBCPP_COMPRESSED_TRIPLE(T1, Name1, T2, Name2, T3, Name3) \
_LIBCPP_NO_UNIQUE_ADDRESS T1 Name1; \
_LIBCPP_NO_UNIQUE_ADDRESS T2 Name2; \
_LIBCPP_NO_UNIQUE_ADDRESS T3 Name3
#endif // _LIBCPP_ABI_NO_COMPRESSED_PAIR_PADDING
_LIBCPP_END_NAMESPACE_STD
#endif // _LIBCPP___MEMORY_COMPRESSED_PAIR_H
Reference in New Issue View Git Blame Copy Permalink