tryEvaluateString was returning an std::optional, but the other try* API
was not. Update tryEvaluateObjectSize and tryEvaluateStrLen to return an
std::optional<uint64_t>.
The conversion code always ended up just getting the type of Src from
the Src argument itself, with no virtual users of this, so there is no
point in also providing this API hook. Fix the documentation as well,
since it seems DestAddr must have been similarly removed at some point
in the past from the API but was still documented.
Also fixes CIR to actually return the casted value!
ptrauth intrinsic to safely construct relative ptr for swift coroutines.
A ptrauth intrinsic for swift co-routine support that allows creation of
signed pointer
from offset stored at address relative to the pointer.
Following C-like pseudo code (ignoring keys,discriminators) explains its
operation:
let rawptr = PACauth(inputptr);
return PACsign( rawptr + signextend64( *(int32*)(rawptr+addend) ))
What: Authenticate a signed pointer, load a 32bit value at offset
'addend' from pointer,
add this value to pointer, sign this new pointer.
builtin: __builtin_ptrauth_auth_load_relative_and_sign
intrinsic: ptrauth_auth_resign_load_relative
This patch adds type-generic rotate builtins that accept any unsigned
integer
type. These builtins provide:
- Support for all unsigned integer types, including _BitInt
- Constexpr evaluation capability
- Automatic normalization of rotation counts modulo the bit-width
- Proper handling of negative rotation counts (converted to equivalent
positive rotations in the opposite direction)
- Implicit conversion support for both arguments for
types with conversion operators.
The builtins follow C23 naming conventions.
Resolves https://github.com/llvm/llvm-project/issues/122819
We proceeded with frontend/clang changes, until we figure out how ABI
for BE should look like. Once it is final, we will proceed with codegen
changes.
In this patch several things addressed:
- Define riscv32be/riscv64be target triples
- Set correct data layout for BE targets
- Handle BE-specific ABI details
- Emit warning for BE case since it is still experimental
Add support for `__builtin_stack_address` builtin. The semantics match
those of GCC's builtin with the same name.
`__builtin_stack_address` returns the starting address of the stack
region that may be used by called functions. It may or may not include
the space used for on-stack arguments passed to a callee (See [GCC
Bug/121013](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=121013)).
Fixes#82632.
Introduce `__builtin_allow_sanitize_check("name")` which returns true if
the specified sanitizer is enabled for the function (after inlining).
Supported sanitizers are "address", "thread", "memory", "hwaddress", and
their "kernel-" variants, matching the names of the
`no_sanitize("name")`
usage.
This builtin enables conditional execution of explicit checks only when
the sanitizer is enabled, respecting `no_sanitize` attributes, even when
used from `always_inline` functions that may be used in sanitized or
no_sanitize functions.
Since we must defer until after inlining and cannot determine the result
statically, Clang must lower to the `llvm.allow.sanitize.*` intrinsics,
which are then resolved by the `LowerAllowCheckPass`.
*Original Motivation:* The Linux kernel has a number of low-level
primitives that use inline assembly not visible to the sanitizers, but
use explicitly inserted checks to avoid coverage loss. Many of those
low-level helpers, however, are also used from so-called `noinstr`
functions, which use `no_sanitize(..)` to prohibit instrumentation;
these are used for very brittle code (such as when the kernel sets up a
task context *before* normal memory is accessible), and any
instrumentation, incl. from explicit instrumentation, is prohibited.
Many such helpers themselves are macros or `always_inline`, however, are
unable to be used from such brittle contexts because they contain
explicit instrumentation. This requires awkward workarounds to avoid the
instrumentation.
The ideal solution is this new builtin, that can be used to determine if
instrumentation is enabled in a given function or not, which the helper
can then use to insert instrumentation only where instrumentation is
allowed.
A recent such case came up in [1], where file-level instrumentation had
already been disabled for KASAN and KCSAN, which had not been necessary
if the new builtin were available.
[1] https://lore.kernel.org/all/20251208-gcov-inline-noinstr-v1-0-623c48ca5714@google.com/
Before this change, constrained fptrunc for __builtin_store_half/halff
always used round.tonearest, ignoring the active pragma STDC FENV_ROUND.
This PR guards builtin emission with CGFPOptionsRAII so the current
rounding mode is propagated to the generated constrained intrinsic.
The ctlz will produce a value in the range [1..bitwidth]. It can't
produce 0. This means the subtract of 1 will not have unsigned wrap.
It also has no signed wrap, but the optimizer can figure that out on its
own.
It's very likely InstCombine will just drop the NUW when it
canonicalizes to Add, but maybe it will be helpful in some case.
Fix#126516
Use AtomicOrdering::Monotonic's atomic loading to let the backend
automatically select the appropriate instructions to match MSVC semantics.
The __builtin_counted_by_ref builtin was previously limited to flexible
array members (FAMs). This change extends it to also support pointer
members that have the 'counted_by' attribute.
The 'counted_by' attribute can be applied to both FAMs and pointer
members:
struct fam_struct {
int count;
int array[] __attribute__((counted_by(count)));
};
struct ptr_struct {
int count;
int *buf __attribute__((counted_by(count)));
};
With this change, __builtin_counted_by_ref works with both:
*__builtin_counted_by_ref(p->array) = size; // FAM - already worked
*__builtin_counted_by_ref(p->buf) = size; // pointer - now works
This enables the same allocation pattern for pointer members that was
previously only available for FAMs:
#define alloc_buf(P, MEMBER, COUNT) ({ \
typeof(P) __p = malloc(sizeof(*__p)); \
__p->MEMBER = malloc(sizeof(*__p->MEMBER) * COUNT); \
*_Generic(__builtin_counted_by_ref(__p->MEMBER), \
void *: &(size_t){0}, \
default: __builtin_counted_by_ref(__p->MEMBER)) = COUNT; \
__p; \
})
The builtin returns:
- A pointer to the count field if the member has 'counted_by'
- void* (null) if the member is an array or pointer without 'counted_by'
- An error for other member types (e.g., int, struct)
This was requested by upstream Linux kernel devs:
https://lore.kernel.org/linux-hardening/202512041215.44484FCACD@keescook/
d076608d58d1ec55016eb747a995511e3a3f72aa moved some deps around to avoid
cycles and left clang/Frontend/FrontendDiagnostic.h as a shim that
simply includes clang/Basic/DiagnosticFrontend.h. This PR inlines it so
that nothing in tree still includes clang/Frontend/FrontendDiagnostic.h.
Doing this will help prevent future layering issues. See #162865.
Frontend already depends on Basic, so no new deps need to be added
anywhere except for places that do strict dep checking.
This PR moves the code that checks whether an LLVM intrinsic should be
generated instead of a call to floating point math functions to a
separate function. This simplifies `EmitBuiltinExpr` in `CGBuiltin.cpp`
and will allow us to reuse the logic in ClangIR.
Currently only __builtin_elementwise_sqrt emits contrained fp intrinsic
and propagates fp options.
This commit adds this support for the rest of elementwise builtins.
The counted_by attribute currently rejects void* members because void
has no defined size. However, the sized_by attribute accepts void* since
it explicitly measures bytes. As a GNU extension, void pointer
arithmetic treats void as having size 1 byte, so counted_by on void*
should behave identically to sized_by (treating the count as bytes).
Allow counted_by on void* as a GNU extension. The implementation
validates this only at declaration time in SemaBoundsSafety.cpp,
emitting a -Wpointer-arith warning that the attribute is treated as a
GNU extension equivalent to sized_by. Both use-site validation and code
generation trust this earlier validation, avoiding redundant checks.
In CodeGen, __builtin_dynamic_object_size now correctly handles
counted_by on void* by treating any CountAttributedType with zero
element size as having 1-byte elements, matching the GNU void pointer
arithmetic semantics.
Add tests validating both Sema diagnostics and CodeGen behavior (correct
byte counts from __builtin_dynamic_object_size). Update existing
counted_by tests to explicitly use -Wpointer-arith to preserve their
original intent of rejecting void* in strict C mode.
Implement code generation for `__builtin_infer_alloc_token()`. The
`AllocToken` pass is now registered to run unconditionally in the
optimization pipeline. This ensures that all instances of the
`llvm.alloc.token.id` intrinsic are lowered to constant token IDs,
regardless of whether `-fsanitize=alloc-token` is enabled. This
guarantees that the builtin always resolves to a token value, providing
a consistent and reliable mechanism for compile-time token querying.
This completes `__builtin_infer_alloc_token(<malloc-args>, ...)` to
allow compile-time querying of the token ID, where the builtin arguments
mirror those normally passed to any allocation function. The argument
expressions are unevaluated operands. For type-based token modes, the
same type inference logic is used as for untyped allocation calls.
For example the ID that is passed to (with `-fsanitize=alloc-token`):
some_malloc(sizeof(Type), ...)
is equivalent to the token ID returned by
__builtin_infer_alloc_token(sizeof(Type), ...)
The builtin provides a mechanism to pass or compare token IDs in code
that needs to be explicitly allocation token-aware (such as inside an
allocator, or through wrapper macros).
A more concrete demonstration of __builtin_infer_alloc_token's use is
enabling type-aware Slab allocations in the Linux kernel:
https://lore.kernel.org/all/20250825154505.1558444-1-elver@google.com/
Notably, any kind of allocation-call rewriting is a poor fit for the
Linux kernel's kmalloc-family functions, which are macros that wrap
(multiple) layers of inline and non-inline wrapper functions. Given the
Linux kernel defines its own allocation APIs, the more explicit builtin
gives the right level of control over where the type inference happens
and the resulting token is passed.
`UnqualPtrTy` didn't always match `llvm::PointerType::getUnqual`:
sometimes it returned a pointer that is not in address space 0 (notably
for SPIRV).
Since `UnqualPtrTy` was used as the "generic" or "default" pointer type,
this patch renames it to `DefaultPtrTy` to avoid confusion with LLVM's
`PointerType::getUnqual`.
Summary:
Right now these enformce alignment, which isn't convenient for the user
on platforms that support unaligned accesses. The options are to either
permit passing the alignment manually, or just assume it's unaligned
unless the user specifies it.
I've added https://github.com/llvm/llvm-project/pull/156057 which should
make the requiested alignment show up on the intrinsic if the user
passed `__builtin_assume_aligned`, however that's only with
optimizations. This shouldn't cause issues unless the backend
categorically decides to reject an unaligned access.
Summary:
This patch exposes `__builtin_masked_gather` and
`__builtin_masked_scatter` to clang. These map to the underlying
intrinsic relatively cleanly, needing only a level of indirection to
take a vector of indices and a base pointer to a vector of pointers.
Summary:
The added bit counting builtins for vectors used `cttz` and `ctlz`,
which is consistent with the LLVM naming convention. However, these are
clang builtins and implement exactly the `__builtin_ctzg` and
`__builtin_clzg` behavior. It is confusing to people familiar with other
other builtins that these are the only bit counting intrinsics named
differently. This includes the additional operation for the undefined
zero case, which was added as a `clzg` extension.
Summary:
The interface here is nearly indentical to the already added masked
loads and stores. These bind to very similar intrinsics so we add them
here.
Summary:
Boolean vectors as implemented in clang can be bit-casted to an integer
that is rounded up to the next primitive sized integer. Users can do
this themselves, but since the counting bits are very likely to be used
with bitmasks like this and the generic forms are expected to be
generic it seems reasonable that we handle this case directly.
This changes a bunch of places which use getAs<TagType>, including
derived types, just to obtain the tag definition.
This is preparation for #155028, offloading all the changes that PR used
to introduce which don't depend on any new helpers.
Summary:
Clang has support for boolean vectors, these builtins expose the LLVM
instruction of the same name. This differs from a manual load and select
by potentially suppressing traps from deactivated lanes.
Fixes: https://github.com/llvm/llvm-project/issues/107753
These builtins are modeled on the clzg/ctzg builtins, which accept an
optional second argument. This second argument is returned if the first
argument is 0. These builtins unconditionally exhibit zero-is-undef
behaviour, regardless of target preference for the other ctz/clz
builtins. The builtins have constexpr support.
Fixes#154113
Now that #149310 has restricted lifetime intrinsics to only work on
allocas, we can also drop the explicit size argument. Instead, the size
is implied by the alloca.
This removes the ability to only mark a prefix of an alloca alive/dead.
We never used that capability, so we should remove the need to handle
that possibility everywhere (though many key places, including stack
coloring, did not actually respect this).
Writing something like this:
__builtin_dynamic_object_size((0, p->array), 0)
is equivalent to writing this:
__builtin_dynamic_object_size(p->array, 0)
though the former will give a warning about the first value being
unused.
__enqueue_kernel_varargs' last parameter is in addrspace(5), but CodeGen
currently misses this qualifier. This commit fixes the code to preserve
the qualifier by referencing Alloca, which has its casts removed, rather
than TmpPtr.
For long enough _BitInt types we use different types for memory,
storing-loading and other operations. Makes sure it is correct for mixed
sign __builtin_mul_overflow cases. Using pointer element type as a
result type doesn't work, because it will be "in-memory" type which is
usually bigger than "operations" type and that caused crashes because
clang was trying to emit trunc to a bigger type.
Fixes https://github.com/llvm/llvm-project/issues/144771
ArrayRef has a constructor that accepts std::nullopt. This
constructor dates back to the days when we still had llvm::Optional.
Since the use of std::nullopt outside the context of std::optional is
kind of abuse and not intuitive to new comers, I would like to move
away from the constructor and eventually remove it.
This patch takes care of the clang side of the migration.
These are identified by misc-include-cleaner. I've filtered out those
that break builds. Also, I'm staying away from llvm-config.h,
config.h, and Compiler.h, which likely cause platform- or
compiler-specific build failures.
This extends https://github.com/llvm/llvm-project/pull/138577 to more UBSan checks, by changing SanitizerDebugLocation (formerly SanitizerScope) to add annotations if enabled for the specified ordinals.
Annotations will use the ordinal name if there is exactly one ordinal specified in the SanitizerDebugLocation; otherwise, it will use the handler name.
Updates the tests from https://github.com/llvm/llvm-project/pull/141814.
---------
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
With pointer authentication it becomes non-trivial to correctly load the
vtable pointer of a polymorphic object.
__builtin_get_vtable_pointer is a function that performs the load and
performs the appropriate authentication operations if necessary.
The patch introduce __builtin_spirv_generic_cast_to_ptr_explicit which
is lowered to the llvm.spv.generic.cast.to.ptr.explicit intrinsic.
The SPIR-V builtins are now split into 3 differents file:
BuiltinsSPIRVCore.td,
BuiltinsSPIRVVK.td for Vulkan specific builtins, BuiltinsSPIRVCL.td for
OpenCL specific builtins
and BuiltinsSPIRVCommon.td for common ones.
The patch also introduces a new header defining its SPIR-V friendly
equivalent (__spirv_GenericCastToPtrExplicit_ToGlobal,
__spirv_GenericCastToPtrExplicit_ToLocal and
__spirv_GenericCastToPtrExplicit_ToPrivate). The functions are declared
as aliases to the new builtin allowing C-like languages to have a
definition to rely on as well as gaining proper front-end diagnostics.
The motivation for the header is to provide a stable binding for
applications or library (such as SYCL) and allows non SPIR-V targets to
provide an implementation (via libclc or similar to how it is done for
gpuintrin.h).
This reverts commit 894a0dd57f81211f9e431d9e84f2856d34f46993 with
tests fixed.
This patch is part of a stack that teaches Clang to generate Key Instructions
metadata for C and C++.
RFC:
https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668
The feature is only functional in LLVM if LLVM is built with CMake flag
LLVM_EXPERIMENTAL_KEY_INSTRUCTIONs. Eventually that flag will be removed.
