Summary:
This PR changes the handling of the emitted kernels when targeting a CPU
to be a pointer struct.
The old handling emitted a standard function prototype, this
necessitated a target specific ABI to call it because the signature
differed with the number of arguments. Instead, this PR emits a void
pointer to a naturally aligned struct, this is what APIs like `pthreads`
assert.
This allows us to remove all the complexity around launching host
kernels and just pass the argument list.
This PR implements Function Multi-versioning on AIX using `__attribute__
((target_clones(<feature-list>)))`.
Initially, we will only support specifying a cpu in the version list.
Feature strings (such as "altivec" or "isel") on target_clones will be
implemented in a future PR.
Accepted syntax:
```
__attribute__((target_clones(<OPTIONS>)))
```
where `<OPTIONS>` is a comma separated list of strings, each string is
either:
1) the default string `"default"`
2) a cpu string `"cpu=<CPU>"`, where `<CPU>`is a value accepted by the
`-mcpu` flag.
For example, specifying the following on a function
```
__attribute__((target_clones("default", "cpu=power8", "cpu=power9")))
int foo(int x) { return x + 1; }
```
Would generate 3 versions of `foo`: (1) `foo.default`, (2)
`foo.cpu_power8`, and (3) `foo.cpu_power9`,
an IFUNC `foo`, and the resolver function `foo.resolver`, for the IFUNC,
that chooses one of the three versions at runtime.
---------
Co-authored-by: Wael Yehia <wyehia@ca.ibm.com>
Store alloca element types alongside alloca pointers in Clang's CodeGen,
eliminating the need to query getAllocatedType() later.
Changes:
- CodeGenFunction.h: Modified DominatingLLVMValue::saved_type to track
type Uses Type==nullptr as sentinel for "not saved" instead of separate
bool. Fixes a bug where it previously tried to use the addrspacecasted
value constructor and later cast it to an alloca.
- CGHLSLRuntime.cpp: Changed OutputSemantic map to store (Value*, Type*)
pairs to trivially remove getAllocatedType calls.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
The `sycl_kernel_entry_point` attribute facilitates the generation of an
offload kernel entry point function based on the parameters and body
of the attributed function. This change extends the behavior of that
attribute to support integration with a SYCL runtime library through
an interface that communicates symbol names and kernel arguments
for the generated offload kernel entry point functions.
Consider the following function declared with the
`sycl_kernel_entry_point` attribute with a call to this function
occurring in the implementation of a SYCL kernel invocation function
such as `sycl::handler::single_task()`.
```c++
template<typename KernelName, typename KernelType>
[[clang::sycl_kernel_entry_point(KernelName)]]
void kernel_entry_point(KernelType kernel) {
kernel();
}
```
The body of the above function specifies the parameters and body of the
generated offload kernel entry point. Clearly, a call to the above
function by a SYCL kernel invocation function is not intended to execute
the body as written. Previously, code generation emitted an empty
function body so that calls to the function had no effect other than to
trigger the generation of the offload kernel entry point. The function
body is therefore available to hook for SYCL library support and is now
substituted with a call to a (SYCL library provided) function template
or variable template named `sycl_kernel_launch()` with the kernel
name type passed as the first template argument, the symbol name
of the offload kernel entry point passed as a string literal for the first
function argument, and the function parameters passed as the
remaining explicit function arguments. Given a call like this:
```c++
kernel_entry_point<struct KN>([]{})
```
the body of the instantiated `kernel_entry_point()` specialization would
be substituted as follows with "kernel-symbol-name" substituted for the
generated symbol name and `kernel` forwarded.
```c++
sycl_kernel_launch<KN>("kernel-symbol-name", kernel)
```
Name lookup and overload resolution for the `sycl_kernel_launch()`
function is performed at the point of definition of the
`sycl_kernel_entry_point` attributed function (or the point of
instantiation for an instantiated function template specialization). If
overload resolution fails, the program is ill-formed.
Implementation of the `sycl_kernel_launch()` function might require
additional information provided by the SYCL library. This is facilitated
by removing the previous prohibition against use of the
`sycl_kernel_entry_point` attribute with a non-static member function.
If the `sycl_kernel_entry_point` attributed function is a non-static
member function, then overload resolution for the `sycl_kernel_launch()`
function template may select a non-static member function in which case,
`this` will be implicitly passed as the implicit object argument.
If a `sycl_kernel_entry_point` attributed function is a non-static
member function, use of `this` in a potentially evaluated expression is
prohibited in the definition since `this` is not a kernel argument and
will not be available within the generated offload kernel entry point
function. The attribute cannot be applied to a function with an
explicit object parameter.
---------
Co-authored-by: Mariya Podchishchaeva <mariya.podchishchaeva@intel.com>
fixes#159438
This patch adds `MatrixElementExpr`, a new AST node for HLSL matrix
element and swizzle access (e.g. M._m00, M._11_22_33).
It introduces a shared `ElementAccessExprBase` used by both matrix and
vector swizzle expressions, updates Sema to parse and validate
zero-based and one-based accessors, detects duplicates for l-value
checks, and emits improved diagnostics. CodeGen is updated to lower
scalar and multi-element accesses consistently, and full AST
serialization, dumping, and tooling support is included. This
implementation reflects the updated
[RFC](https://github.com/llvm/wg-hlsl/pull/357/files) for HLSL matrix
accessor semantics.
The conversion code always ended up just getting the type of Src from
the Src argument itself, with no virtual users of this, so there is no
point in also providing this API hook. Fix the documentation as well,
since it seems DestAddr must have been similarly removed at some point
in the past from the API but was still documented.
Also fixes CIR to actually return the casted value!
This just added unnecessary work to the IR, since they are only used for
load and store, which just causes some IR noise. Tests updated by UTC
script to remove the extra lines.
The return value of `getIsCounterPair(S)` was `std::pair<bool,bool>` but
its `hasExec` wasn't used. So, introduce just the `hasSkipCounter(S)`.
It'd be better to name `getExecSkipCounterExistence` if
`std::pair<bool,bool>` would be resurrected in the future.
MCDCCondBitmapAddr is moved from `CodeGenFunction` into `MCDCState` and
created for each Decision.
In `maybeCreateMCDCCondBitmap`, Allocate bitmaps for all valid Decisions
and emit them order by ID, to prevent nondeterminism.
`getBranchCounterPair()` allocates an additional Counter to SkipPath in
`SingleByteCoverage`.
`IsCounterEqual()` calculates the comparison with rewinding counter
replacements.
`NumRegionCounters` is updated to take additional counters in account.
`incrementProfileCounter()` has a few additiona arguments.
- `UseSkipPath=true`, to specify setting counters for SkipPath. It
assumes `UseSkipPath=false` is used together.
- `UseBoth` may be specified for marking another path. It introduces the
same effect as issueing `markStmtAsUsed(!SkipPath, S)`.
`llvm-cov` discovers counters in `FalseCount` to allocate `MaxCounterID`
for empty profile data.
https://discourse.llvm.org/t/rfc-integrating-singlebytecoverage-with-branch-coverage/82492
Resumes: #112730
Depends on: #112698#112702#112724
fixes#166206
- Add swizzle support if row index is constant
- Add test cases
- Add new AST type
- Add new LValue for Matrix Row Type
- TODO: Make the new LValue a dynamic index version of ExtVectorElt
If the 'counted_by' value is signed, we will incorrectly allow accesses
when the value is negative. This has obvious bad effects as it will
allow accessing a huge swath of unallocated memory.
Also clarify and rearrange the parameters to make them more
perspicuous.
Fixes: #170987.
Refactor the logic for inferring allocated types out of `CodeGen` and
into a new reusable component in `clang/AST/InferAlloc.h`.
This is a preparatory step for implementing `__builtin_infer_alloc_token`.
By moving the type inference heuristics into a common place, it can be
shared between the existing allocation-call instrumentation and the new
builtin's implementation.
Adds support for elementwise and aggregate splat casting struct types
with bitfields. Replacing existing Flattening function which used to
produce a list of GEPs representing a flattened object with one that
produces a list of LValues representing a flattened object. The LValues
can be used by EmitStoreThroughLValue and EmitLoadOfLValue, ensuring
bitfields are properly loaded and stored. This also simplifies the code
in the elementwise and aggregate splat casting functions.
Closes#125986
This change implements the fuse directive, `#pragma omp fuse`, as specified in the OpenMP 6.0, along with the `looprange` clause in clang.
This change also adds minimal stubs so flang keeps compiling (a full implementation in flang of this directive is still pending).
---------
Co-authored-by: Roger Ferrer Ibanez <roger.ferrer@bsc.es>
In SPIR-V, kernel arguments are not allowed to be in the Generic AS, in
both Intel's internal SPIR-V offloading implementation as well as
HIPSPV, `CrossWorkgroup` AS1 is used. Do the same for OMPSPV.
Currently with Generic AS the `llvm-spirv` translator blows up if we are
using it, and if not, the GPU runtime blows up.
To get the existing logic to set the correct AS to kick in, we need to
know if the function is a kernel or not at the time we first create the
function that may end up as the kernel.
I use the existing `arrangeSYCLKernelCallerDeclaration` function to do
the right kernel ABI computation, but since the function is not specific
to SYCL anymore because I merged all the device kernel clang attributes
into one.
Rename the function to be accurate to the current behavior,
`arrangeDeviceKernelCallerDeclaration`.
---------
Signed-off-by: Sarnie, Nick <nick.sarnie@intel.com>
This implements support for [named
loops](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3355.htm) for
C2y.
When parsing a `LabelStmt`, we create the `LabeDecl` early before we parse
the substatement; this label is then passed down to `ParseWhileStatement()`
and friends, which then store it in the loop’s (or switch statement’s) `Scope`;
when we encounter a `break/continue` statement, we perform a lookup for
the label (and error if it doesn’t exist), and then walk the scope stack and
check if there is a scope whose preceding label is the target label, which
identifies the jump target.
The feature is only supported in C2y mode, though a cc1-only option
exists for testing (`-fnamed-loops`), which is mostly intended to try
and make sure that we don’t have to refactor this entire implementation
when/if we start supporting it in C++.
---------
Co-authored-by: Balazs Benics <benicsbalazs@gmail.com>
In 29992cfd628ed5b968ccb73b17ed0521382ba317 (#145967) support was added
for "trap reasons" on traps emitted in UBSan in trapping mode (e.g.
`-fsanitize-trap=undefined`). This improved the debugging experience by
attaching the reason for trapping as a string on the debug info on trap
instructions. Consumers such as LLDB can display this trap reason string
when the trap is reached.
A limitation of that patch is that the trap reason string is hard-coded
for each `SanitizerKind` even though the compiler actually has much more
information about the trap available at compile time that could be shown
to the user.
This patch is an incremental step in fixing that. It consists of two
main steps.
**1. Introduce infrastructure for building trap reason strings**
To make it convenient to construct trap reason strings this patch
re-uses Clang's powerful diagnostic infrastructure to provide a
convenient API for constructing trap reason strings. This is achieved
by:
* Introducing a new `Trap` diagnostic kind to represent trap diagnostics
in TableGen files.
* Adding a new `Trap` diagnostic component. While this part probably
isn't technically necessary it seemed like I should follow the existing
convention used by the diagnostic system.
* Adding `DiagnosticTrapKinds.td` to describe the different trap
reasons.
* Add the `TrapReasonBuilder` and `TrapReason` classes to provide an
interface for constructing trap reason strings and the trap category.
Note this API while similar to `DiagnosticBuilder` has different
semantics which are described in the code comments. In particular the
behavior when the destructor is called is very different.
* Adding `CodeGenModule::BuildTrapReason()` as a convenient constructor
for the `TrapReasonBuilder`.
This use of the diagnostic system is a little unusual in that the
emitted trap diagnostics aren't actually consumed by normal diagnostic
consumers (e.g. the console). Instead the `TrapReasonBuilder` is just
used to format a string, so in effect the builder is somewhat analagous
to "printf". However, re-using the diagnostics system in this way brings
a several benefits:
* The powerful diagnostic templating languge (e.g. `%select`) can be
used.
* Formatting Clang data types (e.g. `Type`, `Expr`, etc.) just work
out-of-the-box.
* Describing trap reasons in tablegen files opens the door for
translation to different languages in the future.
* The `TrapReasonBuilder` API is very similar to `DiagnosticBuilder`
which makes it easy to use by anyone already familiar with Clang's
diagnostic system.
While UBSan is the first consumer of this new infrastructure the intent
is to use this to overhaul how trap reasons are implemented in the
`-fbounds-safety` implementation (currently exists downstream).
**2. Apply the new infrastructure to UBSan checks for arithmetic
overflow**
To demonstrate using `TrapReasonBuilder` this patch applies it to UBSan
traps for arithmetic overflow. The intention is that we would
iteratively switch to using the `TrapReasonBuilder` for all UBSan traps
where it makes sense in future patches.
Previously for code like
```
int test(int a, int b) { return a + b; }
```
The trap reason string looked like
```
Undefined Behavior Sanitizer: Integer addition overflowed
```
now the trap message looks like:
```
Undefined Behavior Sanitizer: signed integer addition overflow in 'a + b'
```
This string is much more specific because
* It explains if signed or unsigned overflow occurred
* It actually shows the expression that overflowed
One possible downside of this approach is it may blow up Debug info size
because now there can be many more distinct trap reason strings. To
allow users to avoid this a new driver/cc1 flag
`-fsanitize-debug-trap-reasons=` has been added which can either be
`none` (disable trap reasons entirely), `basic` (use the per
`SanitizerKind` hard coded strings), and `detailed` (use the new
expressive trap reasons implemented in this patch). The default is
`detailed` to give the best out-of-the-box debugging experience. The
existing `-fsanitize-debug-trap-reasons` and
`-fno-sanitize-debug-trap-reasons` have been kept for compatibility and
are aliases of the new flag with `detailed` and `none` arguments passed
respectively.
rdar://158612755
This reintroduces `Type.h`, having earlier been renamed to `TypeBase.h`,
as a redirection to `TypeBase.h`, and redirects most users to include
the former instead.
This is a preparatory patch for being able to provide inline definitions
for `Type` methods which would otherwise cause a circular dependency
with `Decl{,CXX}.h`.
Doing these operations into their own NFC patch helps the git rename
detection logic work, preserving the history.
This patch makes clang just a little slower to build (~0.17%), just
because it makes more code indirectly include `DeclCXX.h`.
This is a preparatory patch, to be able to provide inline definitions
for `Type` functions which depend on `Decl{,CXX}.h`. As the latter also
depends on `Type.h`, this would not be possible without some
reorganizing.
Splitting this rename into its own patch allows git to track this as a
rename, and preserve all git history, and not force any code
reformatting.
A later NFC patch will reintroduce `Type.h` as redirection to
`TypeBase.h`, rewriting most places back to directly including `Type.h`
instead of `TypeBase.h`, leaving only a handful of places where this is
necessary.
Then yet a later patch will exploit this by making more stuff inline.
This changes a bunch of places which use getAs<TagType>, including
derived types, just to obtain the tag definition.
This is preparation for #155028, offloading all the changes that PR used
to introduce which don't depend on any new helpers.
The cleanup structs expect that pointers and (u)int64_t have the same
alignment requirements, which isn't true on sparc32, which causes
SIGBUSes.
See also: https://github.com/llvm/llvm-project/issues/66620
This is a major change on how we represent nested name qualifications in
the AST.
* The nested name specifier itself and how it's stored is changed. The
prefixes for types are handled within the type hierarchy, which makes
canonicalization for them super cheap, no memory allocation required.
Also translating a type into nested name specifier form becomes a no-op.
An identifier is stored as a DependentNameType. The nested name
specifier gains a lightweight handle class, to be used instead of
passing around pointers, which is similar to what is implemented for
TemplateName. There is still one free bit available, and this handle can
be used within a PointerUnion and PointerIntPair, which should keep
bit-packing aficionados happy.
* The ElaboratedType node is removed, all type nodes in which it could
previously apply to can now store the elaborated keyword and name
qualifier, tail allocating when present.
* TagTypes can now point to the exact declaration found when producing
these, as opposed to the previous situation of there only existing one
TagType per entity. This increases the amount of type sugar retained,
and can have several applications, for example in tracking module
ownership, and other tools which care about source file origins, such as
IWYU. These TagTypes are lazily allocated, in order to limit the
increase in AST size.
This patch offers a great performance benefit.
It greatly improves compilation time for
[stdexec](https://github.com/NVIDIA/stdexec). For one datapoint, for
`test_on2.cpp` in that project, which is the slowest compiling test,
this patch improves `-c` compilation time by about 7.2%, with the
`-fsyntax-only` improvement being at ~12%.
This has great results on compile-time-tracker as well:
This patch also further enables other optimziations in the future, and
will reduce the performance impact of template specialization resugaring
when that lands.
It has some other miscelaneous drive-by fixes.
About the review: Yes the patch is huge, sorry about that. Part of the
reason is that I started by the nested name specifier part, before the
ElaboratedType part, but that had a huge performance downside, as
ElaboratedType is a big performance hog. I didn't have the steam to go
back and change the patch after the fact.
There is also a lot of internal API changes, and it made sense to remove
ElaboratedType in one go, versus removing it from one type at a time, as
that would present much more churn to the users. Also, the nested name
specifier having a different API avoids missing changes related to how
prefixes work now, which could make existing code compile but not work.
How to review: The important changes are all in
`clang/include/clang/AST` and `clang/lib/AST`, with also important
changes in `clang/lib/Sema/TreeTransform.h`.
The rest and bulk of the changes are mostly consequences of the changes
in API.
PS: TagType::getDecl is renamed to `getOriginalDecl` in this patch, just
for easier to rebasing. I plan to rename it back after this lands.
Fixes#136624
Fixes https://github.com/llvm/llvm-project/issues/43179
Fixes https://github.com/llvm/llvm-project/issues/68670
Fixes https://github.com/llvm/llvm-project/issues/92757
Now that #149310 has restricted lifetime intrinsics to only work on
allocas, we can also drop the explicit size argument. Instead, the size
is implied by the alloca.
This removes the ability to only mark a prefix of an alloca alive/dead.
We never used that capability, so we should remove the need to handle
that possibility everywhere (though many key places, including stack
coloring, did not actually respect this).
This extends https://github.com/llvm/llvm-project/pull/138577 to more UBSan checks, by changing SanitizerDebugLocation (formerly SanitizerScope) to add annotations if enabled for the specified ordinals.
Annotations will use the ordinal name if there is exactly one ordinal specified in the SanitizerDebugLocation; otherwise, it will use the handler name.
Updates the tests from https://github.com/llvm/llvm-project/pull/141814.
---------
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
This patch is part of a stack that teaches Clang to generate Key Instructions
metadata for C and C++.
When returning a value, stores to the `retval` allocas and branches to `return`
block are put in the same atom group. They are both rank 1, which could in
theory introduce an extra step in some optimized code. This low risk currently
feels an acceptable for keeping the code a bit simpler (as opposed to adding
scaffolding to make the store rank 2).
In the case of a single return (no control flow) the return instruction inherits
the atom group of the branch to the return block when the blocks get folded
togather.
RFC:
https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668
The feature is only functional in LLVM if LLVM is built with CMake flag
LLVM_EXPERIMENTAL_KEY_INSTRUCTIONs. Eventually that flag will be removed.
The InstrProf headers are very expensive. Avoid including them in all of
CodeGen/ by moving the CodeGenPGO member behind a unqiue_ptr.
This reduces clang build time by 0.8%.
This is a scoped helper similar to ApplyDebugLocation that creates a new source
location atom group which instructions can be added to.
A source atom is a source construct that is "interesting" for debug stepping
purposes. We use an atom group number to track the instruction(s) that implement
the functionality for the atom, plus backup instructions/source locations.
This patch is part of a stack that teaches Clang to generate Key Instructions
metadata for C and C++.
RFC:
https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668
The feature is only functional in LLVM if LLVM is built with CMake flag
LLVM_EXPERIMENTAL_KEY_INSTRUCTIONs. Eventually that flag will be removed.
This generalizes the debug info annotation code from https://github.com/llvm/llvm-project/pull/139149 and moves it into a helper function, SanitizerAnnotateDebugInfo().
Future work can use 'ApplyDebugLocation ApplyTrapDI(*this, SanitizerAnnotateDebugInfo(Ordinal));' to add annotations to additional checks.
The 'counted_by' attribute is now available for pointers in structs.
It generates code for sanity checks as well as
__builtin_dynamic_object_size()
calculations. For example:
struct annotated_ptr {
int count;
char *buf __attribute__((counted_by(count)));
};
If the pointer's type is 'void *', use the 'sized_by' attribute, which
works similarly to 'counted_by', but can handle the 'void' base type:
struct annotated_ptr {
int count;
void *buf __attribute__((sized_by(count)));
};
If the 'count' field member occurs after the pointer, use the
'-fexperimental-late-parse-attributes' flag during compilation.
Note that 'counted_by' cannot be applied to a pointer to an incomplete
type, because the size isn't known.
struct foo;
struct annotated_ptr {
int count;
struct foo *buf __attribute__((counted_by(count))); /* invalid */
};
Signed-off-by: Bill Wendling <morbo@google.com>
This fixes emitting undefined behavior where a 64-bit generic
pointer is written to a 32-bit slot allocated for a private pointer.
This can be seen in test/CodeGenOpenCL/amdgcn-automatic-variable.cl's
wrong_pointer_alloca.
