Add new elementwise popcount builtin to support HLSL function
'countbits'.
elementwise popcount only accepts integer types.
Add hlsl intrinsic 'countbits'
Closes#99094
This reverts commit 4a63f4d301c0e044073e1b1f8f110015ec1778a1.
It was reverted because of a buildbot breakage, but the fix-forward has
landed (https://github.com/llvm/llvm-project/pull/109023).
HLSL inlines all its functions by default. This uses the alwaysinline
attribute to make the alwaysinliner pass inline any function not
explicitly marked noinline by the user or autogeneration. The
alwayslinline marking takes place in `SetLLVMFunctionAttributesForDefinitions`
where all other inlining interactions are determined.
The outermost entry function is marked noinline because there's no
reason to inline it. Any user calls to an entry function will instead call
the internal mangled version of the entry function.
Adds tests for function and constructor inlining and augments some
existing tests to verify correct inlining of implicitly created
functions as well.
Incidentally restore RUN line that I believe was mistakenly removed as
part of #88918Fixes#89282
There's currently no code path that can reach this crash, but:
```
Instruction *Inst = cast<llvm::Instruction>(Call.getScalarVal());
```
fails if the call returns `void`. This could happen if a builtin for
something like `void sincos(double, double*, double*)` is added to
clang.
Instead, use the `llvm::CallBase` returned from `EmitCall()` to set the
TBAA metadata, which should exist no matter the return type.
This implements our original design now that LLVM is comfortable with
structs and arrays of scalable vector types. All SVE ACLE intrinsics
already use struct types so the effect of this change is purely the
types used for alloca and function parameters.
There should be no C/C++ user visible change with this patch.
This patch enable the function multiversion(FMV) and `target_clones`
attribute for RISC-V target.
The proposal of `target_clones` syntax can be found at the
https://github.com/riscv-non-isa/riscv-c-api-doc/pull/48 (which has
landed), as modified by the proposed
https://github.com/riscv-non-isa/riscv-c-api-doc/pull/85 (which adds the
priority syntax).
It supports the `target_clones` function attribute and function
multiversioning feature for RISC-V target. It will generate the ifunc
resolver function for the function that declared with target_clones
attribute.
The resolver function will check the version support by runtime object
`__riscv_feature_bits`.
For example:
```
__attribute__((target_clones("default", "arch=+ver1", "arch=+ver2"))) int bar() {
return 1;
}
```
the corresponding resolver will be like:
```
bar.resolver() {
__init_riscv_feature_bits();
// Check arch=+ver1
if ((__riscv_feature_bits.features[0] & BITMASK_OF_VERSION1) == BITMASK_OF_VERSION1) {
return bar.arch=+ver1;
} else {
// Check arch=+ver2
if ((__riscv_feature_bits.features[0] & BITMASK_OF_VERSION2) == BITMASK_OF_VERSION2) {
return bar.arch=+ver2;
} else {
// Default
return bar.default;
}
}
}
```
Remove '_' in fatbin and gpubin symbol suffixes when missing TU hash ID.
Internalize gpubin symbol so that it is not unresolved at link-time when
symbol is not relocatable.
HLSL allows implicit conversions to truncate vectors to scalar
pr-values. These conversions are scored as vector truncations and should
warn appropriately.
This change allows forming a truncation cast to a pr-value, but not an
l-value. Truncating a vector to a scalar is performed by loading the
first element of the vector and disregarding the remaining elements.
Fixes#102964
Some switch statements require all SVE builtin types to be manually
specified. This patch refactors the SVE_*_TYPE macros so that such code
can be generated during preprocessing.
I've tried to establish a minimal interface that covers all types where
no special information is required and then created a set of macros that
are dedicated to specific datatypes (i.e. int, float).
This patch is groundwork to simplify the changing of SVE tuple types to
become struct based as well as work to support the FP8 ACLE.
Adds target codegen info class for DirectX. For now it always translates
`__hlsl_resource_t` handle to `target("dx.TypedBuffer", i32, 1, 0, 1)`
(`RWBuffer<int>`). More work is needed to determine the actual target
exp type and parameters based on the resource handle attributes.
Part 1/2 of #95952
partially fixes#70078
### Changes
- Implemented `sign` clang builtin
- Linked `sign` clang builtin with `hlsl_intrinsics.h`
- Added sema checks for `sign` to `CheckHLSLBuiltinFunctionCall` in
`SemaChecking.cpp`
- Add codegen for `sign` to `EmitHLSLBuiltinExpr` in `CGBuiltin.cpp`
- Add codegen tests to `clang/test/CodeGenHLSL/builtins/sign.hlsl`
- Add sema tests to `clang/test/SemaHLSL/BuiltIns/sign-errors.hlsl`
### Related PRs
- https://github.com/llvm/llvm-project/pull/101987
- https://github.com/llvm/llvm-project/pull/101988
### Discussion
- Should there be a `usign` intrinsic that handles the unsigned cases?
Update codegen for func param with transparent_union attr to be that of
the first union member.
This is a followup to #101738 to fix non-ppc codegen and closes#76773.
This patch implements the intrinsics of the form
floatNxM_t vamin[q]_fN(floatNxM_t vn, floatNxM_t vm);
floatNxM_t vamax[q]_fN(floatNxM_t vn, floatNxM_t vm);
as defined in https://github.com/ARM-software/acle/pull/324
---------
Co-authored-by: Hassnaa Hamdi <hassnaa.hamdi@arm.com>
This patch is the frontend implementation of the coroutine elide
improvement project detailed in this discourse post:
https://discourse.llvm.org/t/language-extension-for-better-more-deterministic-halo-for-c-coroutines/80044
This patch proposes a C++ struct/class attribute
`[[clang::coro_await_elidable]]`. This notion of await elidable task
gives developers and library authors a certainty that coroutine heap
elision happens in a predictable way.
Originally, after we lower a coroutine to LLVM IR, CoroElide is
responsible for analysis of whether an elision can happen. Take this as
an example:
```
Task foo();
Task bar() {
co_await foo();
}
```
For CoroElide to happen, the ramp function of `foo` must be inlined into
`bar`. This inlining happens after `foo` has been split but `bar` is
usually still a presplit coroutine. If `foo` is indeed a coroutine, the
inlined `coro.id` intrinsics of `foo` is visible within `bar`. CoroElide
then runs an analysis to figure out whether the SSA value of
`coro.begin()` of `foo` gets destroyed before `bar` terminates.
`Task` types are rarely simple enough for the destroy logic of the task
to reference the SSA value from `coro.begin()` directly. Hence, the pass
is very ineffective for even the most trivial C++ Task types. Improving
CoroElide by implementing more powerful analyses is possible, however it
doesn't give us the predictability when we expect elision to happen.
The approach we want to take with this language extension generally
originates from the philosophy that library implementations of `Task`
types has the control over the structured concurrency guarantees we
demand for elision to happen. That is, the lifetime for the callee's
frame is shorter to that of the caller.
The ``[[clang::coro_await_elidable]]`` is a class attribute which can be
applied to a coroutine return type.
When a coroutine function that returns such a type calls another
coroutine function, the compiler performs heap allocation elision when
the following conditions are all met:
- callee coroutine function returns a type that is annotated with
``[[clang::coro_await_elidable]]``.
- In caller coroutine, the return value of the callee is a prvalue that
is immediately `co_await`ed.
From the C++ perspective, it makes sense because we can ensure the
lifetime of elided callee cannot exceed that of the caller if we can
guarantee that the caller coroutine is never destroyed earlier than the
callee coroutine. This is not generally true for any C++ programs.
However, the library that implements `Task` types and executors may
provide this guarantee to the compiler, providing the user with
certainty that HALO will work on their programs.
After this patch, when compiling coroutines that return a type with such
attribute, the frontend checks that the type of the operand of
`co_await` expressions (not `operator co_await`). If it's also
attributed with `[[clang::coro_await_elidable]]`, the FE emits metadata
on the call or invoke instruction as a hint for a later middle end pass
to elide the elision.
The original patch version is
https://github.com/llvm/llvm-project/pull/94693 and as suggested, the
patch is split into frontend and middle end solutions into stacked PRs.
The middle end CoroSplit patch can be found at
https://github.com/llvm/llvm-project/pull/99283
The middle end transformation that performs the elide can be found at
https://github.com/llvm/llvm-project/pull/99285
Converts existing resource attributes `[[hlsl::resource_class(..)]]` and
`[[is_rov]]` from declaration attributes to type attributes.
During type attribute processing all HLSL resource type attributes are
validated and collected by `SemaHLSL`
(`SemaHLSL::handleResourceTypeAttr`). At the end of the declaration they
are be combined into a single `HLSLAttributedResourceType` instance
(`SemaHLSL::ProcessResourceTypeAttributes`) that wraps the original type
and stores all of the necessary information about the resource.
`SemaHLSL` will also need to short-term-store the `TypeLoc` information
for the newly created type that will be grabbed by `TypeSpecLocFiller`
soon after it is created.
Updates all places that expected resource attributes on declarations
like resource binding diagnostic, builtin types in
HLSLExternalSemaSource, or codegen.
Also includes implementation of
`TreeTransform<Derived>::TransformHLSLAttributedResourceType` that
enables the use of attributed resource types inside templates.
Fixes#104861
Part 2/2
Clang should only emit an available_externally vtable when there are no
unused virtual inline functions. Currently, if such such a function is
declared without inline inside the class, but is defined inline outside
the class, Clang might emit the vtable as available_externally. This
happens because Clang only considers the declarations of vtable entries,
but not the definitions. This patch addresses this by inspecting the
definitions in addition to the declarations.
Add nuw attribute to inbounds GEPs where the expression used to form the
GEP is an addition of unsigned indices.
Relands #105496, which was reverted because it exposed a miscompilation
arising from #98608. This is now fixed by #106512.
[P2641R4](https://wg21.link/P2641R4)
This new builtin function is declared `consteval`. Support for
`-fexperimental-new-constant-interpreter` will be added in a later
patch.
---------
Co-authored-by: cor3ntin <corentinjabot@gmail.com>
This potentially affects platforms that support comdats other than ELF,
COFF, or wasm, but that is the intention of the FIXME, and if they don't
want this behavior, they probably shouldn't advertise comdat support.
This commits add the WaveIsFirstLane() hlsl intrinsinc. This intrinsic
uses the convergence intrinsincs for the SPIR-V backend. On the DXIL
side, I'm not sure what the strategy is for convergence, so I
implemented that like in DXC: a normal builtin function.
Signed-off-by: Nathan Gauër <brioche@google.com>
This patch reduces the memory usage for import lists by employing
memory-efficient data structures.
With this patch, an import list for a given destination module is
basically DenseSet<uint32_t> with each element indexing into the
deduplication table containing tuples of:
{SourceModule, GUID, Definition/Declaration}
In one of our large applications, the peak memory usage goes down by
9.2% from 6.120GB to 5.555GB during the LTO indexing step.
This patch addresses several sources of space inefficiency associated
with std::unordered_map:
- std::unordered_map<GUID, ImportKind> takes up 16 bytes because of
padding even though ImportKind only carries one bit of information.
- std::unordered_map uses pointers to elements, both in the hash table
proper and for collision chains.
- We allocate an instance of std::unordered_map for each
{Destination Module, Source Module} pair for which we have at least
one import. Most import lists have less than 10 imports, so the
metadata like the size of std::unordered_map and the pointer to the
hash table costs a lot relative to the actual contents.
HLSL output parameters are denoted with the `inout` and `out` keywords
in the function declaration. When an argument to an output parameter is
constructed a temporary value is constructed for the argument.
For `inout` pamameters the argument is initialized via copy-initialization
from the argument lvalue expression to the parameter type. For `out`
parameters the argument is not initialized before the call.
In both cases on return of the function the temporary value is written
back to the argument lvalue expression through an implicit assignment
binary operator with casting as required.
This change introduces a new HLSLOutArgExpr ast node which represents
the output argument behavior. The OutArgExpr has three defined children:
- An OpaqueValueExpr of the argument lvalue expression.
- An OpaqueValueExpr of the copy-initialized parameter.
- A BinaryOpExpr assigning the first with the value of the second.
Fixes#87526
---------
Co-authored-by: Damyan Pepper <damyanp@microsoft.com>
Co-authored-by: John McCall <rjmccall@gmail.com>
Summary:
This patch proposes new llvm types for RISCV vector tuples represented
as `TargetExtType` which contains both `LMUL` and `NF`(num_fields)
information and keep it all the way down to `selectionDAG` to match the
corresponding `MVT`(support in the following patch).
Detail:
Currently we have built-in C types for RISCV vector tuple type, e.g.
`vint32m1x2_t`, however it's is represented as structure of scalable
vector types, i.e. `{<vscale x 2 x i32>, <vscale x 2 x i32>}`. It loses
the information for num_fields(NF) as struct is flattened during
`selectionDAG`, thus it makes it not possible to handle inline assembly
of vector tuple type, it also makes the calling convention of vector
tuple types handing not strait forward and hard to realize the
allocation code, i.e. `RVVArgDispatcher`.
The llvm IR for the example above is then represented as
`target("riscv.vector.tuple", <vscale x 8 x i8>, 2)` in which the first
type parameter is the equivalent size scalable vecotr of i8 element
type, the following integer parameter is the `NF` of the tuple.
The new RISCV specific vector insert/extract intrinsics are also added
as `llvm.riscv.vector.insert` and `llvm.riscv.vector.extract` to handle
tuple type subvector insertion/extraction since the generic ones only
operates on `VectorType` but not `TargetExtType`.
There are total of 32 llvm types added for each `VREGS * NF <= 8`, where
`VREGS` is the vector registers needed for each `LMUL` and `NF` is
num_fields.
The name of types are:
```
target("riscv.vector.tuple", <vscale x 1 x i8>, 2) // LMUL = mf8, NF = 2
target("riscv.vector.tuple", <vscale x 1 x i8>, 3) // LMUL = mf8, NF = 3
target("riscv.vector.tuple", <vscale x 1 x i8>, 4) // LMUL = mf8, NF = 4
target("riscv.vector.tuple", <vscale x 1 x i8>, 5) // LMUL = mf8, NF = 5
target("riscv.vector.tuple", <vscale x 1 x i8>, 6) // LMUL = mf8, NF = 6
target("riscv.vector.tuple", <vscale x 1 x i8>, 7) // LMUL = mf8, NF = 7
target("riscv.vector.tuple", <vscale x 1 x i8>, 8) // LMUL = mf8, NF = 8
target("riscv.vector.tuple", <vscale x 2 x i8>, 2) // LMUL = mf4, NF = 2
target("riscv.vector.tuple", <vscale x 2 x i8>, 3) // LMUL = mf4, NF = 3
target("riscv.vector.tuple", <vscale x 2 x i8>, 4) // LMUL = mf4, NF = 4
target("riscv.vector.tuple", <vscale x 2 x i8>, 5) // LMUL = mf4, NF = 5
target("riscv.vector.tuple", <vscale x 2 x i8>, 6) // LMUL = mf4, NF = 6
target("riscv.vector.tuple", <vscale x 2 x i8>, 7) // LMUL = mf4, NF = 7
target("riscv.vector.tuple", <vscale x 2 x i8>, 8) // LMUL = mf4, NF = 8
target("riscv.vector.tuple", <vscale x 4 x i8>, 2) // LMUL = mf2, NF = 2
target("riscv.vector.tuple", <vscale x 4 x i8>, 3) // LMUL = mf2, NF = 3
target("riscv.vector.tuple", <vscale x 4 x i8>, 4) // LMUL = mf2, NF = 4
target("riscv.vector.tuple", <vscale x 4 x i8>, 5) // LMUL = mf2, NF = 5
target("riscv.vector.tuple", <vscale x 4 x i8>, 6) // LMUL = mf2, NF = 6
target("riscv.vector.tuple", <vscale x 4 x i8>, 7) // LMUL = mf2, NF = 7
target("riscv.vector.tuple", <vscale x 4 x i8>, 8) // LMUL = mf2, NF = 8
target("riscv.vector.tuple", <vscale x 8 x i8>, 2) // LMUL = m1, NF = 2
target("riscv.vector.tuple", <vscale x 8 x i8>, 3) // LMUL = m1, NF = 3
target("riscv.vector.tuple", <vscale x 8 x i8>, 4) // LMUL = m1, NF = 4
target("riscv.vector.tuple", <vscale x 8 x i8>, 5) // LMUL = m1, NF = 5
target("riscv.vector.tuple", <vscale x 8 x i8>, 6) // LMUL = m1, NF = 6
target("riscv.vector.tuple", <vscale x 8 x i8>, 7) // LMUL = m1, NF = 7
target("riscv.vector.tuple", <vscale x 8 x i8>, 8) // LMUL = m1, NF = 8
target("riscv.vector.tuple", <vscale x 16 x i8>, 2) // LMUL = m2, NF = 2
target("riscv.vector.tuple", <vscale x 16 x i8>, 3) // LMUL = m2, NF = 3
target("riscv.vector.tuple", <vscale x 16 x i8>, 4) // LMUL = m2, NF = 4
target("riscv.vector.tuple", <vscale x 32 x i8>, 2) // LMUL = m4, NF = 2
```
RFC:
https://discourse.llvm.org/t/rfc-support-riscv-vector-tuple-type-in-llvm/80005
