This PR improves legalization process of SPIR-V instructions. Namely, it
introduces validation and fixing of bit width of scalar registers as a
part of pre-legalizer. A test case is added that demonstrates ability to
legalize instructions with non 8/16/32/64 bit width both with and
without vendor-specific SPIR-V extension
(SPV_INTEL_arbitrary_precision_integers). In the case of absence of the
extension, a generated SPIR-V code will fallback to 8/16/32/64 bit width
in OpTypeInt, but SPIR-V Backend still is able to legalize operations
with original integer sizes.
This PR implements insertion of OpGenericCastToPtr using builtin
functions (both opencl `to_global|local|private` and `__spirv_`
wrappers), and improves type inference.
This change makes sure the preferred switch condition int type size
remains the same throughout CodeGen optimizations.
The change fixes running several OpenCL applications with -O2 or higher
opt levels, and fixes Basic/stream/stream_max_stmt_exceed.cpp DPC++ E2E
test with -O2.
For i64 this avoids loading a 64-bit value into register, for smaller registers this just avoids an immediate operand.
For i8+i16, limit to one use case as we save fewer bytes and these can be wasted entirely on extra register moves.
Fixes#67709
The wasm backend fetches the tan runtime lib call in
`llvm/include/llvm/IR/RuntimeLibcalls.def` via `StaticLibcallNameMap()`,
but ignores the runtime function because a function sinature mapping is
not specified in RuntimeLibcallSignatureTable(). The fix is to specify
the function signatures for float32-128.
This is a fix for a build break reported on PR
https://github.com/llvm/llvm-project/pull/94559#issuecomment-2159923215.
Those BitVectors get expensive on targets like AMDGPU with thousands of
registers, and RegAliasIterator is also expensive.
We can move all liveness calculations to use RegUnits instead to speed
it up for targets where RegAliasIterator is expensive, like AMDGPU.
On targets where RegAliasIterator is cheap, this alternative can be a little more expensive, but I believe the tradeoff is worth it.
The case-list of the switches generated by this pass were not
"deterministic" (based on allocation patterns).
This is because the CaseList order relied on an unordered_set order.
Using the sorted exit target list for those should solve the problem.
Fixes#94961
Signed-off-by: Nathan Gauër <brioche@google.com>
There were four tests in Driver that actually tested bits of Driver and
bits of CodeGen, and therefore had target restrictions. Rework those
four tests into one Driver test (with no target restrictions) and two
target-specific CodeGen tests.
It seems that we have `B` extension again:
https://github.com/riscv/riscv-b
According to the spec, `B` extension represents the collection of
the `Zba`, `Zbb`, `Zbs` extensions.
Though it hasn't been ratified, I set its version to `1.0`.
Lower global references to ptrauth constants into `@AUTH` `MCExpr`'s.
The logic is common for MachO and ELF - test both.
---------
Co-authored-by: Ahmed Bougacha <ahmed@bougacha.org>
Relanding this PR now that
https://github.com/llvm/llvm-project/pull/90503 has merged. with `FTAN`
landing in
[TargetLoweringBase.cpp:L1021](https://github.com/llvm/llvm-project/blob/main/llvm/lib/CodeGen/TargetLoweringBase.cpp#L1020C23-L1021C63
) There is now a llvm tan intrinsic 32\64\128 Expand case for all llvm
backends.
In LLVM, the `llvm.experimental.constrained.cos` and
`llvm.experimental.constrained.sin` intrinsics are used for performing
cosine and sine calculations with additional constraints on
floating-point operations. This behavior is expected for all
floating-point math intrinsics. This change adds these constraints for
the `tan` intrinsic.
- `Builtins.td` - replace TanF128 with F16F128MathTemplate
- `CGBuiltin.cpp` - map existing tan builtins to `tan` and
`constrained_tan` intrinsic
- `ConstrainedOps.def` map tan and constrained_tan to an ISDOpcode.
resolves #91421
---------
Co-authored-by: Farzon Lotfi <farzon@farzon.com>
LSR will generate chains of related instructions with a known increment
between them. With SVE, in the case of the test case, this can include
increments like 'vscale * 16 + 8'. The idea of this patch is if we have
a '+8' increment already calculated in the chain, we can generate a
(legal) '+ vscale*16' addressing mode from it, allowing us to use the
'[x16, #1, mul vl]' addressing mode instructions.
In order to do this we keep track of the known 'bases' when generating
chains in GenerateIVChain, checking for each if the accumulated
increment expression from the base neatly folds into a legal addressing
mode. If they do not we fall back to the existing LeftOverExpr, whether
it is legal or not.
This is mostly orthogonal to #88124, dealing with the generation of
chains as opposed to rest of LSR. The existing vscale addressing mode
work has greatly helped compared to the last time I looked at this,
allowing us to check that the addressing modes are indeed legal.
In a similar way to how we push vector adds into extends, this pushed
'mul(zext,zext)' into 'zext(mul(zext,zext))' if the extend can be done
in two or more steps.
https://alive2.llvm.org/ce/z/WjU7Kr
Refactor the pass to only support `IntrinsicInst` calls.
`ReplaceWithVecLib` used to support instructions, as AArch64 was using
this pass to replace a vectorized frem instruction to the fmod vector
library call (through TLI).
As this replacement is now done by the codegen (#83859), there is no
need for this pass to support instructions.
Additionally, removed 'frem' tests from:
- AArch64/replace-with-veclib-armpl.ll
- AArch64/replace-with-veclib-sleef-scalable.ll
- AArch64/replace-with-veclib-sleef.ll
Such testing is done at codegen level:
- #83859
This moves the combine of fdiv by constant to fmul out of an
'if (Options.UnsafeFPMath || Flags.hasAllowReciprocal()' block,
so that it triggers if the divide is exact. An extra check for
Recip.isDenormal() is added as multiple places make reference
to it being unsafe or slow on certain platforms.
We currently only constant fold binop(bitcast(c1),bitcast(c2)) if c1 and c2 are both bitcasted and from the same type.
This patch relaxes this assumption to allow the constant build vector to originate from different types (and allow cases where only one operand was bitcasted).
We still ensure we bitcast back to one of the original types if both operand were bitcasted (we assume that if we have a non-bitcasted constant then its legal to keep using that type).
The pr description in #94008 mismatches with the code.
> + When VT is smaller than ShiftVT, it is safe to use trunc.
> + When VT is larger than ShiftVT, it is safe to use zext iff
`is_zero_poison` is true (i.e., `opcode == ISD::CTTZ_ZERO_UNDEF`). See
also the counterexample `src_shl_cttz2 -> tgt_shl_cttz2` in the alive2
proofs.
Closes#94824.
In some case, constant can survive early constant folding optimization
because they are hidden behind several layers of type changes.
E.g., consider the following sequence (extracted from the arm test that
this commit changes):
```
t2: v1f16 = BUILD_VECTOR ConstantFP:f16<APFloat(0)>
t4: v1f16 = insert_vector_elt t2, ConstantFP:f16<APFloat(0)>, Constant:i32<0>
t5: f16 = bitcast t4
t6: f32 = fp_extend t5
```
Because the constant (APFloat(0)) is hidden behind a <1 x ty> type, all
the constant folding that normally happen for scalar nodes when using
`SelectionDAG::getNode` are blocked.
As a result the constant manages to survive as an actual conversion
instruction down to the select phase:
```
t11: f32 = fp16_to_fp Constant:i32<0>
```
With the change in this patch, we try to do constant folding one more
time during dag combine, which in the motivating example result in the
much better sequence:
```
t7: ch = CopyToReg t0, Register:f32 %0, ConstantFP:f32<0.000000e+00>
```
Note: I'm sure we have this problem in a lot of other places. Generally
speaking I believe SDISel is not that good with <1 x ty> compared to
pure scalar. However, I only changed what I could easily test.
When compiling the following code:
```cpp
#include <stdio.h>
#include <stdlib.h>
#include <stddef.h>
#include <stdbool.h>
int main() {
int a;
float f;
scanf("%d", &a);
scanf("%f", &f);
a += (int)f;
return a;
}
```
for `-march=rv32ima_zbb` we get a libcall:
```
call scanf
lw a0, -20(s0)
call __fixsfsi
mv a1, a0
```
When we try to use GlobalISel we get this error:
```
error in backend: unable to legalize instruction: %9:_(s32) = G_FPTOSI %8:_(s32) (in function: main)
```
(Here is a link to a reproducer in Godblot:
https://godbolt.org/z/f67vEEb41 )
The goal of this PR is to do a libcall for the legalization of
`G_FPTOSI` and `G_FPTOUI` instead of doing a fallback to Selection DAG
to do the same libcall later.
This PR continues https://github.com/llvm/llvm-project/pull/94467 and
contains fixes in emission of type intrinsics, constant recording and
corresponding test cases:
* type-deduce-global-dup.ll -- fix of integer constant emission on
32-bit platforms and correct type deduction for globals
* type-deduce-simple-for.ll -- fix of GEP translation (there was an
issue previously that led to incorrect translation/broken logic of
for-range implementation)
This PR also:
* fixes a cast between identical storage classes and updates the test
case to include validation run by spirv-val,
* ensures that Bitcast for pointers satisfies the requirement that the
address spaces must match and adds the corresponding test case,
* improve encode in Tablegen and decode in code of dependencies between
SPIR-V entities and required capability/extensions,
* prevent emission of identical OpTypePointer instructions.
This pattern is an obscured way to express saturating a signed value
into a smaller unsigned value.
If (setltu, X, 256) is true, then the value is already in the desired
range so we can pick X. If it's false, we select (sext (setgt X, 0))
which is 0 for negative values and all ones for positive values. The all
ones value when truncated to the final type will still be all ones like
we want.
Prior to this change, `SelectionDAGBuilder` was producing `SDNode`s of
the form: `f32 = extract_vector_elt <1 x bfloat|half>, i32 0` when
lowering phis of `<1 x bfloat|half>` and running on a target that
promotes this type to `f32` (like some x86 or AMDGPU targets.)
This construct is invalid since this type of node only allows type
extensions for integer types.
It went unotice because the `extract_vector_elt` node is later broken
down in `bitcast` followed by `bf16_to_fp|fp_extend`. However, when the
argument of the phi is a constant we were crashing because the existing
code would try to constant fold this `extract_vector_elt` into a
any_ext.
This patch fixes this by using a proper decomposition for `<1 x
bfloat|half>`:
```
bfloat|half = bitcast <1 x blfoat|half>
float = fp_extend bfloat|half
```
This change should be NFC for the non-constant-folding cases and fix the
SDISel crashes (reported in
https://github.com/llvm/llvm-project/issues/94449) for the folding
cases.
Note: The change on the arm test is a missing fp16 to f32 constant folding
exposed by this patch. I'll push a separate improvement for that.
This PR depends on https://github.com/llvm/llvm-project/pull/90260
We changed the order in which functions are outlined in Machine
Outliner.
The formula for priority is found via a black-box Bayesian optimization
toolbox. Using this formula for sorting consistently reduces the
uncompressed size of large real-world mobile apps. We also ran a few
benchmarks using LLVM test suites, and showed that sorting by priority
consistently reduces the text segment size.
|run (CTMark/) |baseline (1)|priority (2)|diff (1 -> 2)|
|----------------|------------|------------|-------------|
|lencod |349624 |349264 |-0.1030% |
|SPASS |219672 |219480 |-0.0874% |
|kc |271956 |251200 |-7.6321% |
|sqlite3 |223920 |223708 |-0.0947% |
|7zip-benchmark |405364 |402624 |-0.6759% |
|bullet |139820 |139500 |-0.2289% |
|consumer-typeset|295684 |290196 |-1.8560% |
|pairlocalalign |72236 |72092 |-0.1993% |
|tramp3d-v4 |189572 |189292 |-0.1477% |
This is part of an enhanced version of machine outliner -- see
[RFC](https://discourse.llvm.org/t/rfc-enhanced-machine-outliner-part-1-fulllto-part-2-thinlto-nolto-to-come/78732).
