Currently, WebAssembly/WASI target does not provide direct support for
code coverage.
This patch set fixes several issues to unlock the feature. The main
changes are:
1. Port `compiler-rt/lib/profile` to WebAssembly/WASI.
2. Adjust profile metadata sections for Wasm object file format.
- [CodeGen] Emit `__llvm_covmap` and `__llvm_covfun` as custom sections
instead of data segments.
- [lld] Align the interval space of custom sections at link time.
- [llvm-cov] Copy misaligned custom section data if the start address is
not aligned.
- [llvm-cov] Read `__llvm_prf_names` from data segments
3. [clang] Link with profile runtime libraries if requested
See each commit message for more details and rationale.
This is part of the effort to add code coverage support in Wasm target
of Swift toolchain.
Reverse the generic foldVSelectToSignBitSplatMask fold on AVX512 targets where we can use the SETCC result directly in predicated moves/instructions.
Fixes#109272
If each 128-bit vXi8 lane is shifting the same constant value, we can pre-compute the 8 valid shift results and use PSHUFB to act as a LUT with the shift amount.
Fixes#110317
ARM uses complex encoding of immediate values using small number of
bits. As a result, some values cannot be represented as immediate
operands, they need to be synthesized in a register. This change
implements legalization of such constants with loading values from
constant pool.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
[MIR] Serialize virtual register flags
This introduces target-specific vreg flag serialization. Flags are represented as `uint8_t` and the `TargetRegisterInfo` override provides methods `getVRegFlagValue` to deserialize and `getVRegFlagsOfReg` to serialize.
The memcpy inline limit has been 16 for a long time, this patch makes
the memmove inline limit the same, allowing small-constant sized
memmoves to be emitted inline. The 16 is the number of registers stored,
which equates to a limit of 256 bytes.
When `andn` is available, we should avoid switching `s &= ~(z & ~y);` into `s &= ~z | y;`
This patch turns this assembly from:
```
foo:
not rcx
and rsi, rdx
andn rax, rsi, rdi
or rcx, rdx
and rax, rcx
ret
```
into:
```
foo:
and rsi, rdx
andn rcx, rdx, rcx
andn rax, rsi, rdi
andn rax, rcx, rax
ret
```
Fixes#108731
ReplaceWithVeclib.cpp would construct overload name using all the
arguments in the intrinsic, but overloads should only be constructed
from arguments for which isVectorIntrinsicWithOverloadTypeAtArg returns
true, including the return type first (index -1).
Additionally,
- skip when `Intrinsic::not_intrinsic`, otherwise
`isVectorIntrinsicWithOverloadTypeAtArg` asserts for some
IntrinsicCalls.
Unblocks translation for pow and atan2 intrinsics.
Fixes#111093
The purpose of this optimization is to make the VL argument, for
instructions that have a VL argument, as small as possible. This is
implemented by visiting each instruction in reverse order and checking
that if it has a VL argument, whether the VL can be reduced.
By putting this pass before VSETVLI insertion, we see three kinds of
changes to generated code:
1. Eliminate VSETVLI instructions
2. Reduce the VL toggle on VSETVLI instructions that also change vtype
3. Reduce the VL set by a VSETVLI instruction
The list of supported instructions is currently whitelisted for safety.
In the future, we could add more instructions to `isSupportedInstr` to
support even more VL optimization.
We originally wrote this pass because vector GEP instructions do not
take a VL, which leads us to emit code that uses VL=VLMAX to implement
GEP in the RISC-V backend. As a result, some of the vector instructions
will write to lanes, specifically between the intended VL and VLMAX,
that will never be read. As an alternative to this pass, we considered
adding a vector predicated GEP instruction, but this would not fit well
into the intrinsic type system since GEP has a variable number of
arguments, each with arbitrary types. The second approach we considered
was to put this pass after VSETVLI insertion, but we found that it was
more difficult to recognize optimization opportunities, especially
across basic block boundaries -- the data flow analysis was also a bit
more expensive and complex.
While this pass solves the GEP problem, we have expanded it to handle
more cases of VL optimization, and there is opportunity for the analysis
to be improved to enable even more optimization. We have a few follow up
patches to post, but figured this would be a good start.
---------
Co-authored-by: Craig Topper <craig.topper@sifive.com>
Co-authored-by: Kito Cheng <kito.cheng@sifive.com>
Lowering fixed-size BUILD_VECTORS without Neon may introduce stack
spills, leading to more stores/reloads than if the stores were not
merged. In some cases, it can also prevent using paired store
instructions.
In the future, we may want to relax when SVE is available, but
currently, the SVE lowerings for BUILD_VECTOR are limited to a few
specific cases.
This reverts commit 3f9998af4f79e95fe8be615df9d6b898008044b9.
It breaks downstream tests with egregious numerical differences.
Unfortunately no upstream tests are broken, but the fact that
a prior iteration of the commit (pre-optimization) does work
with our downstream tests (coming from the Triton repo) supports
the claim that the final version of the commit is incorrect.
Reverting now so that the original author can evaluate.
Both input and output of ballot are lane-masks:
result is lane-mask with 'S32/S64 LLT and SGPR bank'
input is lane-mask with 'S1 LLT and VCC reg bank'.
Ballot copies bits from input lane-mask for
all active lanes and puts 0 for inactive lanes.
GlobalISel did not set 0 in result for inactive lanes
for non-constant input.
This allows us to emit wide generic and scratch memory accesses when we
do not have alignment information. In cases where accesses happen to be
properly aligned or where generic accesses do not go to scratch memory,
this improves performance of the generated code by a factor of up to 16x
and reduces code size, especially when lowering memcpy and memmove
intrinsics.
Also: Make the use of the FeatureUnalignedScratchAccess feature more
consistent: FeatureUnalignedScratchAccess and EnableFlatScratch are now
orthogonal, whereas, before, code assumed that the latter implies the
former at some places.
Part of SWDEV-455845.
FMINNM/FMAXNM instructions of AArch64 follow IEEE754-2008. We can use
them to canonicalize a floating point number. And
FMINNUM_IEEE/FMAXNUM_IEEE is used by something like expanding
FMINIMUMNUM/FMAXIMUMNUM, so let's define them.
---------
Co-authored-by: Your Name <you@example.com>
- add degrees builtin
- link degrees api in hlsl_intrinsics.h
- add degrees intrinsic to IntrinsicsDirectX.td
- add degrees intrinsic to IntrinsicsSPIRV.td
- add lowering from clang builtin to dx/spv intrinsics in CGBuiltin.cpp
- add semantic checks to SemaHLSL.cpp
- add expansion of directx intrinsic to llvm fmul for DirectX in
DXILIntrinsicExpansion.cpp
- add mapping to spir-v intrinsic in SPIRVInstructionSelector.cpp
- add test coverage:
- degrees.hlsl -> check hlsl lowering to dx/spv degrees intrinsics
- degrees-errors.hlsl/half-float-only-errors -> check semantic warnings
- hlsl-intrinsics/degrees.ll -> check lowering of spir-v degrees
intrinsic to SPIR-V backend
- DirectX/degrees.ll -> check expansion and scalarization of directx
degrees intrinsic to fmul
Resolves#99104
Before CUDA 12.3 `ptxas` did not recognize that the trap instruction
terminates a basic block. Instead, it would assume that control flow
continued to the next instruction. The next instruction could be in the
block that's lexically below it. This would lead to phantom CFG edges
being created within ptxas.
[NVPTX: Lower unreachable to exit to allow ptxas to accurately
reconstruct the
CFG.](1ee4d880e8)
added the LowerUnreachable pass to NVPTX to work around this. Several
other WAR patches followed.
This bug in `ptxas` was fixed in CUDA 12.3 and is thus impossible to
encounter when targeting PTX ISA v8.3+
This commit reverts the WARs for the `ptxas` bug when targeting PTX ISA
v8.3+
CC @maleadt
- add additional lowering for directx backend in CGBuiltin.cpp
- add directx intrinsic to IntrinsicsDirectX.td
- add semantic check of arguments in SemaHLSL.cpp
- add mapping to DXIL op in DXIL.td
- add testing of semantics in WaveGetLaneIndex-errors.hlsl
- add testing of dxil lowering in WaveGetLaneIndex.ll
Resolves#70105
In [[NVPTX] Improve lowering of
v4i8](cbafb6f2f5)
@Artem-B add the ability to lower ISD::BUILD_VECTOR with bfi PTX
instructions. @Artem-B did this because:
([source](https://github.com/llvm/llvm-project/pull/67866#discussion_r1343066911))
> Under the hood byte extraction/insertion ends up as BFI/BFE
instructions, so we may as well do that in PTX, too.
https://godbolt.org/z/Tb3zWbj9b
However, the example that @Artem-B linked was targeting sm_52. On modern
architectures, ptxas uses prmt.b32.
[Example](https://godbolt.org/z/Ye4W1n84o).
Thus, remove uses of NVPTXISD::BFI in favor of NVPTXISD::PRMT.
Rather than invariantly running `F->verify()` when asserts are enabled,
run machine IR verification in LIT tests only.
Swap `CHECK-PERF` and `CHECK-SIZE` in `code_placement_ext_tsp_large.ll`.
Remove `={0,1,true,false}` from flags in tests.
