In preparation for refactoring the instruction size checks being made by
PAuth-related code, switch all instruction emission in AArch64AsmPrinter
to using EmitToStreamer function.
Introduce a single-operand overload of `EmitToStreamer(MCInst)`, as the
only MCStreamer passed as the first argument is actually `*OutStreamer`.
To decrease the number of code lines changed due to clang-format, do not
touch the existing calls to two-argument EmitToStreamer function so far.
There are llvm intrinsics which we do not expect to actually represent
code after lowering or which are not implemented yet but can be found in
a customer's LLVM IR input. We do not want translation to crash when
these llvm intrinsics are found, and this PR fixes the issue with
translation crash for some known cases, aligned with Khronos Translator.
This PR reworks implementation of OpSpecConstantOp with ptr-cast
operation (PtrCastToGeneric, GenericCastToPtr). Previous implementation
didn't take into account a lot of use cases, including multiple
inclusion of pointers, reference to a pointer from OpName, etc. A
reproducer is attached as a new test case.
This PR also fixes wrong type inference for IR patterns which generate
new virtual registers without SPIRV type. Previous implementation
assumed always that result has the same address space as a source that
is not the fact, and, for example, led to impossibility to emit a
ptr-cast operation in the reproducer, because wrong type inference
rendered source and destination with the same address space, eliminating
translation of G_ADDRSPACE_CAST.
This PR improves type inference and fixes inconsistency between
previously deduced element type of a pointer and function's return type.
It fixes https://github.com/llvm/llvm-project/issues/109401 by ensuring
that OpPhi is consistent with respect to operand types.
For register constraints that require specific register ranges, the
width of the range should match the type of the associated
parameter/return value. With this PR, we error out when that is not the
case. Previously, these cases would hit assertions or llvm_unreachables.
The handling of register constraints that require only a single register
remains more lenient to allow narrower non-vector types for the
associated IR values. For example, constraining an i16 or i8 value to a
32-bit register is still allowed.
Fixes#101190.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
After #110144, we can finish off #110129 and fold f16 vfmv.f.s into a
flh.
vfmv.f.s is only available for f16 with zvfh, which in turn requires
zfhmin so we can use flh.
bf16 has no vfmv.f.s so the extract_vector_elt is lowered as an integer
in #110144, and gets the existing integer vmv.x.s fold.
```
define internal void @foo() {
ret void
}
define i64 @main() {
ret i64 add (i64 ptrtoint (ptr @foo to i64), i64 -2147483626)
}
```
When `foo` is a local symbol, `foo` and `main` are in the same section,
and `offset` is near INT32_MIN, referencing `foo+offset` in `main` with
RIP-relative addressing needs `leaq .text+offset1(%rip), %rax` where
`offset1 < offset`, and `offset1` might underflow.
(https://discourse.llvm.org/t/arithmetic-referencing-dso-local-function-causes-compilation-error-on-linux-x64/80033):
Don't use RIP-relative addressing if the negative offset is near
INT32_MIN. Arbitrarily reuse the magic number in isOffsetSuitableForCodeModel to
guard against the edge case when `address(current_instruction)-foo < 4GiB-16MiB`.
If the difference is larger than 4GiB-16MiB, `ret i64 add (i64 ptrtoint
(ptr @foo to i64), i64 -2**32+256MiB)` would still cause the assembly
issue, such cases are unrealistic.
Pull Request: https://github.com/llvm/llvm-project/pull/98438
Now that we support 'any' type in the value stack in the checker, this
uses it in more places.
When an instruction pops multiple values, rather than popping in one by
one and generating multiple error messages, it adds them to a vector and
pops them at once. When the type to be popped is not clear, it pops
'any', at least makes sure there are correct number of values on the
stack. So for example, in case of `table.fill`, which expects `[i32 t
i32]` (where t is the type of the elements in the table), it pops them
at once, generating an error message like
```console
error: type mismatch, expected [i32, externref, i32] but got [...]
```
In case the table is invalid so we don't know the type, it tries to pop
an 'any' instead, popping whatever value there is:
```console
error: type mismatch, expected [i32, any, i32] but got [...]
```
Checks done on other instructions based on the register info are already
popping and pushing types in vectors, after #110094:
a52251675f/llvm/lib/Target/WebAssembly/AsmParser/WebAssemblyAsmTypeCheck.cpp (L515-L536)
This also pushes 'any' in case the type to push is unclear. For example,
`local/global.set` pushes a value of the type specified in the local or
global, but in case that local or global is invalid, we push 'any'
instead, which will match with whatever type.
The objective of all these is not to make one instruction's error
propragate continuously into subsequent instructions. This also matches
Wabt's behavior.
This also renames `checkAndPopTypes` to just `popTypes`, to be
consistent with a single-element version `popType`. `popType(s)` also
does type checks.
This replaces some of the most frequent offenders of using a DenseMap that
cause a malloc, where the typical element-count is small enough to fit in
an initial stack allocation.
Most of these are fairly obvious, one to highlight is the collectOffset
method of GEP instructions: if there's a GEP, of course it's going to have
at least one offset, but every time we've called collectOffset we end up
calling malloc as well for the DenseMap in the MapVector.
Alongside something like:
vpternlogq zmm0, zmm2, zmm1, 64
We will now have a comment on the right like:
# zmm0 = zmm0 & zmm2 & ~zmm1
This makes it easy to tell at a glance what sort of truth table the
instruction will provide.
If we know vlen is a multiple of 16, we don't need any alignment
padding.
I wrote the code so that it would generate the minimum amount of padding
if the stack align was 32 or larger or if RVVBitsPerBlock was smaller
than half the stack alignment.
This fixes issue #109250
The issue happens during the `MachineBlockPlacement` pass. The block,
whose address was previously not taken, is deemed redundant by the pass
and subsequently replaced using
`MachineBasicBlock::ReplaceUsesOfBlockWith` in `BranchFolding`.
ReplaceUsesOfBlockWith only replaces uses in the terminator. However,
`expandPostRAPseudo` introduces new block uses when expanding catchrets.
These uses do not get replaced, which results in undefined label errors
later on.
Marking the block addresss as taken prevents the replacement of the
block, without also replacing non-terminator uses.
This patch fixes:
llvm/lib/Target/SystemZ/SystemZISelLowering.cpp:9858:18: error:
using the result of an assignment as a condition without parentheses
[-Werror,-Wparentheses]
Allocating wwm-registers and per-thread VGPR operands
together imposes many challenges in the way the
registers are reused during allocation. There are
times when regalloc reuses the registers of regular
VGPRs operations for wwm-operations in a small range
leading to unwantedly clobbering their inactive lanes
causing correctness issues that are hard to trace.
This patch splits the VGPR allocation pipeline further
to allocate wwm-registers first and the regular VGPR
operands in a separate pipeline. The splitting would
ensure that the physical registers used for wwm
allocations won't take part in the next allocation
pipeline to avoid any such clobbering.
PRs #69924 and #72140 modified SIInstrInfo::isBasicBlockPrologue to skip
over EXEC modifications and spills when allocating VGPRs. But treating
VGPR spills as part of the prologue can confuse the register allocator
as in #109294, so restrict it to SGPR spills, which were inserted during
SGPR allocation which is done in an earlier pass.
Fixes: #109294
Fixes: SWDEV-485841
This patch was reverted because of a failing C test.
It now has being solved and can be merged into main again
This patch adds these intrinsics:
// Variants are also available for: _s8
svuint8x4_t svluti4_zt_u8_x4(uint64_t zt0, svuint8x2_t zn)
__arm_streaming __arm_in("zt0");
according to PR#324[1]
[1]ARM-software/acle#324
/llvm-project/llvm/lib/Target/AMDGPU/AMDGPUMCResourceInfo.cpp:26:16:
error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture]
auto GOCS = [this, FuncName, &OutContext](StringRef Suffix) {
^~~~~
/llvm-project/llvm/lib/Target/AMDGPU/AMDGPUMCResourceInfo.cpp:64:27:
error: lambda capture 'this' is not used [-Werror,-Wunused-lambda-capture]
auto assignMaxRegSym = [this, &OutContext](MCSymbol *Sym, int32_t RegCount) {
^~~~~
2 errors generated.
Converts AMDGPUResourceUsageAnalysis pass from Module to MachineFunction
pass. Moves function resource info propagation to to MC layer (through
helpers in AMDGPUMCResourceInfo) by generating MCExprs for every
function resource which the emitters have been prepped for.
Fixes https://github.com/llvm/llvm-project/issues/64863
Change the names of the TableGen features to match the names used by
AMDGPUSubtarget. "Addressable" refers to the amount that can be accessed
by a single workgroup. Add some explanatory comments. NFC.
…me lowering
SME instructions can only be used in streaming mode. PTRUE for
predicated counter and the ld/st pair can be used when:
sve2.1 is available or
sme2 available in function in streaming mode.
Previously the frame lowering only checking if sme2 available when
building the machine instruction.
This fix checks if sme2 is available and is subtarget in streaming mode
Remove the following intrinsics which can be trivially replaced with an
`addrspacecast`
* llvm.nvvm.ptr.gen.to.global
* llvm.nvvm.ptr.gen.to.shared
* llvm.nvvm.ptr.gen.to.constant
* llvm.nvvm.ptr.gen.to.local
* llvm.nvvm.ptr.global.to.gen
* llvm.nvvm.ptr.shared.to.gen
* llvm.nvvm.ptr.constant.to.gen
* llvm.nvvm.ptr.local.to.gen
Also, cleanup the NVPTX lowering of `addrspacecast` making it more
concise.
This was reverted to avoid conflicts while reverting #107655. Re-landing
unchanged.
This adds the TTI predicate for conveying the availability of fast
`popcnt`, which subsequently allows passes like `LoopIdiomRecognize` to
collapse known popcount patterns. Since SPIR-V natively exposes
`OpBitcount`, it seems preferable to compress the resulting code, and
retain the information, even if a concrete target might have to expand
back into a loop structure.
Previously we used an empty MachinePointerInfo. I checked a few other
targets like X86, ARM, and AArch64 and they all appear to use correct
MachinePointerInfo.
When we're lowering to a split sequence, we only need one
materialization of the zero constant. Our codegen looks something like
this:
vmv.v.i v24, 0
vmerge.vim v8, v24, -1, v0
vmv1r.v v0, v16
vmerge.vim v16, v24, -1, v0
Note: Doing this specific case since it was pointed out in
https://github.com/llvm/llvm-project/pull/110164#discussion_r1778268391,
but it's worth noting that we have the same basic problem (over costing
split operations with split invariant terms) at multiple places through
this file.
Calling into BasicTTI is not always safe. In particular, BasicTTI does
not have a full legalization implementation (vector widening is
missing), and falls back on scalarization. The problem is that
scalarization for <N x i1> vectors is cost in terms of the cast API and
we can end up in an infinite recursive cycle.
The "right" fix for this would be teach BasicTTI how to model the full
legalization state machine, but several attempts at doing so have
resulted in dead ends or undesirable cost changes for targets I don't
understand.
This patch instead papers over the issue by avoiding the call to the
base class when dealing with an i1 source or dest. This doesn't
necessarily produce correct costs, but it should at least return
something semi-sensible and not crash.
Fixes https://github.com/llvm/llvm-project/issues/108708
