SIInstrInfo::resultDependsOnExec assumes that operand 0 of a comparison
is always the destination of the instruction. This is not true for
instructions in VOPC form where it is "src0". This led to a crash in
machine-cse.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
According the
[spec](https://github.com/XUANTIE-RV/thead-extension-spec/releases/tag/2.3.0),
the operation of `th.ext rd, rs1, msb, lsb` is
reg[rd] := sign_extend(reg[rs1][msb:lsb])
The spec doesn't specify if lsb is greater than msb.
I don't think lsb can be greater than msb. So that If the shift-right
amount is greater than msb, we can set lsb equal to msb to extract the
bit rs1[msb] and sign-extend it.
This change introduces a new pattern for lowering kernel byval
parameters in `NVPTXLowerArgs`. Each byval argument is wrapped in a call
to a new intrinsic, `@llvm.nvvm.internal.addrspace.wrap`. This intrinsic
explicitly equates to no instructions and is removed during operation
legalization in SDAG. However, it allows us to change the addrspace of
the arguments to 101 to reflect the fact that they will occupy this
space when lowered by `LowerFormalArgs` in `NVPTXISelLowering`.
Optionally, if a generic pointer to a param is needed, a standard
`addrspacecast` is used. This approach offers several advantages:
- Exposes addrspace optimizations: By using a standard `addrspacecast`
back to generic space we allow InferAS to optimize this instruction,
potentially sinking it through control flow or in other ways unsupported
by `NVPTXLowerArgs`. This is demonstrated in several existing tests.
- Clearer, more consistent semantics: Previously an `addrspacecast` from
generic to param space was implicitly a no-op. This is problematic
because it's not reciprocal with the inverse cast, violating LLVM
semantics. Further it is very confusing given the existence of
`cvta.to.param`. After this change the cast equates to this instruction.
- Allow for the removal of all nvvm.ptr.* intrinsics: In a follow-up
change the nvvm.ptr.gen.to.param and nvvm.ptr.param.to.gen intrinsics
may be removed.
This patch adds the following Dense Math Facility bfloat16
floating-point calculation instructions: dmxvbf16gerx2,
dmxvbf16gerx2pp,dmxvbf16gerx2pn, dmxvbf16gerx2np, dmxvbf16gerx2nn,
pmdmxvbf16gerx2, pmdmxvbf16gerx2pp, pmdmxvbf16gerx2pn,
pmdmxvbf16gerx2np, pmdmxvbf16gerx2nn, along with their corresponding
intrinsics and tests.
Test how these intrinsics are handled by the NVPTX backend. Currently, these intrinsics are lowered to sequential reductions by the ExpandReductions pass.
fixes#135654
In #128613 we added safe guards to prevent the lowering of just any
intrinsic in the backend. We used `DiagnosticInfoUnsupported` to do
this.
What we found was when using `opt` the diagnostic print function was
called but when using clang the diagnostic message was used.
Printing message in the clang version means we miss valuable debugging
information like function name and function type when LLVMContext was
only needed to call `getBestLocationFromDebugLoc`.
There are a few potential fixes
1. Write a custom DiagnosticInfoUnsupported so we can change the Message
just for DirectX. Too heavy handed so rejected.
2. Add the function name to the Message in DirectX code. Very simple one
line change. Downside is when using opt you see the function name twice.
But makes the clang-dxc bugs more actionable.
3. change CodeGenAction.cpp to always use the print function and not the
message directly. Downside is a bunch of innacurate information shows up
in the message if you don't specify `-debug-info-kind=standalone`.
4. add some book keeping to know which function called the intrinsic
keep a map of these so we can pass the calling function to
`DiagnosticInfoUnsupported` instead of the intrinsic. This would only be
useful if we had debug info so we could distinguish different uses of
the intrinsic by line\col number. We would also need to change from
iterating on every function to doing something like a LazyCallGraph
which is a nonstarter.
5. pick a different means of doing a Diagnostic error, because other
uses of `DiagnosticInfoUnsupported` error when we are in the body of a
function not when we see one being used like in the intrinsic case.
This PR went with a combo of option 2 & 5. Its low code change that also
only impacts the DirectX backend.
The PR will fix the issue
https://github.com/llvm/llvm-project/issues/122728
This patch addresses the signed/zero extension of poison by using a
poison value of the extended type instead of a constant zero of the
extended type.
This patch removes UB from some tests for MachinePipeliner. This patch
fixes following cases.
- Branching on an `undef` value.
- Using `undef`/`null` as a pointer operand of a load/store.
There are other tests of pipeliner that contain the same UB, but for
now, this patch fixes particularly unstable cases when I developed
pipeliner.
addr matching was the only gatekeeper for starting selecting G_LOAD
and G_STORE using SDAG patterns.
* Introduce a complex renderer gi_addr for addr. In this patch only
the existing functionality has been implemented. The renderer's name is
the same as in SDAG: selectAddr. Apparently the type of
GIComplexOperandMatcher doesn't matter as RISCV also uses s32 for
both 64 and 32 bit pointers.
* X86SelectAddress now is used for both: pattern matching and manual
selection. As a result it accumulates all the code that previously was
distributed among different selection functions.
* Replace getLoadStoreOp with getPtrLoadStoreOp in Load/Store
selector as GlobalISel matcher or emitter can't map the pointer type
into i32/i64 types used in SDAG patterns for pointers. So the load and
store selection of pointers is still manual. getLoadStoreOp is still
present because it is used in G_FCONSTANT lowering that requires extra
efforts to select it using SDAG patterns.
* Since truncating stores are not supported, we custom legalize them by
matching types of store and MMO.
* Introduce a constant pool flag in X86AddressMode because otherwise
we need to introduce a GlobalISel copy for X86ISelAddressMode.
* Also please notice in the tests that GlobalISel prefers to fold memory
operands immediately comparing to SDAG. The reason is that GlobalISel
doesn't have target hooks in GIM_CheckIsSafeToFold. Or maybe another
check on profitability is required along with safety check that is
currently not present.
When SPIRV-Tools is enabled, tests that specify the target environment
`vulkan1.3` in the validation step fail. This is because SPIRV-Tools
seems to have [grown some capability checks for vulkan
1.3](7e41c7130f).
The failing tests are:
- `CodeGen/SPIRV/hlsl-intrinsics/SV_GroupIndex.ll`
- `CodeGen/SPIRV/hlsl-intrinsics/smoothstep.ll`
For now, the fix is to XFAIL these tests to unblock the pipeline.
Issue #136344 tracks the long-term solution for this.
If XRivosVizip is available, the ri.vunzip2a and ri.vunzip2b can be used
to the concatenation and register deinterleave shuffle. This patch only
effects the intrinsic lowering (and thus scalable vectors because the
fixed vectors go through shuffle lowering).
Note that this patch is restricted to e64 for staging purposes only. e64
is obviously profitable (i.e. we remove a vcompress). At e32 and below,
our alternative is a vnsrl instead, and we need a bit more complexity
around lowering with fractional LMUL before the ri.vunzip2a/b versions
becomes always profitable. I'll post the followup change once this
lands.
We can't assume MBBI is still pointing at MBB if we've already expanded
a probe. We need to re-query the MBB from MBBI. Fixes#135206
Co-authored-by: Craig Topper <craig.topper@sifive.com>
This avoids the need to have special handling at every use site.
Unfortunately this means we unnecessarily emit AssertZext in the DAG
(where we already directly understand the range of the intrinsic), andt
we regress in undefined cases as we don't fold out asserts on undef.
If we have a build vector which could be either a splat or a scalar
insert, prefer the scalar insert. At high LMUL, this reduces vector
register pressure (locally, the use will likely still be aligned), and
the amount of work performed for the splat.
Reducing churn in advance of a change which makes better use of these.
Note that for very short fixed length shuffles, we already use the
vizip family instructions - but mostly in the form of zipeven/zipodd.
While fp128 operations are not natively supported in hardware, emulation
for them is supported by nvcc. This change adds basic support for
fp128 as a storage type allowing for lowering of IR containing these
types.
Fixes: https://github.com/llvm/llvm-project/issues/95471
On targets that support v_cvt_pk_f16_f32 instruction, if we make v2f64
-> v2f16 Legal, we will generate the following sequence of instructions:
v_cvt_f32_f64_e32 v1, s[6:7]
v_cvt_f32_f64_e32 v2, s[4:5]
v_cvt_pk_f16_f32 v1, v2, v1
It possibly returns imprecise results due to double rounding. This patch
fixes the issue by not setting the conversion Legal. While we may still
expect the above sequence of code when unsafe fpmath is set, I hope
https://github.com/llvm/llvm-project/pull/134738 can address that
performance concern.
Fixes: SWDEV-523856
This extends the DAG combine introduced in 336b2909 to handle the case
where the prior value is defined by a vmv.s.x instead of a vmv.v.x. If
the vrgather splats the single source element, and has no passthru we
can replace it with a vmv.v.x - which will in turn usually get folded
into a vmerge if a select follows.
fixes#135719
LLVM 3.7 did not have a freeze instruction
Further this instruction is really only used as syntactic sugar
in LLVM's optimizer passes to not aggressively optimize things that
could be undef or poison ie x*2 to x+x.
Most backends treat it as a no-op so we will do the same
by removing it and replacing its uses with its input.
This change unifies the NVVM intrinsics for floating point absolute
value into two new overloaded intrinsics "llvm.nvvm.fabs.*" and
"llvm.nvvm.fabs.ftz.*". Documentation has been added specifying the
semantics of these intrinsics to clarify how they differ from
"llvm.fabs.*". In addition, support for these new intrinsics is
extended to cover the f16 variants.
Propagation to poison in function `SDValue
SelectionDAG::getNode(unsigned Opcode, const SDLoc &DL, EVT VT,SDValue
N1, SDValue N2, const SDNodeFlags Flags) ` if one of the input is
poison.
The patch also revert the test cases
llvm/test/CodeGen/X86/pr119158.ll
llvm/test/CodeGen/X86/half.ll
which are mentioned in
https://github.com/llvm/llvm-project/pull/125883#discussion_r2021390919
---------
Co-authored-by: Amy Kwan <amy.kwan1@ibm.com>
