SITargetLowering::adjustWritemask calls SelectionDAG::UpdateNodeOperands
to update an EXTRACT_SUBREG node in-place to refer to a new IMAGE_LOAD
instruction, before we delete the old IMAGE_LOAD instruction. But in
UpdateNodeOperands can do CSE on the fly and return a different
EXTRACT_SUBREG node, so the original EXTRACT_SUBREG node would still
exist and would refer to the old deleted IMAGE_LOAD instruction. This
caused errors like:
t31: v3i32,ch = <<Deleted Node!>> # D:1
This target-independent node should have been selected!
UNREACHABLE executed at lib/CodeGen/SelectionDAG/InstrEmitter.cpp:1209!
Fix it by detecting the CSE case and replacing all uses of the original
EXTRACT_SUBREG node with the CSE'd one.
This finishes the work of replacing OperandMatchResultTy with
ParseStatus, started in D154101.
As a drive-by change, rename some RegNo variables to just Reg
(a leftover from the days when RegNo had 'unsigned' type).
Now that the old backend is gone, clean-up a few things that no longer make sense and tidy up the file a bit.
Depends on D158710
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D158714
Remove CodeGen leftovers from the old combiner backend and adapt the API to fit the new backend better.
It's now quite a bit closer to how InstructionSelector works.
- `CombinerInfo` is now a simple "options" struct.
- `Combiner` is now the base class of all TableGen'd combiner implementation.
- Many fields have been moved from derived classes into that class.
- It has been refactored to create & own the Observer and Builder.
- `tryCombineAll` TableGen'd method can now be renamed, which allows targets to implement the actual `tryCombineAll` call manually and do whatever they want to do before/after it.
Note: `CombinerHelper` needs to be mutable because none of its methods are const. This can be revisited later.
Depends on D158710
Reviewed By: aemerson, dsanders
Differential Revision: https://reviews.llvm.org/D158713
This was requiring all fast math flags, which is practically
useless. This wouldn't fire using all the standard OpenCL fast math
flags. This only needs afn nnan and ninf.
https://reviews.llvm.org/D158904
Irritatingly, atomic_store had operands in the opposite order from
regular store. This made it difficult to share patterns between
regular and atomic stores.
There was a previous incomplete attempt to move atomic_store into the
regular StoreSDNode which would be better.
I think it was a mistake for all atomicrmw to swap the operand order,
so maybe it's better to take this one step further.
https://reviews.llvm.org/D123143
The MatchTable-based GlobalISel Combiner backend is the new default. There are no in-tree users left of the old backend.
- Removed implementation of old MatchDAG-based Combiner, including tests, the backend itself and all supporting code.
- Renamed MatchTable backend to `GlobalISelCombinerEmitter.cpp` + removed "-matchtable" from its CL option.
- no need to have a verbose name as it's the only backend left now.
Reviewed By: aemerson
Differential Revision: https://reviews.llvm.org/D158710
This means llvm-mc should now build without depending on the target
CodeGen libraries.
Fix up a few includes in RISCV, AMDGPU, and X86 MCA to avoid transitive
deps on CodeGen.
Fixes#64166
There are really two rounding modes, so only return the standard
values if both modes are the same. Otherwise, return a bitmask
representing the two modes.
Annoyingly the register doesn't use the same values as FLT_ROUNDS. Use
a simple integer table we can shift into to convert.
https://reviews.llvm.org/D153158
Expand-Atomic pass emits the CAS loop for FP operations
which limits the optimizations offered by atomic optimizer.
Moving atomic optimizer before expand-atomics allows
better codegen.
Reviewed By: arsenm, #amdgpu
Differential Revision: https://reviews.llvm.org/D157265
Reduction and Scan are implemented using `Iterative`
and `DPP` strategy for `float` type.
Reviewed By: arsenm, #amdgpu
Differential Revision: https://reviews.llvm.org/D156301
Protect from accidental passing of an invalid MCFixupKind value which
can cause an out-of-bounds access in the array.
Reviewed by: arsenm
Differential Revision: https://reviews.llvm.org/D158725
The sync pipeline should always contain the candidate ID. If it doesn't
something's gone awry. assert on that.
Reviewed by: jrbyrnes
Differential Revision: https://reviews.llvm.org/D158845
This provides a uniform way to lower into the relevant instructions across all generations.
Differential Revision: https://reviews.llvm.org/D158468
Change-Id: I1f7ba4b15ee470738535cf1c7d177a11fc471e43
D150312 added a TODO:
TODO: consider renaming the field `StartAtCycle` and `Cycles` to
`AcquireAtCycle` and `ReleaseAtCycle` respectively, to stress the
fact that resource allocation is now represented as an interval,
relatively to the issue cycle of the instruction.
This patch implements that TODO. This naming clarifies how to use these
fields in the scheduler. In addition it was confusing that `StartAtCycle` was
singular but `Cycles` was plural. This renaming fixes this inconsistency.
This commit as previously reverted since it missed renaming that came
down after rebasing. This version of the commit fixes those problems.
Differential Revision: https://reviews.llvm.org/D158568
D150312 added a TODO:
TODO: consider renaming the field `StartAtCycle` and `Cycles` to
`AcquireAtCycle` and `ReleaseAtCycle` respectively, to stress the
fact that resource allocation is now represented as an interval,
relatively to the issue cycle of the instruction.
This patch implements that TODO. This naming clarifies how to use these
fields in the scheduler. In addition it was confusing that `StartAtCycle` was
singular but `Cycles` was plural. This renaming fixes this inconsistency.
This commit as previously reverted since it missed renaming that came
down after rebasing. This version of the commit fixes those problems.
Differential Revision: https://reviews.llvm.org/D158568
D150312 added a TODO:
TODO: consider renaming the field `StartAtCycle` and `Cycles` to
`AcquireAtCycle` and `ReleaseAtCycle` respectively, to stress the
fact that resource allocation is now represented as an interval,
relatively to the issue cycle of the instruction.
This patch implements that TODO. This naming clarifies how to use these
fields in the scheduler. In addition it was confusing that `StartAtCycle` was
singular but `Cycles` was plural. This renaming fixes this inconsistency.
Differential Revision: https://reviews.llvm.org/D158568
We were basing the defer the fast case to codegen based on the fdiv
itself, and not looking for a foldable sqrt input.
https://reviews.llvm.org/D158127
Introducing rsq contract flags is wrong, and also requires some level
of approximate functions. AMDGPUCodeGenPrepare already should handle
the f32 cases with appropriate flags, and I don't see how new
situations to handle would arise during legalization (other than cases
involving the rcp intrinsic, which instcombine tries to
handle). AMDGPUCodeGenPrepare does need to learn better handling of
rcp/rsq for f64 though, which we never bothered to handle well.
Removes another obstacle to correctly lowering sqrt.
https://reviews.llvm.org/D158099
Summary:
Emit .actual_access metadata for the deduced argument access qualifier,
and .access for kernel_arg_access_qual.
Reviewers:
arsenm
Differential Revision:
https://reviews.llvm.org/D157451
Avoiding warnings like this when building with GCC:
warning: enumeral mismatch in conditional expression:
'llvm::AMDGPUISD::NodeType' vs 'llvm::ISD::NodeType'
[-Wenum-compare]
