Add custom lowering for `BR_JT` DAG nodes to the `brx.idx` PTX
instruction ([PTX ISA 9.7.13.4. Control Flow Instructions: brx.idx]
(https://docs.nvidia.com/cuda/parallel-thread-execution/#control-flow-instructions-brx-idx)).
Depending on the heuristics in DAG selection, `switch` statements may
now be lowered using `brx.idx`.
Note: this fixes the previous issue in #102400 by adding the isBarrier
attribute to BRX_END
Prior to this patch, SelectionDAG generated aligned move onto stacks for
AVX registers when the function was marked as a no-realign-stack
function. This lead to misalignment between the stack and the
instruction generated. This patch fixes the issue. There was a similar
issue reported for `extractelement` which was fixed in
a6614ec5b7c1dbfc4b847884c5de780cf75e8e9c
Co-authored-by: Manish Kausik H <hmamishkausik@gmail.com>
This is used by PreISelIntrinsicLowering. With this change,
PreISelIntrinsicLowering does not have to assume that there were changes
just because we encountered a VP intrinsic.
lowerConstantIntrinsics does an RPO traveral, which doesn't reach dead
blocks. Remove the assertion that all intrinsics are lowered, because
some intrinsics might remain.
Always match ABD patterns pre-legalization, and use TargetLowering::expandABD to expand again during legalization.
abdu(lhs, rhs) -> sub(xor(sub(lhs, rhs), usub_overflow(lhs, rhs)), usub_overflow(lhs, rhs))
Alive2: https://alive2.llvm.org/ce/z/dVdMyv
REAPPLIED: Fix regression issue with "abs(ext(x) - ext(y)) -> zext(abd(x, y))" fold failing after type legalization
Machine block placement might remove nodes from the function but does
not update the dominator tree accordingly. Instead of renumbering (which
might crash due to accessing removed blocks), set the domtree to null to
make clear that it is invalid at this point.
Fixup of #102107.
For a __thread variable x, when emulated TLS is enabled and there is an
access to x, the compiler first looks up the symbol __emutls_v.x within
the module. However, the issue arises with an alias y of x, the compiler
still tries to look up __emutls_v.y instead of __emutls_v.x. As a
result, the lookup returns a nullptr, causing the compiler to crash. The
purpose of this MR (Merge Request) is to ensure that in emulated TLS,
before checking __emutls_v.y, the compiler first identifies which global
value y is an alias of.
The resulting add is nuw if either the gep was nuw or it was
nusw+nneg. Previously only inbounds+nneg was handled.
Test via wasm load offsets, which seems to most directly expose
these SDAG flags.
This hint map is not required whenever a new register is added, in fact,
at -O0, it is not used at all. Growing this map is quite expensive, as
SmallVectors are not trivially copyable.
Grow this map only when hints are actually added to avoid multiple grows
and grows when no hints are added at all.
This patch fixes https://github.com/llvm/llvm-project/issues/17204.
If a base pointer is used in a function, and it is clobbered by an
instruction (typically an inline asm), current register allocator can't
handle this situation, so BP becomes garbage after those instructions.
It can also occur to FP in theory.
We can spill and reload FP/BP registers around those instructions. But
normal spill/reload instructions also use FP/BP, so we can't spill them
into normal spill slots, instead we spill them into the top of stack by
using SP register.
expandVectorPredication may change code, even if the intrinsic itself
remains in the code. Report changes whenever such an intrinsic is
encountered, because code could have been changed.
Another follow-up fix for #101652 to fix expensive-checks-only failure.
The dominator tree gained an optimization to use block numbers instead
of a DenseMap to store blocks. Given that machine basic blocks already
have numbers, expose these via appropriate GraphTraits. For debugging,
block number epochs are added to MachineFunction -- this greatly helps
in finding uses of block numbers after RenumberBlocks().
In a few cases where dominator trees are preserved across renumberings,
the dominator tree is updated to use the new numbers.
Emit an optimization remark when objects in the stack frame may cause
hazards in a streaming mode function. The analysis requires either the
`aarch64-stack-hazard-size` or `aarch64-stack-hazard-remark-size` flag
to be set by the user, with the former flag taking precedence.
Always match ABD patterns pre-legalization, and use TargetLowering::expandABD to expand again during legalization.
abdu(lhs, rhs) -> sub(xor(sub(lhs, rhs), usub_overflow(lhs, rhs)), usub_overflow(lhs, rhs))
Alive2: https://alive2.llvm.org/ce/z/dVdMyv
dependencies in same cycle
Dependency checks were insufficient when reordering instructions with
physical register dependencies (i.e. Anti/Output dependencies). This
could result in generating incorrect code.
This addresses a TODO where we can fold a vselect to it's true operand
if the boolean is known to be all trues, by factoring out the logic from
extractBooleanFlip which checks TLI.getBooleanContents.
Since ce0c205813c74b4225180ac8a6e40fd52ea88229, we are doing that if a
single (LTO) compilation contains more than one compile unit, but the
same thing can happen if the non-lto and single-cu lto compilations,
typically when the CU ends up (nearly) empty. In my case, this happened
when LTO emptied two compile units.
Note that the source file name is already a part of the hash, so this
can only happen when a single file is compiled and linked twice into the
same application (most likely with different preprocessor defintiions).
While not exactly common, this pattern is used by some C code to
implement "templates".
The 2017 patch already hinted at the possibility of doing this
unconditionally, and this patch implements that. While the DWARF spec
hints at the option of using the type signature hashing algorithm for
the DWO_id purposes, AFAICT it does not actually require it, so I
believe this change is still conforming.
The relevant section of the spec is in Section 3.1.2 "Skeleton
Compilation Unit Entries" (in non-normative text):
```
The means of determining a compilation unit ID does not need to be
similar or related to the means of determining a type unit signature.
However, it should be suitable for detecting file version skew or other
kinds of mismatched files and for looking up a full split unit in a
DWARF package file (see Section 7.3.5 on page 190).
```
sizeWithoutDebug() iterates over the entire block, but all we do is to
check whether the number is not equal to one. In times of debug records,
simply check whether the first and last instructions are equal.
This reverts commit 667598d84b16d1789ce90b231565e9e7bfdbe77d and fixes failed tests: llvm/test/CodeGen/X86/nomerge.ll and llvm/test/MC/AArch64/local-bounds-single-trap.ll.
1. It fixes the problem that llvm.trap() not getting the nomerge
attribute.
2. It sets nomerge flag for the node if the instruction has nomerge
arrtibute.
This is a copy of https://reviews.llvm.org/D146164. This only attempts
to fix `nomerge` for `__builtin_trap()`, `__debugbreak()`,
`__builtin_verbose_trap()`, not working for non-trap builtins.
Fixes#53011
Currently, the LowerConstantIntrinsics pass does an RPO traversal of
every function... only to find that many functions don't have constant
intrinsics (is.constant, objectsize). In the CodeGen pipeline, there is
already a pre-isel intrinsic lowering pass, which iterates over
intrinsic declarations and lowers all users. Call
lowerConstantIntrinsics from this pass to avoid the extra iteration over
the entire IR and the RPO traversal.
ucmp can be promoted with either sext or zext. RISC-V and LoongArch
prefer sext for promoting i32 to i64 unless the inputs are known to be
zero extended already.
This patch uses the existing SExtOrZExtPromotedOperands function that is
used by SETCC promotion to intelligently handle this.
