This PR resolves https://github.com/llvm/llvm-project/issues/144513
The modification include five pattern :
1.vselect Cond, 0, 0 → 0
2.vselect Cond, -1, 0 → bitcast Cond
3.vselect Cond, -1, x → or Cond, x
4.vselect Cond, x, 0 → and Cond, x
5.vselect Cond, 000..., X -> andn Cond, X
1-4 have been migrated to DAGCombine. 5 still in x86 code.
The reason is that you cannot use the andn instruction directly in
DAGCombine, you can only use and+xor, which will introduce optimization
order issues. For example, in the x86 backend, select Cond, 0, x →
(~Cond) & x, the backend will first check whether the cond node of
(~Cond) is a setcc node. If so, it will modify the comparison operator
of the condition.So the x86 backend cannot complete the optimization of
andn.In short, I think it is a better choice to keep the pattern of
vselect Cond, 000..., X instead of and+xor in combineDAG.
For commit, the first is code changes and x86 test(note 1), the second
is tests in other backend(node 2).
---------
Co-authored-by: Simon Pilgrim <llvm-dev@redking.me.uk>
This patch allows urem by a constant to be expanded more efficiently to
avoid the need for expensive udiv instructions. This is part of the
resolution to issue #118090
Always try to fold freeze(op(....)) -> op(freeze(),freeze(),freeze(),...).
This patch proposes we drop the opt-in limit for opcodes that are allowed to push a freeze through the op to freeze all its operands, through the tree towards the roots.
I'm struggling to find a strong reason for this limit apart from the DAG freeze handling being immature for so long - as we've improved coverage in canCreateUndefOrPoison/isGuaranteedNotToBeUndefOrPoison it looks like the regressions are not as severe.
Hopefully this will help some of the regression issues in #143102 etc.
After 901e1390c9778a191256335d37802bc631c2d183 (#127770), the DAG
combine would transform `fma(x, 0.0, 1.0)` into `1.0` if
`-fp-contract=fast` was enabled, in addition to when 'x' is marked
nnan/ninf.
It's only valid in the latter case, not the former, so delete the extra
condition.
CTTZ/CTLZ_ZERO_UNDEF nodes can only create poison if the source value is zero - so check with isKnownNeverZero
Pulled out of #146361 and reapplied now that #146490 has landed.
This keeps getting forgotten (e.g. #66603) - so make a point of adding
it here to make it clear instead of relying on the implicit default of
returning true.
Although nice to have to prove the freeze can be moved, this can fail
immediately after freeze(op(...)) -> op(freeze(),freeze(),...) creation
if any of the new freeze nodes now prevents value tracking from seeing
through to the source values (e.g. shift amounts/element indices are in
bounds etc.).
This will allow us to remove the isGuaranteedNotToBeUndefOrPoison checks
inside canCreateUndefOrPoison that were discussed on #146361
Remove Linker Optimization Hints (LOHs) from outlining candidates
instead of simply preventing outlining if LOH labels are found in the
candidate. This will improve the effectiveness of the machine outliner
when LOHs are enabled (which is the default).
In
https://discourse.llvm.org/t/loh-conflicting-with-machineoutliner/83279/1
it was observed that the machine outliner is much more effective when
LOHs are disabled. Rather than completely disabling LOH, this PR aims to
keep LOH in most places and removing them from outlined functions where
it could be illegal. Note that we are conservatively removing all LOHs
from outlined functions for simplicity, but I believe we could retain
LOHs that are in the intersection of all candidates.
It should be ok to remove these LOHs since these blocks are being
outlined anyway, which will harm performance much more than the gain
from keeping the LOHs.
Neither the arithmetic value or overflow result can create undef/poison from regular operands values.
We have complete test coverage for all ADDO/SUBO nodes, 32-bit codegen handles the _CARRY variants but until #145939 lands AND DAGCombiner::visitFREEZE handles multiple results we can't see any codegen change.
Pulled out of #145939
PR which introduced the bug:
https://github.com/llvm/llvm-project/pull/131837.
Fixes a crash around dead registers which started in f5c62ee0fa04 by
verifying that the reused incoming register is also virtual.
Patch 3/4 adding bitcode support, though the final patch doesn't depend on this
one.
Prior to this patch, a Key Instructions function inlined into a
Not-Key-Instructions function fell back to Not-Key-Instructions handling.
In order to fully support inlining mixed modes we need to run
`computeKeyInstructions` (in case there's a Key Instructions scope) and
`findForceIsStmtInstrs` (in case there's a Not-Key-Instructions scope) on all
functions. This has a slight performance cost for all configurations - see PR
for details.
Patch 2/4 adding bitcode support.
A non-key-instructions function inlined into a key-instructions function uses
non-key-instructions is_stmt placement (without `findForceIsStmtInstrs`).
A key-instructions function inlined into a non-key-instructions function
currently results in falling back to non-key-instructions for the inlined scope
too.
Both of these concessions (not using `findForceIsStmtInstrs` in the 1st case,
and not using Key Instructions for the inlined scope in the 2nd) are for
performance reasons; to do the right thing we'd need to run both
`findForceIsStmtInstrs` and `computeKeyInstructions` - in case that's
controversial I've got a separate PR for that: PR 144103.
Remove `UnsafeFPMath` in `visitFMULForFMADistributiveCombine`,
`visitFSUBForFMACombine` and `visitFDIV`.
All affected tests are fixed by add fast math flags manually.
Propagate fast math flags when lowering fdiv in NVPTX backend, so it can
produce optimized dag when `unsafe-fp-math` is absent.
The insertion point of COPY isn't always optimal and could eventually
lead to a worse block layout, see the regression test in the first
commit.
This change affects many architectures but the amount of total
instructions in the test cases seems too be slightly lower.
MCExpr::print has an optional MCAsmInfo argument, which is error-prone
when omitted. MCExpr::print and the convenience helper operator<< are
discouraged to use. Switch to MCAsmInfo::printExpr instead. Use the
target-specific MCAsmInfo if available.
`enum VariantKind` is deprecated. Targets are encouraged to use their
own relocation specifier constants. MCSymbolRefExpr::create callers with
a VK_None argument should switch to the overload with a VariantKind
parameter.
Reapply "[NFC][DebugInfo][DWARF] Create new low-level dwarf library (#…
(#145959)
This reapplies cbf781f0bdf2f680abbe784faedeefd6f84c246e, with fixes for
the shared-library build and the unconventional sanitizer-runtime build.
Original Description:
This is the culmination of a series of changes described in [1].
Although somewhat large by line count, it is almost entirely mechanical,
creating a new library in DebugInfo/DWARF/LowLevel. This new library has
very minimal dependencies, allowing it to be used from more places than
the normal DebugInfo/DWARF library--in particular from MC.
1.
https://discourse.llvm.org/t/rfc-debuginfo-dwarf-refactor-into-to-lower-and-higher-level-libraries/86665/2
This changes the final stage of InstrRef, i.e. the TransferTracker
(which combines the values locations with the variable values), so that
it treats a DEBUG_VALUE of an EntryValue just like a DEBUG_VALUE of a
constant: a location that is never clobbered and can be propagated to
subsequent BBs as long as no other DEBUG_VALUE intrinsics updated the
variable.
We add two tests here:
1. `entry_value_clobbered_stack_copy` that saves a register on the
stack, uses this register as an entry value DBG_VALUE location, and then
clobbers it. Prior to this patch, this test would crash because we would
try to describe a new location for the variable in terms of what was
saved on the stack, and use an invalid expression to do so. This is not
needed as an EntryValue can never be clobbered.
2. `entry_value_gets_propagated`, that tests that an EntryValue
DBG_VALUE is propagated in a diamond-shaped CFG.
This patch is trying to reland
https://github.com/llvm/llvm-project/pull/77938 but also fixes the bug
with InstrRef based LiveDebugValues, where entry values were not being
propagated in a diamond-shaped CFG.
When CSEing a load with an existing load with different range
metadata, clear the range metadata on the existing
load.
This is conservative, alternatively we could calculate new range
metadata using MDNode::getMostGenericRange. Without a test case I wasn't
sure it was worth it.
MDnode::getMostGenericRange takes a non-const MDNode*, but all of
SelectionDAG
uses const MDNode*. A const_cast will need to be used somewhere or
we need to make the codebase consistent about whether MDNode pointers
should be const or not.
I'm sure this isn't the only place that needs to be updated to handle
the CSE.
Fixes#145363.
This is especially helpful for AArch64, which simplifies ands + cmp to tst.
Alive2: https://alive2.llvm.org/ce/z/LLgcJJ
---------
Co-authored-by: Simon Pilgrim <llvm-dev@redking.me.uk>
Followup to #143760 which handled ISD::VSELECT
I've moved ISD::SELECT/VSELECT under the "No poison except from flags
(which is handled above)" subgroup to try to remind people that these
can have poison generating FMFs (NINF/NNAN), even though this hasn't
been well explained anywhere I can find :(
Helps with regressions from #145939
In the pre-legalizer combiner, there exists a bug with UseVectorTruncate
match-apply optimization. When the destinations' types do not match the
vector element type of the G_UNMERGE_VALUES instruction, the resulting
collapsed truncate does not preserve original functional behavior. This
commit introduces a simple type check to ensure that the destination
types match the vector element type.
This is the culmination of a series of changes described in [1].
Although somewhat large by line count, it is almost entirely mechanical,
creating a new library in DebugInfo/DWARF/LowLevel. This new library has
very minimal dependencies, allowing it to be used from more places than
the normal DebugInfo/DWARF library--in particular from MC.
I am happy to put it in another location, or to structure it differently
if that makes sense. Some have suggested in BinaryFormat, but it is not
a great fit there. But if that makes more sense to the reviewers, I can
do that.
Another possibility would be to use pass-through headers to allow
clients who don't care to depend only on DebugInfo/DWARF. This would be
a much less invasive change, and perhaps easier for clients. But also a
system that hides details.
Either way, I'm open.
1.
https://discourse.llvm.org/t/rfc-debuginfo-dwarf-refactor-into-to-lower-and-higher-level-libraries/86665/2
Unlike other formats, the GOFF object file format uses a 2 dimensional structure
to define the location of data. For example, the equivalent of the ELF .text
section is made up of a Section Definition (SD) and a class (Element Definition;
ED). The name of the SD symbol depends on the application, while the class has
the predefined name C_CODE/C_CODE64 in AMODE31 and AMODE64 respectively.
Data can be placed into this structure in 2 ways. First, the data (in a text
record) can be associated with an ED symbol. To refer to data, a Label
Definition (LD) is used to give an offset into the data a name. When binding,
the whole data is pulled into the resulting executable, and the addresses
given by the LD symbols are resolved.
The alternative is to use a Part Definition (PR). In this case, the data (in
a text record) is associated with the part. When binding, only the data of
referenced PRs is pulled into the resulting binary.
Both approaches are used. SD, ED, and PR elements are modeled by nested
MCSectionGOFF instances, while LD elements are associated with MCSymbolGOFF
instances.
At the binary level, a record called "External Symbol Definition" (ESD) is used. The
ESD has a type (SD, ED, PR, LD), and depending on the type a different subset of
the fields is used.
This patch focuses on generic DAG combines, plus an AMDGPU-target-specific one
that is closely connected.
The generic DAG combine is based on a part of PR #105669 by rgwott, which was
adapted from work by jrtc27, arichardson, davidchisnall in the CHERI/Morello
LLVM tree. I added some parts and removed several disjuncts from the
reassociation condition:
- `isNullConstant(X)`, since there are address spaces where 0 is a perfectly
normal value that shouldn't be treated specially,
- `(YIsConstant && ZOneUse)` and `(N0OneUse && ZOneUse && !ZIsConstant)`, since
they cause regressions in AMDGPU.
For SWDEV-516125.
## Purpose
This patch is one in a series of code-mods that annotate LLVM’s public
interface for export. This patch annotates the remaining LLVM CodeGen
and CodeGenTypes library interfaces that were missed in, or modified
since, previous patches. The annotations currently have no meaningful
impact on the LLVM build; however, they are a prerequisite to support an
LLVM Windows DLL (shared library) build.
## Background
This effort is tracked in #109483. Additional context is provided in
[this
discourse](https://discourse.llvm.org/t/psa-annotating-llvm-public-interface/85307),
and documentation for `LLVM_ABI` and related annotations is found in the
LLVM repo
[here](https://github.com/llvm/llvm-project/blob/main/llvm/docs/InterfaceExportAnnotations.rst).
## Overview
The bulk of these changes were generated automatically using the
[Interface Definition Scanner (IDS)](https://github.com/compnerd/ids)
tool, followed formatting with `git clang-format`.
The following manual adjustments were also applied after running IDS:
- Explicitly instantiate `CallLowering::setArgFlags` template method
instances in `CodeGen/GlobalISel/CallLowering.h` and annotate them with
`LLVM_ABI`. These methods are already explicitly instantiated in
`lib/CodeGen/GlobalISel/CallLowering.cpp` but were not `extern` declared
in the header.
- Annotate several explicit template instantiations with
`LLVM_EXPORT_TEMPLATE`.
- Include `llvm/CodeGen/Passes.h` from
`llvm/lib/CodeGen/GlobalMergeFunctions.cpp` to pick up the declaration
of `llvm::createGlobalMergeFuncPass` with the `LLVM_ABI` annotation
(instead of adding `LLVM_ABI` to the function definition in this file)
## Validation
Local builds and tests to validate cross-platform compatibility. This
included llvm, clang, and lldb on the following configurations:
- Windows with MSVC
- Windows with Clang
- Linux with GCC
- Linux with Clang
- Darwin with Clang
In Ada, a record type can have a non-constant size, and a field can
appear at a non-constant bit offset in a record.
To support this, this patch changes DIType to record the size and offset
using metadata, rather than plain integers. In addition to a constant
offset, both DIVariable and DIExpression are now supported here.
One thing of note in this patch is the choice of how exactly to
represent a non-constant bit offset, with the difficulty being that
DWARF 5 does not support this. DWARF 3 did have a way to support a
non-constant byte offset, combined with a constant bit offset within the
byte, but this was deprecated in DWARF 4 and removed from DWARF 5.
This patch takes a simple approach: a DWARF extension allowing the use
of an expression with DW_AT_data_bit_offset. There is a corresponding
DWARF issue, see https://dwarfstd.org/issues/250501.1.html. The main
reason for this approach is that it keeps API simplicity: just a single
value is needed, rather than having separate data describing the byte
offset and the bit within the byte.
Always let SimplifyDemandedVectorElts fold either side of a
VECTOR_SHUFFLE to UNDEF if no elements are demanded from that side.
For a single use this could be done by SimplifyDemandedVectorElts
already, but in case the operand had multiple uses we did not eliminate
the use.
- Add a comment explaining TrackedRegs cache in handleNewDebugValue
- Remove a comment which is not meaningful since
https://reviews.llvm.org/D4919
- Expand the subject of a comment to match the code it describes
Now that the loop vectorizer emits just a single
llvm.vector.[de]interleaveN intrinsic after #141865, we can remove the
need to recognise recursively [de]interleaved intrinsics.
No in-tree target currently has instructions to emit an interleaved
access with a factor > 8, and I'm not aware of any other passes that
will emit recursive interleave patterns, so this code is effectively
dead.
Some tests have been converted from the recursive form to a single
intrinsic, and some others were deleted that are no longer needed, e.g.
to do with the recursive tree.
This closes off the work started in #139893.
A case found from https://github.com/rust-lang/rust/issues/142752:
https://llvm.godbolt.org/z/T7ce9saWh.
We should emit `@bar_0` for the following code:
```llvm
target triple = "x86_64-unknown-linux-gnu"
@rel_0 = private unnamed_addr constant [1 x i32] [
i32 trunc (i64 sub (i64 ptrtoint (ptr @bar_0 to i64), i64 ptrtoint (ptr @rel_0 to i64)) to i32)]
@bar_0 = internal unnamed_addr constant ptr @foo_0, align 8
@foo_0 = external global ptr, align 8
define void @foo(ptr %arg0) {
store ptr @bar_0, ptr %arg0, align 8
ret void
}
```
We can not trust that !HasFallThrough implies that there is not
fallthrough exit in cases when analyzeBranch failed.
Adding a new blockNeverFallThrough helper to make the tests on
!HasFallThrough safe by also checking IsBrAnalyzable. We also
try to prove no-fallthrough by inspecting the successor list. If
the textual successor isn't in the successor list we know that
there is no fallthrough.
The bug has probably been around for years. Found it when
working on an out-of-tree target.
