Replace the DenseMap from blocks to their innermost loop a vector
indexed by block numbers, when possible. Supporting number updates is
not trivial as we don't store a list of basic blocks, so this is not
implemented.
NB: I'm generally not happy with the way loops are stored. As I think
that there's room for improvement, I don't want to touch the
representation at this point.
Pull Request: https://github.com/llvm/llvm-project/pull/103400
Add infrastructure to identify conditional branches on values loaded from
memory. Such branches are likely to be harder to predict accurately since
branch history (probably) provides little useful information.
This analysis walks the def-use chain from the branch condition to find
loads that feed into it. Several cases are excluded from consideration:
- Loads from constant pools (predictable values)
- Dereferenceable invariant loads (loop-invariant)
- Branches with biased probability (null checks, etc.)
- Loads not "close in program time" to the branch (must be in the same
basic block with no intervening calls)
The analysis is disabled by default behind -enable-early-ifcvt-data-dependent.
Reduce the size of a DomTreeNodeBase from 80 to 56 bytes by not storing
the children in a SmallVector. Instead, store children as forward-linked
list. This also avoids extra allocations for nodes with many children.
Additionally, DomTreeNodeBase is now trivially destructible.
A lot of code depends on the order of nodes in the dominator tree, so
make sure that the order is the same when inserting nodes. (Not having
to do this would save 8 bytes per node.)
NewGVN uses the order of nodes in the dominator tree in a way that is
not entirely clear to me (https://reviews.llvm.org/D28129). I kept the
semantics as, but now this is the only external user of
addChild/removeChild, which actually should be private.
https://llvm-compile-time-tracker.com/compare.php?from=263802c56b4db3fc9b6ed9fd313499cb03ca44da&to=43e0c0c5b663b3a4067252fc0addbaccefd0014d&stat=instructions:u
This changes `MCRegUnit` type from `unsigned` to `enum class : unsigned`
and inserts necessary casts.
The added `MCRegUnitToIndex` functor is used with `SparseSet`,
`SparseMultiSet` and `IndexedMap` in a few places.
`MCRegUnit` is opaque to users, so it didn't seem worth making it a
full-fledged class like `Register`.
Static type checking has detected one issue in
`PrologueEpilogueInserter.cpp`, where `BitVector` created for
`MCRegister` is indexed by both `MCRegister` and `MCRegUnit`.
The number of casts could be reduced by using `IndexedMap` in more
places and/or adding a `BitVector` adaptor, but the number of casts *per
file* is still small and `IndexedMap` has limitations, so it didn't seem
worth the effort.
Pull Request: https://github.com/llvm/llvm-project/pull/167943
Bug relates to `early-if-predicator` and `early-ifcvt` passes. If
virtual register has "killed" flag in both basic blocks to be merged
into head, both instructions in head basic block will have "killed" flag
for this register. It makes MIR incorrect.
Example:
```
bb.0: ; if
...
%0:intregs = COPY $r0
J2_jumpf %2, %bb.2, implicit-def dead $pc
J2_jump %bb.1, implicit-def dead $pc
bb.1: ; if.then
...
S4_storeiri_io killed %0, 0, 1
J2_jump %bb.3, implicit-def dead $pc
bb.2: ; if.else
...
S4_storeiri_io killed %0, 0, 1
J2_jump %bb.3, implicit-def dead $pc
```
After early-if-predicator will become:
```
bb.0:
%0:intregs = COPY $r0
S4_storeirif_io %1, killed %0, 0, 1
S4_storeirit_io %1, killed %0, 0, 1
```
Having `killed` flag set twice in bb.0 for `%0` is an incorrect MIR.
Use nonstatic member instead. This requires explicit conversions, but
many will go away as we continue converting unsigned to Register.
In a few places where it was simple, I changed unsigned to Register.
Change eraseNode to require that the basic block is still contained
inside the function. This is a preparation for using numbers of basic
blocks inside the dominator tree, which are invalid for blocks that are
not inside a function.
This reverts commit ab58b6d58edf6a7c8881044fc716ca435d7a0156.
In `CodeGen/Generic/MachineBranchProb.ll`, `llc` crashed with dumped MIR
when targeting PowerPC. Move test to `llc/new-pm`, which is X86
specific.
Prepare for new pass manager version of `MachineDominatorTreeAnalysis`.
We may need a machine dominator tree version of `DomTreeUpdater` to
handle `SplitCriticalEdge` in some CodeGen passes.
Fixes#82659
There are some functions, such as `findRegisterDefOperandIdx` and `findRegisterDefOperand`, that have too many default parameters. As a result, we have encountered some issues due to the lack of TRI parameters, as shown in issue #82411.
Following @RKSimon 's suggestion, this patch refactors 9 functions, including `{reads, kills, defines, modifies}Register`, `registerDefIsDead`, and `findRegister{UseOperandIdx, UseOperand, DefOperandIdx, DefOperand}`, adjusting the order of the TRI parameter and making it required. In addition, all the places that call these functions have also been updated correctly to ensure no additional impact.
After this, the caller of these functions should explicitly know whether to pass the `TargetRegisterInfo` or just a `nullptr`.
After a pass calls addRequired<X>() it is strange to call
getAnalysisIfAvailable<X>() because analysis X should always be
available. Use getAnalysis<X>() instead.
This patch adds a heuristic to skip if-conversion if the condition has a
high chance of being predictable.
If the condition is in a loop, consider it predictable if the condition
itself or all its operands are loop-invariant. E.g. this considers a load
from a loop-invariant address predictable; we were unable to prove that it
doesn't alias any of the memory-writes in the loop, but it is likely to
read to same value multiple times.
This is a relatively crude heuristic, but it helps to prevent excessive
if-conversion in multiple workloads in practice.
Reviewed By: apostolakis
Differential Revision: https://reviews.llvm.org/D141639
The `ReversePredicate` should have made sure the reverse predicate is
supported by target, but the check comes from early function and might
be invalid by any mistake. So it's better to double confirm it here.
Differential Revision: https://reviews.llvm.org/D149586
This change initializes the members TSI, LI, DT, PSI, and ORE pointer feilds of the SelectOptimize class to nullptr.
Reviewed By: LuoYuanke
Differential Revision: https://reviews.llvm.org/D148303
Make forward declaration possible to reduce amount of dependencies and reduce
re-compilation burden caused by further patches.
Reviewed By: spatel
Differential Revision: https://reviews.llvm.org/D140539
Adds a target hook canPredicatePredicatedInstr(const MachineInstr&) that
assumes an instruction is already predicated and returns true if it can
be predicated again, used by the early if-conversion pass in order to
iterate multiple times on architectures supporting predicate logic.
No test added since there is no upstream target that can take advantage.
Differential Revision: https://reviews.llvm.org/D139981
This was stored in LiveIntervals, but not actually used for anything
related to LiveIntervals. It was only used in one check for if a load
instruction is rematerializable. I also don't think this was entirely
correct, since it was implicitly assuming constant loads are also
dereferenceable.
Remove this and rely only on the invariant+dereferenceable flags in
the memory operand. Set the flag based on the AA query upfront. This
should have the same net benefit, but has the possible disadvantage of
making this AA query nonlazy.
Preserve the behavior of assuming pointsToConstantMemory implying
dereferenceable for now, but maybe this should be changed.
This reverts commit 7f230feeeac8a67b335f52bd2e900a05c6098f20.
Breaks CodeGenCUDA/link-device-bitcode.cu in check-clang,
and many LLVM tests, see comments on https://reviews.llvm.org/D121169
This extends the early-ifcvt pass to avoid a few more cases where the resulting
select instructions would have matching operands. Additionally, we now use TII
to determine "sameness" of the operands so that as TII gets smarter, so too
will ifcvt.
The attached test case was bugpoint-reduced down from CINT2000/252.eon in the
test-suite. See: https://clang.godbolt.org/z/WvnrcrGEn
Differential Revision: https://reviews.llvm.org/D101508
