The old LoopUnswitch pass unswitched selects, but the changes were never
ported to the new SimpleLoopUnswitch.
We unswitch by turning:
```
S = select %cond, %a, %b
```
into:
```
head:
br %cond, label %then, label %tail
then:
br label %tail
tail:
S = phi [ %a, %then ], [ %b, %head ]
```
Unswitch selects are always nontrivial, since the successors do not exit
the loop and the loop body always needs to be cloned.
Unswitch selects always need to freeze the conditional if the
conditional could be poison or undef. Selects don't propagate
poison/undef, and branches on poison/undef causes UB.
Reviewed By: nikic, kachkov98, vitalybuka
Differential Revision: https://reviews.llvm.org/D138526
Seems to be causing a bug in CorrelatedValuePropegation. Reverting
while the issue is investigated.
This reverts commit 6c667abf3294d61e4fbe1238e1755c79f7547f1b.
Partial Inlining identifies basic blocks that can be outlined into a
function. It is possible that an unreachable basic block is marked for
outlining. During costing of the outlined region, such unreachable basic
blocks are included as well. However, the CodeExtractor eliminates such
unreachable basic blocks and emits outlined function without them.
Thus, during costing of the outlined function, it is possible that the
cost of the outlined function comes out to be lesser than the cost of
outlined region, which triggers an assert.
Assertion `OutlinedFunctionCost >= Cloner.OutlinedRegionCost && "Outlined
function cost should be no less than the outlined region"' failed.
This patch adds code to eliminate unreachable blocks from the function
body before passing it on to be inlined. It also adds a test that checks
for behaviour of costing in case of unreachable basic blocks.
Discussion: https://discourse.llvm.org/t/incorrect-costing-in-partialinliner-if-ir-has-unreachable-basic-blocks/70163
Reviewed By: fhahn
Differential Revision: https://reviews.llvm.org/D149130
This broke two lit tests:
LLVM :: Transforms/LoopSimplify/merge-exits.ll
LLVM :: Transforms/PhaseOrdering/X86/vector-reductions.ll
see comment on the code review.
> Differential Revision: https://reviews.llvm.org/D149934
This reverts commit 2ba4cfd56769ab50c9c6f432f93265d7793bd1f2.
The interop types use the number of dependencies in the function
interface. Every other function uses an `i32` to count the number of
dependencies except for the initialization function. This leads to
codegen issues when the rest of the compiler passes in an `i32` that
then creates an invalid call. Fix this to be consistent with the other
uses.
Reviewed By: tianshilei1992
Differential Revision: https://reviews.llvm.org/D150156
This reverts commit 1ddfd1c8186735c62b642df05c505dc4907ffac4.
The original commit causes a Chrome build assertion failure with
ThinLTO: https://crbug.com/1443635
Fix the last runtime issue as some sequent comparisons need be spilted.
For the origin equal comparisons chain, the new spilted Icmp chain will
still be end with equal, while for the new not-equal comparisons chain,
the new spilted Icmp chain will still be end with equal, so should address
this carefully, see detail wih case partial_sequent_ne
Thanks for @aeubanks, @glandium and @ayzhao report the runtime issue
and carefully examine.
Fix https://github.com/llvm/llvm-project/issues/59740.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D141188
Adds an LTO option to indicate that whether we are linking with an
allocator that supports hot/cold operator new interfaces. If not,
at the start of the LTO backends any existing memprof hot/cold
attributes are removed from the IR, and we also remove memprof metadata
so that post-LTO inlining doesn't add any new attributes.
This is done via setting a new flag in the module summary index. It is
important to communicate via the index to the LTO backends so that
distributed ThinLTO handles this correctly, as they are invoked by
separate clang processes and the combined index is how we communicate
information from the LTO link. Specifically, for distributed ThinLTO the
LTO related processes look like:
```
# Thin link:
$ lld --thinlto-index-only obj1.o ... objN.o -llib ...
# ThinLTO backends:
$ clang -x ir obj1.o -fthinlto-index=obj1.o.thinlto.bc -c -O2
...
$ clang -x ir objN.o -fthinlto-index=objN.o.thinlto.bc -c -O2
```
It is during the thin link (lld --thinlto-index-only) that we have
visibility into linker dependences and want to be able to pass the new
option via -Wl,-supports-hot-cold-new. This will be recorded in the
summary indexes created for the distributed backend processes
(*.thinlto.bc) and queried from there, so that we don't need to know
during those individual clang backends what allocation library was
linked. Since in-process ThinLTO and regular LTO also use a combined
index, for consistency we query the flag out of the index in all LTO
backends.
Additionally, when the LTO option is disabled, exit early from the
MemProfContextDisambiguation handling performed during LTO, as this is
unnecessary.
Depends on D149117 and D149192.
Differential Revision: https://reviews.llvm.org/D149215
The generic cast to `BinaryOperator` can break if `V` is not a
`BinaryOperator` (i.e a `ConstantExpr`). This occurs in things like
PPC linux build.
This reverts commit fe733f54da6faca95070b36b1640dbca3e43d396.
The patch implements a helper function that matches and fold the following cases in the InstCombine pass:
bswap(logic_op(x, bswap(y))) -> logic_op(bswap(x), y)
bswap(logic_op(bswap(x), y)) -> logic_op(x, bswap(y))
bswap(logic_op(bswap(x), bswap(y))) -> logic_op(x, y) in multiuse case, which still reduces the number of instructions.
The helper function accepts bswap and bitreverse intrinsics. This patch folds the bswap cases and remain the bitreverse optimization for the future
Differential Revision: https://reviews.llvm.org/D149699
Fold the following case on IR InstCombine pass. This patch includes the new test cases for this optimization
bswap(logic_op(x, bswap(y))) -> logic_op(bswap(x), y)
bswap(logic_op(bswap(x), y)) -> logic_op(x, bswap(y))
bswap(logic_op(bswap(x), bswap(y))) -> logic_op(x, y) with multi-use
Differential Revision: https://reviews.llvm.org/D149577
This reverts commit 8b8466fd31e5a194fd8ba7a73a0f23d32f164318.
This is causing size regressions with -Oz and FullLTO. Revert while I
come up with a repro.
This reverts commit f09807ca9dda2f588298d8733e89a81105c88120, restoring
bfe7205975a63a605ff3faacd97fe4c1bf4c19b3 and follow on fix
e3e6bc699574550f2ed1de07f4e5bcdddaa65557, now that the nondeterminism
has been addressed by D149924.
Differential Revision: https://reviews.llvm.org/D141077
Multiple cases of instability in the cloning behavior occurred due to
iteration of maps indexed by pointers. Fix by changing the maps to
MapVector. This necessitated adding DenseMapInfo specializations for the
structure types used in the keys.
These were found while trying to commit patch 3 of the cloning
(bfe7205975a63a605ff3faacd97fe4c1bf4c19b3), but the second one turned
out to be in code committed in patch 2, but just exposed by a new test
added with patch 3. Specifically, the iteration in identifyClones().
Added the portion of the new test cases from patch 3 that only relied on
the already committed changes and exposed the issue.
Differential Revision: https://reviews.llvm.org/D149924
To generate cast instructions, the result type is needed. To allow
creating widened casts without underlying instruction, introduce a new
VPWidenCastRecipe that also holds the result type.
This functionality will be used in a follow-up patch to
implement truncateToMinimalBitwidths as VPlan-to-VPlan transform.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D149081
There are a few inaccuracies with how FuncSpec handles global
variables.
When specialisation on non-const global variables is disabled (the
default) the pass could nevertheless perform some specializations,
e.g. on a constant GEP expression, or on a SSA variable, for which the
Solver has determined it has the value of a global variable.
When specialisation on non-const global variables is enabled, the pass
would skip non-scalars, e.g. a global array, but this should be
completely inconsequential, a pointer is a pointer.
Reviewed By: SjoerdMeijer
Differential Revision: https://reviews.llvm.org/D149476
Change-Id: Ic73051b2f8602587306760bf2ec552e5860f8d39
Fix bug constants and sub instructions
When finding constants in a chain starting with the RHS operator of
sub instructions, we were negating the constant before zero extending
it, which is incorrect.
Unfortunately, I was unable to find a simple way to implement this
transformation correctly, so for now I just disabled this optimization
for constants that feed into the RHS of a sub.
Resolves#62379
Transformation from alive2.llvm.org:
define i16 @src(i8 %a, i8 %b, i8 %c) {
entry:
%0 = sub nuw nsw i8 %c, %a
%1 = sub nuw nsw i8 %b, %0
%2 = zext i8 %1 to i16
ret i16 %2
}
Before/Bad:
define i16 @tgt(i8 %a, i8 %b, i8 %c) {
entry:
%0 = zext i8 %a to i16
%1 = zext i8 %b to i16
%c_neg = sub i8 0, %c
%c_zext = zext i8 %c_neg to i16
%2 = sub i16 0, %0
%3 = sub i16 %1, %2
%4 = add i16 %3, %c_zext
ret i16 %4
}
Correct:
define i16 @tgt(i8 %a, i8 %b, i8 %c) {
entry:
%0 = zext i8 %a to i16
%1 = zext i8 %b to i16
%c_zext = zext i8 %c to i16
%c_neg = sub i16 0, %c_zext
%2 = sub i16 0, %0
%3 = sub i16 %1, %2
%4 = add i16 %3, %c_neg
ret i16 %4
}
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D149507
insts.
If the vectorizable GEP node is built, which should not be scheduled,
and at least one node is a non-gep instruction, need to insert the
vectorized instructions before the last instruction in the list, not
before the first one, otherwise the instructions may be emitted in the
wrong order.
This reverts commit bfe7205975a63a605ff3faacd97fe4c1bf4c19b3, and follow
on fix e3e6bc699574550f2ed1de07f4e5bcdddaa65557, due to some remaining
instability exposed by the bot enabling expensive checks:
https://lab.llvm.org/buildbot/#/builders/42/builds/9842
The asm in a naked function may reasonably expect the argument registers and the
return address register (if present) to be live.
When using -pg and -finstrument-functions, functions are instrumented by adding
a function call to `_mcount/__cyg_profile_func_enter/__cyg_profile_func_enter_bare`/etc,
which will clobber these registers. If the return address register is clobbered,
the function will be unable to return to the caller, possibly causing an
infinite loop.
```
__attribute__((naked)) void g() {
#if defined(__arm__)
__asm__("bx lr");
#else
__asm__("ret");
#endif
}
int main() { g(); }
```
It seems that the only one reasonable way to handle the combination is to
disable instrumenting for naked functions.
GCC PR: https://gcc.gnu.org/PR109707
Close https://github.com/llvm/llvm-project/issues/62504
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D149721
ugt/uge/sgt/sge test for the extension to isImpliedCondOperands
case from issue 62441 and 61393:
(X >> Z) <=(u) Y ==> X <=(to u) Y and (X > Y +_{nuw} 1) ==> X != Y
Differential Revision: https://reviews.llvm.org/D149512
This reverts commit 6fbf022908c104a380fd1854fb96eafc64509366, restoring
commit bf6ff4fd4b735afffc65f92a4a79f6610e7174c3 with a fix for a bot
failure due to a previously unstable iteration order.
Differential Revision: https://reviews.llvm.org/D141077
This patch adds a new preheader block the VPlan to place SCEV expansions
expansions like the trip count. This preheader block is disconnected
at the moment, as the bypass blocks of the skeleton are not yet modeled
in VPlan.
The preheader block is executed before skeleton creation, so the SCEV
expansion results can be used during skeleton creation. At the moment,
the trip count expression and induction steps are expanded in the new
preheader. The remainder of SCEV expansions will be moved gradually in
the future.
D147965 will update skeleton creation to use the steps expanded in the
pre-header to fix#58811.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D147964
The step is already expanded in the VPlan. Use this expansion instead.
This is a step towards modeling fixing up IV users in VPlan.
It also fixes a crash casued by SCEV-expanding the Step expression in
fixupIVUsers, where the IR is in an incomplete state
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D147963
This reverts commit bf6ff4fd4b735afffc65f92a4a79f6610e7174c3.
There is a bot failure where we are getting the correct remarks output
but in a different order. I'll need to investigate to see where we are
having nondeterministic behavior.
Applies cloning decisions to the IR, cloning functions and updating
calls. For Regular LTO, the IR is updated directly during function
assignment, whereas for ThinLTO it is recorded in the summary index
(a subsequent patch will apply to the IR via the index during the
ThinLTO backend.
The function assignment and cloning proceeds greedily, and we create new
clones as needed when we find an incompatible assignment of function
clones to callsite clones (i.e. when different callers need to invoke
different combinations of callsite clones).
Depends on D140949.
Differential Revision: https://reviews.llvm.org/D141077
Re-land D145441 with data layout upgrade code fixed to not break OpenMP.
This reverts commit 3f2fbe92d0f40bcb46db7636db9ec3f7e7899b27.
Differential Revision: https://reviews.llvm.org/D149776
Argument promotion mostly works on functions with more than one caller (otherwise the function would be inlined or is dead), so there's a good chance that performing this increases code size since we introduce loads at every call site. If any caller is marked minsize, bail.
We could compare the number of loads/stores removed from the function with the number of loads introduced in callers, but that's TODO.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D149768
