The current version of the test doesn't use any of the loads, so they
can be removed together with the mask of the interleave group.
Use some loaded values and store them, to prevent the mask from being
optimized away.
This is basically a copy and paste of the same logic we do in
`computeKnownBits` but adapts it for just `isKnownNonZero`.
Differential Revision: https://reviews.llvm.org/D157801
If the masked gathers can be reordered, it may produce strided access
pattern and the reordering does not affect common reodering, better to
try to reorder masked gathers for better performance.
Differential Revision: https://reviews.llvm.org/D157009
* Changes the default value of FuncSpecMaxIters from 1 to 10.
This allows specialization of recursive functions.
* Adds an option to control the maximum codesize growth per function.
* Measured ~45% performance uplift for SPEC2017:548.exchange2_r on
AWS Graviton3.
Differential Revision: https://reviews.llvm.org/D145819
SCEVExpander tries to reuse existing instruction with the same
SCEV expression. However, doing this replacement blindly is not
safe, because the instruction might be more poisonous.
What we were already doing is to drop poison-generating flags on
the reused instruction. But this is not the only way that more
poison can be introduced. The poison-generating flag might not
be directly on the reused instruction, or the poison contribution
might come from something like 0 * %var, which folds to 0 but can
still introduce poison.
This patch fixes the issue in a principled way, by determining which
values can contribute poison to the SCEV expression, and then
checking whether any additional values can contribute poison to the
instruction being reused. Poison-generating flags are dropped if
doing that enables reuse.
This is a pretty big hammer and does cause some regressions in
tests, but less than I would have expected. I wasn't able to come
up with a less intrusive fix that still satisfies the correctness
requirements.
Fixes https://github.com/llvm/llvm-project/issues/63763.
Fixes https://github.com/llvm/llvm-project/issues/63926.
Fixes https://github.com/llvm/llvm-project/issues/64333.
Fixes https://github.com/llvm/llvm-project/issues/63727.
Differential Revision: https://reviews.llvm.org/D158181
Refresh of the generic scheduling model to use A510 instead of A55.
Main benefits are to the little core, and introducing SVE scheduling information.
Changes tested on various OoO cores, no performance degradation is seen.
Differential Revision: https://reviews.llvm.org/D156799
When the 'int Four = Two;' is sunk into the 'case 0:' block,
the debug value for 'Three' is set incorrectly to 'poison'.
Reviewed By: aprantl
Differential Revision: https://reviews.llvm.org/D158171
AAIndirectCallInfo will collect information and specialize indirect call
sites. It is similar to our IndirectCallPromotion but runs as part of
the Attributor (so with assumed callee information). It also expands
more calls and let's the rest of the pipeline figure out what is UB, for
now. We use existing call promotion logic to improve the result,
otherwise we rely on the (implicit) function pointer cast.
This effectively "fixes" #60327 as it will undo the type punning early
enough for the inliner to work with the (now specialized, thus direct)
call.
Fixes: https://github.com/llvm/llvm-project/issues/60327
When updating the return type of deoptimize call during inline, we need
to drop incompatible return attributes. This bug was exposed once we
relaxed the contraint of adding the attributes through D156844. With
that change deoptimize (are not willreturn) will start having return
attributes added to it.
Fixes https://github.com/llvm/llvm-project/issues/64804.
Differential Revision: https://reviews.llvm.org/D158286
Currently after widening br(WC && (c1 && c2)) we insert assume of
(c1 && c2) which is joined to WC by And operation.
But we are going to support more flexible form of widenable branches
where WC could be placed arbitrary in the expression tree, e.g:
br(c1 && (c2 && WC)).
In that case we won't have (c1 && c2) in the IR. So we need to add
explicit (c1 && c2) and then create an assumption of it.
Reviewed By: anna
Differential Revision: https://reviews.llvm.org/D157502
Before, we allowed the condition to be simplified to a simple constant
only, otherwise we assumed all successors are live. Now we allow
multiple constants, and mark the default successor as dead accordingly.
We know that __kmpc_alloc_shared is by construction matched with a
unique __kmpc_free_shared. Making the compiler aware of these facts
helps to avoid mallocs/allocas.
Fixes: https://github.com/llvm/llvm-project/issues/64551
The entry point function is called as a regular function. Among other things, it
can be inlined, which would violate the semantics of entry_value in the IR.
Differential Revision: https://reviews.llvm.org/D158108
swifterror pointers can only be used as pointer operands of load & store
instructions (and as swifterror argument of a call). Sinking loads or
stores with swifterror pointer operands would require introducing a
select of of the pointer operands, which isn't allowed.
Check for this condition in canSinkInstructions.
Reviewed By: aschwaighofer
Differential Revision: https://reviews.llvm.org/D158083
When a convergencectrl token is passed to a convergent call, and the called
function in turn calls the entry intrinsic, the intrinsic is now now replaced
with the convergencectrl token.
The spec requires the following check:
A call from function F to function G can be inlined only if:
- at least one of F or G does not make any convergent calls, or,
- both F and G make the same kind of convergent calls: controlled or
uncontrolled.
But this change does not implement this complete check. A proper implemenation
require a whole new analysis that identifies convergence in every function. For
now, we skip that and just do a cursory check for the entry intrinsic. The
underlying assumption is that in a compiler flow that fully implements
convergence control tokens, there is no mixing of controlled and uncontrolled
convergent operations in the whole program.
This is a reboot of the original change D85606 by
Nicolai Haehnle <nicolai.haehnle@amd.com>.
Reviewed By: arsenm, nhaehnle
Differential Revision: https://reviews.llvm.org/D152431
This just expands on the existing logic that worked for `Or` and
applies it to any binop where `0` is the identity value on the RHS
i.e: `add`, `or`, `xor`, `shl`, etc...
Proofs For Some: https://alive2.llvm.org/ce/z/XZo6JD
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D148414
AFAICT, the trunc is not needed for correctness/performance and just
blocks what should be handlable cases.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D148413
The actual callsite we are adding to doesn't need to be
`willreturn`/`nounwind`, only ever instructions between the callsite
and the return.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D156844
We can do this by just querying attribute in the callsite itself. This
is both cleaner code and produces bette results.
Differential Revision: https://reviews.llvm.org/D156843
This check is redundant as it is covered by the call to
isPotentiallyReachable.
Depends on D155726.
Differential Revision: https://reviews.llvm.org/D155718
Add test coverage for sinking/hoisting loads/stores with swifterror
pointers. Currently this isn't handled correctly by SimplifyCFG and
causes a verifier error.
Relative to the previous attempt, this also adjusts RegionInfo
verification to allow unreachable predecessors.
-----
If a block in the CHR region has an unreachable predecessor, then
there will be no edge from that predecessor to the newly cloned
block. However, a phi node entry for it will be left behind. Make
sure that these incoming blocks get dropped as well.
Fixes https://github.com/llvm/llvm-project/issues/64594.
Differential Revision: https://reviews.llvm.org/D157621
