35408 Commits

Author SHA1 Message Date
Alexey Bataev
bc8c4bbd79
[SLP][TTI][X86]Add addsub pattern cost estimation. (#76461)
SLP/TTI do not know about the cost estimation for addsub pattern,
supported by X86. Previously the support for pattern detection was added
(seeTTI::isLegalAltInstr), but the cost still did not estimated
properly.
2023-12-27 15:57:21 -05:00
Craig Topper
7f1c8fc25a
[InstCombine] Use ConstantInt::getSigned to sign extend -2 for large types. (#76464)
Using ContantInt::get will zero extend.

Fixes #76441
2023-12-27 12:27:12 -08:00
Yingwei Zheng
aacff347af
[InstCombine] Simplify icmp pred (sdiv exact X, C), (sdiv exact Y, C) into icmp pred X, Y when C is positive (#76409)
Alive2: https://alive2.llvm.org/ce/z/u49dQ9
It will improve the codegen of `std::_Vector_base<T>::~_Vector_base()` when `sizeof(T)` is not a power of 2.

NOTE: We can also fold `icmp signed-pred (sdiv exact X, C), (sdiv exact Y, C)` into `icmp signed-pred (sdiv exact Y, C), (sdiv exact X, C)` when C is negative. But I don't think it enables more optimizations for real-world applications.
2023-12-27 06:06:16 +08:00
Yingwei Zheng
4358e6e0c5
[FuncAttrs] Infer norecurse for funcs with calls to nocallback callees (#76372)
This patch adds missing `norecurse` attrs to funcs that only call intrinsics with `nocallback` attrs.
Fixes the regression found in https://github.com/dtcxzyw/llvm-opt-benchmark/pull/45#discussion_r1436148743.
The function loses `norecurse` attr because it calls `@llvm.fabs.f64`, which is not marked as `norecurse`.

Since `norecurse` is not a default attribute of intrinsics and it is
ambiguous for intrinsics, I decided to use the existing `callback`
attributes.

> nocallback
This attribute indicates that the function is only allowed to jump back
into caller’s module by a return or an exception, and is not allowed to
jump back by invoking a callback function, a direct, possibly
transitive, external function call, use of longjmp, or other means. It
is a compiler hint that is used at module level to improve dataflow
analysis, dropped during linking, and has no effect on functions defined
in the current module.

See also https://llvm.org/docs/LangRef.html#function-attributes.
2023-12-27 03:16:43 +08:00
Yingwei Zheng
ff76627aeb
[InstCombine] Fix type mismatch between cond and value in foldSelectToCopysign (#76343)
This patch fixes the miscompilation when we try to bitcast a floating point vector into an integer scalar.
2023-12-26 00:04:06 +08:00
Yingwei Zheng
0d454d6e59
[InstCombine] Fold xor of icmps using range information (#76334)
This patch folds xor of icmps into a single comparison using range-based reasoning as `foldAndOrOfICmpsUsingRanges` does.
Fixes #70928.
2023-12-25 07:14:31 +08:00
Craig Topper
d8ddcae547 [LSR] Fix typo in debug message where backspace escape was used instead of new line. 2023-12-24 10:35:27 -08:00
Benjamin Kramer
9423e45987 [ProfileData] Copy CallTargetMaps a bit less. NFCI 2023-12-24 17:48:18 +01:00
Kazu Hirata
1daf2994de [llvm] Use StringRef::contains (NFC) 2023-12-23 22:21:52 -08:00
Florian Hahn
fbcf8a8cbb
[ConstraintElim] Add (UGE, var, 0) to unsigned system for new vars. (#76262)
The constraint system used for ConstraintElimination assumes all
varibles to be signed. This can cause missed optimization in the
unsigned system, due to missing the information that all variables are
unsigned (non-negative).

Variables can be marked as non-negative by adding Var >= 0 for all
variables. This is done for arguments on ConstraintInfo construction and
after adding new variables. This handles cases like the ones outlined in
https://discourse.llvm.org/t/why-does-llvm-not-perform-range-analysis-on-integer-values/74341

The original example shared above is now handled without this change,
but adding another variable means that instcombine won't be able to
simplify examples like https://godbolt.org/z/hTnra7zdY

Adding the extra variables comes with a slight compile-time increase
https://llvm-compile-time-tracker.com/compare.php?from=7568b36a2bc1a1e496ec29246966ffdfc3a8b87f&to=641a47f0acce7755e340447386013a2e086f03d9&stat=instructions:u

stage1-O3    stage1-ReleaseThinLTO    stage1-ReleaseLTO-g  stage1-O0-g
 +0.04%           +0.07%                   +0.05%           +0.02%
stage2-O3    stage2-O0-g    stage2-clang
  +0.05%         +0.05%        +0.05%

https://github.com/llvm/llvm-project/pull/76262
2023-12-23 15:53:48 +01:00
Kazu Hirata
03dc806b12 [Transforms] Use {DenseMap,SmallPtrSet}::contains (NFC) 2023-12-22 14:51:22 -08:00
Yingwei Zheng
345d7b1618
[InstCombine] Fold minmax intrinsic using KnownBits information (#76242)
This patch tries to fold minmax intrinsic by using
`computeConstantRangeIncludingKnownBits`.
Fixes regression in
[_karatsuba_rec:cpython/Modules/_decimal/libmpdec/mpdecimal.c](c31943af16/Modules/_decimal/libmpdec/mpdecimal.c (L5460-L5462)),
which was introduced by #71396.
See also
https://github.com/dtcxzyw/llvm-opt-benchmark/issues/16#issuecomment-1865875756.

Alive2 for splat vectors with undef: https://alive2.llvm.org/ce/z/J8hKWd
2023-12-23 04:41:32 +08:00
Nikita Popov
658b260dbf [Attributor] Don't construct pretty GEPs
Bring this in line with other transforms like ArgPromotion/SROA/
SCEVExpander and always produce canonical i8 GEPs.
2023-12-22 16:48:13 +01:00
Nikita Popov
c16559137c [IndVars] Avoid unnecessary truncate for zext nneg use
When performing sext IV widening, if one of the narrow uses is in
a zext nneg, we can treat it like an sext and avoid the insertion
of a trunc.
2023-12-22 11:30:17 +01:00
Nikita Popov
24e80d4cc5 [IndVars] Move "using namespace" to top-level scope (NFC) 2023-12-22 11:28:54 +01:00
Nikita Popov
54067c5fbe [SROA] Use memcpy if type size does not match store size
The original memcpy also copies the padding, so make sure that
this is still the case after splitting.

Fixes https://github.com/llvm/llvm-project/issues/64081.
2023-12-22 10:19:22 +01:00
Shan Huang
06a9c6738a
[CVP] Fix #76058: missing debug location in processSDiv function (#76118)
This PR fixes #76058.
2023-12-22 09:26:32 +01:00
Mikhail Gudim
411cba215a
Revert "[InstCombine] Extend foldICmpBinOp to add-like or. (#71… (#76167)
…396)"

This reverts commit 8773c9be3d9868288f1f46957945d50ff58e4e91.
2023-12-21 11:41:09 -05:00
Nikita Popov
b8df88b41c [InstCombine] Support zext nneg in gep of sext add fold
Add m_NNegZext() and m_SExtLike() matchers to make doing these kinds
of changes simpler in the future.
2023-12-21 16:38:09 +01:00
Chia
8674a023bc
[InstCombine] fold (Binop phi(a, b) phi(b, a)) -> (Binop a, b) while Binop is commutative. (#75765)
Alive2 proof: https://alive2.llvm.org/ce/z/2P8gq-
This patch closes #73905
2023-12-21 22:47:21 +08:00
Z572
e6d2bb0ed8
[InstCombine] Simplifiy (-x * y * -x) into (x * y * x) (#72953)
fix https://github.com/llvm/llvm-project/issues/72259
proof: https://alive2.llvm.org/ce/z/HsrmTC
2023-12-21 19:13:09 +08:00
boxu.zhang
d3ef867082
[LoopUnroll] Make UnrollMaxUpperBound to be overridable by target (#76029)
The UnrollMaxUpperBound should be target dependent, since different
chips provide different register set which brings different ability of
storing more temporary values of a program. So I add a MaxUpperBound
value in UnrollingPreference which can be override by targets. All uses
of UnrollMaxUpperBound are replaced with UP.MaxUpperBound.
 
The default value is still 8 and the command line argument
'--unroll-max-upperbound' takes final effect if provided.
2023-12-21 09:47:46 +01:00
Ivan R. Ivanov
39f09ec245
Invalidate analyses after running Attributor in OpenMPOpt (#74908)
Using the LoopInfo from OMPInfoCache after the Attributor ran resulted
in a crash due to it being in an invalid state.

---------

Co-authored-by: Ivan Radanov Ivanov <ivanov2@llnl.gov>
2023-12-20 15:01:21 -08:00
Mikhail Gudim
8773c9be3d
[InstCombine] Extend foldICmpBinOp to add-like or. (#71396)
InstCombine canonicalizes `add` to `or` when possible, but this makes
some optimizations applicable to `add` to be missed because they don't
realize that the `or` is equivalent to `add`.

In this patch we generalize `foldICmpBinOp` to handle such cases.
2023-12-20 17:28:57 -05:00
Justin Bogner
1f3d70a95a [Transforms][DXIL] Basic debug output in dxil-upgrade. NFC 2023-12-20 14:06:42 -07:00
Florian Hahn
18170d0f28
[ConstraintElim] Extend AND implication logic to support OR as well. (#76044)
Extend the logic check if an operand of an AND is implied by the other
to also support OR. This is done by checking if !op1 implies op2 or vice
versa.
2023-12-20 18:13:41 +01:00
Nikita Popov
8b8f2ef06e [MergeFunc] Fix comparison of constant expressions
Functions using different constant expressions were incorrectly
merged, because a lot of state was missing from the comparison,
including the opcode, the comparison predicate, the GEP element
type, as well as the inbounds, inrange and nowrap poison flags.
2023-12-20 15:59:02 +01:00
Alexey Bataev
a13148a880 [SLP]Fix PR75995: drop wrapping flags for resized wrapped binops.
If decided to resize the instruction, need to drop wrapping flags from
the resulting vector instructions to avoid incorrect
optimizations/assumptions later.
Fixes PR75995.
2023-12-20 06:51:39 -08:00
Florian Hahn
7cf499c63b
[ConstraintElim] Check if second op implies first for And. (#75750)
Generalize checkAndSecondOpImpliedByFirst to also check if the second
operand implies the first.
2023-12-20 11:58:35 +01:00
Arthur Eubanks
71a9292298 Revert "[SLP]Improve findReusedOrderedScalars processing, NFCI."
This reverts commit 44dc1e0baae7c4b8a02ba06dcf396d3d452aa873.

Causes non-determinism, see #75987.
2023-12-19 16:14:04 -08:00
Nikita Popov
92fc4b482f [InstCombine] Preserve poison in bitcast of insertelement fold
If the base was poison, retain the poison value.
2023-12-19 13:06:04 +01:00
Nikita Popov
67fd4e3408 [InstCombine] Check for poison instead of undef in shuffle transform
This one doesn't seem to make a practical difference because we'd
canonicalize undef -> poison in the relevant cases anywy.
2023-12-19 12:56:52 +01:00
Nikita Popov
f412b78ffc [InstCombine] Return poison if all lanes are poison 2023-12-19 12:43:23 +01:00
Nikita Popov
9d4557920f [InstCombine] Don't treat undef as poison in demanded element simplification
We can only set PoisonElts if the element is poison, not if it is
undef.
2023-12-19 12:26:48 +01:00
Alexey Bataev
00edad17c2 [SLP][NFC]Check for equal opcode preliminary to meet weak strict order
requirement, NFC.

This change does not affect functionality, just fixes the assertions in
some standard c++ library implementations.
2023-12-18 14:12:33 -08:00
Alexey Bataev
a7e10e6603 Revert "[SLP][NFC]Check for equal opcode preliminary to meet weak strict order"
This reverts commit 58a2c4e2f24ffce3966c3988d1a4ca7b04c52244 to fix the
issue detected by https://lab.llvm.org/buildbot/#/builders/233/builds/5424.
2023-12-18 12:35:52 -08:00
Nikita Popov
e400c59beb Revert "[InstCombine] Favour m_Poison in SimplifyDemandedVectorElts"
This reverts commit 318d5bff0b65aa7d52fc7004d49587416f0fb564.

Has incomplete test updates.
2023-12-18 18:08:57 +01:00
Antonio Frighetto
318d5bff0b [InstCombine] Favour m_Poison in SimplifyDemandedVectorElts
A miscompilation issue has been addressed with refined checking.
2023-12-18 17:28:39 +01:00
Nikita Popov
cd54c47424 [InstCombine] Match poison instead of undef in foldVectorBinop()
Some negative tests turn into positive tests, as the differences
between undef and poison propagation allow additional transforms.
2023-12-18 17:01:59 +01:00
Nikita Popov
ddd11537e2 [InstCombine] Match poison instead of undef in binop of same-mask shuffle fold 2023-12-18 16:41:38 +01:00
Nikita Popov
9d25b28b9e [InstCombine] Explicitly canonicalize splat shuffles to use poison RHS
This is usually handled by demanded elements simplification. However,
as that is not supported for scalable vectors, also handle it
explicitly here.
2023-12-18 16:30:40 +01:00
Nikita Popov
a5f3415533 [InstCombine] Replace non-demanded undef vector with poison
If an operand (esp to shufflevector or insertelement) is not
demanded, canonicalize it from undef to poison.
2023-12-18 16:12:37 +01:00
Nikita Popov
d0605e21af [InstCombine] Canonicalize splat shuffles to use poison operand
If the splat shuffle is represented using an undef RHS, replace it
with poison.
2023-12-18 15:57:49 +01:00
Alexey Bataev
58a2c4e2f2 [SLP][NFC]Check for equal opcode preliminary to meet weak strict order
requirement, NFC.

This change does not affect functionality, just fixes the assertions in
some standard c++ library implementations.
2023-12-18 06:42:03 -08:00
Nikita Popov
e93d324adb [InstCombine] Preserve poison in evaluateInDifferentElementOrder()
Don't unnecessarily replace poison with undef.
2023-12-18 15:36:22 +01:00
Nikita Popov
6c9813aa02 [InstCombine] Check for poison instead of undef in shuffle combine
Otherwise we may replace undef with poison.

Note that a lot of tests regressing here already have variants
that use poison instead of undef (often in a separate
inseltpoison file), which is why I'm not adjusting them to the
new pattern.
2023-12-18 15:19:16 +01:00
Paul Walker
dea16ebd26
[LLVM][IR] Replace ConstantInt's specialisation of getType() with getIntegerType(). (#75217)
The specialisation will not be valid when ConstantInt gains native
support for vector types.

This is largely a mechanical change but with extra attention paid to constant
folding, InstCombineVectorOps.cpp, LoopFlatten.cpp and Verifier.cpp to
remove the need to call `getIntegerType()`.

Co-authored-by: Nikita Popov <github@npopov.com>
2023-12-18 11:58:42 +00:00
Nikita Popov
465ecf872e [InstCombine] Rename UndefElts -> PoisonElts (NFC)
In line with updated shufflevector semantics, this represents the
poison elements rather than undef elements now. This commit is a
pure rename, without any logic changes.
2023-12-18 12:36:19 +01:00
Antonio Frighetto
151ddf07a6 [InstCombine] Stop propagating undef when element is demanded
Do not poison `undef` demanded elements in `SimplifyDemandedVectorElts`.
A miscompilation issue has been addressed with refined checking.

Proofs: https://alive2.llvm.org/ce/z/WA5oD5.
2023-12-17 21:41:03 +01:00
Yingwei Zheng
c59ea32f82
[InstCombine] Canonicalize icmp pred (X +/- C1), C2 into icmp pred X, C2 -/+ C1 with nowrap flag implied by with.overflow intrinsic (#75511)
This patch tries to canonicalize the pattern `Overflow | icmp pred Res,
C2` into `Overflow | icmp pred X, C2 +/- C1`, where `Overflow` and `Res`
are return values of `xxx.with.overflow X, C1`.
Alive2: https://alive2.llvm.org/ce/z/PhR_3S

Fixes #75360.
2023-12-16 17:58:57 +08:00