35400 Commits

Author SHA1 Message Date
Kazu Hirata
1daf2994de [llvm] Use StringRef::contains (NFC) 2023-12-23 22:21:52 -08:00
Florian Hahn
fbcf8a8cbb
[ConstraintElim] Add (UGE, var, 0) to unsigned system for new vars. (#76262)
The constraint system used for ConstraintElimination assumes all
varibles to be signed. This can cause missed optimization in the
unsigned system, due to missing the information that all variables are
unsigned (non-negative).

Variables can be marked as non-negative by adding Var >= 0 for all
variables. This is done for arguments on ConstraintInfo construction and
after adding new variables. This handles cases like the ones outlined in
https://discourse.llvm.org/t/why-does-llvm-not-perform-range-analysis-on-integer-values/74341

The original example shared above is now handled without this change,
but adding another variable means that instcombine won't be able to
simplify examples like https://godbolt.org/z/hTnra7zdY

Adding the extra variables comes with a slight compile-time increase
https://llvm-compile-time-tracker.com/compare.php?from=7568b36a2bc1a1e496ec29246966ffdfc3a8b87f&to=641a47f0acce7755e340447386013a2e086f03d9&stat=instructions:u

stage1-O3    stage1-ReleaseThinLTO    stage1-ReleaseLTO-g  stage1-O0-g
 +0.04%           +0.07%                   +0.05%           +0.02%
stage2-O3    stage2-O0-g    stage2-clang
  +0.05%         +0.05%        +0.05%

https://github.com/llvm/llvm-project/pull/76262
2023-12-23 15:53:48 +01:00
Kazu Hirata
03dc806b12 [Transforms] Use {DenseMap,SmallPtrSet}::contains (NFC) 2023-12-22 14:51:22 -08:00
Yingwei Zheng
345d7b1618
[InstCombine] Fold minmax intrinsic using KnownBits information (#76242)
This patch tries to fold minmax intrinsic by using
`computeConstantRangeIncludingKnownBits`.
Fixes regression in
[_karatsuba_rec:cpython/Modules/_decimal/libmpdec/mpdecimal.c](c31943af16/Modules/_decimal/libmpdec/mpdecimal.c (L5460-L5462)),
which was introduced by #71396.
See also
https://github.com/dtcxzyw/llvm-opt-benchmark/issues/16#issuecomment-1865875756.

Alive2 for splat vectors with undef: https://alive2.llvm.org/ce/z/J8hKWd
2023-12-23 04:41:32 +08:00
Nikita Popov
658b260dbf [Attributor] Don't construct pretty GEPs
Bring this in line with other transforms like ArgPromotion/SROA/
SCEVExpander and always produce canonical i8 GEPs.
2023-12-22 16:48:13 +01:00
Nikita Popov
c16559137c [IndVars] Avoid unnecessary truncate for zext nneg use
When performing sext IV widening, if one of the narrow uses is in
a zext nneg, we can treat it like an sext and avoid the insertion
of a trunc.
2023-12-22 11:30:17 +01:00
Nikita Popov
24e80d4cc5 [IndVars] Move "using namespace" to top-level scope (NFC) 2023-12-22 11:28:54 +01:00
Nikita Popov
54067c5fbe [SROA] Use memcpy if type size does not match store size
The original memcpy also copies the padding, so make sure that
this is still the case after splitting.

Fixes https://github.com/llvm/llvm-project/issues/64081.
2023-12-22 10:19:22 +01:00
Shan Huang
06a9c6738a
[CVP] Fix #76058: missing debug location in processSDiv function (#76118)
This PR fixes #76058.
2023-12-22 09:26:32 +01:00
Mikhail Gudim
411cba215a
Revert "[InstCombine] Extend foldICmpBinOp to add-like or. (#71… (#76167)
…396)"

This reverts commit 8773c9be3d9868288f1f46957945d50ff58e4e91.
2023-12-21 11:41:09 -05:00
Nikita Popov
b8df88b41c [InstCombine] Support zext nneg in gep of sext add fold
Add m_NNegZext() and m_SExtLike() matchers to make doing these kinds
of changes simpler in the future.
2023-12-21 16:38:09 +01:00
Chia
8674a023bc
[InstCombine] fold (Binop phi(a, b) phi(b, a)) -> (Binop a, b) while Binop is commutative. (#75765)
Alive2 proof: https://alive2.llvm.org/ce/z/2P8gq-
This patch closes #73905
2023-12-21 22:47:21 +08:00
Z572
e6d2bb0ed8
[InstCombine] Simplifiy (-x * y * -x) into (x * y * x) (#72953)
fix https://github.com/llvm/llvm-project/issues/72259
proof: https://alive2.llvm.org/ce/z/HsrmTC
2023-12-21 19:13:09 +08:00
boxu.zhang
d3ef867082
[LoopUnroll] Make UnrollMaxUpperBound to be overridable by target (#76029)
The UnrollMaxUpperBound should be target dependent, since different
chips provide different register set which brings different ability of
storing more temporary values of a program. So I add a MaxUpperBound
value in UnrollingPreference which can be override by targets. All uses
of UnrollMaxUpperBound are replaced with UP.MaxUpperBound.
 
The default value is still 8 and the command line argument
'--unroll-max-upperbound' takes final effect if provided.
2023-12-21 09:47:46 +01:00
Ivan R. Ivanov
39f09ec245
Invalidate analyses after running Attributor in OpenMPOpt (#74908)
Using the LoopInfo from OMPInfoCache after the Attributor ran resulted
in a crash due to it being in an invalid state.

---------

Co-authored-by: Ivan Radanov Ivanov <ivanov2@llnl.gov>
2023-12-20 15:01:21 -08:00
Mikhail Gudim
8773c9be3d
[InstCombine] Extend foldICmpBinOp to add-like or. (#71396)
InstCombine canonicalizes `add` to `or` when possible, but this makes
some optimizations applicable to `add` to be missed because they don't
realize that the `or` is equivalent to `add`.

In this patch we generalize `foldICmpBinOp` to handle such cases.
2023-12-20 17:28:57 -05:00
Justin Bogner
1f3d70a95a [Transforms][DXIL] Basic debug output in dxil-upgrade. NFC 2023-12-20 14:06:42 -07:00
Florian Hahn
18170d0f28
[ConstraintElim] Extend AND implication logic to support OR as well. (#76044)
Extend the logic check if an operand of an AND is implied by the other
to also support OR. This is done by checking if !op1 implies op2 or vice
versa.
2023-12-20 18:13:41 +01:00
Nikita Popov
8b8f2ef06e [MergeFunc] Fix comparison of constant expressions
Functions using different constant expressions were incorrectly
merged, because a lot of state was missing from the comparison,
including the opcode, the comparison predicate, the GEP element
type, as well as the inbounds, inrange and nowrap poison flags.
2023-12-20 15:59:02 +01:00
Alexey Bataev
a13148a880 [SLP]Fix PR75995: drop wrapping flags for resized wrapped binops.
If decided to resize the instruction, need to drop wrapping flags from
the resulting vector instructions to avoid incorrect
optimizations/assumptions later.
Fixes PR75995.
2023-12-20 06:51:39 -08:00
Florian Hahn
7cf499c63b
[ConstraintElim] Check if second op implies first for And. (#75750)
Generalize checkAndSecondOpImpliedByFirst to also check if the second
operand implies the first.
2023-12-20 11:58:35 +01:00
Arthur Eubanks
71a9292298 Revert "[SLP]Improve findReusedOrderedScalars processing, NFCI."
This reverts commit 44dc1e0baae7c4b8a02ba06dcf396d3d452aa873.

Causes non-determinism, see #75987.
2023-12-19 16:14:04 -08:00
Nikita Popov
92fc4b482f [InstCombine] Preserve poison in bitcast of insertelement fold
If the base was poison, retain the poison value.
2023-12-19 13:06:04 +01:00
Nikita Popov
67fd4e3408 [InstCombine] Check for poison instead of undef in shuffle transform
This one doesn't seem to make a practical difference because we'd
canonicalize undef -> poison in the relevant cases anywy.
2023-12-19 12:56:52 +01:00
Nikita Popov
f412b78ffc [InstCombine] Return poison if all lanes are poison 2023-12-19 12:43:23 +01:00
Nikita Popov
9d4557920f [InstCombine] Don't treat undef as poison in demanded element simplification
We can only set PoisonElts if the element is poison, not if it is
undef.
2023-12-19 12:26:48 +01:00
Alexey Bataev
00edad17c2 [SLP][NFC]Check for equal opcode preliminary to meet weak strict order
requirement, NFC.

This change does not affect functionality, just fixes the assertions in
some standard c++ library implementations.
2023-12-18 14:12:33 -08:00
Alexey Bataev
a7e10e6603 Revert "[SLP][NFC]Check for equal opcode preliminary to meet weak strict order"
This reverts commit 58a2c4e2f24ffce3966c3988d1a4ca7b04c52244 to fix the
issue detected by https://lab.llvm.org/buildbot/#/builders/233/builds/5424.
2023-12-18 12:35:52 -08:00
Nikita Popov
e400c59beb Revert "[InstCombine] Favour m_Poison in SimplifyDemandedVectorElts"
This reverts commit 318d5bff0b65aa7d52fc7004d49587416f0fb564.

Has incomplete test updates.
2023-12-18 18:08:57 +01:00
Antonio Frighetto
318d5bff0b [InstCombine] Favour m_Poison in SimplifyDemandedVectorElts
A miscompilation issue has been addressed with refined checking.
2023-12-18 17:28:39 +01:00
Nikita Popov
cd54c47424 [InstCombine] Match poison instead of undef in foldVectorBinop()
Some negative tests turn into positive tests, as the differences
between undef and poison propagation allow additional transforms.
2023-12-18 17:01:59 +01:00
Nikita Popov
ddd11537e2 [InstCombine] Match poison instead of undef in binop of same-mask shuffle fold 2023-12-18 16:41:38 +01:00
Nikita Popov
9d25b28b9e [InstCombine] Explicitly canonicalize splat shuffles to use poison RHS
This is usually handled by demanded elements simplification. However,
as that is not supported for scalable vectors, also handle it
explicitly here.
2023-12-18 16:30:40 +01:00
Nikita Popov
a5f3415533 [InstCombine] Replace non-demanded undef vector with poison
If an operand (esp to shufflevector or insertelement) is not
demanded, canonicalize it from undef to poison.
2023-12-18 16:12:37 +01:00
Nikita Popov
d0605e21af [InstCombine] Canonicalize splat shuffles to use poison operand
If the splat shuffle is represented using an undef RHS, replace it
with poison.
2023-12-18 15:57:49 +01:00
Alexey Bataev
58a2c4e2f2 [SLP][NFC]Check for equal opcode preliminary to meet weak strict order
requirement, NFC.

This change does not affect functionality, just fixes the assertions in
some standard c++ library implementations.
2023-12-18 06:42:03 -08:00
Nikita Popov
e93d324adb [InstCombine] Preserve poison in evaluateInDifferentElementOrder()
Don't unnecessarily replace poison with undef.
2023-12-18 15:36:22 +01:00
Nikita Popov
6c9813aa02 [InstCombine] Check for poison instead of undef in shuffle combine
Otherwise we may replace undef with poison.

Note that a lot of tests regressing here already have variants
that use poison instead of undef (often in a separate
inseltpoison file), which is why I'm not adjusting them to the
new pattern.
2023-12-18 15:19:16 +01:00
Paul Walker
dea16ebd26
[LLVM][IR] Replace ConstantInt's specialisation of getType() with getIntegerType(). (#75217)
The specialisation will not be valid when ConstantInt gains native
support for vector types.

This is largely a mechanical change but with extra attention paid to constant
folding, InstCombineVectorOps.cpp, LoopFlatten.cpp and Verifier.cpp to
remove the need to call `getIntegerType()`.

Co-authored-by: Nikita Popov <github@npopov.com>
2023-12-18 11:58:42 +00:00
Nikita Popov
465ecf872e [InstCombine] Rename UndefElts -> PoisonElts (NFC)
In line with updated shufflevector semantics, this represents the
poison elements rather than undef elements now. This commit is a
pure rename, without any logic changes.
2023-12-18 12:36:19 +01:00
Antonio Frighetto
151ddf07a6 [InstCombine] Stop propagating undef when element is demanded
Do not poison `undef` demanded elements in `SimplifyDemandedVectorElts`.
A miscompilation issue has been addressed with refined checking.

Proofs: https://alive2.llvm.org/ce/z/WA5oD5.
2023-12-17 21:41:03 +01:00
Yingwei Zheng
c59ea32f82
[InstCombine] Canonicalize icmp pred (X +/- C1), C2 into icmp pred X, C2 -/+ C1 with nowrap flag implied by with.overflow intrinsic (#75511)
This patch tries to canonicalize the pattern `Overflow | icmp pred Res,
C2` into `Overflow | icmp pred X, C2 +/- C1`, where `Overflow` and `Res`
are return values of `xxx.with.overflow X, C1`.
Alive2: https://alive2.llvm.org/ce/z/PhR_3S

Fixes #75360.
2023-12-16 17:58:57 +08:00
Mariusz Borsa
7e4ae28645
[Sanitizers] Don't inline unpoisoning of small stacks when inlining disabled (#75555)
When ASan.MaxInlinePoisoningSize == 0 , it means that no shadow memory
operations should be made via inlined instrumentation code,
but only via calls to shadow setting functions. This change fixes one
violation of this, which happened when the function allocas count
was small, i.e. less than 5 - in the code modifying the shadow just
before ret instruction.
We now explicitly check ASan.MaxInlinePoisoningSize , and if it's 0 then
we disallow inlining. It is required for the instrumentation
emitting code suitable for handling by ABI implementation.

rdar://119513720

Co-authored-by: Mariusz Borsa <m_borsa@apple.com>
2023-12-15 17:33:54 -08:00
Florian Hahn
cb56ba6350
[VPlan] Unswitch cond in replaceUsesWithIf in optimizeInductions (NFC)
As suggested post-commit for a00227197, unswitch the condition in
replaceUsesWithIf to simplify the check.
2023-12-15 20:26:36 +00:00
Arthur Eubanks
0d948827d7
[Instrumentation][X86] Limit setting large section flag to medium/large code models (#75542)
In #74514 and #74778 we marked various instrumentation-added sections as
large. This causes an extra PT_LOAD segment if using the small code
model. Since people using the small code model presumably aren't hitting
relocation limits, disable this when using the small code model to avoid
the extra segment.

This uses Module::getCodeModel() which isn't necessarily reliable since
it reads module metadata (which right now only the clang frontend sets),
but it would be nice to get to a point where we reliably put this sort
of information (e.g. PIC/code model/etc) in the IR. This requires
duplicating the existing tests since opt/llc currently don't set these
metadata. If we get to a point where they do set the code model metadata
based on command line arguments then we can deduplicate these tests.
2023-12-15 09:46:59 -08:00
Florian Hahn
9277ef12c0
[VPlan] Remove stale comment from optimizeInductions (NFC).
As suggested post-commit for a00227197, remove the stale comment,
SetVector is no longer used here.
2023-12-15 17:35:13 +00:00
Reid Kleckner
3e16152ebc [SLP] Fix OOB GEP index access for a no-op GEP
Issue is covered by existing test
llvm/test/Transforms/SLPVectorizer/RISCV/phi-const.ll

See issue #75632 for ideas for how we could catch these more easily in
the future.
2023-12-15 17:33:06 +00:00
Florian Hahn
b1bfe221e6
[VPlan] Remove unneeded getNumUsers calls in replaceAllUsesWith (NFC).
As suggested post-commit for a00227197, replace unnecessary getNumUsers
calls by boolean variable to indicate if users changed. Note that this
also requires an early exit to detect the case where a value is replaced
by itself.
2023-12-15 13:43:15 +00:00
Yingwei Zheng
af2d740d2d
[InstCombine] Treat lshr nneg as ashr in getBinOpsForFactorization (#75521)
This patch reinterprets `lshr nneg C, X` as `ashr nneg C, X` to allow
more factorization opportunities.
Fixes #70582.
2023-12-15 16:32:18 +08:00
Mircea Trofin
ed10fba1b2
[ThinLTO] Allow importing based on a workload definition (#74545)
An example of a "workload definition" would be "the transitive closure of functions actually called to satisfy a RPC request", i.e. a (typically significantly) smaller subset of the transitive closure (static + possible indirect call targets) of callees. This means this workload definition is a type of flat dynamic profile.

Producing one is not in scope - it can be produced offline from traces, or from sample-based profiles, etc.

This patch adds awareness to ThinLTO of such a concept. A workload is defined as a root and a list of functions. All function references are by-name (more readable than GUIDs). In the case of aliases, the expectation is the list contains all the alternative names.

The workload definitions are presented to the linker as a json file, containing a dictionary. The keys are the roots, the values are the list of functions.

The import list for a module defining a root will be the functions listed for it in the profile.

Using names this way assumes unique names for internal functions, i.e. clang's `-funique-internal-linkage-names`.

Note that the behavior affects the entire module where a root is defined (i.e. different workloads best be defined in different modules), and does not affect modules that don't define roots.
2023-12-14 15:10:48 -08:00