28806 Commits

Author SHA1 Message Date
David Green
46541a3636 [ARM] Add a extra MVE low-trip-count loop. NFC
This makes use of half floats, which makes the masked stores expensive.
2024-05-23 21:50:47 +01:00
Matt Arsenault
2d61692d4b
Attributor: Do not assume function context in AANoCapture (#91462)
If the calling function has the null_pointer_is_valid attribute, somehow
a null constant reaches here. I'm not sure why exactly, it doesn't
happen for other types of constants.

Fixes #87856
2024-05-23 19:31:33 +02:00
Billy Zhu
910292c3ac
[LLVM][Coroutines] Check variable decl scope instead of optimization level for hoisted DbgDeclare Loc (#92978)
Minor patch following up on
https://github.com/llvm/llvm-project/pull/75402.

The more generalized version of [this
error](https://github.com/llvm/llvm-project/pull/75104#issuecomment-1853497609)
is whenever we have a debug variable created in one subprogram scope
inlined into another subprogram scope. So instead of checking
optimization level, it is safer to just check whether the subprogram
scope of the variable matches the subprogram scope of the hoisted
position.
2024-05-23 10:01:37 -07:00
Chuanqi Xu
44430de0f1
[Coroutines] Always set the calling convention of generated resuming call from 'llvm.coro.await.suspend.handle' as fast (#93167)
See the post commit message in
https://github.com/llvm/llvm-project/pull/89751

We met a regression due to a change of calling convention of this patch.
Previously, the calling convention of indirect resume calls is always
fast.

And in this patch, although we tried to take care of it in the cloner,
we forget the case that we have to update the resuming calls in the ramp
functions. So this is the root cause of the downstream failure.

This patch tries to mark the generated resuming calls as fast
immediately after they got created to make sure the calling convention
is correct.
2024-05-23 22:32:54 +08:00
Nikita Popov
27392a35ef
[SCCP] Don't allow undef ranges when performing operations (#93163)
When performing some range operation (e.g. and) on a constant range that
includes undef, we currently just ignore the undef value, which is
obviously incorrect. Instead, we can do one of two things:
 * Say that the result range also includes undef.
 * Treat undef as a full range.

This patch goes with the second approach -- I'd expect it to be a bit
better overall, e.g. it allows preserving the fact that a zext of a
range with undef isn't a full range.

Fixes https://github.com/llvm/llvm-project/issues/93096.
2024-05-23 15:14:34 +02:00
Chuanqi Xu
8d4d85de1e Revert "[Coroutines] Always set the calling convention of generated resuming call from 'llvm.coro.await.suspend.handle' as fast"
This reverts commit 31f1590e4fb324c43dc36199587c453e27b6f6fa.

It looks like some bots are not happy about the FileChecks
2024-05-23 18:31:55 +08:00
Chuanqi Xu
31f1590e4f [Coroutines] Always set the calling convention of generated resuming call from 'llvm.coro.await.suspend.handle' as fast
See the post commit message in
https://github.com/llvm/llvm-project/pull/89751

We met a regression due to a change of calling convention of this patch.
Previously, the calling convention of indirect resume calls is always
fast.

And in this patch, although we tried to take care of it in the cloner,
we forget the case that we have to update the resuming calls in the
ramp functions. So this is the root cause of the downstream failure.

This patch tries to mark the generated resuming calls as fast
immediately after they got created to make sure the calling convention
is correct.
2024-05-23 18:12:57 +08:00
Nikita Popov
5e06050efd [SCCP] Add tests for #93096 (NFC) 2024-05-23 11:53:51 +02:00
Matt Arsenault
d81170873c
AtomicExpand: Preserve metadata when expanding partword RMW (#89769)
This will be important for AMDGPU in a future patch.
2024-05-23 10:04:47 +02:00
Nikita Popov
f64732195c [FunctionAttrs] Regenerate test checks (NFC) 2024-05-23 08:45:38 +02:00
Nikita Popov
ca478bc6cc
[SCEV] Support ule/sle exit counts via widening (#92206)
If we have an exit condition of the form IV <= Limit, we will first try
to convert it into IV < Limit+1 or IV-1 < Limit based on range info (in
icmp simplification). If that fails, we try to convert it to IV < Limit
+ 1 based on controlling exits in non-infinite loops.

However, if all else fails, we can still determine the exit count by
rewriting to ext(IV) < ext(Limit) + 1, where the zero/sign extension
ensures that the addition does not overflow.

Proof: https://alive2.llvm.org/ce/z/iR-iYd
2024-05-23 07:54:08 +02:00
Johannes Doerfert
5ec91b392d
[AttributorLight] Without liveness checks, look at all functions (#91004) 2024-05-23 07:28:07 +02:00
Freddy Ye
aa4069ea96
Revert "[X86] Remove knl/knm specific ISAs supports (#92883)" (#93123)
This reverts commit 282d2ab58f56c89510f810a43d4569824a90c538.
2024-05-23 10:25:23 +08:00
Freddy Ye
282d2ab58f
[X86] Remove knl/knm specific ISAs supports (#92883)
Cont. patch after https://github.com/llvm/llvm-project/pull/75580
2024-05-23 09:46:44 +08:00
Daniel Thornburgh
8baf96f306
Revert "[IR] Avoid creating icmp/fcmp constant expressions" (#93087)
Reverts llvm/llvm-project#92885 due to LLDB CI breakages.
2024-05-22 11:27:55 -07:00
Alexey Bataev
554c47c8e9 [SLP]Fix undef poison vector values shuffles with poisonous vectors.
If trying to find vector value in shuffling of the extractelements and
one of the vector values is undef value, need to generate real mask value
for such vector and either undef vector, or incoming second vector, if
  non-poisonous.
2024-05-22 10:41:57 -07:00
Sirraide
c44fa3e8a9
[Clang] Refactor __attribute__((assume)) (#84934)
This is a followup to #81014 and #84582: Before this patch, Clang 
would accept `__attribute__((assume))` and `[[clang::assume]]` as 
nonstandard spellings for the `[[omp::assume]]` attribute; this 
resulted in a potentially very confusing name clash with C++23’s 
`[[assume]]` attribute (and GCC’s `assume` attribute with the same
semantics).

This pr replaces every usage of `__attribute__((assume))`  with 
`[[omp::assume]]` and makes `__attribute__((assume))` and 
`[[clang::assume]]` alternative spellings for C++23’s `[[assume]]`; 
this shouldn’t cause any problems due to differences in appertainment
and because almost no-one was using this variant spelling to begin
with (a use in libclc has already been changed to use a different
attribute).
2024-05-22 17:58:48 +02:00
Alexey Bataev
30d484fa99 [SLP]Fix a crash when trying to convert masked gather nodes to strided.
Need to check if the loads node is masked gather. Only vectorized loads
can be converted to strided.
2024-05-22 08:08:56 -07:00
Nikita Popov
0748a98ab0 [InstCombine] Handle ConstantFoldCompareInstOperands() failure
This function will return nullptr instead of returning a constant
expression now, so be sure to handle that.

Fixes https://github.com/llvm/llvm-project/issues/93017.
2024-05-22 14:33:50 +02:00
Florian Hahn
ba0e871db8
[ConstraintElim] Look through SExt with precond Op sge 0.
Look through SExt with a precondition that the operand is signed
positive.

https://alive2.llvm.org/ce/z/zvVVHj
2024-05-22 13:11:04 +01:00
Paul Walker
25c021a863 [NFC] Fix typo in llvm/test/Transforms/Util/add-TLI-mappings.ll 2024-05-22 10:37:06 +00:00
Carlos Alberto Enciso
3c5738f3ec
Revert "[indvars] Missing variables at Og (#88270)" (#93016)
This reverts commit 89e1f7784be40bea96d5e65919ce8d34151c1d69.

https://github.com/llvm/llvm-project/pull/88270#discussion_r1609559724
https://github.com/llvm/llvm-project/pull/88270#discussion_r1609552972

Main concerns from @nikic are the interaction between the
'IndVars' and 'LoopDeletion' passes, increasing build times
and adding extra complexity.
2024-05-22 11:36:10 +01:00
Simon Pilgrim
0873b4ca29 [LoopVectorize] optimal-epilog-vectorization-profitability.ll - fix LABLE -> LABEL typo
Typo identified in #91854
2024-05-22 11:07:24 +01:00
Florian Hahn
a56e6dfd2e
[LV] Add test for header mask and invariant compare cost-modeling.
Additional test coverage for the VPlan-based cost model work.
2024-05-22 09:57:35 +01:00
Sander de Smalen
1015f51dd9
[AArch64] NFC: Rename -force-streaming-compatible-sve to -force-streaming-compatible (#92774)
The behaviour of the flag should be equivalent to
__arm_streaming_compatible.

At the moment, the name suggests that '-force-streaming-compatible-sve'
on its own (i.e. without specifying `+sve`) enables the compiler to use
the streaming-compatible subset of SVE instructions, but the semantics
merely are that the function can be called with either PSTATE.SM=0 or
PSTATE.SM=1.
2024-05-22 07:58:54 +01:00
Carlos Alberto Enciso
89e1f7784b
[indvars] Missing variables at Og (#88270)
https://bugs.llvm.org/show_bug.cgi?id=51735
https://github.com/llvm/llvm-project/issues/51077

In the given test case:
 ```
4 ...
 5 void bar() {
 6   int End = 777;
 7   int Index = 27;
 8   char Var = 1;
 9   for (; Index < End; ++Index)
10     ;
11   nop(Index);
12 }
13 ...
```
Missing local variable `Index` after loop `Induction Variable Elimination`.
When adding a breakpoint at line `11`, LLDB does not have information
on the variable. But it has info on `Var` and `End`.
2024-05-22 07:09:50 +01:00
Nikita Popov
108575f02e
[IR] Avoid creating icmp/fcmp constant expressions (#92885)
Do not create icmp/fcmp constant expressions in IRBuilder etc anymore,
i.e. treat them as "undesirable". This is in preparation for removing
them entirely.

Part of:
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179
2024-05-22 07:40:08 +02:00
Alan Zhao
0c8bc08868
Reapply "[coro][CoroSplit] Use llvm.lifetime.end to compute putting objects on the frame vs the stack (#90265) (#91372)
This reverts commit 924384161ffceda08099536dd07a953299a69b53.

This reland addresses the performance regressions seen in #90265 by
retaining the original definition of
`isPotentiallyReachableFromMany(...)` instead of reimplementing it with
`isManyPotentiallyReachableFromMany(...)`.

Fixes #86580
2024-05-21 09:48:03 -07:00
Jeffrey Byrnes
ea43a30899
[AMDGPU] Vectorize more 16 bit shuffles (#90648)
In the case of larger vectors, we should still prefer the vectorized
version (i.e. shufflevector vs extract/insert chains).

In arithmetic chains, vectorization results in chains of packed math
instructions (as opposed to unpack/repack & scalarized arithmetic):
https://godbolt.org/z/c5onaf6G5

In chains with PHIs, vectorization again removes the unnecessary pack /
repack code around BBs: https://godbolt.org/z/vz7zYzvhs
2024-05-21 09:21:36 -07:00
Nikita Popov
d0e0205bfc [InstCombine] Check for poison instead of undef in single shuffle fold
Otherwise we'll convert undef to poison. Alive2 was already flagging
the existing test8 test as a miscompile.
2024-05-21 16:03:20 +02:00
Nikita Popov
8f1c984325 [InstCombine] Check for poison instead of undef in shuffle of unop transform
Otherwise this may not actually be a single-source shuffle.
2024-05-21 15:43:50 +02:00
Nikita Popov
b8e3d80216 [InstCombine] Add test for incorrect shuffle of unop transform (NFC) 2024-05-21 15:42:26 +02:00
Nikita Popov
ecd269e830 [InstCombine] Check for poison instead of undef in splat shuffle fold
We can't canonicalize these to a splat shuffle, as doing so would
convert undef -> poison.
2024-05-21 15:21:31 +02:00
Nikita Popov
557a0be3af [InstCombine] Add test for splat shuffle miscompile (NFC) 2024-05-21 15:21:31 +02:00
Nikita Popov
263224e448 [InstCombine] Require poison operand in canEvaluateShuffled transform
This transform works on single-source shuffles, which require that
the second operand is poison, not undef. Otherwise we may convert
undef to poison.

Fixes https://github.com/llvm/llvm-project/issues/92887.
2024-05-21 15:00:01 +02:00
Nikita Popov
f78b1a4086 [InstCombine] Add test for #92887 (NFC) 2024-05-21 15:00:01 +02:00
Florian Hahn
1b377dbeb7
[LAA] Check accesses don't overlap early to determine NoDep (#92307)
Use getStartAndEndForAccess to compute the start and end of both src 
and sink (factored out to helper in bce3680f45b57f). If they do not
overlap (i.e. SrcEnd <= SinkStart || SinkEnd <= SrcStart), there is no
dependence, regardless of stride.

PR: https://github.com/llvm/llvm-project/pull/92307
2024-05-21 11:00:11 +01:00
Florian Hahn
352dc7d4bb
[LV] Propagate PredicatedBBsAfterVectorization to predecessors.
This fixes some cases where predicated BBs where missed previously,
leading to under-estimating the cost of those blocks.
2024-05-21 10:27:32 +01:00
Nikita Popov
74f1e29133 [Tests] Regenerate test checks (NFC)
To minimize future diffs.
2024-05-21 10:28:40 +02:00
hev
1e86e92428
[LoongArch] Enable interleaved vectorization (#92629)
This PR enables interleaved vectorization for LoongArch, with a default
interleaving factor of `2`.
2024-05-21 15:31:02 +08:00
Ramkumar Ramachandra
63d81311a2
VectorCombine: add tests written for InstSimplify (#92776)
2141907 (InstSimplify: increase shufflevector test coverage) was
recently merged as a pre-commit test for some work that was misguided.
It turns out that InstSimplify can never work on those tests, but the
tests are useful nevertheless; move them to VectorCombine to support the
development of VectorCombine::foldShuffleToIdentity.
2024-05-21 08:06:34 +01:00
Nikita Popov
9c78481717
[SeparateConstOffsetFromGEP] Support multiple indices in reorderGEP (#92339)
The code was essentially already ready to handle multiple indices -- we
only need to adjust the non-negative index check to check all indices,
instead of only the first one.
2024-05-21 08:37:18 +02:00
zhongyunde 00443407
f2e787e703 [InstCombine] Add some tests with reduced flag sets, NFC
We remove the combine for fmuladd with fast flag on PR90434,
and add these tests with reduced flags to show they have identity
results.
2024-05-21 14:33:39 +08:00
zhongyunde 00443407
2631531764 [InstCombine] Removing the combine of fmuladd with fast flag
We should treat fmuladd like an fma intrinsic, and any regressions need to be
addressed by dealing with fma/fmuladd in other contexts.
2024-05-21 14:33:39 +08:00
Noah Goldstein
2232843160 [ValueTracking] Recognize X op (X != 0) as non-zero
The ops supported are: `add`, `sub`, `xor`, `or`, `umax`, `uadd.sat`

Proofs: https://alive2.llvm.org/ce/z/8ZMSRg

The `add` case actually comes up in SPECInt, the rest are here mostly
for completeness.

Closes #88579
2024-05-20 15:25:40 -05:00
Noah Goldstein
2a45f89aee [ValueTracking] Add tests for isKnowNonZero of X op (X != 0); NFC 2024-05-20 15:25:40 -05:00
David Green
285f1392da [VectorCombine] Some more tests for different cmp's and fp consts. NFC 2024-05-20 18:27:33 +01:00
Mingming Liu
e33db249b5
Reland "[ThinLTO] Populate declaration import status except for distributed ThinLTO under a default-off new option" (#92718)
The original PR is reviewed in
https://github.com/llvm/llvm-project/pull/88024, and this PR adds one
line (b9f04d199d)
to fix test

Limit to one thread for in-process ThinLTO to test `LLVM_DEBUG` log.
- This should fix build bot failure like
https://lab.llvm.org/buildbot/#/builders/259/builds/4727 and
https://lab.llvm.org/buildbot/#/builders/9/builds/43876
- I could repro the failure and see interleaved log messages by using
`-thinlto-threads=all`

**Original Commit Message:**

The goal is to populate `declaration` import status if a new flag
`-import-declaration` is on.

* For in-process ThinLTO, the `declaration` status is visible to backend
`function-import` pass, so `FunctionImporter::importFunctions` should
read the import status and be no-op for declaration summaries.
Basically, the postlink pipeline is updated to keep its current behavior
(import definitions), but not updated to handle `declaration` summaries.
Two use cases ([better call-graph
sort](https://discourse.llvm.org/t/rfc-for-better-call-graph-sort-build-a-more-complete-call-graph-by-adding-more-indirect-call-edges/74029#support-cross-module-function-declaration-import-5)
or [cross-module
auto-init](https://github.com/llvm/llvm-project/pull/87597#discussion_r1556067195))
would use this bit differently.

* For distributed ThinLTO, the `declaration` status is not serialized to
bitcode. As discussed, https://github.com/llvm/llvm-project/pull/87600
will do this.
2024-05-20 08:55:31 -07:00
Krzysztof Drewniak
8b22bb8a62
[DivRemPairs] Do not freeze poisons that can't be undef (#92627)
Per comments in DivRemPairs, the rewrite from
```llvm
    %div = div %X, %Y
    %rem = rem %X, %Y
```
to
```llvm
    %div = div %X, %Y
    %.mul = mul %div, %Y
    %rem = sub %X, %mul
```
is unsound when %X or %Y are undef.

However, it is known to be sound if %X or %Y are poison but can't be
undef, since both the pre- and post-rewrite %rem are `poison`.

Additionally, proofs: https://alive2.llvm.org/ce/z/xtNQ8j

A comment in the pass listed a TODO for changing a usage of
isGuaranteedNotToBeUndefOrPoison() in the pass to something that only
detects undef. Such a function has been implemented since the time that
TODO was written, but has not been used.

Therefore, this commit updates DivRemPairs to use
isGuaranteedNotToBeUndef() instead.
2024-05-20 10:44:18 -05:00
Krzysztof Drewniak
02f1a99203
[DivRemPairs] Pre-commit tests for PR #92627 (#92628)
The tests are added to a new AMDGPU/ subdirectory since I found the
missed optimization while hacking on AMDGPU code. Also, this ensures
that AMDGPU, which uses DivRemPass, is being checked for existing
expected behavior.
2024-05-20 10:28:20 -05:00