38004 Commits

Author SHA1 Message Date
Kazu Hirata
890c4bece2
[memprof] Use SmallVector for InlinedCallStack (NFC) (#114599)
We can stay within 8 inlined elements more than 99% of the time while
building a large application.
2024-11-01 19:52:11 -07:00
vporpo
083369fd99
[SandboxVec][Legality] Per opcode checks (#114145)
This patch adds more opcode-specific legality checks.
2024-11-01 15:04:03 -07:00
Thurston Dang
e549ec529c
[msan] Add handleIntrinsicByApplyingToShadow; support NEON tbl/tbx (#114490)
This adds a general function that handles intrinsics by applying the
intrinsic to the shadows, and applies it to the specific case of Arm
NEON TBL/TBX intrinsics.

This also updates the tests from
https://github.com/llvm/llvm-project/pull/114462
2024-11-01 14:58:45 -07:00
Florian Hahn
17bad1a9da
[LV] Bail out on header phis in shouldConsiderInvariant.
This fixes an infinite recursion in rare cases.

Fixes https://github.com/llvm/llvm-project/issues/113794.
2024-11-01 20:51:25 +00:00
Han-Kuan Chen
a795a18bba
[SLP][REVEC] VF should be scaled when ScalarTy is FixedVectorType. (#114551) 2024-11-02 03:03:52 +08:00
Simon Pilgrim
718d50d6d0 [VectorCombine] foldPermuteOfBinops - prefer the new fold for matching costs.
Minor tweak to #114101 - as we're reducing the instruction count, we should prefer the fold if the old/new costs are the same.
2024-11-01 17:28:37 +00:00
Yingwei Zheng
a77dedcacb
[InstSimplify][InstCombine][ConstantFold] Move vector div/rem by zero fold to InstCombine (#114280)
Previously we fold `div/rem X, C` into `poison` if any element of the
constant divisor `C` is zero or undef. However, it is incorrect when
threading udiv over an vector select:
https://alive2.llvm.org/ce/z/3Ninx5
```
define <2 x i32> @vec_select_udiv_poison(<2 x i1> %x) {
  %sel = select <2 x i1> %x, <2 x i32> <i32 -1, i32 -1>, <2 x i32> <i32 0, i32 1>
  %div = udiv <2 x i32> <i32 42, i32 -7>, %sel
  ret <2 x i32> %div
}
```
In this case, `threadBinOpOverSelect` folds `udiv <i32 42, i32 -7>, <i32
-1, i32 -1>` and `udiv <i32 42, i32 -7>, <i32 0, i32 1>` into
`zeroinitializer` and `poison`, respectively. One solution is to
introduce a new flag indicating that we are threading over a vector
select. But it requires to modify both `InstSimplify` and
`ConstantFold`.

However, this optimization doesn't provide benefits to real-world
programs:

https://dtcxzyw.github.io/llvm-opt-benchmark/coverage/data/zyw/opt-ci/actions-runner/_work/llvm-opt-benchmark/llvm-opt-benchmark/llvm/llvm-project/llvm/lib/IR/ConstantFold.cpp.html#L908

https://dtcxzyw.github.io/llvm-opt-benchmark/coverage/data/zyw/opt-ci/actions-runner/_work/llvm-opt-benchmark/llvm-opt-benchmark/llvm/llvm-project/llvm/lib/Analysis/InstructionSimplify.cpp.html#L1107

This patch moves the fold into InstCombine to avoid breaking numerous
existing tests.

Fixes #114191 and #113866 (only poison-safety issue).
2024-11-01 22:56:22 +08:00
Yingwei Zheng
e577f14b67
[InstCombine] Use m_NotForbidPoison when folding (X u< Y) ? -1 : (~X + Y) --> uadd.sat(~X, Y) (#114345)
Alive2: https://alive2.llvm.org/ce/z/mTGCo-
We cannot reuse `~X` if `m_AllOnes` matches a vector constant with some
poison elts. An alternative solution is to create a new not instead of
reusing `~X`. But it doesn't worth the effort because we need to add a
one-use check.

Fixes https://github.com/llvm/llvm-project/issues/113869.
2024-11-01 22:18:44 +08:00
Wang Qiang
b77e40265c
[llvm][NFC] Fix typos: replace “avaliable” with “available” across various files (#114524)
This pull request corrects multiple occurrences of the typo "avaliable"
to "available" across the LLVM and Clang codebase. These changes improve
the clarity and accuracy of comments and documentation. Specific
modifications are in the following files:

1. clang-tools-extra/clang-tidy/readability/FunctionCognitiveComplexityCheck.cpp:
Updated comments in readability checks for cognitive complexity.
2. llvm/include/llvm/ExecutionEngine/Orc/ExecutionUtils.h: Corrected
documentation for JITDylib responsibilities.
3. llvm/include/llvm/Target/TargetMacroFusion.td: Fixed descriptions for
FusionPredicate variables.
4. llvm/lib/CodeGen/SafeStack.cpp: Improved comments on DominatorTree
availability.
5. llvm/lib/Target/RISCV/RISCVSchedSiFive7.td: Enhanced resource usage
descriptions for vector units.
6. llvm/lib/Transforms/Scalar/LoopIdiomRecognize.cpp: Updated invariant
description in shift-detect idiom logic.
7. llvm/test/MC/ARM/mve-fp-registers.s: Amended ARM MVE register
availability notes.
8. mlir/lib/Bytecode/Reader/BytecodeReader.cpp: Adjusted forward
reference descriptions for bytecode reader operations.

These changes have no impact on code functionality, focusing solely on
documentation clarity.

Co-authored-by: wangqiang <wangqiang1@kylinos.cn>
2024-11-01 13:25:04 +00:00
Han-Kuan Chen
e4aeeba84c
[SLP][REVEC] When ScalarTy is FixedVectorType, the insertion index should consider the number of elements of ScalarTy. (#114526) 2024-11-01 21:17:57 +08:00
David Sherwood
4ed7bcb4a6
[VPlan][NFC] Add new getMiddleBlock interface to VPlan (#113558)
This work is in preparation for PRs #112138 and #88385 where
the middle block is not guaranteed to be the immediate successor
to the region block. I've simply add new getMiddleBlock()
interfaces to VPlan that for now just return

cast<VPBasicBlock>(VectorRegion->getSingleSuccessor())

Once PR #112138 lands we'll need to do more work to discover
the middle block.
2024-11-01 10:50:52 +00:00
Nuno Lopes
344d972736 AssumeBundleBuilder: switch placeholder from undef to poison [NFC] 2024-11-01 10:12:10 +00:00
Yingwei Zheng
f16bff1261
[GVN][NewGVN][Local] Handle attributes for function calls after CSE (#114011)
This patch intersects attributes of two calls to avoid introducing UB.
It also skips incompatible call pairs in GVN/NewGVN. However, I cannot
provide negative tests for these changes.

Fixes https://github.com/llvm/llvm-project/issues/113997.
2024-11-01 12:44:33 +08:00
Lei Wang
bef3b54ea1
[InstrPGO] Avoid using global variable to fix potential data race (#114364)
In https://github.com/llvm/llvm-project/pull/109837, it sets a global
variable(`PGOInstrumentColdFunctionOnly`) in PassBuilderPipelines.cpp
which introduced a data race detected by TSan. To fix this, I decouple
the flag setting, the flags are now set
separately(`instrument-cold-function-only-path` is required to be used
with `--pgo-instrument-cold-function-only`).
2024-10-31 21:28:13 -07:00
Yingwei Zheng
96b14f2ccb
[Reland][InstCombine] Fix FMF propagation in foldSelectIntoOp (#114499)
Relands #114356. Compared to the last version, this patch only merges
poison-generating/nsz flags from the select to fix LV regression in
`llvm/test/Transforms/PhaseOrdering/AArch64/predicated-reduction.ll`.
2024-11-01 12:22:57 +08:00
Ruiling, Song
54d31bde32
Reapply "StructurizeCFG: Optimize phi insertion during ssa reconstruction (#101301)" (#114347)
This reverts commit be40c723ce2b7bf2690d22039d74d21b2bd5b7cf.
2024-11-01 08:29:59 +08:00
Florian Hahn
3b4c45e4e5
[VPlan] Fix long comment added in b021464d35ca (NFC).
Fix formatting of comment added in b021464d35ca.
2024-10-31 21:05:00 +00:00
Florian Hahn
b021464d35
[VPlan] Introduce scalar loop header in plan, remove VPLiveOut. (#109975)
Update VPlan to include the scalar loop header. This allows retiring
VPLiveOut, as the remaining live-outs can now be handled by adding
operands to the wrapped phis in the scalar loop header.

Note that the current version only includes the scalar loop header, no
other loop blocks and also does not wrap it in a region block.

PR: https://github.com/llvm/llvm-project/pull/109975
2024-10-31 21:36:44 +01:00
Igor Kudrin
454abad7b0
[CFI][LowerTypeTests] Fix indirect call with alias (#113987)
This is a fixed version of #106185, which was reverted in #113978 due to
a buildbot failure.

Motivation example:
```
> cat test.cpp
extern "C" [[gnu::weak]] void f() {}
void alias() __attribute__((alias("f")));
int main() { auto p = alias; p(); }
> clang test.cpp -fsanitize=cfi-icall -flto=thin -fuse-ld=lld
> ./a.out
[1]    1868 illegal hardware instruction  ./a.out
```

If the address of a function was only taken through its alias, the
function was not considered exported and therefore was not included in
the CFI jumptable. This resulted in `@llvm.type.test()` being lowered to
`false`, and consequently the indirect call to the function was
eventually optimized to `ubsantrap()`.
2024-10-31 13:29:07 -07:00
gulfemsavrun
d183dc7c24
Revert "[InstCombine] Fix FMF propagation in foldSelectIntoOp" (#114458)
Reverts llvm/llvm-project#114356 because it caused test failures.
https://lab.llvm.org/buildbot/#/builders/190/builds/8601

https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-base-linux-x64/b8732549597609293617/overview
2024-10-31 13:21:52 -07:00
Alexey Bataev
e05def081e
[SLP]Do not vectorize code in EH and non-returning blocks
The code in EH and non-returning blocks can be skipped by the
vectorizer, since it does not add to the perfromance, just consumes
compile/link time.

Reviewers: RKSimon

Reviewed By: RKSimon

Pull Request: https://github.com/llvm/llvm-project/pull/112221
2024-10-31 13:50:02 -04:00
Alexey Bataev
19a34dded7
[SLP]Do not account external uses in EH block and in non-returning blocks
No need to account the cost of the external uses in EH and non-returning
basic blocks.

Reviewers: RKSimon

Reviewed By: RKSimon

Pull Request: https://github.com/llvm/llvm-project/pull/112045
2024-10-31 13:23:43 -04:00
Alexey Bataev
e7080fd735 [SLP]Extra check if the intruction matked for removal, must be replaced in reduction ops
If the instruction is vectorized and it is a part of the reduced values
gather/buildvector node, it should replaced in reduced operation
instructions before removal properly, to avoid compiler crash.

Fixes #114371
2024-10-31 09:59:35 -07:00
Yingwei Zheng
cf1963afad
[InstCombine] Fix FMF propagation in foldSelectIntoOp (#114356)
Closes https://github.com/llvm/llvm-project/issues/113423.
2024-10-31 23:26:45 +08:00
Kenji Mouri / 毛利 研二
7e877fc0ac
[Reland][TLI] Add support for hypot libcall. (#114343)
This patch adds basic support for `hypot`. Constant folding support will
be submitted in a subsequent patch.

Related issue: https://github.com/llvm/llvm-project/issues/113711

Note: It's my first time contributing to the LLVM with encouragement
from one of my friends, @fawdlstty. I learned a lot from
https://github.com/llvm/llvm-project/pull/99611, and thanks for that.

Note: I had created the same PR and merged
(https://github.com/llvm/llvm-project/pull/113724), but reverted caused
by the merging issue. (The CI issue happened in 3 A.M. at my timezone.
So, I need to fall asleep again after I replied about why issue
happened.) So, I rebased to the latest main branch and recreate the PR
and hope I won't have the third time to create the same PR.

I hope @arsenm can help me review the code again. I’m sorry for that.

Kenji Mouri
2024-10-31 07:50:29 -07:00
Hari Limaye
b396921d0c
[SCCP] Handle llvm.vscale intrinsic calls (#114033)
Teach SCCP to compute a constant range for calls to llvm.vscale
intrinsics.
2024-10-31 12:22:15 +00:00
Simon Pilgrim
92af82a48d
[VectorCombine] Fold "shuffle (binop (shuffle, shuffle)), undef" --> "binop (shuffle), (shuffle)" (#114101)
Add foldPermuteOfBinops - to fold a permute (single source shuffle) through a binary op that is being fed by other shuffles.

Fixes #94546
Fixes #49736
2024-10-31 10:58:09 +00:00
Dmitry Chernenkov
d924a9ba03 Revert "[InstrPGO] Support cold function coverage instrumentation (#109837)"
This reverts commit e517cfc531886bf6ed64b4e7109bb3141ac7f430.
2024-10-31 10:55:17 +00:00
Paul Kirth
b01e2a8b56
[llvm] Allow always dropping all llvm.type.test sequences
Currently, the `DropTypeTests` parameter only fully works with phi nodes
and llvm.assume instructions. However, we'd like CFI to work in
conjunction with FatLTO, in so far as the bitcode section should be able
to contain the CFI instrumentation, while any incompatible bits are
dropped when compiling the object code.

To do that, we need to drop the llvm.type.test instructions everywhere,
and not just their uses in phi nodes. This patch updates the
LowerTypeTest pass so that uses are removed, and replaced with `true` in
all cases, and not just in phi nodes.

Addressing this will allow us to fix #112053 by modifying the FatLTO
pipeline.

Reviewers: pcc, nikic

Reviewed By: pcc

Pull Request: https://github.com/llvm/llvm-project/pull/112787
2024-10-30 16:56:30 -07:00
gulfemsavrun
36d5692570
Revert "[TLI] Add support for hypot libcall." (#114312)
Reverts llvm/llvm-project#113724
2024-10-30 15:10:29 -07:00
Artem Pianykh
84a78abdf5
[NFC][Utils] Extract CloneFunctionAttributesInto from CloneFunctionInto (#112976)
This patch is a part of step-by-step refactoring of CloneFunctionInto.
The goal is to extract reusable pieces out of it that will be later used
to optimize function cloning e.g. in coroutine processing.

Extracted from #109032 (commit 2)
2024-10-30 13:23:43 -07:00
Artem Pianykh
1c2824e3a4
[NFC][Coro] Add helpers for coro cloning with a TimeTraceScope (#112948)
A helper (2 overloads) that consolidates corocloner creation and the
actual cloning. The helpers create a TimeTraceScope to make it easier to
see how long the cloning takes.

Extracted from #109032 (commit 1)
2024-10-30 13:23:20 -07:00
Florian Hahn
5bd1af5abc
[LV] Directly store VPlan in InnerLoopVectorizer (NFC).
The current VPlan is already passed to multiple functions and more in
the future. Store it once directly in InnerLoopVectorizer.
2024-10-30 18:39:50 +00:00
Kenji Mouri / 毛利 研二
feb2d867fa
[TLI] Add support for hypot libcall. (#113724)
This patch adds basic support for `hypot`. Constant folding support will
be submitted in a subsequent patch.

Related issue: https://github.com/llvm/llvm-project/issues/113711

Note: It's my first time contributing to the LLVM with encouragement
from one of my friends, @fawdlstty. I learned a lot from
https://github.com/llvm/llvm-project/pull/99611, and thanks for that.

Kenji Mouri
2024-10-30 10:34:32 -07:00
Steven Perron
f405c683ba
[OPT] Search whole BB for convergence token. (#112728)
The spec for llvm.experimental.convergence.entry says that is must be in
the entry block for a function, and must preceed any other convergent
operation. It does not have to be the first instruction in the entry
block.

Inlining assumes that the call to llvm.experimental.convergence.entry
will be the first instruction after any phi instructions. This commit
modifies inlining to search the entire block for the call.
2024-10-30 11:19:23 -04:00
Mel Chen
8420dbf2b9
[VPlan] Refine the constructor of VPWidenIntrinsicRecipe. nfc (#113890)
Infers member MayReadFromMemory, MayWriteToMemory, and
MayHaveSideEffects based on intrinsic attributes.

---------

Co-authored-by: Florian Hahn <flo@fhahn.com>
2024-10-30 12:22:28 +08:00
Piotr Fusik
3c02fea737
[LV][NFC] Remove stray semicolons (#114057) 2024-10-30 04:07:14 +01:00
vporpo
ca998b071e
[SandboxVec][Legality] Check wrap flags (#113975) 2024-10-29 15:37:03 -07:00
Florian Hahn
680901ed80
[VPlan] Implement VPHeaderPHIRecipe::computeCost.
Fill out computeCost implementations for various header PHI recipes,
matching the legacy cost model for now.
2024-10-29 21:04:31 +00:00
Hari Limaye
e19a5fc6d3
[FuncSpec] Improve accounting of specialization codesize growth (#113448)
Only accumulate the codesize increase of functions that are actually
specialized, rather than for every candidate specialization that we
analyse.

This fixes a subtle bug where prior analysis of candidate
specializations that were deemed unprofitable could prevent subsequent
profitable candidates from being recognised.
2024-10-29 11:53:12 +00:00
Jay Foad
2443549b85
[IR] Remove some uses of StructType::setBody. NFC. (#113685)
It is simple to create the struct body up front, now that we have
transitioned to opaque pointers.
2024-10-29 11:44:53 +00:00
Hari Limaye
06664fdc76
[FuncSpec] Enable SpecializeLiteralConstant by default (#113442)
Enable specialization on literal constant arguments by default in
Function Specialization.

---------

Co-authored-by: Alexandros Lamprineas <alexandros.lamprineas@arm.com>
2024-10-29 11:41:25 +00:00
Yingwei Zheng
18311093ab
[InstCombine] Do not fold shufflevector(select) if the select condition is a vector (#113993)
Since `shufflevector` is not element-wise, we cannot do fold it into
select when the select condition is a vector.
For shufflevector that doesn't change the length, it doesn't crash, but
it is still a miscompilation: https://alive2.llvm.org/ce/z/s8saCx

Fixes https://github.com/llvm/llvm-project/issues/113986.
2024-10-29 10:39:07 +08:00
c8ef
0c1c37bfbe
[TLI] Add support for the tgamma libcall. (#113791)
This patch adds the `tgamma` libcall.
2024-10-29 10:08:38 +08:00
vporpo
a461869db3
[SandboxIR][Pass] Implement Analyses class (#113962)
The Analyses class provides a way to pass around commonly used Analyses
to SandboxIR passes throught `runOnFunction()` and `runOnRegion()`
functions.
2024-10-28 18:00:52 -07:00
Igor Kudrin
757d0e4764
Revert "[CFI][LowerTypeTests] Fix indirect call with alias" (#113978)
Reverts llvm/llvm-project#106185

This is breaking Sanitizer bots:
https://lab.llvm.org/buildbot/#/builders/66/builds/5449/steps/8/logs/stdio
2024-10-28 16:13:32 -07:00
David Majnemer
902acde341 [InstCombine] Optimize away certain additions using modular arithmetic
We can turn:
```
  %add = add i8 %arg, C1
  %and = and i8 %add, C2
  %cmp = icmp eq i1 %and, C3
```

into:
```
  %and = and i8 %arg, C2
  %cmp = icmp eq i1 %and, (C3 - C1) & C2
```

This is only worth doing if the sequence is the sole user of the addition
operation.
2024-10-28 22:51:35 +00:00
Matthias Braun
5903c6af44
InstCombine: Fold shufflevector(select) and shufflevector(phi) (#113746)
- Transform `shufflevector(select(c, x, y), C)` to
  `select(c, shufflevector(x, C), shufflevector(y, C))` by re-using
  the `FoldOpIntoSelect` helper.
- Transform `shufflevector(phi(x, y), C)` to
  `phi(shufflevector(x, C), shufflevector(y, C))` by re-using the
  `foldOpInotPhi` helper.
2024-10-28 15:35:17 -07:00
vporpo
bf4b31ad54
[SandboxVec][Legality] Check Fastmath flags (#113967) 2024-10-28 15:32:20 -07:00
vporpo
5ea694816b
[SandboxVec][Legality] Check opcodes and types (#113741) 2024-10-28 14:05:58 -07:00