103646 Commits

Author SHA1 Message Date
Noah Goldstein
809b1d834d [KnownBits] Return 0 for poison {s,u}div inputs
It seems consistent to always return zero for known poison rather than
varying the value. We do the same elsewhere.

Differential Revision: https://reviews.llvm.org/D150922
2023-06-06 15:14:10 -05:00
David Green
2a8df8d0b9 [AArch64][SVE] Add one-use-check to EitherVSelectOrPassthruPatFrags
As pointed out in D149968 vselect predicate patterns could do with a one-use
check to prevent multiple operations being created. This updates the
EitherVSelectOrPassthruPatFrags pattern frags used in creating predicates
min/max.

Differential Revision: https://reviews.llvm.org/D151080
2023-06-06 21:10:32 +01:00
Guozhi Wei
84bcfa0e1b [GVN] Improve PRE on load instructions
This patch implements the enhancement proposed by
https://github.com/llvm/llvm-project/issues/59312.

Suppose we have following code

   v0 = load %addr
   br %LoadBB

LoadBB:
   v1 = load %addr
   ...

PredBB:
   ...
   br %cond, label %LoadBB, label %SuccBB

SuccBB:
   v2 = load %addr
   ...

Instruction v1 in LoadBB is partially redundant, edge (PredBB, LoadBB) is a
critical edge. SuccBB is another successor of PredBB, it contains another load
v2 which is identical to v1. Current GVN splits the critical edge
(PredBB, LoadBB) and inserts a new load in it. A better method is move the load
of v2 into PredBB, then v1 can be changed to a PHI instruction.

If there are two or more similar predecessors, like the test case in the bug
entry, current GVN simply gives up because otherwise it needs to split multiple
critical edges. But we can move all loads in successor blocks into predecessors.

Differential Revision: https://reviews.llvm.org/D141712
2023-06-06 19:45:34 +00:00
Jessica Clarke
db1d72b155 [RISCV] Don't persist invalid feature state on .option arch error
Otherwise subsequent .option arch, +foo directives (but not -, since
those have their own separate validation) fail the parseFeatureBits
check, leading to cascading errors.

Reviewed By: luismarques, MaskRay

Differential Revision: https://reviews.llvm.org/D152273
2023-06-06 20:15:12 +01:00
Ellis Hoag
1117b9a284 [InstrProf] Use BalancedPartitioning to order temporal profiling trace data
In [0] we described an algorithm called //BalancedPartitioning// (bp) to consume function traces [1] and compute a function order that reduces the number of page faults during startup.

This patch adds the `order` command to the `llvm-profdata` tool which uses bp to output a function order that can be passed to the linker via `--symbol-ordering-file=`.

Special thanks to Sergey Pupyrev and Julian Mestre for designing this balanced partitioning algorithm.

[0] https://discourse.llvm.org/t/rfc-temporal-profiling-extension-for-irpgo/68068
[1] https://reviews.llvm.org/D147287

Reviewed By: spupyrev

Differential Revision: https://reviews.llvm.org/D147812
2023-06-06 11:59:57 -07:00
Craig Topper
58b2d652af [RISCV] Add special case to selectImm for constants that can be created with (ADD (SLLI C, 32), C).
Where C is a simm32.

This costs an extra temporary register, but avoids a constant pool.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D152236
2023-06-06 11:59:12 -07:00
Simon Pilgrim
9a81b69757 [AArch64] Regenerate tests with missing immediate hex asm comments
Reduces diff in a future commit
2023-06-06 19:44:28 +01:00
Simon Pilgrim
a279a09ab9 Revert rG98061013e01207444cfd3980 - [X86] X86FixupVectorConstantsPass - attempt to replace full width fp vector constant loads with broadcasts on AVX+ targets
Reverting while we address an existing issue exposed by this (Issue #63108)
2023-06-06 18:44:24 +01:00
Simon Pilgrim
78de45fd4a Revert rGab4b924832ce26c21b88d7f82fcf4992ea8906bb - [X86] X86FixupVectorConstantsPass - attempt to replace full width integer vector constant loads with broadcasts on AVX2+ targets
Reverting while we address an existing issue exposed by this (Issue #63108)
2023-06-06 18:07:33 +01:00
Jolanta Jensen
a963dbb5ac [SVE ACLE] Extend existing aarch64_sve_mul combines to also act on aarch64_sve_mul_u.
Differential Revision: https://reviews.llvm.org/D152004
2023-06-06 15:26:33 +00:00
Mikhail Goncharov
df3a8f3760 Revert "Reland [MergeICmps] Adapt to non-eq comparisons, bugfix"
Causes miscompile. See https://reviews.llvm.org/D141188.

This reverts commit fb2c98a929aa65603e9d984307a41325e577e9d3
2023-06-06 16:26:52 +02:00
Jessica Clarke
2c11768eee [RISCV] Make .option arch parser less mind-bending
Currently the early-return flow in the infinite loop makes it hard to
find the non-error termination points amongst the sea of errors. Rewrite
it with a more conventional control flow that has a clear loop guard (in
place of one of the early returns) and a break (in place of the other),
and with greater code reuse.

This has a small effect on the errors given for malformed input, as seen
in the affected test, and is probably more helpful as a result. Note
that we also bail early now if parseComma fails, as is standard.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D152192
2023-06-06 14:56:20 +01:00
Andrew Ng
744e589caa [llvm-objcopy][ELF] Preserve sh_link to .symtab when applicable
This change to llvm-objcopy preserves the ELF section sh_link to .symtab
so long as none of the symbol table indices have been changed.
Previously, any invocation of llvm-objcopy including a "no-op" would
clear any section sh_link to .symtab.

Differential Revision: https://reviews.llvm.org/D150859
2023-06-06 14:35:04 +01:00
Jay Foad
a4a3ac10cb [AMDGPU] Remove extract_subvector patterns
Removing them seems to slightly increase code quality as well as
simplifying both the tablegen and C++ parts of the code.

Differential Revision: https://reviews.llvm.org/D149853
2023-06-06 14:04:50 +01:00
Ricardo Jesus
3a87c15026 [AArch64][NFC] Normalise name of indexed forms of SQRDMLAH/SQRDMLSH
Most indexed vector instructions are suffixed with v<N><TY>_indexed.

SQRDMLAH/SQRDMLSH are the exception, being suffixed with <TY>_indexed
instead, which can complicate matching them slightly.

Differential Revision: https://reviews.llvm.org/D152161
2023-06-06 13:02:36 +00:00
Simon Pilgrim
85b77b13e3 [GlobalISel][X86] Add G_IMPLICIT_DEF / G_CONSTANT legalization handling 2023-06-06 11:45:22 +01:00
Thorsten Schütt
60b8019ea0 [GlobalIsel][X86] Legalize G_ANYEXT, G_SEXT, and G_ZEXT
Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D152243
2023-06-06 12:22:09 +02:00
wangpc
26e41a80d0 [RISCV] Handle "o" inline asm memory constraint
This is the same as D100412.

We just found the same crash when we tried to compile some packages
like mariadb, php, etc.

For constraint "o", it means "A memory operand is allowed, but
only if the address is offsettable". So I think it can be handled
just like constraint "m" for RISCV target.

And we print verbose information when unsupported constraints occur.

Reviewed By: asb

Differential Revision: https://reviews.llvm.org/D151979
2023-06-06 17:50:40 +08:00
Florian Hahn
8f781b96e2
Revert "[VPlan] Mark recurrence recipes as not having side-effects."
This reverts commit 02369b75fdd7b5fc5d9b47f1b60587c225918511.

At the moment, live-outs used *only* for the resume values in the scalar
loop are not modeled in VPlan yet. This means first-order recurrence
recipes could be removed, when a scalar epilogue is required and the
only use of a FOR is outside the loop.

Keep treating recurrence recipes as having side-effects for now, to
avoid them being removed.

Fixes #62954.
2023-06-06 11:35:26 +02:00
Florian Hahn
f47084ecfb
[LV] Use force-vector-width for X86 recurrence test.
This makes sure that all tests that can be vectorized in the file are
vectorized.
2023-06-06 11:27:35 +02:00
Florian Hahn
4c51a45e80
[LV] Add test for #62954. 2023-06-06 11:20:22 +02:00
Carl Ritson
6afc4b0629 [AMDGPU] WQM: Ensure exact mode placement before branches
Fix for D151797 where the change accidentally allowed exit to
exact mode between branch instructions.

Reviewed By: dstuttard

Differential Revision: https://reviews.llvm.org/D152228
2023-06-06 18:11:35 +09:00
Martin Storsjö
a499d675ae [AArch64] Make .arch without extra features actually take effect
This fixes PR32873 / https://github.com/llvm/llvm-project/issues/32220.

Differential Revision: https://reviews.llvm.org/D151982
2023-06-06 11:50:04 +03:00
Martin Storsjö
4b8d9abca7 [AArch64] Complete the list of extensions supported by .arch and .arch_extension
This brings the list of extensions supported here up to date
with what is supported by current git versions of binutils.

Also add a comment to AArch64TargetParser to remind people to
consider adding new ones to the list supported in assembly.

In the case of the "rdma" extension, there's a slight surprise:
LLVM knows of the extension under the name "rdm", while binutils
has it named "rdma". However, binutils appears to accept any
abbreviated prefix of an arch extension, so it does accept the
form "rdm" too even if it formally considers it called "rdma".

Support both spellings for the extensions here, for simplicity.

Differential Revision: https://reviews.llvm.org/D151981
2023-06-06 11:50:03 +03:00
Serge Pavlov
10e7899818 [FPEnv] Get rid of extra moves in fpenv calls
If intrinsic `get_fpenv` or `set_fpenv` is lowered to the form where FP
environment is represented as a region in memory, extra moves can
appear. For example the code:

  define void @func_01(ptr %ptr) {
    %env = call i256 @llvm.get.fpenv.i256()
    store i256 %env, ptr %ptr
    ret void
  }

produces DAG:

  ch = get_fpenv_mem ch, memory_region
  val: i256, ch = load ch, memory_region
  ch = store ch, ptr, val

In this case the extra moves can be avoided if `get_fpenv_mem` got
pointer to the memory where the FP environment should be finally placed.

This change implement such optimization for this use case.

Differential Revision: https://reviews.llvm.org/D150437
2023-06-06 14:54:52 +07:00
Christian Ulmann
2544d91956 llvm-extract: Replace IFuncs with declarations
This commit ensures that llvm-extract does not copy all IFuncs into the
resulting modules. Before this change, ifuncs were not modified which
could cause the emission unexpected IR files.

Reviewed By: darthscsi

Differential Revision: https://reviews.llvm.org/D152148
2023-06-06 07:18:33 +00:00
Carl Ritson
7275637505 [AMDGPU] Pre-commit test for D152228 (NFC) 2023-06-06 16:00:20 +09:00
Luo, Yuanke
787f3008be [X86] Pre-commit test case for D152227. 2023-06-06 14:56:45 +08:00
Luo, Yuanke
60b7dbb670 [X86] Add test cases for D152227. 2023-06-06 14:24:46 +08:00
Paulo Matos
9571a28ee4 [WebAssembly] Add tests ensuring rotates persist
Due to the nature of WebAssembly, it's always better to keep
rotates instead of trying to optimize it. Commit 9485d983
disabled the generation of fsh for rotates, however these
tests ensure that future changes don't change the behaviour for
the Wasm backend that tends to have different optimization
requirements than other architectures. Also see:
https://github.com/llvm/llvm-project/issues/62703

Differential Revision: https://reviews.llvm.org/D152126
2023-06-06 07:48:35 +02:00
khei4
116670d192 [InstCombine] add overflow checking on Add ~X + C --> (C-1) - X
Differential Revision: https://reviews.llvm.org/D152088
2023-06-06 12:24:45 +09:00
khei4
0505fcdccd [InstCombine] precommit test for D152088(NFC)
Differential Revision: https://reviews.llvm.org/D152089
2023-06-06 12:24:45 +09:00
Ben Shi
b1f0cb89c1 [AVR][NFC][test] Supplement more tests of 8-bit rotation
Reviewed By: Patryk27, jacquesguan

Differential Revision: https://reviews.llvm.org/D152129
2023-06-06 11:24:18 +08:00
Jianjian GUAN
77da27b5e3 [RISCV] Improve selection for vector fpclass.
Since vfclass intruction will only set one single bit in the result, so if we only want to check 1 fp class, we could use vmseq to do it.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D151967
2023-06-06 10:24:24 +08:00
Matt Arsenault
ecf30c31fb AMDGPU: Fix broken test 2023-06-05 20:44:59 -04:00
Matt Arsenault
d065b1d65b AutoUpgrade: Fix crash when tbaa has an empty argument
Produce a verifier error instead.
2023-06-05 20:44:58 -04:00
Johannes Doerfert
cb17c48fdd [Attributor] Identify and remove no-op fences
The logic and implementation follows the removal of no-op barriers. If
the fence is not making updates visible, either to the world or the
current thread, it is not needed. Said differently, the fences we remove
do not establish synchronization (happens-before) edges.
This allows us to eliminate some of the regression caused by:
  https://reviews.llvm.org/D145290
2023-06-05 17:14:00 -07:00
NAKAMURA Takumi
d3777f20c5 test/AMDGPU: REQUIRES asserts (D148184) 2023-06-06 08:55:46 +09:00
Johannes Doerfert
532356e82d [Attributor] Merge ranges by expansion, avoid unknown ranges
Different offsets can be handled by expansion rather than defaulting to
an unknown offset. Thus, [4,4] & [8,8] will result in [4, 12] rather
than [unknown, unknown].
2023-06-05 16:53:46 -07:00
Johannes Doerfert
87d13b8776 [Attributor][NFC] Precommit vector write range tests 2023-06-05 16:53:45 -07:00
Johannes Doerfert
8f4fadd1b4 [OpenMP] Use "kernel" attribute consistently 2023-06-05 16:33:53 -07:00
Johannes Doerfert
dbbe9b3776 [Attributor] Create AAMustProgress for the mustprogress attribute
Derive the mustprogress attribute based on the willreturn attribute
or the fact that all callers are mustprogress.

Differential Revision: https://reviews.llvm.org/D94740
2023-06-05 16:33:52 -07:00
Matt Arsenault
30bd96fa17 AMDGPU: Add baseline test for undoing mul add 1 reassociation
Add some tests for combines to undo regressions caused by
0cfc6510323fbb5a56a5de23cbc65f7cc30fd34c.
2023-06-05 18:44:17 -04:00
Matt Arsenault
b25c001ad3 AMDGPU: Fold zext into result of v_mad_u16 on high zeroing targets
Avoids regressions in future patch.
2023-06-05 18:41:07 -04:00
Matt Arsenault
db08f9a2d5 AMDGPU: Add baseline 16-bit mad matching tests 2023-06-05 18:41:07 -04:00
Matt Arsenault
cb4b7340b0 AMDGPU: Convert test to generated checks 2023-06-05 18:41:06 -04:00
Craig Topper
b64ddae8a2 [RISCV] Lower experimental_get_vector_length intrinsic to vsetvli for some cases.
This patch lowers to vsetvli when the AVL is i32 or XLenVT and
the VF is a power of 2 in the range [1, 64]. VLEN=32 is not supported
as we don't have a valid type mapping for that. VF=1 is not supported
with Zve32* only.

The element width is used to set the SEW for the vsetvli if possible.
Otherwise we use SEW=8.

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D150824
2023-06-05 15:02:11 -07:00
Craig Topper
4157bfb230 [RISCV] Add RISCVISD nodes for vfwadd/vfwsub.
Add a DAG combine to form these from FADD_VL/FSUB_VL and FP_EXTEND_VL.

This makes it similar to other widening ops and allows us to handle
using the same FP_EXTEND_VL for both operands.

Differential Revision: https://reviews.llvm.org/D151969
2023-06-05 14:12:47 -07:00
Jessica Clarke
bacb14b9f3
[RISCV][test] Fix a couple of whitespace issues 2023-06-05 21:25:04 +01:00
David Blaikie
5e74b2e8bb llvm-dwarfdump --verify: Add support for .debug_str_offsets[.dwo]
Had a couple of issues lately causing corrupted strings due to
problematic str_offsets (overflow due to >4GB .debug_str.dwo section in
a dwp and the dwp tool silently overflowing the 32 bit offsets updated
in the .debug_str_offsets.dwo section, and then more recently two CUs in
a dwo caused the dwp tool to reapply the offset adjustment twice
corrupting str_offsets.dwo as well) - so let's check that the offsets
are valid.

This assumes no suffix merging - if anyone implements that, then this
checking should just be removed for the most part (we could still check
the offsets are within the bounds of .debug_str[.dwo], but nothing more
- any offset in the range would be valid, the offsets wouldn't have to
land at the start of a string)
2023-06-05 19:59:37 +00:00