53213 Commits

Author SHA1 Message Date
David Green
9e1a49cba7 [AArch64][ARM] Add tests for frem power2 lowering. NFC 2024-05-09 17:15:35 +01:00
Sander de Smalen
aa9d467aba
Revert "[AArch64] NFC: Add RUN lines for streaming-compatible code." (#91599)
Reverts llvm/llvm-project#90617
2024-05-09 16:03:16 +01:00
Momchil Velikov
139e0aa68d
Revert "[AArch64] Add intrinsics for multi-vector to ZA array vector accumulators" (#91597)
Reverts llvm/llvm-project#88266  due to test failures

error: 'expected-error' diagnostics seen but not expected: 
(frontend): '-fsyntax-only' action ignored; '-emit-llvm' action
specified previously
2024-05-09 15:03:52 +01:00
Momchil Velikov
e88ba6d975
[AArch64] Add intrinsics for multi-vector to ZA array vector accumulators (#88266)
According to the specification in
https://github.com/ARM-software/acle/pull/309 this adds the intrinsics

void_svadd_za16_vg1x2_f16(uint32_t slice, svfloat16x2_t zn)
__arm_streaming __arm_inout("za");
void_svadd_za16_vg1x4_f16(uint32_t slice, svfloat16x4_t zn)
__arm_streaming __arm_inout("za");
void_svsub_za16_vg1x2_f16(uint32_t slice, svfloat16x2_t zn)
__arm_streaming __arm_inout("za");
void_svsub_za16_vg1x4_f16(uint32_t slice, svfloat16x4_t zn)
__arm_streaming __arm_inout("za");

as well as the corresponding `bf16` variants.
2024-05-09 14:30:53 +01:00
Hongbin Jin
ed3a60c796
[RISCV][GlobalISel] Fix selectShiftMask when shift mask is created from G_AND (#89602)
This patch fixes cases where G_AND creating the shift mask is eliminated
if one of its source operands is a constant, resulting from an incorrect
predicate.
2024-05-09 09:18:10 -04:00
Matt Arsenault
6a8d30b1c1
DAG: Skip 0 sign handling in minimum/maximum lowering for _ieee case (#91326)
dc9664a8adae17f2083fbcc8e96cfce606c56d57 changed the documentation to
assume these order -0 as less than +0.
2024-05-09 14:41:13 +02:00
Simon Pilgrim
8b400de79e
[X86] Enable TuningSlowDivide64 on Barcelona/Bobcat/Bulldozer/Ryzen Families (#91277)
Despite most AMD cpus having a lower latency for i64 divisions that converge early, we are still better off testing for values representable as i32 and performing a i32 division if possible.

All AMD cpus appear to have been missed when we added the "idivq-to-divl" attribute - this patch now matches Intel cpu behaviour (and the x86-64/v2/3/4 levels).

Unfortunately the difference in code scheduling means I've had to stop using the update_llc_test_checks script and just use old-fashioned CHECK-DAG checks for divl/divq pairs.

Fixes #90985
2024-05-09 13:15:01 +01:00
Simon Pilgrim
e0d8dbc1dc [X86] opt-shuff-tstore.ll - regenerate checks 2024-05-09 13:13:20 +01:00
Jay Foad
6eb9e214b3
RFC: [AMDGPU] Check subtarget features for consistency (#86957)
Implement GCNSubtarget::checkSubtargetFeatures as a canonical place to
check subtarget features for consistency and diagnose any
inconsistencies. To start with, the implementation just checks that
either wavefrontsize32 or wavefrontsize64 is selected.

checkSubtargetFeatures is called at the start of instruction selection.
This is pretty arbitrary. It is just a convenient point at which we have
access to the subtarget that we're going to use for codegenning a
particular function.
2024-05-09 11:37:28 +01:00
Lukacma
643c38333f
[AArch64] Remove EXT instr before UZP when extracting elements from vector (#91328)
Assembly generated for getting odd/even elements from vector contained
extra EXT instruction. This was due to way llvm constructs DAGs when
vector_shuffling from larger type to smaller. This patch optimises DAG
in these situations, allowing for correct assembly to be emitted.
2024-05-09 11:10:02 +01:00
Jay Foad
2cbfe4a823
[AMDGPU] Remove duplicate -mtriple options in tests (#91576) 2024-05-09 10:59:25 +01:00
Craig Topper
5baf58b628
[RISCV] Improve use of BSETI/BCLRI in constant materialization. (#91546)
We failed to use BSETI when bit 31 was set and a few bits above bit 31
were set. We also failed to use multiple BSETI when the low 32 bits were
zero.

I've removed the special cases for constants 0x80000000-0xffffffff and
wrote a more generic algorithm for BSETI.

I've rewritten the BCLRI handling to be similar to the new BSETI
algorithm. This picks up cases where bit 31 is 0 and only a few high
bits are 0.
2024-05-08 21:53:15 -07:00
Nikita Popov
3a3aeb8eba
[PPCMergeStringPool] Avoid replacing constant with instruction (#88846)
String pool merging currently, for a reason that's not entirely clear to
me, tries to create GEP instructions instead of GEP constant expressions
when replacing constant references. It only uses constant expressions in
cases where this is required. However, it does not catch all cases where
such a requirement exists. For example, the landingpad catch clause has
to be a constant.

Fix this by always using the constant expression variant, which also
makes the implementation simpler.

Additionally, there are some edge cases where even replacement with a
constant GEP is not legal. The one I am aware of is the
llvm.eh.typeid.for intrinsic, so add a special case to forbid
replacements for it.

Fixes https://github.com/llvm/llvm-project/issues/88844.
2024-05-09 13:27:20 +09:00
Luke Lau
8f4f34f103 [RISCV] Add test for vmerge.vvm that could have splat sunk. NFC 2024-05-09 12:14:57 +08:00
Felix (Ting Wang)
ea126aebdc
[PowerPC] Tune AIX shared library TLS model at function level (#84132)
Under some circumstance (library loaded with the main program), TLS
initial-exec model can be applied to local-dynamic access(es). We
could use some simple heuristic to decide the update at function level:
* If there is equal or less than a number of TLS local-dynamic access(es)
in the function, use TLS initial-exec model. (the threshold which default to
1 is controlled by hidden option)
2024-05-09 09:50:36 +08:00
Craig Topper
36d8b37dfa [RISCV] Add another missed Zbs constant materialization test. NFC
This can be LI+BCLRI+BCLRI.
2024-05-08 16:20:07 -07:00
Craig Topper
c0b5a96ac4 [RISCV] Add tests where we could use Zbs instructions in constant materialization. NFC 2024-05-08 16:04:54 -07:00
David Green
fcf945f4ed
[DAG] Fold add(mul(add(A, CA), CM), CB) -> add(mul(A, CM), CM*CA+CB) (#90860)
This is useful when the inner add has multiple uses, and so cannot be
canonicalized by pushing the constants down through the mul. This patch
adds patterns for both `add(mul(add(A, CA), CM), CB)` and with an extra add
`add(add(mul(add(A, CA), CM), B) CB)` as the second can come up when
lowering geps.
2024-05-08 22:11:18 +01:00
Simon Pilgrim
965f3ca3dc
[GISel] Fold bitreverse(shl/srl(bitreverse(x),y)) -> srl/shl(x,y) (#91355)
Sibling patch to #89897
2024-05-08 21:58:59 +01:00
David Green
9047331f1b [AArch64] Add some additional add mul imm tests with multiple uses. NFC 2024-05-08 20:34:37 +01:00
Philip Reames
4298fc5eb5
[RISCV] Move strength reduction of mul X, 3/5/9*2^N to combine (#89966)
This moves our last major category tablegen driven multiply strength
reduction into the post legalize combine framework. The one slightly
tricky bit is making sure that we use a leading shl if we can form a
slli.uw, and trailing shl otherwise. Having the trailing shl is critical
for shNadd matching, and folding any following sext.w.

As can be seen in the TD deltas, this allows us to kill off both the
actual multiply patterns and the explicit add (mul X, C) Y patterns. The
later are now handled by the generic shNadd matching code, with the
exception of the THead only C=200 case because we don't (yet) have a
multiply expansion with two shNadd + a shift.

---------

Co-authored-by: Yingwei Zheng <dtcxzyw@qq.com>
2024-05-08 10:13:01 -07:00
Simon Pilgrim
5636eb89bd [X86] combineBlendOfPermutes - allow whole-lane permutation on AVX1 targets.
dd4bf22b9380e797362fac1415a1796da338b2db fixed #91433 but meant we couldn't use vperm2f128 to permute entire 128-bit lanes - if the new 256-bit permutation mask can be scaled to 2x128-bit elements, then we can still fold.
2024-05-08 17:05:45 +01:00
Juergen Ributzka
29b7eb8400
[llvm][stackmaps] Include pristine registers for liveness computation. (#90529)
Users of stackmaps and patchpoints need to add all pristine registers to
the
spill set, even so they don't need to be all preserved.

This fixes the liveness computation for stackmaps to include pristine
registers.

This fixes rdar://21228337.
2024-05-08 08:58:55 -07:00
Matt Arsenault
b5afda8d76 AMDGPU: Add some more ctlz_zero_undef tests 2024-05-08 16:13:22 +02:00
Xiang Li
665af09a86
[DirectX backend] emits metadata for DXIL version. (#88350)
Emit named metadata "dx.version" for DXIL version.

Default to DXIL 1.0
2024-05-08 06:40:06 -07:00
Paul Walker
c84c74e678 [LLVM][CodeGen][SVE] Add tests for vector extracts from unpacked types. 2024-05-08 12:53:04 +00:00
Thorsten Schütt
737e0bcfe3
[GlobalIsel] combine ext of trunc with flags (#87115)
https://github.com/llvm/llvm-project/pull/85592

https://discourse.llvm.org/t/rfc-add-nowrap-flags-to-trunc/77453

https://github.com/llvm/llvm-project/pull/88609
2024-05-08 14:27:02 +02:00
Felix (Ting Wang)
19220110ac
[PowerPC][AIX] Refactor existing logic to handle non-zero offsets for aix-small-local-dynamic-tls (#89182)
To enable optimized small local-dynamic access sequence for non-zero
offsets, this patch refactors existing
2a50921553798d2db52ca6330c89f0f8a5bc2215.
2024-05-08 18:37:51 +08:00
YunQiang Su
8f21294897
MIPS: Use pcrel|sdata4 for eh_frame (#91291)
Gas uses encoding DW_EH_PE_absptr for PIC, and gnu ld converts it to
DW_EH_PE_sdata4|DW_EH_PE_pcrel.
LLD doesn't have this workarounding, thus complains
```
  relocation R_MIPS_32 cannot be used against local symbol; recompile with -fPIC
  relocation R_MIPS_64 cannot be used against local symbol; recompile with -fPIC
```

So, let's generates asm/obj files with `DW_EH_PE_sdata4|DW_EH_PE_pcrel`
encoding. In fact, GNU ld supports such OBJs well.

For N64, maybe we should use sdata8, while GNU ld doesn't support it
well, and in fact sdata4 is enough now. So we just ignore the `Large`
for `MCObjectFileInfo::initELFMCObjectFileInfo`. Maybe we should switch
back to sdata8 once GNU LD supports it well.

Fixes: #58377.
2024-05-08 17:30:14 +08:00
Simon Pilgrim
dd4bf22b93 [X86] combineBlendOfPermutes - don't introduce lane-crossing permutes without AVX2 support.
Fixes #91433
2024-05-08 10:22:58 +01:00
Adrian Kuegel
bafbe39778
[NVPTX] Add support for atomic add for bf16 type (#89586)
atom.add.noftz.bf16 is supported since SM 9.0 and PTX 7.8
2024-05-08 10:22:02 +02:00
Luke Lau
bbd6a2d85c
[RISCV] Convert implicit_def tuples to noreg in post-isel peephole (#91173)
If a segmented load has an undefined passthru then it will be selected
as a reg_sequence with implicit_def operands, which currently slips
through the implicit_def -> noreg peephole.

This patch fixes this so we're able to infer if the passthru is
undefined without the need for looking through vreg definitions with
MachineRegisterInfo, which will help with moving RISCVInsertVSETVLI to
LiveIntervals in #70549
2024-05-08 15:38:13 +08:00
Farzon Lotfi
3e82442ff7
[SPIRV] Add tan intrinsic part 3 (#90278)
This change is an implementation of #87367's investigation on supporting
IEEE math operations as intrinsics.
Which was discussed in this RFC:
https://discourse.llvm.org/t/rfc-all-the-math-intrinsics/78294

If you want an overarching view of how this will all connect see:
https://github.com/llvm/llvm-project/pull/90088
Changes:
- `llvm/docs/GlobalISel/GenericOpcode.rst` - Document the `G_FTAN`
opcode
-  `llvm/include/llvm/IR/Intrinsics.td` - Create the tan intrinsic
- `llvm/include/llvm/Support/TargetOpcodes.def` - Create a `G_FTAN`
Opcode handler
- `llvm/include/llvm/Target/GenericOpcodes.td` - Define the `G_FTAN`
Opcode
- `llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp` Map the tan intrinsic
to `G_FTAN` Opcode
- `llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp` - Map the
`G_FTAN` opcode to the GLSL 4.5 and openCL tan instructions.
- `llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp` - Define `G_FTAN` as a
legal spirv target opcode.
2024-05-08 00:57:39 -04:00
Craig Topper
ef84452571
[DAGCombiner] Be more careful about looking through extends and truncates in mergeTruncStores. (#91375)
Previously we recursively looked through extends and truncates on both
SourceValue and WideVal.

SourceValue is the largest source found for each of the stores we are
combining. WideVal is the source for the current store.

Previously we could incorrectly look through a (zext (trunc X)) pair and
incorrectly believe X to be a good source.

I think we could also look through a zext on one store and a sext on
another store and arbitrarily pick one of the extends as the final
source.

With this patch we only look through one level of extend or truncate.
And we don't look through extends/truncs on both SourceValue and WideVal
at the same time.

This may lose some optimization cases, but keeps everything we had tests
for.

Fixes #90936.
2024-05-07 21:17:50 -07:00
Farzon Lotfi
85ef6b7c36
[DXIL] Add tan intrinsic part 2 (#90277)
This change is an implementation of #87367's investigation on supporting
IEEE math operations as intrinsics.
Which was discussed in this RFC:
https://discourse.llvm.org/t/rfc-all-the-math-intrinsics/78294

If you want an overarching view of how this will all connect see:
https://github.com/llvm/llvm-project/pull/90088

Changes:
-  `llvm/include/llvm/IR/Intrinsics.td` - Create the tan intrinsic
- `llvm/lib/Target/DirectX/DXIL.td` - Map `int_tan` (the tan intrinsic)
to the equivalent DXIL Op.
2024-05-08 00:10:08 -04:00
Thorsten Schütt
812c3025ec
[GlobalIsel][AArch64] legalize ptr add (#89218)
LLVM ERROR: unable to legalize instruction: %275:_(<4 x p0>) = G_PTR_ADD
%268:_, %274:_(<4 x s64>) (in function: prepare_for_pass)
2024-05-08 05:35:23 +02:00
Luke Lau
48b6f4a182
[RISCV] Rewrite spill-fpr-scalar.ll test to not use vsetvli. NFC (#91428)
It was relying on the fact that vsetvlis have side effects to prevent
reordering, but #91319 proposes to remove the side effects. This reworks
it to use volatile loads and stores instead.
2024-05-08 11:33:05 +08:00
Liao Chunyu
f4d2f7a3b7
[RISCV] Codegen support for XCVbi extension (#89719)
spec:
https://github.com/openhwgroup/cv32e40p/blob/master/docs/source/instruction_set_extensions.rst#immediate-branching-operations

Contributors: @CharKeaney, @jeremybennett, @lewis-revill,
@NandniJamnadas,
@PaoloS02, @simonpcook, @xingmingjie, @realqhc, @PhilippvK,@melonedo
2024-05-08 11:22:16 +08:00
Jinsong Ji
2dade0041a
[Analysis] Attribute Range should not prevent tail call optimization (#91122)
- Remove Range attr when comparing for tailcall
- Add test for testcall with range
2024-05-07 22:02:10 -04:00
Stanislav Mekhanoshin
2a3903fa0e
[AMDGPU] Prevent FMINIMUM and FMAXIMUM beeing fully scalarized (#91378)
This is the same logic as with FMINNUM_IEEE/FMAXNUM_IEEE.
2024-05-07 14:20:13 -07:00
Shilei Tian
0b50d095bc
[AMDGPU] Don't optimize agpr phis if the operand doesn't have subreg use (#91267)
If the operand doesn't have any subreg use, the optimization could
potentially
generate `V_ACCVGPR_READ_B32_e64` with wrong register class. The
following example demonstrates the issue.

Input MIR:

```
bb.0:
  %0:sgpr_32 = S_MOV_B32 0
  %1:sgpr_128 = REG_SEQUENCE %0:sgpr_32, %subreg.sub0, %0:sgpr_32, %subreg.sub1, %0:sgpr_32, %subreg.sub2, %0:sgpr_32, %subreg.sub3
  %2:vreg_128 = COPY %1:sgpr_128
  %3:areg_128 = COPY %2:vreg_128, implicit $exec

bb.1:
  %4:areg_128 = PHI %3:areg_128, %bb.0, %6:areg_128, %bb.1
  %5:areg_128 = PHI %3:areg_128, %bb.0, %7:areg_128, %bb.1
  ...
```

Output of current implementation:

```
bb.0:
  %0:agpr_32 = V_ACCVGPR_WRITE_B32_e64 0, implicit $exec
  %1:agpr_32 = V_ACCVGPR_WRITE_B32_e64 0, implicit $exec
  %2:agpr_32 = V_ACCVGPR_WRITE_B32_e64 0, implicit $exec
  %3:agpr_32 = V_ACCVGPR_WRITE_B32_e64 0, implicit $exec
  %4:areg_128 = REG_SEQUENCE %0:agpr_32, %subreg.sub0, %1:agpr_32, %subreg.sub1, %2:agpr_32, %subreg.sub2, %3:agpr_32, %subreg.sub3
  %5:vreg_128 = V_ACCVGPR_READ_B32_e64 %4:areg_128, implicit $exec
  %6:areg_128 = COPY %46:vreg_128

bb.1:
  %7:areg_128 = PHI %6:areg_128, %bb.0, %9:areg_128, %bb.1
  %8:areg_128 = PHI %6:areg_128, %bb.0, %10:areg_128, %bb.1
  ...
```

The problem is the generated `V_ACCVGPR_READ_B32_e64` instruction.
Apparently the operand `%4:areg_128` is not valid for this.

In this patch, we don't count the none-subreg use because
`V_ACCVGPR_READ_B32_e64` can't handle none-32-bit operand.

Fixes: SWDEV-459556
2024-05-07 16:44:00 -04:00
Maryam Moghadas
9a28814f59
[PowerPC] Spill non-volatile registers required for traceback table (#71115)
On AIX we need to spill all [rfv]N-[rfv]31 when a function clobbers
[rfv]N so that the traceback table contains accurate information.
2024-05-07 16:23:37 -04:00
Xiang Li
5e9dd8827b
[DirectX] remove string function attribute DXIL not allowed (#90778)
Remove string function attribute other than
"waveops-include-helper-lanes" and "fp32-denorm-mode".

Move DXILPrepareModulePass after DXILTranslateMetadataPass since
DXILTranslateMetadataPass needs to use attribute like hlsl.numthreads.

Fixes #90773
2024-05-07 16:19:52 -04:00
Brendan Dahl
1a2a1fbd7c
[WebAssembly] Implement prototype f32.load_f16 instruction. (#90906)
Adds a builtin and intrinsic for the f32.load_f16 instruction.

The instruction loads an f16 value from memory and puts it in an f32.
Specified at:

29a9b9462c/proposals/half-precision/Overview.md

Note: the current spec has f32.load_f16 as opcode 0xFD0120, but this is
incorrect and will be changed to 0xFC30 soon.
2024-05-07 11:33:10 -07:00
Simon Pilgrim
7198b8a39a [X86] x86-64-extend-shift.ll - regenerate checks 2024-05-07 17:27:58 +01:00
Simon Pilgrim
63ceb9afc6 [X86] sext-subreg.ll - regenerate checks 2024-05-07 17:27:58 +01:00
Petr Hosek
8bcb073705
[Clang] -fseparate-named-sections option (#91028)
When set, the compiler will use separate unique sections for global
symbols in named special sections (e.g. symbols that are annotated with
__attribute__((section(...)))). Doing so enables linker GC to collect
unused symbols without having to use a different section per-symbol.
2024-05-07 09:18:55 -07:00
Ulrich Weigand
de117dd533 [SystemZ] Add some more atomic load/store tests
Verify atomic load/store of f128 on z14 where the type
lives in VRs.
2024-05-07 16:57:17 +02:00
Simon Pilgrim
41dd07bf5c [AArch64] Add test coverage for bitreverse(logicalshift(bitreverse(x),y)) -> logicalshift(x,y) fold
DAG already performs this fold (#89897), GISel is currently missing it (patch incoming)
2024-05-07 15:55:41 +01:00
Matt Arsenault
f548c4d83c AMDGPU: Add mode register use to s_getreg_b32
This should fix reading the wrong mode after setting the mode.
Ideally we would have separate pseudos for the case that we know
does not read mode.
2024-05-07 16:06:51 +02:00