319 Commits

Author SHA1 Message Date
Jeremy Morse
087172258a [DebugInfo][RemoveDIs] Handle non-instr debug-info in GlobalISel (#75228)
The RemoveDIs project is aiming to eliminate debug intrinsics like
dbg.value and dbg.declare from LLVM, and replace them with DPValue objects
attached to instructions. ISel is one of the "terminals" where that
information needs to be converted into MIR format: this patch implements
support for that in GlobalISel. We aim for the output of LLVM to be
identical with/without RemoveDIs debug-info.

This patch should be NFC, as we're handling the same data about variables
stored in a different format -- it now appears in a DPValue object rather
than as an intrinsic. To that end, I've refactored the handling of
dbg.values into a dedicated function, and call it whenever a dbg.value or a
DPValue is encountered. dbg.declare is handled in a similar way.

Testing: adding the --try-experimental-debuginfo-iterators switch to llc
causes it to try and convert to the "new" debug-info format if it's built
in (LLVM_EXPERIMENTAL_DEBUGINFO_ITERATORS=On), and it'll be covered by our
buildbot. One test has a few extra wildcard-regexes added: this is because
there's some extra data printed about attached debug-info, which is safe to
ignore.
2024-01-23 15:04:08 +00:00
Simon Pilgrim
a2a0089ac3
[X86] movsd/movss/movd/movq - add support for constant comments (#78601)
If we're loading a constant value, print the constant (and the zero upper elements) instead of just the shuffle mask.

This did require me to move the shuffle mask handling into addConstantComments as we can't handle this in the MC layer.
2024-01-19 14:21:26 +00:00
Evgenii Kudriashov
ef35da825f
[X86][GlobalISel] Add instruction selection for G_SELECT (#70753) 2023-12-12 16:08:08 +01:00
Tobias Stadler
373c343a77 Reland: [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND
Reland 3686a0b after fixing an exposed miscompile in #68840

Differential Revision: https://reviews.llvm.org/D159140
2023-11-02 00:18:19 +01:00
Craig Topper
9a7c26a399
[GISel] Restrict G_BSWAP to multiples of 16 bits. (#70245)
This is consistent with the IR verifier and SelectionDAG's getNode.

Update tests accordingly. I tried to keep some coverage of non-pow2 when
possible. X86 didn't like a G_UNMERGE_VALUES from s48 to 3 s16 that got
created when I tried s48.
2023-10-30 10:27:57 -07:00
Guozhi Wei
9a091de7fe [X86, Peephole] Enable FoldImmediate for X86
Enable FoldImmediate for X86 by implementing X86InstrInfo::FoldImmediate.

Also enhanced peephole by deleting identical instructions after FoldImmediate.

Differential Revision: https://reviews.llvm.org/D151848
2023-10-27 19:47:23 +00:00
Evgenii Kudriashov
0a8f54c3fe
[X86][GlobalISel] Add legalization of 64-bit G_ICMP for i686 (#69478) 2023-10-25 22:30:42 +02:00
Mogball
3fb5b18e81 Revert 24633ea and 760e7d0 "Enable FoldImmediate for X86"
This reverts commits 24633eac38d46cd4b253ba53258165ee08d886cd
and 760e7d00d142ba85fcf48c00e0acc14a355da7c3.

I have confirmed that these commits are introducing a new crash in the
peephole optimizer. I have minimized a test case, which you can find
below.

```llvmir
; ModuleID = 'bugpoint-reduced-simplified.bc'
source_filename = "/mnt/big/modular/Kernels/mojo/Mogg/MOGG.mojo"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

declare dso_local void @foo({ { ptr, [4 x i64], [4 x i64], i1 }, { ptr, [4 x i64], [4 x i64], i1 } }, { ptr }, { ptr, i64, i8 })

define dso_local void @bad_fn(ptr %0, ptr %1, ptr %2) {
  %4 = load i64, ptr null, align 8
  %5 = insertvalue [4 x i64] poison, i64 12, 1
  %6 = insertvalue [4 x i64] %5, i64 poison, 2
  %7 = insertvalue [4 x i64] %6, i64 poison, 3
  %8 = insertvalue { ptr, [4 x i64], [4 x i64], i1 } poison, [4 x i64] %7, 1
  %9 = insertvalue { ptr, [4 x i64], [4 x i64], i1 } %8, [4 x i64] poison, 2
  %10 = insertvalue { ptr, [4 x i64], [4 x i64], i1 } %9, i1 poison, 3
  %11 = icmp ne i64 %4, 1
  %12 = or i1 false, %11
  %13 = select i1 %12, i64 %4, i64 0
  %14 = zext i1 %12 to i64
  %15 = insertvalue [4 x i64] poison, i64 12, 1
  %16 = insertvalue [4 x i64] %15, i64 poison, 2
  %17 = insertvalue [4 x i64] %16, i64 %13, 3
  %18 = insertvalue [4 x i64] poison, i64 %14, 3
  %19 = icmp eq i64 0, 0
  %20 = icmp eq i64 0, 0
  %21 = icmp eq i64 %13, 0
  %22 = and i1 %20, %19
  %23 = select i1 %22, i1 %21, i1 false
  %24 = select i1 %23, i1 %12, i1 false
  %25 = insertvalue { ptr, [4 x i64], [4 x i64], i1 } poison, [4 x i64] %17, 1
  %26 = insertvalue { ptr, [4 x i64], [4 x i64], i1 } %25, [4 x i64] %18, 2
  %27 = insertvalue { ptr, [4 x i64], [4 x i64], i1 } %26, i1 %24, 3
  %28 = insertvalue { { ptr, [4 x i64], [4 x i64], i1 }, { ptr, [4 x i64], [4 x i64], i1 } } undef, { ptr, [4 x i64], [4 x i64], i1 } %10, 0
  %29 = insertvalue { { ptr, [4 x i64], [4 x i64], i1 }, { ptr, [4 x i64], [4 x i64], i1 } } %28, { ptr, [4 x i64], [4 x i64], i1 } %27, 1
  br label %31

30:                                               ; preds = %3
  br label %softmax_pass

31:                                               ; preds = %31
  %exitcond.not.i = icmp eq i64 poison, 3
  br i1 %exitcond.not.i, label %37, label %31

32:                                               ; preds = %31
  br i1 poison, label %34, label %33

33:                                               ; preds = %32
  br label %34

34:                                               ; preds = %33, %32
  br i1 poison, label %35, label %36

35:                                               ; preds = %34
  br label %softmax_pass

36:                                               ; preds = %34
  br i1 poison, label %37, label %.critedge.i

37:                                               ; preds = %36
  br i1 poison, label %38, label %.critedge.i

38:                                               ; preds = %37
  br i1 poison, label %40, label %39

39:                                               ; preds = %38
  br label %40

40:                                               ; preds = %39, %38
  br i1 poison, label %.lr.ph28.i, label %._crit_edge.i

.lr.ph28.i:                                       ; preds = %40
  br label %41

41:                                               ; preds = %51, %.lr.ph28.i
  br i1 poison, label %.thread, label %42

42:                                               ; preds = %41
  br i1 poison, label %43, label %44

43:                                               ; preds = %42
  br label %45

44:                                               ; preds = %42
  br label %45

45:                                               ; preds = %44, %43
  br i1 poison, label %46, label %.thread

46:                                               ; preds = %45
  br label %47

.thread:                                          ; preds = %45, %41
  br label %47

47:                                               ; preds = %.thread, %46
  br i1 poison, label %51, label %48

48:                                               ; preds = %47
  br i1 poison, label %49, label %50

49:                                               ; preds = %48
  br label %51

50:                                               ; preds = %48
  br label %51

51:                                               ; preds = %50, %49, %47
  call void @foo({ { ptr, [4 x i64], [4 x i64], i1 }, { ptr, [4 x i64], [4 x i64], i1 } } %29, { ptr } poison, { ptr, i64, i8 } poison)
  br i1 poison, label %._crit_edge.i, label %41

._crit_edge.i:                                    ; preds = %51, %40
  br label %softmax_pass

.critedge.i:                                      ; preds = %37, %36
  br i1 poison, label %.lr.ph.i, label %softmax_pass

.lr.ph.i:                                         ; preds = %.lr.ph.i, %.critedge.i
  store { ptr, [4 x i64], [4 x i64], i1 } %10, ptr poison, align 8
  br i1 poison, label %.lr.ph.i, label %softmax_pass

softmax_pass:                                     ; preds = %.lr.ph.i, %.critedge.i, %._crit_edge.i, %35, %30
  ret void
}
```
2023-10-24 07:08:38 +00:00
Evgenii Kudriashov
cc455033d4
[X86][GlobalISel] Reorganize shift scalar tests (NFC) (#68232)
Removed duplicated tests from GlobalISel directory
2023-10-24 02:00:30 +02:00
Guozhi Wei
760e7d00d1 [X86, Peephole] Enable FoldImmediate for X86
Enable FoldImmediate for X86 by implementing X86InstrInfo::FoldImmediate.

Also enhanced peephole by deleting identical instructions after FoldImmediate.

Differential Revision: https://reviews.llvm.org/D151848
2023-10-17 16:22:42 +00:00
Jay Foad
7b3bbd83c0 Revert "[CodeGen] Really renumber slot indexes before register allocation (#67038)"
This reverts commit 2501ae58e3bb9a70d279a56d7b3a0ed70a8a852c.

Reverted due to various buildbot failures.
2023-10-09 12:31:32 +01:00
Jay Foad
2501ae58e3
[CodeGen] Really renumber slot indexes before register allocation (#67038)
PR #66334 tried to renumber slot indexes before register allocation, but
the numbering was still affected by list entries for instructions which
had been erased. Fix this to make the register allocator's live range
length heuristics even less dependent on the history of how instructions
have been added to and removed from SlotIndexes's maps.
2023-10-09 11:44:41 +01:00
Tobias Stadler
305fbc1b32 Revert "[GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND"
This reverts commit 3686a0b611c65f0d7190345b8e3e73cdca9fa657.
This seems to have broken some sanitizer tests:
https://lab.llvm.org/buildbot/#/builders/184/builds/7721
2023-09-29 03:35:40 +02:00
Tobias Stadler
3686a0b611 [GlobalISel] LegalizationArtifactCombiner: Elide redundant G_AND
The legalizer currently generates lots of G_AND artifacts.
For example between boolean uses and defs there is always a G_AND with a mask of 1, but when the target uses ZeroOrOneBooleanContents, this is unnecessary.
Currently these artifacts have to be removed using post-legalize combines.
Omitting these artifacts at their source in the artifact combiner has a few advantages:
- We know that the emitted G_AND is very likely to be useless, so our KnownBits call is likely worth it.
- The G_AND and G_CONSTANT can interrupt e.g. G_UADDE/... sequences generated during legalization of wide adds which makes it harder to detect these sequences in the instruction selector (e.g. useful to prevent unnecessary reloading of AArch64 NZCV register).
- This cleans up a lot of legalizer output and even improves compilation-times.
AArch64 CTMark geomean: `O0` -5.6% size..text; `O0` and `O3` ~-0.9% compilation-time (instruction count).

Since this introduces KnownBits into code-paths used by `O0`, I reduced the default recursion depth.
This doesn't seem to make a difference in CTMark, but should prevent excessive recursive calls in the worst case.

Reviewed By: aemerson

Differential Revision: https://reviews.llvm.org/D159140
2023-09-29 02:11:57 +02:00
Mirko Brkusanin
72e3713009 [IRTranslator] Set NUW flag for inbounds gep and load/store offsets
Patch by: Acim Maravic

Differential Revision: https://reviews.llvm.org/D159515
2023-09-22 16:16:28 +02:00
pvanhout
490a867f16 [GlobalISel] Also set dead flags of implicit defs added by BuildMI
BuildMI automatically adds the implicit operands of the
instruction. This meant we couldn''t set the dead flag on
dead implicit defs in that case.

Fix it by introducing an opcode to mark a given implicit
def as dead.

Fixes #64565

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D157515
2023-08-11 08:38:37 +02:00
Matt Arsenault
1ca0808db2 GlobalISel: Don't expand stacksave/stackrestore in IRTranslator
In some (likely invalid edge cases anyway), it's not correct to
directly copy the stack pointer register.
2023-08-09 18:33:55 -04:00
pvanhout
c3cfbbc416 [GlobalISel] Add dead flags to implicit defs in ISel
Checks for implicit defs that are unused within a pattern and mark them as dead.

This is done directly at the TableGen level forr efficiency.
The instructions are directly created with the "dead" operand and no further analysis is needed later.

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D157273
2023-08-09 14:20:51 +02:00
Vitaly Buka
a496c8be6e Revert "[CodeGen]Allow targets to use target specific COPY instructions for live range splitting"
And dependent commits.

Details in D150388.

This reverts commit 825b7f0ca5f2211ec3c93139f98d1e24048c225c.
This reverts commit 7a98f084c4d121244ef7286bc6503b6a181d446e.
This reverts commit b4a62b1fa546312d882fa12dfdcd015177d66826.
This reverts commit b7836d856206ec39509d42529f958c920368166b.

No conflicts in the code, few tests had conflicts in autogenerated CHECKs:
llvm/test/CodeGen/Thumb2/mve-float32regloops.ll
llvm/test/CodeGen/AMDGPU/fix-frame-reg-in-custom-csr-spills.ll

Reviewed By: alexfh

Differential Revision: https://reviews.llvm.org/D156381
2023-07-26 22:13:32 -07:00
Yashwant Singh
b7836d8562 [CodeGen]Allow targets to use target specific COPY instructions for live range splitting
Replacing D143754. Right now the LiveRangeSplitting during register allocation uses
TargetOpcode::COPY instruction for splitting. For AMDGPU target that creates a
problem as we have both vector and scalar copies. Vector copies perform a copy over
a vector register but only on the lanes(threads) that are active. This is mostly sufficient
however we do run into cases when we have to copy the entire vector register and
not just active lane data. One major place where we need that is live range splitting.

Allowing targets to use their own copy instructions(if defined) will provide a lot of
flexibility and ease to lower these pseudo instructions to correct MIR.

- Introduce getTargetCopyOpcode() virtual function and use if to generate copy in Live range
 splitting.
- Replace necessary MI.isCopy() checks with TII.isCopyInstr() in register allocator pipeline.

Reviewed By: arsenm, cdevadas, kparzysz

Differential Revision: https://reviews.llvm.org/D150388
2023-07-07 22:29:50 +05:30
Simon Pilgrim
c63be92fc8 [GlobalISel][X86] Regenerate add/sub legalization tests 2023-07-06 14:09:11 +01:00
Simon Pilgrim
8269fd2db5 [GlobalIsel][X86] Add initial scalar G_MUL/G_SMULH/G_UMULH instruction selection handling
Reuse the existing div/rem selection code to also handle mul/imul to support G_MUL/G_SMULH/G_UMULH, as they have a similar pattern using rDX/rAX for mulh/mul results, plus the AH/AL support for i8 multiplies.
2023-07-02 12:56:41 +01:00
Thorsten Schütt
e0e998f8d8 [GlobalIsel][X86]] Legalize G_CONSTANT_FOLD_BARRIER
Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D153684
2023-06-24 17:19:51 +02:00
Thorsten Schütt
68ed9d9472 [GlobalIsel][X86] G_STORE extension
Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D153643
2023-06-24 05:51:30 +02:00
Simon Pilgrim
fb60dda189 [GlobalIsel][X86] selectDivRem - fix typo in 64-bit AH handling code
This function was lifted from fast-isel, and still referred to the Instruction::SRem/URrem opcodes, instead of the G_SREM/G_UREM opcodes.

But it turns out these aren't necessary at all as only the G_SREM/G_UREM codepaths will use the AH register for DivRemResultReg anyhow.
2023-06-18 17:37:17 +01:00
Simon Pilgrim
46479ea785 [GlobalIsel][X86] Regenerate srem/urem select test coverage 2023-06-18 17:06:32 +01:00
Simon Pilgrim
0342aefa32 [GlobalISel][X86] Add handling of scalar G_UADDO/G_USUBO opcodes
This finally allows x86 globalisel to lower addition/subtraction of illegal types without fallback :)
2023-06-16 15:15:27 +01:00
Simon Pilgrim
91afb4b769 [GlobalISel][X86] Add handling of scalar G_USUBE opcodes
Extend the G_UADDE handling to also support G_USUBE
2023-06-16 13:31:50 +01:00
Simon Pilgrim
cad1eefc7d [GlobalISel][X86] Add s8/s16/s64 handling of G_UADDE opcodes 2023-06-16 11:45:43 +01:00
Simon Pilgrim
29293e6f9d [GlobalISel][X86] Add i128 add/sub test coverage for narrowing on x86_64 triples 2023-06-15 22:40:42 +01:00
Simon Pilgrim
46b8cbbe01 [GlobalIsel][X86] Avoid IMPLICIT_DEF in irregular types for CTTZ/CTLZ legalization tests
IMPLICIT_DEF can lead to some pretty weird G_UNMERGE_VALUES of long lists of s1 values, try to use a real source value instead
2023-06-15 22:28:38 +01:00
Simon Pilgrim
78a0b2be83 [GlobalIsel][X86] Regenerate legalize-add.mir with common CHECK prefix 2023-06-14 15:01:18 +01:00
Simon Pilgrim
35def3034f [GlobalIsel][X86] Regenerate legalize-phi.mir 2023-06-13 10:40:23 +01:00
Simon Pilgrim
3d34f7be73 [GlobalIsel][X86] Rename x86_64-select-fcmp.mir to select-fcmp.mir and add 32-bit test coverage
x86_64 was being used as shorthand for SSE2
2023-06-12 12:41:27 +01:00
Simon Pilgrim
22c17c6a1f [GlobalIsel][X86] Move G_FCMP getActionDefinitionsBuilder out of setLegalizerInfo64bit and add 32-bit support
We were using x86_64-only support as a SSE2 proxy
2023-06-12 12:18:37 +01:00
Simon Pilgrim
6a12ab874a [GlobalIsel][X86] Regenerate legalize-fcmp.mir 2023-06-12 12:18:36 +01:00
Simon Pilgrim
1a576aa09d [GlobalIsel][X86] Rename x86_64-legalize-fcmp to legalize-fcmp
32-bit support will be added shortly - x86_64 was being used a shorthand for SSE2
2023-06-12 12:18:36 +01:00
Simon Pilgrim
26706807c9 [GlobalIsel][X86] Ensure bit count legalizer patterns keep matching result + input scalar types 2023-06-11 18:56:28 +01:00
Thorsten Schütt
24f49deacf [GlobalIsel][X86] Legalize G_FREEZE
Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D152501
2023-06-10 07:33:02 +02:00
Simon Pilgrim
0662167c5b [GlobalIsel][X86] Update legalization of G_PTR_ADD
Replace the legacy legalizer versions

Add test coverage for 32-bit targets and non-constant ptr offsets
2023-06-09 13:27:25 +01:00
Simon Pilgrim
b8f053f5d7 [GlobalIsel][X86] Add 32-bit test coverage to zero count tests
This shows a current problem with G_CTTZ_ZERO_UNDEF result legalizations
2023-06-09 10:36:32 +01:00
Simon Pilgrim
2717d98a1e [GlobalIsel][X86] legalize-select.mir - add x86-64 test coverage 2023-06-09 10:36:32 +01:00
Thorsten Schütt
0b771c679a [GlobalIsel][X86] Legalize G_SELECT
with bug fixes

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D152445
2023-06-08 18:54:25 +02:00
Simon Pilgrim
85b77b13e3 [GlobalISel][X86] Add G_IMPLICIT_DEF / G_CONSTANT legalization handling 2023-06-06 11:45:22 +01:00
Thorsten Schütt
60b8019ea0 [GlobalIsel][X86] Legalize G_ANYEXT, G_SEXT, and G_ZEXT
Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D152243
2023-06-06 12:22:09 +02:00
Simon Pilgrim
c2926c6c4d [GlobalISel][X86] Regenerate legalize-undef.mir 2023-06-05 14:41:40 +01:00
Simon Pilgrim
b28bc5f5ad [GlobalISel][X86] Add 128/256/512-bit vector and/or/xor test coverage
Based off the legalize-add-v*.mir tests
2023-06-05 12:08:22 +01:00
Simon Pilgrim
dbd3695092 [GlobalISel][X86] Add illegal types and 32-bit target scalar and/or/xor test coverage
Based off the legalize-add.mir tests
2023-06-05 12:08:22 +01:00
Simon Pilgrim
346ee549e5 [GlobalISel][X86] Add G_CTTZ_ZERO_UNDEF/G_CTTZ legalization handling
G_CTTZ_ZERO_UNDEF is always legal using the BSF instruction, G_CTTZ requires the BMI1 TZCNT instruction
2023-06-05 11:18:27 +01:00
Simon Pilgrim
9424a54201 [GlobalIsel][X86] Update legalization of G_AND/G_OR/G_XOR
Replace the legacy G_AND/G_OR/G_XOR legalizer, this handles all scalar promotion and vector clamping (allows AVX1 to handle 256-bit logic ops).
2023-06-04 11:44:27 +01:00