2469 Commits

Author SHA1 Message Date
Christian Kissig
730df5a437
[Support] Add KnownBits::computeForSubBorrow (#67788)
- [Support] Add KnownBits::computeForSubBorrow
- [CodeGen] Implement USUBC, USUBO_CARRY, and SSUBO_CARRY with
KnownBits::computeForSubBorrow
- [CodeGen] Compute unknown bits for Carry/Borrow for ADD/SUB
- [CodeGen] Compute known bits of Carry/Borrow for UADDO, SADDO, USUBO,
and SSUBO

Fixes #65893

---------

Co-authored-by: Shafik Yaghmour <shafik@users.noreply.github.com>
2023-10-18 13:48:47 +01:00
Pierre van Houtryve
c464fea779
[DAG] Constant fold FMAD (#69324)
This has very little effect on codegen in practice, but is a nice to
have I think.

See #68315
2023-10-18 07:46:24 +02:00
Simon Pilgrim
2f329d88bc [DAG] foldConstantFPMath - accept ArrayRef<SDValue> Ops instead of explicit N1/N2 ops
First step towards adding unary/ternary fp ops handling, and not just binops
2023-10-17 16:31:46 +01:00
Björn Pettersson
4acb96c99f
[SelectionDAG] Tidy up around endianness and isConstantSplat (#68212)
The BuildVectorSDNode::isConstantSplat function could depend on
endianness, and it takes a bool argument that can be used to indicate
if big or little endian should be considered when internally casting
from a vector to a scalar. However, that argument is default set to
false (= little endian). And in many situations, even in target
generic code such as DAGCombiner, the endianness isn't specified when
using the function.

The intent with this patch is to highlight that endianness doesn't
matter, depending on the context in which the function is used.

In DAGCombiner the code is slightly refactored. Back in the days when
the code was written it wasn't possible to request a MinSplatBits
size when calling isConstantSplat. Instead the code re-expanded the
found SplatValue to match with the EltBitWidth. Now we can just
provide EltBitWidth as MinSplatBits and remove the logic for doing
the re-expand.

While being at it, tidying up around isConstantSplat, this patch also
adds an explicit check in BuildVectorSDNode::isConstantSplat to break
out from the loop if trying to split an on VecWidth into two halves.
Haven't been able to prove that there could be miscompiles involved
if not doing so. There are lit tests that trigger that scenario,
although I think they happen to later discard the returned SplatValue
for other reasons.
2023-10-16 14:53:53 +02:00
Nikita Popov
d4300154b6 Revert "[ValueTracking] Remove by-ref computeKnownBits() overloads (NFC)"
This reverts commit b5743d4798b250506965e07ebab806a3c2d767cc.

This causes some minor compile-time impact. Revert for now, better
to do the change more gradually.
2023-10-16 14:04:09 +02:00
Nikita Popov
b5743d4798 [ValueTracking] Remove by-ref computeKnownBits() overloads (NFC)
Remove the old overloads that accept KnownBits by reference, in
favor of those that return it by value.
2023-10-16 13:00:31 +02:00
Kazu Hirata
e9fa18878c [SelectionDAG] Fix an unused variable warning
This patch fixes:

  llvm/lib/CodeGen/SelectionDAG/SelectionDAG.cpp:10832:12: error:
  variable 'Changed' set but not used
  [-Werror,-Wunused-but-set-variable]
2023-10-06 09:27:35 -07:00
Ben Mudd
6d6b395b53 [DebugInfo][SelectionDAG] Add debug info salvaging for TRUNC nodes
This patch adds support for salvaging TRUNC nodes during SelectionDAG,
fixing LLVM issue #63076:
  https://github.com/llvm/llvm-project/issues/63076

Reviewed in: https://github.com/llvm/llvm-project/pull/66922
2023-10-06 16:10:33 +01:00
Luke Lau
481df274d3
[RISCV][SelectionDAG] Sign extend splats of i32 in getConstant on RV64 (#67027)
We get better constant materialization if we sign extend the value to be
splatted for i32 on RV64 instead of zero extending it.
2023-10-03 12:57:39 +01:00
elhewaty
9103b1d68d
[DAG] Extend the computeOverflowForSignedSub/computeOverflowForUnsignedSub implementations with ConstantRange (#67890)
- Add tests for computeOverflowFor*Sub functions
- extend the computeOverflowForSignedSub/computeOverflowForUnsignedSub
implementations with ConstantRange (#37109)
2023-10-01 14:57:34 +01:00
Simon Pilgrim
8b36d082c4 [DAG] getNode() - fold (zext (trunc x)) -> x iff the upper bits are known zero - add SRL support
This is part of the work to address the D155472 regressions, there's a number of issues with generalizing this fold which is why I'm just adding SRL support atm.

Differential Revision: https://reviews.llvm.org/D159533
2023-09-24 13:40:07 +01:00
Simon Pilgrim
b61b2426ac [DAG] getNode() - remove oneuse limit from (zext (trunc (assertzext x))) -> (assertzext x) fold (REAPPLIED)
Noticed on D159533 and I've finally dealt with the x86 regressions - MatchingStackOffset wasn't peeking through AssertZext nodes while trying to find CopyFromReg/Load sources, it was only removing them if they were part of a (trunc (assertzext x)) pattern.

Reapplied after being reverted at 4389252c58b783ce5b - which should be addressed by D159537 / 6d2679992e58b
2023-09-22 11:01:38 +01:00
Fangrui Song
4389252c58 Revert "[DAG] getNode() - remove oneuse limit from (zext (trunc (assertzext x))) -> (assertzext x) fold"
This reverts commit 05926a5a557878aa233ac8431b3acddf54422e58.

Caused AArch64 crash

 #12 0x00007f09eec09181 skipExtensionForVectorMULL(llvm::SDNode*, llvm::SelectionDAG&)
 #13 0x00007f09eec08289 llvm::AArch64TargetLowering::LowerMUL(llvm::SDValue, llvm::SelectionDAG&) const
 #14 0x00007f09eec1a3fd llvm::AArch64TargetLowering::LowerOperation(llvm::SDValue, llvm::SelectionDAG&) const
 #15 0x00007f09dc8586a7 (anonymous namespace)::VectorLegalizer::LowerOperationWrapper(llvm::SDNode*, llvm::SmallVectorImpl<llvm::SDValue>&)
2023-09-22 00:14:31 -07:00
Simon Pilgrim
05926a5a55 [DAG] getNode() - remove oneuse limit from (zext (trunc (assertzext x))) -> (assertzext x) fold
Noticed on D159533 and I've finally deal with the x86 regressions - MatchingStackOffset wasn't peeking through AssertZext nodes while trying to find CopyFromReg/Load sources, it was only removing them if they were part of a (trunc (assertzext x)) pattern.
2023-09-21 12:07:49 +01:00
Noah Goldstein
6d6314ba64 [DAGCombiner] Extend combineFMulOrFDivWithIntPow2 to work for non-splat float vecs
Do so by extending `matchUnaryPredicate` to also work for
`ConstantFPSDNode` types then encapsulate the constant checks in a
lambda and pass it to `matchUnaryPredicate`.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D154868
2023-09-20 13:28:24 -05:00
Simon Pilgrim
b2ffc867ad [DAG] getNode() - begin generalizing the (zext (trunc (assertzext x))) -> (assertzext x) fold.
We'll need to generalize this fold to check for any zero upperbits to address some of the D155472 regressions, but this exposes a number of issues. For now, just use the general MaskedValueIsZero test instead of the assertzext.
2023-09-18 15:32:31 +01:00
Arthur Eubanks
0a1aa6cda2
[NFC][CodeGen] Change CodeGenOpt::Level/CodeGenFileType into enum classes (#66295)
This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future change to the params of
TargetMachine.

This matches other nearby enums.

For downstream users, this should be a fairly straightforward
replacement,
e.g. s/CodeGenOpt::Aggressive/CodeGenOptLevel::Aggressive
or s/CGFT_/CodeGenFileType::
2023-09-14 14:10:14 -07:00
Simon Pilgrim
e6b85c3027 [DAG] FoldSetCC - add missing icmp(X,undef) -> isTrueWhenEqual case (REAPPLIED)
Followup to D59363 which failed to handle the icmp(X,undef) -> isTrueWhenEqual case - similar to llvm::ConstantFoldCompareInstruction

As discussed on the review, this is affecting some previously reduced test cases, but will also prevent reductions from relying on this inconsistent behaviour in the future.

Reapplied after reversion at e1e3c75c7dad72 with a tweak to the pseudo-probe-peep.ll test

Differential Revision: https://reviews.llvm.org/D158068
2023-09-13 12:33:39 +01:00
Simon Pilgrim
e1e3c75c7d Revert rG6c56cf71ee82ec3a28e0dfc2b751bd10c16929da "[DAG] FoldSetCC - add missing icmp(X,undef) -> isTrueWhenEqual case"
Need to address a missed test change
2023-09-13 11:27:47 +01:00
Simon Pilgrim
6c56cf71ee [DAG] FoldSetCC - add missing icmp(X,undef) -> isTrueWhenEqual case
Followup to D59363 which failed to handle the icmp(X,undef) -> isTrueWhenEqual case - similar to llvm::ConstantFoldCompareInstruction

As discussed on the review, this is affecting some previously reduced test cases, but will also prevent reductions from relying on this inconsistent behaviour in the future.

Differential Revision: https://reviews.llvm.org/D158068
2023-09-13 11:01:58 +01:00
Mohamed Atef
741c127817 [SelectionDAG] Add computeOverflowForSignedMul / computeOverflowForUnsignedMul overflow handlers
Support signed multiplication
Support unsigned multiplication

Differential Revision: https://reviews.llvm.org/D159406
2023-09-07 10:03:18 +01:00
Simon Pilgrim
84447c044f [DAG] Add SelectionDAG::isADDLike helper. NFC.
Make the DAGCombine helper global so we can more easily reuse it.
2023-09-06 16:54:25 +01:00
Ting Wang
71be020dda [SelectionDAG][PowerPC] Memset reuse vector element for tail store
On PPC there are instructions to store element from vector(e.g.
stxsdx/stxsiwx), and these instructions can be leveraged to avoid tail
constant in memset and constant splat array initialization.

This patch tries to explore these opportunities.

Reviewed By: shchenz

Differential Revision: https://reviews.llvm.org/D138883
2023-09-06 01:52:38 -04:00
Simon Pilgrim
5463503ae1 [DAG] Move scalar BITCAST constant folds from getNode to FoldConstantArithmetic 2023-09-05 13:11:20 +01:00
Matt Arsenault
b14e83d1a4 IR: Add llvm.exp10 intrinsic
We currently have log, log2, log10, exp and exp2 intrinsics. Add exp10
to fix this asymmetry. AMDGPU already has most of the code for f32
exp10 expansion implemented alongside exp, so the current
implementation is duplicating nearly identical effort between the
compiler and library which is inconvenient.

https://reviews.llvm.org/D157871
2023-09-01 19:45:03 -04:00
Simon Pilgrim
15b561ed38 [DAG] Move STEP_VECTOR constant fold from getNode to FoldConstantArithmetic 2023-09-01 15:47:37 +01:00
Simon Pilgrim
1d47d5d67c [DAG] Move F16<->FP constant folds from getNode to FoldConstantArithmetic 2023-09-01 15:47:36 +01:00
Simon Pilgrim
4b9c2cf0a7 [DAG] Move INT<->FP constant folds from getNode to FoldConstantArithmetic 2023-09-01 14:02:02 +01:00
Daniel Paoliello
0c5c7b52f0 Emit the CodeView S_ARMSWITCHTABLE debug symbol for jump tables
The CodeView `S_ARMSWITCHTABLE` debug symbol is used to describe the layout of a jump table, it contains the following information:

* The address of the branch instruction that uses the jump table.
* The address of the jump table.
* The "base" address that the values in the jump table are relative to.
* The type of each entry (absolute pointer, a relative integer, a relative integer that is shifted).

Together this information can be used by debuggers and binary analysis tools to understand what an jump table indirect branch is doing and where it might jump to.

Documentation for the symbol can be found in the Microsoft PDB library dumper: 0fe89a942f/cvdump/dumpsym7.cpp (L5518)

This change adds support to LLVM to emit the `S_ARMSWITCHTABLE` debug symbol as well as to dump it out (for testing purposes).

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D149367
2023-08-31 12:06:50 -07:00
Simon Pilgrim
376050db9f [DAG] Move some unary constant folds from getNode() to FoldConstantArithmetic()
We need to clean up some type handling before the remainder (int<->fp and bitcasts) can be moved over.
2023-08-30 13:59:28 +01:00
Craig Topper
299b1b4071 [SelectionDAG][RISCV] Teach getConstant to use SPLAT_VECTOR_PARTS if vXi64 SPLAT_VECTOR is legal but i64 scalars are not.
That matches how such a SPLAT_VECTOR would have been type legalized
so assume it is ok to use for creating constants after type legalization.

Still need some improvements to SPLAT_VECTOR lowering.

This overlaps with some of what D158742 was trying to fix.

Reviewed By: luke

Differential Revision: https://reviews.llvm.org/D158870
2023-08-29 09:22:17 -07:00
Luke Lau
8f1d1e2b61 [SDAG] Add computeKnownBits support for ISD::SPLAT_VECTOR_PARTS
We can work out the known bits for a given lane by concatenating the known bits of each scalar operand.

In the description of ISD::SPLAT_VECTOR_PARTS in ISDOpcodes.h it says that the
total size of the scalar operands must cover the output element size, but I've
added a stricter assertion here that the total width of the scalar operands
must be exactly equal to the element size. It doesn't seem to trigger, and I'm
not sure if there any targets that use SPLAT_VECTOR_PARTS for anything other
than v4i32 -> v2i64 splats.

We also need to include it in isTargetCanonicalConstantNode, otherwise
returning the known bits introduces an infinite combine loop.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D158852
2023-08-28 10:35:58 +01:00
Arthur Eubanks
0a4fc4ac1c Revert "Emit the CodeView S_ARMSWITCHTABLE debug symbol for jump tables"
This reverts commit 8d0c3db388143f4e058b5f513a70fd5d089d51c3.

Causes crashes, see comments in https://reviews.llvm.org/D149367.

Some follow-up fixes are also reverted:

This reverts commit 636269f4fca44693bfd787b0a37bb0328ffcc085.
This reverts commit 5966079cf4d4de0285004eef051784d0d9f7a3a6.
This reverts commit e7294dbc85d24a08c716d9babbe7f68390cf219b.
2023-08-25 18:34:15 -07:00
Daniel Paoliello
8d0c3db388 Emit the CodeView S_ARMSWITCHTABLE debug symbol for jump tables
The CodeView `S_ARMSWITCHTABLE` debug symbol is used to describe the layout of a jump table, it contains the following information:

* The address of the branch instruction that uses the jump table.
* The address of the jump table.
* The "base" address that the values in the jump table are relative to.
* The type of each entry (absolute pointer, a relative integer, a relative integer that is shifted).

Together this information can be used by debuggers and binary analysis tools to understand what an jump table indirect branch is doing and where it might jump to.

Documentation for the symbol can be found in the Microsoft PDB library dumper: 0fe89a942f/cvdump/dumpsym7.cpp (L5518)

This change adds support to LLVM to emit the `S_ARMSWITCHTABLE` debug symbol as well as to dump it out (for testing purposes).

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D149367
2023-08-25 10:19:17 -07:00
Kazu Hirata
134115618a [CodeGen] Use isAllOnesConstant and isNullConstant (NFC) 2023-08-20 22:56:40 -07:00
Jeffrey Byrnes
d26a06728d [DAG] NFC: Add getBitcastedExtOrTrunc
Simple function which scalarizes Ops then ExtOrTruncs them according to function parameters

Differential Revision: https://reviews.llvm.org/D157733

Change-Id: Ie5215069228f7bf530cd2dbb4bd17cbf409e046a
2023-08-17 14:29:17 -07:00
Paul Walker
566065207b [SelectionDAG] Use TypeSize variant of ComputeValueVTs to compute correct offsets for scalable aggregate types.
Differential Revision: https://reviews.llvm.org/D157872
2023-08-16 11:56:31 +00:00
Noah Goldstein
2549ec1866 [SelectionDAG] Improve isKnownToBeAPowerOfTwo
Add additional cases for:
select, vselect, {u,s}{min,max}, and, casts, rotl, rotr

And improve handling of constants and shifts.

Differential Revision: https://reviews.llvm.org/D156778
2023-08-16 02:00:15 -05:00
Noah Goldstein
ac485e4072 [SelectionDAG] Add/Improve cases in isKnownNeverZero
1) Handle casts a bit more cleanly just with a loop rather than with
   recursion.

2) Add additional cases for smin/smax

3 ) For shifts we can also deduce non-zero if the maximum shift amount
    on the known 1s is non-zero.

Differential Revision: https://reviews.llvm.org/D156777
2023-08-16 02:00:15 -05:00
David Green
de775f264d [DAG] Add constant SPLAT handling in getNodes SIGN_EXTEND_INREG
This helps simplify constant splats a little. Without this the code in
llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp#L14072 always returns the
existing node.

Differential Revision: https://reviews.llvm.org/D157259
2023-08-08 10:27:55 +01:00
Matt Arsenault
0efdf3baf5 DAG: Remove getTargetIndex as it's unused
Fixes #29973
2023-08-05 09:20:09 -04:00
Bjorn Pettersson
4ce7c4a92a [llvm] Drop some typed pointer handling/bitcasts
Differential Revision: https://reviews.llvm.org/D157016
2023-08-03 22:54:33 +02:00
Simon Pilgrim
076bee1020 [DAG] getNode() - fold (zext (trunc (assertzext x))) -> (assertzext x)
If the pre-truncated value was the same width as the extension, and the assertzext guarantees that the extended bits are already zero, then skip the zext/trunc 'zero_extend_inreg' pattern.

Addresses several regressions noticed in D155472
2023-07-31 10:43:11 +01:00
Zhongyunde
05aae0839f Reland [AArch64][NFC] Call the API getVScaleRange directly
Use the maximum 64 for BitWidth of getVScaleRange to avoid returning an empty range.

the previous changes bring in a Buildbot failure because MinSVEVectorSize = MinSVEVectorSize.
    error: explicitly assigning value of variable of type 'unsigned int' to itself [-Werror,-Wself-assign]

Reviewed By: sdesmalen, nikic, dmgreen
Differential Revision: https://reviews.llvm.org/D155708
2023-07-26 18:55:31 +08:00
Zhongyunde
ebaac2b2d6 Revert "[AArch64][NFC] Call the API getVScaleRange directly"
This reverts commit 67005c8e6fa9464f8bc436305a422071013ae499.
2023-07-26 16:44:14 +08:00
Zhongyunde
67005c8e6f [AArch64][NFC] Call the API getVScaleRange directly
Use the maximum 64 for BitWidth of getVScaleRange to
avoid returning an empty range.

Reviewed By: sdesmalen, nikic, dmgreen
Differential Revision: https://reviews.llvm.org/D155708
2023-07-26 15:54:04 +08:00
David Green
0c41c59dee [DAG][AArch64] Fix truncated vscale constant types
It appears that vscale values truncated to i1 causes mismatches in the constant
types when created in getNode. https://godbolt.org/z/TaaTo86ne.

Differential Revision: https://reviews.llvm.org/D155626
2023-07-20 09:12:05 +01:00
Matt Arsenault
296e24cd2e DAG: Constant fold frexp nodes
Special casing the nonfinite exponent value everywhere is kind of
annoying.
2023-07-17 17:34:29 -04:00
Simon Pilgrim
4f95821f58 [DAG] SelectionDAG::getNode() - consistently use N1 for first operand. NFCI.
This has been annoying me for years - rename Operand to N1 so it matches all the other getNode() calls, and simplifies my debug watch windows!
2023-07-17 17:17:40 +01:00
Noah Goldstein
a4c461c063 [SelectionDAG] Fill in some more cases in isKnownNeverZero
This mostly copies cases that already exist in ValueTracking, although
it skips the more complex ones. Those can be filled in as needed.

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D149199
2023-07-12 17:17:53 -05:00