162 Commits

Author SHA1 Message Date
Florian Hahn
4a5bcbd560
[ConstraintElim] Store conditional facts as (Predicate, Op0, Op1).
This allows to add facts even if no corresponding ICmp instruction
exists in the IR.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D158837
2023-08-30 10:54:28 +01:00
Florian Hahn
e6260ec49c
[ConstraintElim] Use enum class to differentiate fact/check types (NFC).
Use an enum to clarify the type of fact or check in FactOrCheck, as
suggested in D158837.
2023-08-29 14:23:24 +01:00
Erik Desjardins
66ec5df3a7 [ConstraintElim] fix crash with large constants in mul nsw
Another case of https://github.com/llvm/llvm-project/issues/55085.

The added test would trip an assertion due to calling `getSExtValue()` on a value that doesn't fit in int64_t.

Differential Revision: https://reviews.llvm.org/D158810
2023-08-25 13:20:02 -04:00
Florian Hahn
13ffde316a
[ConstraintElim] Remove dead compares after simplification.
Remove compares after replacing all uses. Cleaning dead compares can
enable additional simplifications when adjusting the position of the
pass slightly. In particular, it seems like the additional dead
instructions may prevent SimplifyCFG performing some folds.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D158760
2023-08-24 22:12:57 +01:00
Yingwei Zheng
92a11eb32c
[ConstraintElim] Add facts implied by MinMaxIntrinsic
Fixes https://github.com/llvm/llvm-project/issues/63896 and https://github.com/rust-lang/rust/issues/113757.
This patch adds facts implied by llvm.smin/smax/umin/umax intrinsics.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D155412
2023-07-24 15:03:34 +08:00
Yingwei Zheng
cce5324994
[ConstraintElim] Store the triple Pred + LHS + RHS in ReproducerEntry instead of CmpInst + Not
This patch represents a condition with `Pred + LHS + RHS` in ReproducerEntry instead of `CmpInst + Not`.
It avoids creating temporary ICmpInsts in D155412.

Reviewed By: nikic, fhahn

Differential Revision: https://reviews.llvm.org/D155782
2023-07-20 23:21:36 +08:00
Antonio Frighetto
a2ba4e8075 [ConstraintElimination] Handle solving-only ICMP_NE predicates
Simplification of non-equality predicates for solving constraint
systems is now supported by checking the validity of related
inequalities and equalities.

Differential Revision: https://reviews.llvm.org/D152684
2023-06-29 21:22:48 +02:00
Florian Hahn
1689c357ae
[ConstraintElim] Allow and check preconditions in doesHold.
Delegate checking of the constraint & its preconditions to the existing
::isValid. This reduces duplication and allows additional optimizations
together with D152730.
2023-06-29 09:17:37 +01:00
Florian Hahn
0ad6879a62
[ConstraintElim] Try to use first cmp to prove second cmp for ANDs.
This patch extends the existing logic to handle cases where we have
branch conditions of the form (AND icmp, icmp) where the first icmp
implies the second. This can improve results in some cases, e.g. if
SimplifyCFG folded conditions from multiple branches to an AND.

The implementation handles this by adding a new type of check
(AndImpliedCheck), which are queued before conditional facts for the same
block.

When encountering AndImpliedChecks during solving, the first condition
is optimistically added to the constraint system, then we check if the
second icmp can be simplified, and finally the newly added entries are
removed.

The reason for doing things this way is to avoid clashes with signed
<-> unsigned condition transfer, which require us to re-order facts to
increase effectiveness.

Reviewed By: nikic, antoniofrighetto

Differential Revision: https://reviews.llvm.org/D151799
2023-06-28 13:19:41 +01:00
Florian Hahn
4b47711ae2
[ConstraintElim] Move condition check logic to helper function (NFC).
This allows easier re-use of the checking logic. Split off from D151799.
2023-06-28 13:02:12 +01:00
Florian Hahn
e6aa29eddd
[ConstraintElim] Move FactOrCheck and State definitions to top. (NFC)
This will enable follow-up refactoring to use the State directly in the
constraint system, reducing the need to pass lots of arguments around.
2023-06-28 09:40:54 +01:00
Florian Hahn
6c25c58a4d
[ConstraintElim] Track and simplify conditions at use.
This patch updates ConstraintElimination to track uses of conditions in
the worklist. This allows simplifying conditions using the context that
holds directly at the condition, instead of where the condition is
defined.

This allows us to catch more cases in practice: there are multiple
code-size changes for CTMark while compile-time remains unchanged:
https://llvm-compile-time-tracker.com/compare.php?from=4b020cca9363bebab4643f89cfa92ab2fcc7976c&to=7a6e84b8f029713c137814cd46eec775d24a54a1&stat=instructions:u

This should help to simplify D151799.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D153660
2023-06-27 17:27:49 +01:00
Antonio Frighetto
1774c14816 [ConstraintElimination] Handle ICMP_EQ predicates
Simplification of equality predicates is now supported by
transferring equalities into inequalities. This is achieved
by separately checking that both `isConditionImplied(A >= B)`
and `isConditionImplied(A <= B)` hold.

Differential Revision: https://reviews.llvm.org/D152067
2023-06-11 15:22:31 +02:00
Antonio Frighetto
420cf63ead [ConstraintElimination] Refactor checkAndReplaceCondition (NFC)
Handling `true` and `false` constant replacements is now abstracted
out into a single lambda function `ReplaceCmpWithConstant`, so as to
reduce code duplication.
2023-06-05 16:54:58 +02:00
Florian Hahn
6bcbb3af05
[ConstraintElim] Move logic to remove stack entry to helper (NFC).
Preparation for follow-up patch that uses the logic in a separate place.
2023-05-31 14:22:45 +01:00
Florian Hahn
78bb06b127
[ConstraintElim] Port mul nuw for unsigned to mul nsw to signed.
Add handling for `mul nsw` for signed systems based on the logic for
`mul nuw` for unsigned.
2023-04-25 20:36:09 +01:00
Florian Hahn
0a0181dc20
[ConstraintElim] Fix integer overflow in ConstraintSystem::negate.
This fixes another integer overflow that was exposed by a variant of the
test case from #62226.
2023-04-21 16:09:46 +01:00
Florian Hahn
626f6ee7f8
[ConstraintElim] Fix integer overflow in getConstraint.
Use SubOverflow to avoid signed integer overflow when combining
coefficients.

Fixes #62226.
2023-04-20 16:14:06 +01:00
Zain Jaffal
721ecc9d41 [ConstraintElimination] Transfer info from sgt %a, %b to ugt %a, %b if %b > 0
Differential Revision: https://reviews.llvm.org/D148326
2023-04-17 09:27:33 +01:00
Zain Jaffal
1d23d60c8d [ConstraintElimination] Add function arguments to constraint system before solving
If there is an optimisation opportunity and the function argument hasn’t been added to constraint
system through previous facts we fail to optimise it.

It might be a good idea to start the constraint system with all the function arguments added to the system

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D144879
2023-04-03 14:16:49 +01:00
Florian Hahn
02885ef3e4
[ConstraintElim] Use GEPOperator instead of GetElementPtrInst.
The logic in ConstraintElimination should trivially apply to GEP
constant expressions as well, so update code to deal with GEPOperator
instead.
2023-03-24 19:16:08 +00:00
Kazu Hirata
c8f9555c4d [Transforms] Use *{Set,Map}::contains (NFC) 2023-03-14 00:24:30 -07:00
Vitaly Buka
bb01b9919f [ConstraintElimination] Fix UB after D145677 2023-03-09 18:18:29 -08:00
Zain Jaffal
7faa8c6ec6 [ConstraintElimination] Fix undefined behaviour in shl decomposition
Add checks to prevent decomposing constants bigger than 64.
relates to https://github.com/llvm/llvm-project/issues/61127

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D145677
2023-03-09 18:17:27 +00:00
Zain Jaffal
2b62774b65
[ConstraintElimination] Decompose or instruction if the constant operand < 2^known_zero_bits of the first operand.
The or operation can be represented as an add instruction.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D142546
2023-03-07 14:11:35 +00:00
Nikita Popov
ffe8f47d72 [IR] Add operator<< overload for CmpInst::Predicate (NFC)
I regularly try and fail to use this while debugging.
2023-03-07 15:10:56 +01:00
Zain Jaffal
07f93d8c2c Recommit "[ConstraintElimination] Change debug output to display variable names."
This reverts commit 02ae7e72b3f00969eeb579a2b4346082827f0b35.

include Value.h in ConstraintSystem.h
2023-02-15 16:38:35 +00:00
Nikita Popov
02ae7e72b3 Revert "Recommit "[ConstraintElimination] Change debug output to display variable names.""
This reverts commit 2a2a6bfcfe8e62886542cb673ac8df349cf26499.

This causes build failures:

    /home/npopov/repos/llvm-project/llvm/lib/Analysis/ConstraintSystem.cpp: In member function ‘llvm::SmallVector<std::__cxx11::basic_string<char> > llvm::ConstraintSystem::getVarNamesList() const’:
    /home/npopov/repos/llvm-project/llvm/lib/Analysis/ConstraintSystem.cpp:118:10: error: invalid use of incomplete type ‘class llvm::Value’
      118 |     if (V->getName().empty())
          |          ^~
    In file included from /home/npopov/repos/llvm-project/llvm/lib/Analysis/ConstraintSystem.cpp:9:
    /home/npopov/repos/llvm-project/llvm/include/llvm/Analysis/ConstraintSystem.h:21:7: note: forward declaration of ‘class llvm::Value’
       21 | class Value;
          |       ^~~~~
    /home/npopov/repos/llvm-project/llvm/lib/Analysis/ConstraintSystem.cpp:119:22: error: invalid use of incomplete type ‘class llvm::Value’
      119 |       OperandName = V->getNameOrAsOperand();
          |                      ^~
    /home/npopov/repos/llvm-project/llvm/include/llvm/Analysis/ConstraintSystem.h:21:7: note: forward declaration of ‘class llvm::Value’
       21 | class Value;
          |       ^~~~~
    /home/npopov/repos/llvm-project/llvm/lib/Analysis/ConstraintSystem.cpp:121:41: error: invalid use of incomplete type ‘class llvm::Value’
      121 |       OperandName = std::string("%") + V->getName().str();
          |                                         ^~
    /home/npopov/repos/llvm-project/llvm/include/llvm/Analysis/ConstraintSystem.h:21:7: note: forward declaration of ‘class llvm::Value’
       21 | class Value;
          |       ^~~~~
2023-02-15 16:36:44 +01:00
Zain Jaffal
2a2a6bfcfe Recommit "[ConstraintElimination] Change debug output to display variable names."
This reverts commit 62d0e1a8541f93dfbf66d982f66da32676df2df7.

remove `dumpWithNames` function
2023-02-15 15:23:53 +00:00
Zain Jaffal
62d0e1a854 Revert "[ConstraintElimination] Change debug output to display variable names."
This reverts commit 869c87ad10e87db7c032c3464338ab9d50916510.

`dumpWithNames` function should be removed
2023-02-15 15:21:46 +00:00
Zain Jaffal
869c87ad10 [ConstraintElimination] Change debug output to display variable names.
Previously when constraint system outputs the rows in the system the variables used are x1,2...n making it hard to infer which ones they relate to in the IR

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D142618
2023-02-15 15:07:48 +00:00
Florian Hahn
f044217392
[ConstraintElim] Add reproducer remarks.
This patch adds an optimization remark for each performed optimization
containing a module that can be used to reproduce the transformation.

The reproducer function contains a series of @llvm.assume calls, one for
each condition currently in scope. For each condition, the operand
instruction are cloned until we reach operands that have an entry in the
constraint system. Those will then be added as function arguments.

The reproducer functions are minimal, that is, they only contain the
conditions required for a given simplification. The resulting IR is very
compact and can be used to verify each transformation individually.

It also provides a python script to extract the IR from the remarks and
create LLVM IR files from it.

Reviewed By: paquette

Differential Revision: https://reviews.llvm.org/D143323
2023-02-14 15:15:57 +00:00
Zain Jaffal
a9d6a86b21 Recommit "[ConstraintElimination] Move Value2Index map to ConstraintSystem (NFC)"
This reverts commit 665ee0cd57f92a112cf8e929d00768e282fb205a.

Fix comments and formatting style.
2023-02-08 14:52:43 +00:00
Zain Jaffal
665ee0cd57 Revert "[ConstraintElimination] Move Value2Index map to ConstraintSystem (NFC)"
This reverts commit 40ffe9c167395256b43846733ab69eec17eead78.

Reverted because some comments where missed in the review https://reviews.llvm.org/D142647
2023-02-08 11:29:59 +00:00
Zain Jaffal
40ffe9c167 [ConstraintElimination] Move Value2Index map to ConstraintSystem (NFC)
Differential Revision: https://reviews.llvm.org/D142647
2023-02-08 11:15:31 +00:00
Florian Hahn
23ce9383ca
[ConstraintElim] Add option to limit number of rows tracked in system.
Once the constraint system grows too large in terms of number of rows,
queries can become very slow. This patch adds a new option to limit the
number of rows tracked.

The python script below can be used to generate worst-case IR with a
chain of conditional branches with N branches.

With this limit, we get the following runtimes:
* python3 generate.py 100:   0.1s
* python3 generate.py 1000:  2s
* python3 generate.py 10000: 4s

Without the limit, the case with 1000 chained conditions takes 20+
seconds.

generate.py:
    import sys

    N = int(sys.argv[1])

    args = []
    checks = []

    for i in range(0, N):
        args.append('i32 %l{}'.format(i))
        checks.append("""
    bb{0}:
      %c{0} = icmp uge i32 %l{0}, 100
      br i1 %c{0}, label %bb{1}, label %exit
    """.format(i, i+1))

    print("""
    define i1 @foo({0}) {{
    {1}

    bb{2}:
      %c{2} = icmp uge i32 %l0, 100
      ret i1 %c{2}

    exit:
      ret i1 false
    }}
    """.format(' ,'.join(args), '\n'.join(checks), N))

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D140926
2023-01-04 13:59:23 +00:00
Florian Hahn
f8d008d19f
[ConstraintElim] Remove legacy pass implementation.
The pass is exclusively used with the new pass manager now, so remove
the legacy PM implementation.
2023-01-04 11:21:12 +00:00
Florian Hahn
8fa81cfad4
[ConstraintElim] Address comments from D137840.
Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D139482
2022-12-07 14:12:53 +00:00
Florian Hahn
9eda78107c
[ConstraintElim] Fix sort order to not comparing insts in different bbs.
Update the sort order to make sure that comesBefore is never used from
conditional facts, which are instructions but may use DFS numbers from
different blocks.

This fixes a crash in the added test on some platforms.
2022-12-06 23:47:24 +00:00
Florian Hahn
6887cfb982
[ConstraintElim] Queue facts and checks directly.
This allows interleaving facts and checks in a single block. In
particular this enables using facts from assumes for conditions in the
same block that come after the assume.

This could be extended to only try to simplify checks at the point where
a condition is used.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D138452
2022-12-05 16:44:20 +00:00
Florian Hahn
6b940588a0
[ConstraintElim] Update comment after recent changes.
Update the comment to reflect the current code.
2022-12-05 12:19:09 +00:00
Florian Hahn
ee605b0acc
[ConstraintElim] Use collectOffset result for chained gep support.
This slightly simplifies the code and addresses a correctness issue
where the index scaling for the precondition was not considered
properly.

Thanks to @nikic for pointing that out in D137840.
2022-12-01 17:01:07 +00:00
Florian Hahn
6a834d2f2b
[ConstraintElim] Use collectOffset instead of custom GEP idx iteration.
Use collectOffset to collect scaled indices and constant offset for GEP
instead of custom code. This simplifies the logic in decomposeGEP and
allows to handle all cases supported by the generic helper.
2022-11-28 16:54:58 +00:00
Nikita Popov
42d9261417 [ConstraintElimination] Use SmallVectorImpl (NFC)
When passing a SmallVector by reference, don't specify its size.
2022-11-14 11:01:15 +01:00
Nikita Popov
e82b5b5bbd [ConstraintElimination] Add Decomposition struct (NFCI)
Replace the vector of DecompEntry with a struct that stores the
constant offset separately. I think this is cleaner than giving the
first element special handling.

This probably also fixes some potential ubsan errors by more
consistently using addWithOverflow/multiplyWithOverflow.
2022-11-14 10:44:16 +01:00
Nikita Popov
30982a595d [ConstraintElimination] Make decompose() infallible
decompose() currently returns a mix of {} and 0 + 1*V on failure.
This changes it to always return the 0 + 1*V form, thus making
decompose() infallible.

This makes the code marginally more powerful, e.g. we now fold
sub_decomp_i80 by treating the constant as a symbolic value.

Differential Revision: https://reviews.llvm.org/D137847
2022-11-14 10:42:04 +01:00
Florian Hahn
8086b0c8a8
[ConstraintElim] Drop bail out for scalable vectors after using getTrue
ConstantInt::getTrue/getFalse can materialize scalable vectors with all
lanes true/false.
2022-11-03 19:05:45 +00:00
Florian Hahn
44d8f80b26
[ConstraintElim] Use ConstantInt::getTrue to create constants (NFC).
Use existing ConstantInt::getTrue/getFalse functionality instead of
custom getScalarConstOrSplat as suggested by @nikic.
2022-11-03 12:20:23 +00:00
Florian Hahn
74d8628cf7
[ConstraintElimination] Skip compares with scalable vector types.
Materializing scalable vectors with boolean values is not implemented
yet. Skip those cases for now and leave a TODO.
2022-11-02 23:57:14 +00:00
Florian Hahn
b478d8b966
[ConstraintElimination] Generate true/false vectors for vector cmps.
This fixes crashes when vector compares can be simplified to true/false.
2022-11-02 18:13:35 +00:00