1362 Commits

Author SHA1 Message Date
DianQK
96ea48ff5d
[SimplifyCFG] Hoist common instructions on Switch.
Sink common instructions are not always performance friendly. We need to implement hoist common instructions on switch instruction to solve the following problem:
```
define i1 @foo(i64 %a, i64 %b, i64 %c, i64 %d) {
start:
  %test = icmp eq i64 %a, %d
  br i1 %test, label %switch_bb, label %exit

switch_bb:                                        ; preds = %start
  switch i64 %a, label %bb0 [
    i64 1, label %bb1
    i64 2, label %bb2
  ]

bb0:                                              ; preds = %switch_bb
  %0 = icmp eq i64 %b, %c
  br label %exit

bb1:                                              ; preds = %switch_bb
  %1 = icmp eq i64 %b, %c
  br label %exit

bb2:                                              ; preds = %switch_bb
  %2 = icmp eq i64 %b, %c
  br label %exit

exit:                                             ; preds = %bb2, %bb1, %bb0, %start
  %result = phi i1 [ false, %start ], [ %0, %bb0 ], [ %1, %bb1 ], [ %2, %bb2 ]
  ret i1 %result
}
```
The pre-commit test is D156617.

Reviewed By: XChy, nikic

Differential Revision: https://reviews.llvm.org/D155711
2023-09-20 07:21:49 +08:00
Kohei Asano
fef8249220
[SimplifyCFG] handle monotonic wrapped case for D150943 (#65882) 2023-09-14 21:26:11 +09:00
Jeremy Morse
e54277fa10 [NFC][RemoveDIs] Use iterators over inst-pointers when using IRBuilder
This patch adds a two-argument SetInsertPoint method to IRBuilder that
takes a block/iterator instead of an instruction, and updates many call
sites to use it. The motivating reason for doing this is given here [0],
we'd like to pass around more information about the position of debug-info
in the iterator object. That necessitates passing iterators around most of
the time.

[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939

Differential Revision: https://reviews.llvm.org/D152468
2023-09-11 20:01:19 +01:00
Jeremy Morse
1d82c765ef [NFC][RemoveDIs] Provide an iterator-taking split-block method
As per the stack of patches this is attached to, allow users of
BasicBlock::splitBasicBlock to provide an iterator for a position, instead
of just an instruction pointer. This is to fit with my proposal for how to
get rid of debug intrinsics [0]. There are other call-sites that would need
to change, but this is sufficient for a stage2clang self host and some
other C++ projects to build identical binaries, in the context of the whole
remove-DIs project.

[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939

Differential Revision: https://reviews.llvm.org/D152545
2023-09-11 17:50:47 +01:00
Jeremy Morse
6942c64e81 [NFC][RemoveDIs] Prefer iterator-insertion over instructions
Continuing the patch series to get rid of debug intrinsics [0], instruction
insertion needs to be done with iterators rather than instruction pointers,
so that we can communicate information in the iterator class. This patch
adds an iterator-taking insertBefore method and converts various call sites
to take iterators. These are all sites where such debug-info needs to be
preserved so that a stage2 clang can be built identically; it's likely that
many more will need to be changed in the future.

At this stage, this is just changing the spelling of a few operations,
which will eventually become signifiant once the debug-info bearing
iterator is used.

[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939

Differential Revision: https://reviews.llvm.org/D152537
2023-09-11 11:48:45 +01:00
Jeremy Morse
4427407a29 [NFC][RemoveDIs] Create a new spelling of the moveBefore method
As outlined in my proposal of how to get rid of debug intrinsics, this
patch adds a moveBefore method that signals the caller /intends/ the order
of moved instructions is to stay the same. This semantic difference has an
effect on debug-info, as it signals whether debug-info needs to move with
instructions or not.

The patch just replaces a few calls to moveBefore with calls to
moveBeforePreserving -- and the latter just calls the former, so it's all
NFC right now. A future patch will add an implementation of
moveBeforePreserving that takes action to correctly preserve debug-info,
but that's tightly coupled with our non-instruction debug-info
representation that's still being reviewed.

[0] https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939

Differential Revision: https://reviews.llvm.org/D156369
2023-09-07 18:37:57 +01:00
Kazu Hirata
83e6931827 [llvm] Use llvm::is_contained (NFC) 2023-09-02 09:32:46 -07:00
Nikita Popov
4eafc9b6ff [IR] Treat callbr as special terminator (PR64215)
isLegalToHoistInto() currently return true for callbr instructions.
That means that a callbr with one successor will be considered a
proper loop preheader, which may result in instructions that use
the callbr return value being hoisted past it.

Fix this by adding callbr to isExceptionTerminator (with a rename
to isSpecialTerminator), which also fixes similar assumptions in
other places.

Fixes https://github.com/llvm/llvm-project/issues/64215.

Differential Revision: https://reviews.llvm.org/D158609
2023-08-25 09:20:18 +02:00
Aleksandr Popov
d6e7c162e1 [NFC][GuardUtils] Add util to extract widenable conditions
This is the next preparation patch to support widenable conditions
widening instead of branches widening.

We've added parseWidenableGuard util which parses guard condition and
collects all checks existing in the expression tree: D157276

Here we are adding util which walks similar way through the expression
tree but looks up for widenable condition without collecting the checks.
Therefore llvm::extractWidenableCondition could parse widenable branches
with arbitrary position of widenable condition in the expression tree.

llvm::parseWidenableBranch which is we are going to get rid of is being
replaced by llvm::extractWidenableCondition where it's possible.

Reviewed By: anna

Differential Revision: https://reviews.llvm.org/D157529
2023-08-18 17:36:05 +02:00
Florian Hahn
b7a95ad467
[SimplifyCFG] Don't sink loads/stores with swifterror pointers.
swifterror pointers can only be used as pointer operands of load & store
instructions (and as swifterror argument of a call). Sinking loads or
stores with swifterror pointer operands would require introducing a
select of of the pointer operands, which isn't allowed.

Check for this condition in canSinkInstructions.

Reviewed By: aschwaighofer

Differential Revision: https://reviews.llvm.org/D158083
2023-08-17 09:59:07 +01:00
Nikita Popov
51dfe3cb3b [IR] Add PHINode::removeIncomingValueIf() (NFC)
Add an API that allows removing multiple incoming phi values based
on a predicate callback, as suggested on D157621.

This makes sure that the removal is linear time rather than quadratic,
and avoids subtleties around iterator invalidation.

I have replaced some of the more straightforward users with the new
API, though there's a couple more places that should be able to use it.

Differential Revision: https://reviews.llvm.org/D158064
2023-08-17 09:09:14 +02:00
Ivan Kosarev
e9df4c9892 [ADT] Support iterating size-based integer ranges.
It seems the ranges start with 0 in most cases.

Reviewed By: dblaikie, gchatelet

Differential Revision: https://reviews.llvm.org/D156135
2023-07-26 16:28:41 +01:00
Teresa Johnson
5986559caa [SimplifyCFG] Guard branch folding by speculate blocks flag
Guard FoldBranchToCommonDest in SimplifyCFG with the SpeculateBlocks
flag as it can also speculate instructions.

This was split out of D155997.

Differential Revision: https://reviews.llvm.org/D156194
2023-07-25 06:46:19 -07:00
Nuno Lopes
9f90669571 [SimplifyCFG] Use poison instead of undef as placeholder [NFC]
This is used in a phi node that is created for which only 1 value is accessed (the non-poison)
2023-07-22 12:44:21 +01:00
Serguei Katkov
1614805eeb Register new assumption in a cache
When new assumption is created it should be registered in assumption cache
or cache should be invalidated.

Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D154601
2023-07-07 10:53:03 +07:00
Nikita Popov
bb3763e497 Revert "[SimplifyCFG] Allow dropping block that only contains ephemeral values"
This reverts commit 20f0c68fd83a0147a8ec1722bd2e848180610288.

https://reviews.llvm.org/D153966#4464594 reports an optimization
regression in Rust.

Additionally this change has caused an unexpected 0.3% compile-time
regression.
2023-06-30 21:24:05 +02:00
Nikita Popov
20f0c68fd8 [SimplifyCFG] Allow dropping block that only contains ephemeral values
Perform the TryToSimplifyUncondBranchFromEmptyBlock() transform if
the block is empty except for ephemeral values. The ephemeral values
will be dropped in that case.

This makes sure that assumes don't block this transforms, as reported
in https://discourse.llvm.org/t/llvm-assume-blocks-optimization/71609.

Differential Revision: https://reviews.llvm.org/D153966
2023-06-30 15:24:01 +02:00
Jie Fu
86f564edc0 [SimplifyCFG] Remove unused variable 'Inc' (NFC)
/data/llvm-project/llvm/lib/Transforms/Utils/SimplifyCFG.cpp:6051:10: error: unused variable 'Inc' [-Werror,-Wunused-variable]
    bool Inc, Wrapped = false;
         ^
1 error generated.
2023-06-26 08:59:37 +08:00
khei4
28d13a6297 [SimplifyCFG] add nsw on BuildLookuptable LinearMap calculation
Differential Revision: https://reviews.llvm.org/D150943
2023-06-26 08:30:23 +09:00
Arthur Eubanks
d49984fa4f [SimplifyCFG] Add option to not speculate blocks
Required for phase ordering changes to not regress Rust code with D145265.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D153391
2023-06-22 08:51:40 -07:00
Arthur Eubanks
3e39cfe5b4 Revert "Revert "InstSimplify: Require instruction be parented""
This reverts commit 0c03f48480f69b854f86d31235425b5cb71ac921.

Going to fix forward size regression instead due to more dependent patches needing to be reverted otherwise.
2023-06-16 13:53:31 -07:00
Arthur Eubanks
0c03f48480 Revert "InstSimplify: Require instruction be parented"
This reverts commit 1536e299e63d7788f38117b0212ca50eb76d7a3b.

Causes large binary size regressions, see comments on https://reviews.llvm.org/rG1536e299e63d7788f38117b0212ca50eb76d7a3b.
2023-06-16 11:24:29 -07:00
Alan Zhao
d6b4f6786b Revert "Revert "InstSimplify: Require instruction be parented""
This reverts commit 00264eac4d0938ae8a0826da38e4777be269124c.

Reason: caused a bunch of bots to break
2023-06-16 10:58:54 -07:00
Alan Zhao
00264eac4d Revert "InstSimplify: Require instruction be parented"
This reverts commit 1536e299e63d7788f38117b0212ca50eb76d7a3b.

Reason: causes a regression in the inliner (see https://crbug.com/1454531 and https://reviews.llvm.org/rG1536e299e63d7788f38117b0212ca50eb76d7a3b#1217141)
2023-06-16 10:36:49 -07:00
Arthur Eubanks
405f91475b [SimplifyCFG] Check optforfuzzing attribute during in the pass implementation
Instead of setting the SimplifyCFGOptions options at the beginning of the pass.

Otherwise it always gets overriden by the pass and the value in SimplifyCFGOptions is ignored.
2023-06-15 13:57:51 -07:00
Matt Arsenault
1536e299e6 InstSimplify: Require instruction be parented
Unlike every other analysis and transform, simplifyInstruction
permitted operating on instructions which are not inserted
into a function. This created an edge case no other code needs
to really worry about, and limited transforms in cases that
can make use of the context function. Only the inliner and a handful
of other utilities were making use of this, so just fix up these
edge cases. Results in some IR ordering differences since
cloned blocks are inserted eagerly now. Plus some additional
simplifications trigger (e.g. some add 0s now folded out that
previously didn't).
2023-06-02 18:14:28 -04:00
Simon Pilgrim
f2a6a97069 Fix MSVC "ignoring return value of function declared with 'nodiscard' attribute" warning. NFC. 2023-05-23 11:40:33 +01:00
khei4
1362dfe165 [SimplifyCFG] add nsw on SwitchToLookupTable index calculation on MinCaseVal subtraction
Differential Revision: https://reviews.llvm.org/D146903
Reviewed By: nikic
2023-05-23 18:02:31 +09:00
khei4
e21a90f091 [SimplifyCFG] add nuw/nsw on BuildLookuptable BitMap shiftwidth calculation
Differential Revision: https://reviews.llvm.org/D150838
2023-05-19 14:10:05 +09:00
Christian Ulmann
794b58b467 [IR] Drop const in DILocation::getMergedLocation
This commit removes constness from DILocation::getMergedLocation and
fixes all its users accordingly.

Having constness on the parameters forced the return type to be const
as well, which does force usage of `const_cast` when the location needs
to be used in metadata nodes.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D149942
2023-05-15 07:21:43 +00:00
Jay Foad
31ec0a6845 [SimplifyCFG] Improve the way hoisting skips over non-matching instructions
D129370 introduced the idea that hoisting could skip over non-matching
instructions and continue to look for matching (hoistable) instructions,
but certain types of mismatch still aborted the whole hoisting attempt.

Fix this by splitting out some of the instruction matching checks into a
helper function.

Also forbid hoisting allocas past stacksave/stackrestore, completing the
fix started in D133730, to avoid regressing tests.

Differential Revision: https://reviews.llvm.org/D149365
2023-04-28 10:03:32 +01:00
Jay Foad
ee88cd82a9 [SimplifyCFG] Remove some unnecessary TTI arguments. NFC.
TTI was already available in the SimplifyCFGOpt class.
2023-04-27 17:05:14 +01:00
Max Kazantsev
9d7785b2e9 [BasicBlockUtils][NFCI] Extract branch inverting to a separate method
The motivation is to make possible branch inverting code reuse.

Patch by Aleksandr Popov!

Differential Revision: https://reviews.llvm.org/D148708
2023-04-21 19:44:13 +07:00
Nikita Popov
53500e333d Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
This exposed another miscompile in GVN, which was fixed by
20e9b31f88149a1d5ef78c0be50051e345098e41.

-----

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-20 14:17:15 +02:00
Krasimir Georgiev
bf7f6b4436 Revert "Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating"
This reverts commit 6f7e5c0f1ac6cc3349a2e1479ac4208465b272c6.

Seems to expose a miscompile in rust, possibly exposing a bug in LLVM
somewhere. Investigation thread over at:
https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/LLVM.20D146629.20breakage
2023-04-19 08:28:48 +00:00
Nikita Popov
6f7e5c0f1a Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
This exposed a miscompile in GVN, which was fixed by D148129.

-----

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-17 14:15:14 +02:00
Nikita Popov
7c78cb4b1f Revert "[SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating"
This reverts commit 78b1fbc63f78660ef10e3ccf0e527c667a563bc8.

This causes or exposes miscompiles in Rust, revert until they
have been investigated.
2023-04-05 17:05:39 +02:00
Nikita Popov
78b1fbc63f [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-04 10:03:45 +02:00
Nikita Popov
11313108ff [SimplifyCFG] Don't merge invoke if this makes immarg non-constant (PR61265)
Don't merge invokes if this replaces constant operands with phis
in a place where this is not legal.

This also disallows converting operand bundles from constant to
non-constant, in line with the restriction we use in other
transforms.

Fixes https://github.com/llvm/llvm-project/issues/61265.

Differential Revision: https://reviews.llvm.org/D146723
2023-03-24 14:31:40 +01:00
Nikita Popov
a5788836b9 [IR] Rename dropUndefImplying to dropUBImplying (NFC)
Clarify that this is only about immediate undefined behavior,
not about undef or poison.
2023-03-22 11:16:22 +01:00
DianQK
6505b9d049
Revert "[SimplifyCFG] Check if the return instruction causes undefined behavior"
This reverts commit b6a0be8ce3114d0c57e7a7d6c3c222986ca506ad.
2023-03-20 06:42:19 +08:00
DianQK
c6e54c7fec
Revert "[SimplifyCFG] Improve the precision of PtrValueMayBeModified"
This reverts commit f890f010f6a70addbd885acd0c8d1b9578b6246f.
2023-03-20 06:42:07 +08:00
luxufan
05ef449600 [SimplifyCFG] Handle MD_noundef when hoisting common codes
Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D144939
2023-03-03 19:02:14 +08:00
Yaxun (Sam) Liu
fbec8f04ab [SimplifyCFG] Do not hoist/sink convergent function calls
Currently SimplifyCFG hoists/sink common instructions in then/else basic blocks
when certain options are enabled, which is the case for default clang optimization
pipelines for -O3. It tries to hoist/sink convergent function calls in divergent
control flow, which causes incorrect ISA generated for GPU, e.g.
https://github.com/ROCm-Developer-Tools/HIP/issues/3172

This patch fixes that by conservatively disable hoisting/sinking common
convergent function calls in then/else blocks.

Reviewed by: Artem Belevich

Differential Revision: https://reviews.llvm.org/D144756
2023-02-28 12:41:56 -05:00
DianQK
f890f010f6
[SimplifyCFG] Improve the precision of PtrValueMayBeModified
The result value of `getelementptr inbounds (TY, null, not zero)` is a poison value. We can think of it as undefined behavior.

> Please let me know if there is anything I don't understand correctly.

Reviewed By: nikic, xbolva00

Differential Revision: https://reviews.llvm.org/D144563
2023-02-25 19:42:59 +08:00
DianQK
b6a0be8ce3
[SimplifyCFG] Check if the return instruction causes undefined behavior
This should fix https://github.com/rust-lang/rust/issues/107681.

Return undefined to a noundef return value is undefined.

Example:

```
define noundef i32 @test_ret_noundef(i1 %cond) {
entry:
  br i1 %cond, label %bb1, label %bb2
bb1:
  br label %bb2
bb2:
  %r = phi i32 [ undef, %entry ], [ 1, %bb1 ]
  ret i32 %r
}
```

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D144319
2023-02-21 21:42:13 +08:00
Kazu Hirata
a28b252d85 Use APInt::getSignificantBits instead of APInt::getMinSignedBits (NFC)
Note that getMinSignedBits has been soft-deprecated in favor of
getSignificantBits.
2023-02-19 23:56:52 -08:00
Kazu Hirata
cbde2124f1 Use APInt::popcount instead of APInt::countPopulation (NFC)
This is for consistency with the C++20-style bit manipulation
functions in <bit>.
2023-02-19 11:29:12 -08:00
DianQK
1235ed9133
Revert "[SimplifyCFG] Check if the return instruction causes undefined behavior"
This reverts commit b6eed9a82e0ce530d94a194c88615d6c272e1854.
2023-02-19 21:08:29 +08:00
DianQK
b6eed9a82e
[SimplifyCFG] Check if the return instruction causes undefined behavior
This should fix https://github.com/rust-lang/rust/issues/107681.

Return undefined to a noundef return value is undefined.

Example:

```
define noundef i32 @test_ret_noundef(i1 %cond) {
entry:
  br i1 %cond, label %bb1, label %bb2
bb1:
  br label %bb2
bb2:
  %r = phi i32 [ undef, %entry ], [ 1, %bb1 ]
  ret i32 %r
}
```

Differential Revision: https://reviews.llvm.org/D144319
2023-02-19 19:42:40 +08:00