899 Commits

Author SHA1 Message Date
Orlando Cazalet-Hyams
c4e764ea24
[RemoveDIs] Update ConvertDebugDeclareToDebugValue after #72276 (#73508)
That patch was missing a recent change to the non-DPValue StoreInst overload.
Add it to the DPValue version. This is tested by
`llvm/tests/Transforms/Mem2Reg/dbg_declare_to_value_conversions.ll`,
which already has `--try-experimental-debuginfo-iterators`. It doesn't
fail currently because until #74090 lands the declares are not converted
to DPValues.

The tests will become "live" once #74090 lands (see for more info).
2023-12-12 14:57:51 +00:00
Orlando Cazalet-Hyams
cb8690ff6f
[RemoveDIs] Handle DPValues in LowerDbgDeclare (#73504)
The tests will become "live" once #74090 lands (see for more info).
2023-12-12 14:28:05 +00:00
Orlando Cazalet-Hyams
2d9d9a1a55
[NFC] Change FindDbgDeclareUsers interface to match findDbgUsers/values (#73498)
This simplifies an upcoming patch to support the RemoveDIs project (tracking
variable locations without using intrinsics).

Next in this series is #73500.
2023-12-12 09:43:58 +00:00
Jeremy Morse
33af16f580
[DebugInfo][RemoveDIs] Final cleanup for enabling non-instr-debuginfo (#74497)
Some final errors have turned up when doing stage2clang builds:
* We can insert before end(), which won't have an attached DPMarker,
thus we can have a nullptr DPMarker in Instruction::insertBefore. Add a
null check there.
* We need to use the iterator-returning form of getFirstNonPHI to ensure
we don't insert any PHIs behind debug-info at the start of a block.
2023-12-05 18:49:27 +00:00
Jeremy Morse
600c12987e
[DebugInfo][RemoveDIs] Reverse order of DPValues from findDbgUsers (#74099)
The order of dbg.value intrinsics appearing in the output can affect the
order of tables in DWARF sections. This means that DPValues, our
dbg.value replacement, needs to obey the same ordering rules. For
dbg.values returned by findDbgUsers it's reverse order of creation (due
to how they're put on use-lists). Produce that order from findDbgUsers
for DPValues.

I've got a few IR files where the order of dbg.values flips, but it's a
fragile test -- ultimately it needs the number of times a DPValue is
handled by findDbgValues to be odd.
2023-12-05 11:59:26 +00:00
Jeremy Morse
f0b5527b79
[DebugInfo][RemoveDIs] Instrument loop-rotate for DPValues (#72997)
Loop-rotate manually maintains dbg.value intrinsics -- it also needs to
manually maintain the replacement for dbg.value intrinsics, DPValue
objects. For the most part this patch adds parallel implementations
using the new type Some extra juggling is needed when loop-rotate hoists
loop-invariant instructions out of the loop: the DPValues attached to
such an instruction need to get rotated but not hoisted. Exercised by
the new test function invariant_hoist in dbgvalue.ll.

There's also a "don't insert duplicate debug intrinsics" facility in
LoopRotate. The value and correctness of this isn't clear, but to
continue preserving behaviour that's now tested in the "tak_dup"
function in dbgvalue.ll.

Other things in this patch include a helper DebugVariable constructor
for DPValues, a insertDebugValuesForPHIs handler for RemoveDIs
(exercised by the new tests), and beefing up the dbg.value checking in
dbgvalue.ll to ensure that each record is tested (and that there's an
implicit check-not).
2023-11-26 22:57:40 +00:00
Sander de Smalen
81b7f115fb
[llvm][TypeSize] Fix addition/subtraction in TypeSize. (#72979)
It seems TypeSize is currently broken in the sense that:

  TypeSize::Fixed(4) + TypeSize::Scalable(4) => TypeSize::Fixed(8)

without failing its assert that explicitly tests for this case:

  assert(LHS.Scalable == RHS.Scalable && ...);

The reason this fails is that `Scalable` is a static method of class
TypeSize,
and LHS and RHS are both objects of class TypeSize. So this is
evaluating
if the pointer to the function Scalable == the pointer to the function
Scalable,
which is always true because LHS and RHS have the same class.

This patch fixes the issue by renaming `TypeSize::Scalable` ->
`TypeSize::getScalable`, as well as `TypeSize::Fixed` to
`TypeSize::getFixed`,
so that it no longer clashes with the variable in
FixedOrScalableQuantity.

The new methods now also better match the coding standard, which
specifies that:
* Variable names should be nouns (as they represent state)
* Function names should be verb phrases (as they represent actions)
2023-11-22 08:52:53 +00:00
Jeremy Morse
f42482def2
[DebugInfo][RemoveDIs] Don't convert debug-intrinsics to Unreachable (#72380)
It might seem obvious, but it's not a good idea to convert a
debug-intrinsic instruction into an UnreachableInst, as this means
things operate differently with and without the -g option. However this
can happen due to the "mutate the next instruction" API calls we make.
With RemoveDIs eliminating debug intrinsics, this behaviour is at risk
of changing, hence this patch ensures we only ever mutate the next _non_
debuginfo instruction into an Unreachable.

The tests instrumented with the --try... flag all exercise this, I've
added some metadata to a SCCP test to ensure it's exercised.
2023-11-20 20:53:24 +00:00
Jeremy Morse
80d3a4c39f
[DebugInfo][RemoveDIs] Add local-utility plumbing for DPValues (#72276)
This patch re-implements a variety of debug-info maintenence functions
to use DPValues instead of DbgValueInst's: supporting the "new"
non-intrinsic representation of debug-info. As per [0], we need to have
parallel implementations of various utilities for a time, and these are
the most fundamental utilities used throughout the compiler.

I've added --try-experimental-debuginfo-iterators to a variety of RUN
lines: this is a flag that turns on "new debug-info" if it's built into
LLVM, and not otherwise. This should ensure that we have the same
behaviour for the same IR inputs, but using a different internal
representation. For the most part these changes affect SROA/Mem2Reg
promotion of dbg.declares into dbg.value intrinsics (now DPValues),
we're leaving dbg.declares as instructions until later in the day.
There's also some salvaging changes made.

I believe the tests that I've added cover almost all the code being
updated here. The only thing I'm not confident about is SimplifyCFG,
which calls rewriteDebugUsers down a variety of code paths. Those
changes can't immediately get full coverage as an additional patch is
needed that updates handling of Unreachable instructions, will upload
that shortly.

[0]
https://discourse.llvm.org/t/rfc-instruction-api-changes-needed-to-eliminate-debug-intrinsics-from-ir/68939/9
2023-11-20 16:56:31 +00:00
Matthias Braun
cb4627d150
Add setBranchWeigths convenience function. NFC (#72446)
Add `setBranchWeights` convenience function to ProfDataUtils.h and use
it where appropriate.
2023-11-16 10:55:19 -08:00
XChy
fc6bdb8549
[SimplifyCFG] Reland transform for redirecting phis between unmergeable BB and SuccBB (#68473)
Reland #67275 with #68953 resolved.
2023-10-28 17:10:20 +08:00
Fangrui Song
8e247b8f47 Replace TypeSize::{getFixed,getScalable} with canonical TypeSize::{Fixed,Scalable}. NFC 2023-10-27 00:30:41 -07:00
Benjamin Kramer
eb67b34740 [IPSCCP] Don't crash on ptrtoint 2023-10-24 14:14:39 +02:00
Carlos Alberto Enciso
f3b20cb16a
[IPSCCP] Variable not visible at Og. (#66745)
https://bugs.llvm.org/show_bug.cgi?id=51559
https://github.com/llvm/llvm-project/issues/50901

IPSCCP pass removes the global variable and does not create a constant
expression for the initializer value.
2023-10-24 06:22:18 +01:00
Nikita Popov
a3bbab1852
[IR] Don't mark experimental.guard as willreturn (#69433)
Control flow does not necessary continue past guard intrinsics, so don't
mark them as willreturn.

This fixes the miscompile in the sdiv-guard.ll test.
2023-10-19 09:01:48 +02:00
Muhammad Omair Javaid
431969ede1 Revert "[SimplifyCFG] Transform for redirecting phis between unmergeable BB and SuccBB (#67275)"
This reverts commit fc86d031fec5e47c6811efd3a871742ad244afdd.

This change breaks LLVM buildbot clang-aarch64-sve-vls-2stage
https://lab.llvm.org/buildbot/#/builders/176/builds/5474
I am going to revert this patch as the bot has been failing for more than a day without a fix.
2023-09-26 15:47:16 +05:00
XChy
fc86d031fe
[SimplifyCFG] Transform for redirecting phis between unmergeable BB and SuccBB (#67275)
This patch extends function TryToSimplifyUncondBranchFromEmptyBlock to
handle the similar cases below.

```llvm
define i8 @src(i8 noundef %arg) {
start:
  switch i8 %arg, label %unreachable [
    i8 0, label %case012
    i8 1, label %case1
    i8 2, label %case2
    i8 3, label %end
  ]

unreachable:
  unreachable

case1:
  br label %case012

case2:
  br label %case012

case012:
  %phi1 = phi i8 [ 3, %case2 ], [ 2, %case1 ], [ 1, %start ]
  br label %end

end:
  %phi2 = phi i8 [ %phi1, %case012 ], [ 4, %start ]
  ret i8 %phi2
}
```
The phis here should be merged into one phi, so that we can better
optimize it:

```llvm
define i8 @tgt(i8 noundef %arg) {
start:
  switch i8 %arg, label %unreachable [
    i8 0, label %end
    i8 1, label %case1
    i8 2, label %case2
    i8 3, label %case3
  ]

unreachable:
  unreachable

case1:
  br label %end

case2:
  br label %end

case3:
  br label %end

end:
  %phi = phi i8 [ 4, %case3 ], [ 3, %case2 ], [ 2, %case1 ], [ 1, %start ]
  ret i8 %phi
}
```
Proof:
[normal](https://alive2.llvm.org/ce/z/vAWi88)
[multiple stages](https://alive2.llvm.org/ce/z/DDBQqp)
[multiple stages 2](https://alive2.llvm.org/ce/z/nGkeqN)
[multiple phi combinations](https://alive2.llvm.org/ce/z/VQeEdp)

And lookup table optimization should convert it into add %arg 1.
This patch just match similar CFG structure and merge the phis in
different cases.

Maybe such transform can be applied to other situations besides switch,
but I'm not sure whether it's better than not merging. Therefore, I only
try it in switch,

Related issue:
#63876

[Migrated](https://reviews.llvm.org/D155940)
2023-09-25 10:13:45 +08:00
Dhruv Chawla
0f152a55d3
[InferAlignment] Implement InferAlignmentPass
This pass aims to infer alignment for instructions as a separate pass,
to reduce redundant work done by InstCombine running multiple times. It
runs late in the pipeline, just before the back-end passes where this
information is most useful.

Differential Revision: https://reviews.llvm.org/D158529
2023-09-20 12:03:36 +05:30
Nikita Popov
7c229f6e85 [GVN] Invalidate MDA when deduplicating phi nodes
Duplicate phi nodes were being directly removed, without
invalidating MDA. This could result in a new phi node being
allocated at the same address, incorrectly reusing a cache entry.

Fix this by optionally allowing EliminateDuplicatePHINodes() to
collect phi nodes to remove into a vector, which allows GVN to
handle removal itself.

Fixes https://github.com/llvm/llvm-project/issues/64598.

Differential Revision: https://reviews.llvm.org/D158849
2023-09-15 07:04:32 +02:00
Serguei Katkov
06dfc8400d [Local] Mark Instruction argument of wouldInstructionBeTriviallyDead as const. NFC.
wouldInstructionBeTriviallyDead is not expected to modify instruction,
so mark argument as const to allow its usage in other non-modifying instructions callers.

Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D157834
2023-08-14 14:49:51 +07:00
Ramkumar Ramachandra
23caf9e9e7 Local: fix debug output of replaceDominatedUsesWith()
The debug output of replaceDominatedUsesWith() prints incorrect
information, and the user is left confused about what exactly was
replaced. Fix this.

Differential Revision: https://reviews.llvm.org/D156318
2023-07-27 13:23:38 +01:00
Nikita Popov
bb3763e497 Revert "[SimplifyCFG] Allow dropping block that only contains ephemeral values"
This reverts commit 20f0c68fd83a0147a8ec1722bd2e848180610288.

https://reviews.llvm.org/D153966#4464594 reports an optimization
regression in Rust.

Additionally this change has caused an unexpected 0.3% compile-time
regression.
2023-06-30 21:24:05 +02:00
Nikita Popov
20f0c68fd8 [SimplifyCFG] Allow dropping block that only contains ephemeral values
Perform the TryToSimplifyUncondBranchFromEmptyBlock() transform if
the block is empty except for ephemeral values. The ephemeral values
will be dropped in that case.

This makes sure that assumes don't block this transforms, as reported
in https://discourse.llvm.org/t/llvm-assume-blocks-optimization/71609.

Differential Revision: https://reviews.llvm.org/D153966
2023-06-30 15:24:01 +02:00
Shubham Sandeep Rastogi
775258d758 Add support for salvaging debug info from icmp instrcuctions.
salvageDebugInfo is a function that allows us to reatin debug info for
instructions that have been optimized out. Currently, it doesn't support
salvaging the debug information from icmp instrcutions, but DWARF
expressions can emulate an icmp by using the DWARF conditional
expressions. This patch adds support for salvaging debug information
from icmp instructions.

Differential Revision: https://reviews.llvm.org/D150216
2023-05-23 15:31:31 -07:00
Nathan Sidwell
03fde97d49 [NFC] Refactor loop metadata movement
* Use 'if (T v = expr)' idiom
* llvm.loop is a fixed metadata ID

Differential Revision: https://reviews.llvm.org/D150109

Reviewed By: kazu
2023-05-09 18:43:59 -04:00
Mingming Liu
b3cb950cf3 [PGO]Implement metadata combine for 'branch_weights' of direct
callsites when none of the instructions folds the rest away.

- Merge cases are added for simplify-cfg {sink,hoist}, based on https://gcc.godbolt.org/z/avGvc38W7 and https://gcc.godbolt.org/z/dbWbjGhaE
- When one instruction folds the others in, do not update branch_weights
  with sum (see test/Transforms/GVN/calls-readonly.ll)

Differential Revision: https://reviews.llvm.org/D148877
2023-04-27 13:04:17 -07:00
OCHyams
c604a2c2c0 Fix buildbots after D140901
Commit 3feea34d77f65f0b68520bb7bf372580a72794ad (D140901) should not have
removed the debug label handling code.

Buildbot: https://lab.llvm.org/buildbot/#/builders/139/builds/39813
2023-04-26 10:26:14 +01:00
OCHyams
3feea34d77 [DebugInfo] Do not delete debug intrinsics with empty metadata operands
A ValueAsMetadata may be replaced with nullptr for several reasons including
deleting (certain) values and value remapping a use-before-def. In the case of
a MetadataAsValue user, handleChangedOperand intercepts and replaces the
metadata with an empty tuple (!{}).

At the moment, an empty metadata operand in a debug intrinsics signals that it
can be deleted.

Given that we end up with empty metadata operands in circumstances where the
Value has been "lost" the current behaviour can lead to incorrect variable
locations. Instead, we should treat empty metadata as meaning "there is no
location for the variable" (the same as we currently treat undef operands).

This patch removes the deletion logic from wouldInstructionBeTriviallyDead.

Related to https://discourse.llvm.org/t/auto-undef-debug-uses-of-a-deleted-value

Reviewed By: StephenTozer

Differential Revision: https://reviews.llvm.org/D140901
2023-04-26 09:58:31 +01:00
Nikita Popov
53500e333d Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
This exposed another miscompile in GVN, which was fixed by
20e9b31f88149a1d5ef78c0be50051e345098e41.

-----

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-20 14:17:15 +02:00
Krasimir Georgiev
bf7f6b4436 Revert "Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating"
This reverts commit 6f7e5c0f1ac6cc3349a2e1479ac4208465b272c6.

Seems to expose a miscompile in rust, possibly exposing a bug in LLVM
somewhere. Investigation thread over at:
https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/LLVM.20D146629.20breakage
2023-04-19 08:28:48 +00:00
Nikita Popov
6f7e5c0f1a Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
This exposed a miscompile in GVN, which was fixed by D148129.

-----

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-17 14:15:14 +02:00
Kazu Hirata
c83c4b58d1 [Transforms] Apply fixes from performance-for-range-copy (NFC) 2023-04-16 08:25:28 -07:00
Nikita Popov
e4251fc6bb [LangRef][Local] dereferenceable metadata violation is UB
I believe !dereferencable violation is immediate undefined behavior,
but this was not explicitly spelled out in LangRef. We already
assume that !dereferenceable is implicitly !noundef and cannot
return poison in isGuaranteedNotToBeUndefOrPoison().

The reason why we made dereferenceable implicitly noundef is that
the purpose of this metadata is to allow speculation, and that
would not be legal on a potential poison pointer.

Differential Revision: https://reviews.llvm.org/D148202
2023-04-14 10:54:01 +02:00
Nikita Popov
7c78cb4b1f Revert "[SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating"
This reverts commit 78b1fbc63f78660ef10e3ccf0e527c667a563bc8.

This causes or exposes miscompiles in Rust, revert until they
have been investigated.
2023-04-05 17:05:39 +02:00
Nikita Popov
78b1fbc63f [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-04 10:03:45 +02:00
Nikita Popov
d68800d15d [Local] Preserve !invariant.load of dominating instruction
Per LangRef:

> If a load instruction tagged with the !invariant.load metadata
> is executed, the memory location referenced by the load has to
> contain the same value at all points in the program where the
> memory location is dereferenceable; otherwise, the behavior is
> undefined.

As invariant.load violation is immediate undefined behavior, it
is sufficient for it to be present on the dominating load (for
the case where K does not move).
2023-04-03 16:05:02 +02:00
Nikita Popov
e20331cec0 [Local] Use combineMetadataForCSE() in patchReplacementInstruction()
patchReplacementInstruction() is used for CSE-style transforms.
Avoid the need to maintain two separate lists of known metadata IDs,
which can and do go out of sync.
2023-04-03 15:30:21 +02:00
Nikita Popov
0b5068695a [Local] Add MD_fpmath to combineMetadataForCSE()
This was present in patchReplacementInstruction() but not
combineMetadataForCSE(). combineMetadata() already knows how to
merge these properly.
2023-04-03 15:27:59 +02:00
Nikita Popov
fc6e91fe81 [Local] Handle size mismatch between pointer/int in copyRangeMetadata()
SROA may convert a wide integer load into a narrow pointer load,
make sure we don't crash. It would not be legal to transfer the
metadata in this case.
2023-03-31 12:20:34 +02:00
luxufan
00b733e200 [Local] Preserve !align if K dominates J and K has a !noundef
Similar to D142687

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D147122
2023-03-31 13:44:56 +08:00
luxufan
9630a9999b [Local] Preserve !nonnull only when K dominate J and K has a !noundef
Similar to D142687

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D146799
2023-03-26 22:25:45 +08:00
Nikita Popov
4923b4dbac [Local] Check for null VH in RecursivelyDeleteTriviallyDeadInstructionsPermissive()
Peculiarly, the non-permissive variant handled this gracefully,
but the permissive one did not.
2023-03-24 12:56:06 +01:00
luxufan
f44c7dec67 [Local] Use most generic range if K does not dominate J or K doesn't have a !noundef
Since D141386 has changed the return value of !range from IUB to poison,
metadata !range shouldn't be preserved even if K dominates J.

If this patch was accepted, I plan to adjust metadata !nonnull as well.
BTW, I found that metadata !noundef is not handled in combineMetadata,
is this intentional?

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D142687
2023-03-23 19:31:55 +08:00
Nikita Popov
a5788836b9 [IR] Rename dropUndefImplying to dropUBImplying (NFC)
Clarify that this is only about immediate undefined behavior,
not about undef or poison.
2023-03-22 11:16:22 +01:00
OCHyams
44e03d6488 [Assignment Tracking] Allow salvaging with variadic expressions [4/x]
Allow the value-component of a dbg.assign to be salvaged using a variadic
DIExpression.

Reviewed By: StephenTozer

Differential Revision: https://reviews.llvm.org/D145915
2023-03-16 09:55:15 +00:00
Yeting Kuo
c7fcae52f6 [InstCombine][debuginfo] Update valueCoversEntireFragment for fixed size fragment and scalable value.
Update valueCoversEntireFragment to handle new case and add regression test.

Reviewed By: sdesmalen, aprantl

Differential Revision: https://reviews.llvm.org/D144472
2023-03-08 15:34:58 +08:00
J. Ryan Stinnett
0bbe6040be [DebugInfo] Remove dbg.addr from Transforms
Part of `dbg.addr` removal
Discussed in https://discourse.llvm.org/t/what-is-the-status-of-dbg-addr/62898

Differential Revision: https://reviews.llvm.org/D144797
2023-03-02 09:29:43 +00:00
luxufan
207854b07d [Local][InstCombine] Handle MD_noundef in combineMetadataCSE
Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D144942
2023-02-28 18:09:04 +08:00
Daniel Woodworth
a33f018b89 [Local][SimplifyCFG][GVN] Handle !nontemporal in combineMetadata
SimplifyCFG currently drops !nontemporal metadata when sinking
common instructions. With this change, SimplifyCFG and similar
transforms will preserve !nontemporal metadata as long as it is
set on both original instructions.

Differential Revision: https://reviews.llvm.org/D144298
2023-02-22 14:47:00 +01:00
Kazu Hirata
cbde2124f1 Use APInt::popcount instead of APInt::countPopulation (NFC)
This is for consistency with the C++20-style bit manipulation
functions in <bit>.
2023-02-19 11:29:12 -08:00