827 Commits

Author SHA1 Message Date
Nikita Popov
20f0c68fd8 [SimplifyCFG] Allow dropping block that only contains ephemeral values
Perform the TryToSimplifyUncondBranchFromEmptyBlock() transform if
the block is empty except for ephemeral values. The ephemeral values
will be dropped in that case.

This makes sure that assumes don't block this transforms, as reported
in https://discourse.llvm.org/t/llvm-assume-blocks-optimization/71609.

Differential Revision: https://reviews.llvm.org/D153966
2023-06-30 15:24:01 +02:00
Shubham Sandeep Rastogi
775258d758 Add support for salvaging debug info from icmp instrcuctions.
salvageDebugInfo is a function that allows us to reatin debug info for
instructions that have been optimized out. Currently, it doesn't support
salvaging the debug information from icmp instrcutions, but DWARF
expressions can emulate an icmp by using the DWARF conditional
expressions. This patch adds support for salvaging debug information
from icmp instructions.

Differential Revision: https://reviews.llvm.org/D150216
2023-05-23 15:31:31 -07:00
Nathan Sidwell
03fde97d49 [NFC] Refactor loop metadata movement
* Use 'if (T v = expr)' idiom
* llvm.loop is a fixed metadata ID

Differential Revision: https://reviews.llvm.org/D150109

Reviewed By: kazu
2023-05-09 18:43:59 -04:00
Mingming Liu
b3cb950cf3 [PGO]Implement metadata combine for 'branch_weights' of direct
callsites when none of the instructions folds the rest away.

- Merge cases are added for simplify-cfg {sink,hoist}, based on https://gcc.godbolt.org/z/avGvc38W7 and https://gcc.godbolt.org/z/dbWbjGhaE
- When one instruction folds the others in, do not update branch_weights
  with sum (see test/Transforms/GVN/calls-readonly.ll)

Differential Revision: https://reviews.llvm.org/D148877
2023-04-27 13:04:17 -07:00
OCHyams
c604a2c2c0 Fix buildbots after D140901
Commit 3feea34d77f65f0b68520bb7bf372580a72794ad (D140901) should not have
removed the debug label handling code.

Buildbot: https://lab.llvm.org/buildbot/#/builders/139/builds/39813
2023-04-26 10:26:14 +01:00
OCHyams
3feea34d77 [DebugInfo] Do not delete debug intrinsics with empty metadata operands
A ValueAsMetadata may be replaced with nullptr for several reasons including
deleting (certain) values and value remapping a use-before-def. In the case of
a MetadataAsValue user, handleChangedOperand intercepts and replaces the
metadata with an empty tuple (!{}).

At the moment, an empty metadata operand in a debug intrinsics signals that it
can be deleted.

Given that we end up with empty metadata operands in circumstances where the
Value has been "lost" the current behaviour can lead to incorrect variable
locations. Instead, we should treat empty metadata as meaning "there is no
location for the variable" (the same as we currently treat undef operands).

This patch removes the deletion logic from wouldInstructionBeTriviallyDead.

Related to https://discourse.llvm.org/t/auto-undef-debug-uses-of-a-deleted-value

Reviewed By: StephenTozer

Differential Revision: https://reviews.llvm.org/D140901
2023-04-26 09:58:31 +01:00
Nikita Popov
53500e333d Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
This exposed another miscompile in GVN, which was fixed by
20e9b31f88149a1d5ef78c0be50051e345098e41.

-----

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-20 14:17:15 +02:00
Krasimir Georgiev
bf7f6b4436 Revert "Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating"
This reverts commit 6f7e5c0f1ac6cc3349a2e1479ac4208465b272c6.

Seems to expose a miscompile in rust, possibly exposing a bug in LLVM
somewhere. Investigation thread over at:
https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/LLVM.20D146629.20breakage
2023-04-19 08:28:48 +00:00
Nikita Popov
6f7e5c0f1a Reapply [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
This exposed a miscompile in GVN, which was fixed by D148129.

-----

After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-17 14:15:14 +02:00
Kazu Hirata
c83c4b58d1 [Transforms] Apply fixes from performance-for-range-copy (NFC) 2023-04-16 08:25:28 -07:00
Nikita Popov
e4251fc6bb [LangRef][Local] dereferenceable metadata violation is UB
I believe !dereferencable violation is immediate undefined behavior,
but this was not explicitly spelled out in LangRef. We already
assume that !dereferenceable is implicitly !noundef and cannot
return poison in isGuaranteedNotToBeUndefOrPoison().

The reason why we made dereferenceable implicitly noundef is that
the purpose of this metadata is to allow speculation, and that
would not be legal on a potential poison pointer.

Differential Revision: https://reviews.llvm.org/D148202
2023-04-14 10:54:01 +02:00
Nikita Popov
7c78cb4b1f Revert "[SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating"
This reverts commit 78b1fbc63f78660ef10e3ccf0e527c667a563bc8.

This causes or exposes miscompiles in Rust, revert until they
have been investigated.
2023-04-05 17:05:39 +02:00
Nikita Popov
78b1fbc63f [SimplifyCFG][LICM] Preserve nonnull, range and align metadata when speculating
After D141386, violation of nonnull, range and align metadata
results in poison rather than immediate undefined behavior,
which means that these are now safe to retain when speculating.
We only need to remove UB-implying metadata like noundef.

This is done by adding a dropUBImplyingAttrsAndMetadata() helper,
which lists the metadata which is known safe to retain on speculation.

Differential Revision: https://reviews.llvm.org/D146629
2023-04-04 10:03:45 +02:00
Nikita Popov
d68800d15d [Local] Preserve !invariant.load of dominating instruction
Per LangRef:

> If a load instruction tagged with the !invariant.load metadata
> is executed, the memory location referenced by the load has to
> contain the same value at all points in the program where the
> memory location is dereferenceable; otherwise, the behavior is
> undefined.

As invariant.load violation is immediate undefined behavior, it
is sufficient for it to be present on the dominating load (for
the case where K does not move).
2023-04-03 16:05:02 +02:00
Nikita Popov
e20331cec0 [Local] Use combineMetadataForCSE() in patchReplacementInstruction()
patchReplacementInstruction() is used for CSE-style transforms.
Avoid the need to maintain two separate lists of known metadata IDs,
which can and do go out of sync.
2023-04-03 15:30:21 +02:00
Nikita Popov
0b5068695a [Local] Add MD_fpmath to combineMetadataForCSE()
This was present in patchReplacementInstruction() but not
combineMetadataForCSE(). combineMetadata() already knows how to
merge these properly.
2023-04-03 15:27:59 +02:00
Nikita Popov
fc6e91fe81 [Local] Handle size mismatch between pointer/int in copyRangeMetadata()
SROA may convert a wide integer load into a narrow pointer load,
make sure we don't crash. It would not be legal to transfer the
metadata in this case.
2023-03-31 12:20:34 +02:00
luxufan
00b733e200 [Local] Preserve !align if K dominates J and K has a !noundef
Similar to D142687

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D147122
2023-03-31 13:44:56 +08:00
luxufan
9630a9999b [Local] Preserve !nonnull only when K dominate J and K has a !noundef
Similar to D142687

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D146799
2023-03-26 22:25:45 +08:00
Nikita Popov
4923b4dbac [Local] Check for null VH in RecursivelyDeleteTriviallyDeadInstructionsPermissive()
Peculiarly, the non-permissive variant handled this gracefully,
but the permissive one did not.
2023-03-24 12:56:06 +01:00
luxufan
f44c7dec67 [Local] Use most generic range if K does not dominate J or K doesn't have a !noundef
Since D141386 has changed the return value of !range from IUB to poison,
metadata !range shouldn't be preserved even if K dominates J.

If this patch was accepted, I plan to adjust metadata !nonnull as well.
BTW, I found that metadata !noundef is not handled in combineMetadata,
is this intentional?

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D142687
2023-03-23 19:31:55 +08:00
Nikita Popov
a5788836b9 [IR] Rename dropUndefImplying to dropUBImplying (NFC)
Clarify that this is only about immediate undefined behavior,
not about undef or poison.
2023-03-22 11:16:22 +01:00
OCHyams
44e03d6488 [Assignment Tracking] Allow salvaging with variadic expressions [4/x]
Allow the value-component of a dbg.assign to be salvaged using a variadic
DIExpression.

Reviewed By: StephenTozer

Differential Revision: https://reviews.llvm.org/D145915
2023-03-16 09:55:15 +00:00
Yeting Kuo
c7fcae52f6 [InstCombine][debuginfo] Update valueCoversEntireFragment for fixed size fragment and scalable value.
Update valueCoversEntireFragment to handle new case and add regression test.

Reviewed By: sdesmalen, aprantl

Differential Revision: https://reviews.llvm.org/D144472
2023-03-08 15:34:58 +08:00
J. Ryan Stinnett
0bbe6040be [DebugInfo] Remove dbg.addr from Transforms
Part of `dbg.addr` removal
Discussed in https://discourse.llvm.org/t/what-is-the-status-of-dbg-addr/62898

Differential Revision: https://reviews.llvm.org/D144797
2023-03-02 09:29:43 +00:00
luxufan
207854b07d [Local][InstCombine] Handle MD_noundef in combineMetadataCSE
Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D144942
2023-02-28 18:09:04 +08:00
Daniel Woodworth
a33f018b89 [Local][SimplifyCFG][GVN] Handle !nontemporal in combineMetadata
SimplifyCFG currently drops !nontemporal metadata when sinking
common instructions. With this change, SimplifyCFG and similar
transforms will preserve !nontemporal metadata as long as it is
set on both original instructions.

Differential Revision: https://reviews.llvm.org/D144298
2023-02-22 14:47:00 +01:00
Kazu Hirata
cbde2124f1 Use APInt::popcount instead of APInt::countPopulation (NFC)
This is for consistency with the C++20-style bit manipulation
functions in <bit>.
2023-02-19 11:29:12 -08:00
Wolfgang Pieb
5d07e0448e [TLS]: Clamp the alignment of TLS global variables if required by the target
Adding a module flag 'MaxTLSAlign' describing the maximum alignment a global TLS
variable can have. Optimizers are prevented from increasing the alignment of such
variables beyond this threshold.

Reviewed By: probinson

Differential Revision: https://reviews.llvm.org/D140123
2023-02-08 10:34:56 -08:00
luxufan
232698dc17 [Local][InstCombine][GVN] Handle !noundef metadata in combineMetadata
Handle !noundef metadata in comhineMetadata. The behavior of violating
!noundef metadata is undefined behavior. So if K dominates J, we can
preserve it uncontionally, otherwise, we preserve it if both K and J
have !noundef metadata been set.

This patch also makes !noundef metadata added in KnownIDs when doing
instruction combine for phi node or global value numbering for loads.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D142801
2023-02-02 14:25:13 +08:00
Felipe de Azevedo Piovezan
055f2f04e6 [mem2reg][debuginfo] Handle op_deref when converting dbg.declare
The conversion of dbg.declare into dbg.values doesn't take into account
the DIExpression attached to the intrinsic. In particular, when
converting:

```
store %val, ptr %alloca
dbg.declare(ptr %alloca, !SomeVar, !DIExpression())
```

Mem2Reg will try to figure out if `%val` has the size of `!SomeVar`. If
it does, then a non-undef dbg.value is inserted:

```
dbg.value(%val, !SomeVar, !DIExpression())
```

This makes sense: the alloca is _the_ address of the variable. So a
store to the alloca is a store to the variable. However, if the
expression in the original intrinsic is a `DW_OP_deref`, this logic is
not applicable:

```
store ptr %val, ptr %alloca
dbg.declare(ptr %alloca, !SomeVar, !DIExpression(DW_OP_deref))
```

Here, the alloca is *not* the address of the variable. A store to the
alloca is *not* a store to the variable. As such, querying whether
`%val` has the same size as `!SomeVar` is meaningless.

This patch addresses the issue by:
1. Allowing the conversion when the expression is _only_ a `DW_OP_deref`
without any other expressions (see code comment).
2. Checking that the expression does not start with a `DW_OP_deref`
before applying the logic that checks whether the value being stored and
the variable have the same length.

Differential Revision: https://reviews.llvm.org/D142160
2023-01-30 11:17:36 -05:00
Stefan Gränitz
3b387d1070 Lift EHPersonalities from Analysis to IR (NFC)
Computing EH-related information was only relevant for analysis passes so far. Lifting it to IR will allow the IR Verifier to calculate EH funclet coloring and validate funclet operand bundles in a follow-up step.

Reviewed By: rnk, compnerd

Differential Revision: https://reviews.llvm.org/D138122
2023-01-27 18:05:13 +01:00
Paul Kirth
68e7c00b03 [llvm][NFC] Rename variables to match style guide in Local.cpp
Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D142579
2023-01-25 23:18:47 +00:00
Nikita Popov
a4898b437d [Local] Preserve range metadata if the type did not change
In copyRangeMetadata() and by extension copyLoadMetadata(),
handle the trivial case where the type did not change, in which
case we can simply preserve the range metadata as is.
2023-01-20 15:28:32 +01:00
Nikita Popov
b3b049a824 [Local] Preserve noundef metadata in copyMetadataForLoad()
If we're only changing the type of the load, preserve the noundef
metadata.
2023-01-19 16:56:09 +01:00
Christian Ulmann
e741b8c2e5 [llvm][ir] Purge MD_prof custom accessors
This commit purges direct accesses to MD_prof metadata and replaces them
with the accessors provided from the utility file wherever possible.
This commit can be seen as the first step towards switching the branch weights to 64 bits.
See post here: https://discourse.llvm.org/t/extend-md-prof-branch-weights-metadata-from-32-to-64-bits/67492

Reviewed By: davidxl, paulkirth

Differential Revision: https://reviews.llvm.org/D141393
2023-01-19 14:26:26 +01:00
Roman Lebedev
333cdd4125
[SimplifyCFG] Reapply: when eliminating unreachable landing pads, mark calls as nounwind
This time the change is in it's least intrusive form since only the return
type in prototype for `removeUnwindEdge()` is changed, since only a single
specific caller need that knowledge.

We really can't recover that knowledge, and `nounwind` knowledge,
(and not just a lack of the unwind edge, aka `call` instead of `invoke`),
is e.g. part of the reasoning in e.g. `mayHaveSideEffects()`.

Note that this is call-site-specific knowledge,
just because some callsite had an `unreachable`
unwind edge, does not mean that all will.
2023-01-13 21:04:17 +03:00
Roman Lebedev
fbcefff9d0
Revert "[SimplifyCFG] When eliminating unreachable landing pads, mark calls as nounwind"
The bool is in the wrong place and might get implicitly converted from
the previous second argument - a pointer. Thinking about it more,
it's not really the best place for that functionality anyways,
only a single caller needs that.

This reverts commit 3c5b1f2d94d021005ce3769a4402d4a4ae843989.
2023-01-13 01:18:56 +03:00
Roman Lebedev
3c5b1f2d94
[SimplifyCFG] When eliminating unreachable landing pads, mark calls as nounwind
We really can't recover that knowledge, and `nounwind` knowledge,
(and not just a lack of the unwind edge, aka `call` instead of `invoke`),
is e.g. part of the reasoning in e.g. `mayHaveSideEffects()`.

Note that this is call-site-specific knowledge,
just because some callsite had an `unreachable`
unwind edge, does not mean that all will.
2023-01-13 00:41:58 +03:00
OCHyams
83f7f86e7d [NFC][Assignment Tracking] Add is/setKillAddress
Unlike D140903 this patch folds in treating an empty metadata address component
of a dbg.assign the same as undef because it was already being treated that way
in the AssignmentTrackingAnalysis pass.

Reviewed By: scott.linder

Differential Revision: https://reviews.llvm.org/D141125
2023-01-12 09:46:01 +00:00
Guillaume Chatelet
8fd5558b29 [NFC] Use TypeSize::geFixedValue() instead of TypeSize::getFixedSize()
This change is one of a series to implement the discussion from
https://reviews.llvm.org/D141134.
2023-01-11 16:49:38 +00:00
Stephen Tozer
c383f4d655 [DebugInfo] Allow non-stack_value variadic expressions and use in DBG_INSTR_REF
Prior to this patch, variadic DIExpressions (i.e. ones that contain
DW_OP_LLVM_arg) could only be created by salvaging debug values to create
stack value expressions, resulting in a DBG_VALUE_LIST being created. As of
the previous patch in this patch stack, DBG_INSTR_REF's syntax has been
changed to match DBG_VALUE_LIST in preparation for supporting variadic
expressions. This patch adds some minor changes needed to allow variadic
expressions that aren't stack values to exist, and allows variadic expressions
that are trivially reduceable to non-variadic expressions to be handled
similarly to non-variadic expressions.

Reviewed by: jmorse

Differential Revision: https://reviews.llvm.org/D133926
2023-01-06 19:31:10 +00:00
OCHyams
775af51209 [DebugInfo] Prefer setKillLocation rather than replacing operands with undef
NFC-ish. There is a functional change but the outputs are semantically
identical. Where we might've before replaced one operand with undef (which
means "this is a kill location marker") the use of `setKillLocation` will
replace all location operands with `undef` (which also means "this is a kill
location marker").

Related to https://discourse.llvm.org/t/auto-undef-debug-uses-of-a-deleted-value

Reviewed By: StephenTozer

Differential Revision: https://reviews.llvm.org/D140904
2023-01-06 10:11:14 +00:00
Owen Anderson
733740b189 Fix a phase-ordering problem in SimplifyCFG.
Switch simplification could sometimes fail to notice when an
intermediate case removal caused the switch condition to become
constant. This would cause the switch to be simplified into a
conditional branch rather than a direct branch.

Most of the time this didn't matter, except that occasionally
downstream parts of SimplifyCFG expect tautological branches to
already have been eliminated. The missed handling in switch
simplification would cause an assertion failure in the downstream
code.

Triggering the assertion failure is fairly sensitive to the exact
order of various simplifications.

Fixes https://github.com/llvm/llvm-project/issues/59768

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D140831
2023-01-04 16:47:13 -07:00
Kazu Hirata
6eb0b0a045 Don't include Optional.h
These files no longer use llvm::Optional.
2022-12-14 21:16:22 -08:00
Fangrui Song
c178ed33bd Transforms/Utils: llvm::Optional => std::optional 2022-12-12 08:29:05 +00:00
Nikita Popov
ce51cae794 [PtrAuth] Use default attributes for some ptrauth intrinsics
This uses the default intrinsic attributes (nofree, nosync,
nocallback, willreturn) for a subset of the ptrauth intrinsics.
Notably, this does not use them for auth and resign, because these
intrinsics are specified to trap and as such are not willreturn.
As far as I understood, the remaining intrinsics may not trap
(or can only trap in cases where behavior is undefined).

Differential Revision: https://reviews.llvm.org/D137557
2022-12-09 10:41:28 +01:00
Kazu Hirata
405fc404bf [ADT] Don't including None.h (NFC)
These source files no longer use None, so they do not need to include
None.h.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-06 20:14:51 -08:00
Nikita Popov
0b20c3034c [IR] Don't assume readnone/readonly intrinsics are willreturn
This removes our "temporary" hack to assume that readnone/readonly
intrinsics are also willreturn. An explicit willreturn annotation,
usually via default intrinsic attributes, is now required.

Differential Revision: https://reviews.llvm.org/D137630
2022-12-06 11:48:50 +01:00
Krzysztof Parzyszek
f3b6dbfda8 Instructions: convert Optional to std::optional 2022-12-04 14:25:11 -06:00