Replace the DenseMap from blocks to their innermost loop a vector
indexed by block numbers, when possible. Supporting number updates is
not trivial as we don't store a list of basic blocks, so this is not
implemented.
NB: I'm generally not happy with the way loops are stored. As I think
that there's room for improvement, I don't want to touch the
representation at this point.
Pull Request: https://github.com/llvm/llvm-project/pull/103400
Block numbers are updated too frequently, which makes it difficult to
keep analyses up to date. Therefore, introduce a second number per basic
block that is used for analyses and is renumbered less often. This frees
analyses from providing somewhat efficient facilities for dealing with
changed block numbers, making it simpler to implement in e.g. LoopInfo
or CycleInfo.
(Currently, "less often" means not at all, but we might want to renumber
after certain passes if the numbering gets too sparse and no analyses
are preserved anyway.)
When we introduced a more general use of block numbers some time ago,
using the existing numbers seemed to be a somewhat obvious choice, but I
now think that this was a bad decision, as it conflates a number that is
used for ordering with a number that should be more stable.
MachineBasicBlock isn't particularly size-optimized and there's a fair
amount of padding where we can add another number.
There should be no performance impact,
If branch folding remoes an EH pad, we're left with a dangling reference
to it from the CxxUnwindMap. We could try to fix this up, but given that
this should be a rare situation, just leave the dead EH pad blocks
around.
Fixes https://github.com/llvm/llvm-project/issues/176421.
These passes run "RenumberBlocks()" on Machine function which renders
the dominator tree analyses in the cache stale (though passes may not
change anything structurally). we need to update the block numbers in
the cached analyses if we are to return PreservedAnalysis::all(). This
is not an issue with legacy PM since none of these passes preserve
dominator tree analyses anyway.
This patch introduces a new virtual method
`TargetInstrInfo::isSafeToMove()` to allow backends to control whether a
machine instruction can be safely moved by optimization passes.
The `BranchFolder` pass now respects this hook when hoisting common
code. By default, all instructions are considered safe to to move.
For LoongArch, `isSafeToMove()` is overridden to prevent
relocation-related instruction sequences (e.g. PC-relative addressing
and calls) from being broken by instruction motion. Correspondingly,
`isSchedulingBoundary()` is updated to reuse this logic for consistency.
Relands #163725
This patch introduces a new virtual method
`TargetInstrInfo::isSafeToMove()` to allow backends to control whether a
machine instruction can be safely moved by optimization passes.
The `BranchFolder` pass now respects this hook when hoisting common
code. By default, all instructions are considered safe to to move.
For LoongArch, `isSafeToMove()` is overridden to prevent
relocation-related instruction sequences (e.g. PC-relative addressing
and calls) from being broken by instruction motion. Correspondingly,
`isSchedulingBoundary()` is updated to reuse this logic for consistency.
Fixes#163681
Depend on #152591 to fix
https://github.com/llvm/llvm-project/issues/149023.
Similar to an EH pad, there is no real advantage in "falling through" to
an indirect target of an INLINEASM_BR. And multiple indirect targets of
inline asm at the end of a function may be rotated infinitely.
Therefore, this patch avoids such optimization on indirect target of
inline asm as fall through.
fbf6271c7da20356d7b34583b3711b4126ca1dbb introduced an assertion failure as
setDebugValueUndef was called on DBG_LABELs, which isn't allowed and doesn't
make sense. Fix by skipping the call for DBG_LABELs and hoisting, in line with
the original behaviour.
Reapply #140091.
branch-folder hoists common instructions from TBB and FBB into their
pred. Without this patch it achieves this by splicing the instructions from TBB
and deleting the common ones in FBB. That moves the debug locations and debug
instructions from TBB into the pred without modification, which is not
ideal. Debug locations are handled in #140063.
This patch handles debug instructions - in the simplest way possible, which is
to just kill (undef) them. We kill and hoist the ones in FBB as well as TBB
because otherwise the fact there's an assignment on the code path is deleted
(which might lead to a prior location extending further than it should).
There's possibly something we could do to preserve some variable locations in
some cases, but this is the easiest not-incorrect thing to do.
Note I had to replace the constant DBG_VALUEs to use registers in the test- it
turns out setDebugValueUndef doesn't undef constant DBG_VALUEs... which feels
wrong to me, but isn't something I want to touch right now.
---
Fix end-iterator-dereference and add test.
Reapply #140091.
branch-folder hoists common instructions from TBB and FBB into their
pred. Without this patch it achieves this by splicing the instructions from TBB
and deleting the common ones in FBB. That moves the debug locations and debug
instructions from TBB into the pred without modification, which is not
ideal. Debug locations are handled in #140063.
This patch handles debug instructions - in the simplest way possible, which is
to just kill (undef) them. We kill and hoist the ones in FBB as well as TBB
because otherwise the fact there's an assignment on the code path is deleted
(which might lead to a prior location extending further than it should).
There's possibly something we could do to preserve some variable locations in
some cases, but this is the easiest not-incorrect thing to do.
Note I had to replace the constant DBG_VALUEs to use registers in the test- it
turns out setDebugValueUndef doesn't undef constant DBG_VALUEs... which feels
wrong to me, but isn't something I want to touch right now.
branch-folder hoists common instructions from TBB and FBB into their
pred. Without this patch it achieves this by splicing the instructions from TBB
and deleting the common ones in FBB. That moves the debug locations and debug
instructions from TBB into the pred without modification, which is not
ideal. Debug locations are handled in pull request 140063.
This patch handles debug instructions - in the simplest way possible, which is
to just kill (undef) them. We kill and hoist the ones in FBB as well as TBB
because otherwise the fact there's an assignment on the code path is deleted
(which might lead to a prior location extending further than it should).
We might be able to do something smarter to preserve some variable locations in
some cases, but this is the easiest not-incorrect thing to do.
This patch is part of a series that adds origin-tracking to the debugify
source location coverage checks, allowing us to report symbolized stack
traces of the point where missing source locations appear.
This patch adds the logic for collecting stack traces in DebugLoc
instances. We do not symbolize the stack traces in this patch - that
only happens when we decide to actually print them, which will be the
responsibility of debugify. The collection happens in the constructor of
a DebugLoc that has neither a valid location nor an annotation; we also
collect an extra stack trace every time we call setDebugLoc, as
sometimes the more interesting point is not where the DebugLoc was
constructed, but where it was applied to an instruction. This takes the
form of a getCopied() method on DebugLoc, which is the identity function
in normal builds, but adds an extra stack trace in origin-tracking
builds.
Part of the coverage-tracking feature, following #107279.
In order for DebugLoc coverage testing to work, we firstly have to set
annotations for intentionally-empty DebugLocs, and secondly we have to
ensure that we do not drop these annotations as we propagate DebugLocs
throughout compilation. As the annotations exist as part of the DebugLoc
class, and not the underlying DILocation, they will not survive a
DebugLoc->DILocation->DebugLoc roundtrip. Therefore this patch modifies
a number of places in the compiler to propagate DebugLocs directly
rather than via the underlying DILocation. This has no effect on the
output of normal builds; it only ensures that during coverage builds, we
do not drop incorrectly annotations and therefore create false
positives.
The bulk of these changes are in replacing
DILocation::getMergedLocation(s) with a DebugLoc equivalent, and in
changing the IRBuilder to store a DebugLoc directly rather than storing
DILocations in its general Metadata array. We also use a new function,
`DebugLoc::orElse`, which selects the "best" DebugLoc out of a pair
(valid location > annotated > empty), preferring the current DebugLoc on
a tie - this encapsulates the existing behaviour at a few sites where we
_may_ assign a DebugLoc to an existing instruction, while extending the
logic to handle annotation DebugLocs at the same time.
branch-folder hoists common instructions from TBB and FBB into their
pred. Without this patch it achieves this by splicing the instructions from TBB
and deleting the common ones in FBB. That moves the debug locations and debug
instructions from TBB into the pred without modification, which is not
ideal. The merged instructions should get merged debug locations for debugging
and PGO purposes, which is handled in this patch. Debug instructions also need
to be handled differently. That'll come in another patch. This issue was found
by @omern1.
EnableTailMerge is false by default and is handled by the pass builder.
Passes are independent of target pipeline options.
This completes the generic `MachineLateOptimization` passes for the NPM
pipeline.
Fixes the "use after poison" issue introduced by #121516 (see
<https://github.com/llvm/llvm-project/pull/121516#issuecomment-2585912395>).
The root cause of this issue is that #121516 introduced "Called Global"
information for call instructions modeling how "Call Site" info is
stored in the machine function, HOWEVER it didn't copy the
copy/move/erase operations for call site information.
The fix is to rename and update the existing copy/move/erase functions
so they also take care of Called Global info.
- Add `MachineBlockFrequencyAnalysis`.
- Add `MachineBlockFrequencyPrinterPass`.
- Use `MachineBlockFrequencyInfoWrapperPass` in legacy pass manager.
- `LazyMachineBlockFrequencyInfo::print` is empty, drop it due to new
pass manager migration.
This reverts commit ab58b6d58edf6a7c8881044fc716ca435d7a0156.
In `CodeGen/Generic/MachineBranchProb.ll`, `llc` crashed with dumped MIR
when targeting PowerPC. Move test to `llc/new-pm`, which is X86
specific.
`BranchFolder::TryTailMergeBlocks(...)` removes unconditional branch
instructions and then recreates them. However, this process loses debug
source location information from the previous branch instruction, even
if tail merging doesn't change IR. This patch preserves the debug
information from the removed instruction and inserts them into the
recreated instruction.
Fixes#94050
https://github.com/llvm/llvm-project/pull/79940 put calls to
recomputeLiveIns into
a loop, to repeatedly call the function until the computation converges.
However,
this repeats a lot of code. This changes moves the loop into a function
to simplify
the handling.
Note that this changes the order in which recomputeLiveIns is called.
For example,
```
bool anyChange = false;
do {
anyChange = recomputeLiveIns(*ExitMBB) || recomputeLiveIns(*LoopMBB);
} while (anyChange);
```
only begins to recompute the live-ins for LoopMBB after the computation
for ExitMBB
has converged. With this change, all basic blocks have a recomputation
of the live-ins
for each loop iteration. This can result in less or more calls,
depending on the
situation.
This is a fix for the regression seen in
https://github.com/llvm/llvm-project/pull/79498
> Currently, the way that recomputeLiveIns works is that it will
recompute the livein registers for that MachineBasicBlock but it matters
what order you call recomputeLiveIn which can result in incorrect
register allocations down the line.
Now we do not recompute the entire CFG but we do ensure that the newly
added MBB do reach convergence.
Currently, the way that recomputeLiveIns works is that it will recompute
the livein registers for that MachineBasicBlock but it matters what
order you call recomputeLiveIn which can result in incorrect register
allocations down the line.
This PR fixes that by simply recomputing the liveins for the entire CFG
until convergence is achieved. This makes it harder to introduce subtle
bugs which alter liveness.
When merging blocks, if the previous block has no any branch instruction
and has one successor, the successor may be SEH landing pad and the
block will always raise exception and nerver fall through to next block.
We can not merge them in such case. isSuccessor should be used to
confirm it can fall through to next block.
When removing an empty machine basic block, all of its successors should
be inherited by its fall through MBB. This keeps CFG as only have one
entry which is required by LiveDebugValues.
Reland #77441 as LiveDebugValues test.
`MergePotentialElts::operator<` asserts that the two elements being
compared are not equal. However, sorting functions are allowed to invoke
the comparison function with equal arguments (though they usually don't
for efficiency reasons).
There is an existing special-case that disables the assert if
_GLIBCXX_DEBUG is used, which may invoke the comparator with equal args
to verify strict weak ordering. I believe libc++ also has strict weak
ordering checks under some options nowadays.
Recently, #71312 was reported, where a change to glibc's qsort_r
implementation can also result in comparison between equal elements.
From what I understood, this is an inefficiency that will be fixed on
the glibc side as well, but I think at this point we should just remove
this assertion.
Fixes https://github.com/llvm/llvm-project/issues/71312.
This reverts commit db6a979ae82410e42430e47afa488936ba8e3025.
Reland D102817 without any change. The previous revert was a mistake.
Differential Revision: https://reviews.llvm.org/D102817
Improve and enable folding of conditional branches with tail calls.
1. Make it so that conditional tail calls can be emitted even when
there are multiple predecessors.
2. Don't guard the transformation behind -Os. The rationale for
guarding it was static-prediction can be affected by whether the
branch is forward of backward. This is no longer true for almost any
X86 cpus (anything newer than `SnB`) so is no longer a meaningful
concern.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D140931
Improve and enable folding of conditional branches with tail calls.
1. Make it so that conditional tail calls can be emitted even when
there are multiple predecessors.
2. Don't guard the transformation behind -Os. The rationale for
guarding it was static-prediction can be affected by whether the
branch is forward of backward. This is no longer true for almost any
X86 cpus (anything newer than `SnB`) so is no longer a meaningful
concern.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D140931
