We add a field `IsOutlined` to indicate whether a MachineFunction
is outlined and set it true for outlined functions in MachineOutliner.
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D146191
Outlining isn't always a win when the saved instruction count is >= 1.
The overhead of representing a new function in the binary depends on
exception metadata and alignment. So parameterize this for local tuning.
Reviewed By: paquette
Differential Revision: https://reviews.llvm.org/D136774
AMDGPU code with enabled address sanitizer generates tons of stack objects (> 200000 in my testcase) and
takes forever to compile due to the time spent on stack slot sharing.
While LiveRange::overlaps method has logarithmic complexity on the number of segments in the involved
liveranges the problem is that when a new interval is assigned to a used color it's tested against
overlapping every other assigned interval for that color.
Instead I decided to join all assigned intervals for a color into a single interval and this allows to
have logarithmic complexity on the number of segments for the joined interval.
This patch reduced time spent on stack slot coloring pass from 628 to 3 seconds on my testcase.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D146057
This was assuming all register operands were assigned to physical registers.
This should ignore the operands which weren't assigned in this run.
Fixes#61134
For IR like:
```
%alloca = alloca ...
dbg.value(%alloca, !myvar, OP_deref(<other_ops>))
```
GlobalISel lowers it to MIR:
```
%some_reg = G_FRAME_INDEX <stack_slot>
DBG_VALUE %some_reg, !myvar, OP_deref(<other_ops>)
```
In other words, if the value of `!myvar` can be obtained by
dereferencing an alloca, in MIR we say that the _location_ of a variable
is obtained by dereferencing register %some_reg (plus some
`<other_ops>`).
We can instead remove the use of `%some_reg`: the location of `!myvar`
_is_ `<stack_slot>` (plus some `<other_ops>`). This patch implements
this transformation, which improves debug information handling in O0, as
these registers hardly ever survive register allocation.
A note about testing: similar to what was done in D76934
(f24e2e9eebde4b7a1d), this patch exposed a bug in the Builder class when
using `-debug`, where we tried to print an incomplete instruction. The
changes in `MachineIRBuilder.cpp` address that.
Differential Revision: https://reviews.llvm.org/D147536
The intrinsics have been out of experimental since 322d0afd875d
("[llvm][mlir] Promote the experimental reduction intrinsics to be
first class intrinsics.", 2020-10-07); update some places that still
referred to them as experimental.
Compiler hits infinite loop in DAGCombine. For force-streaming-compatible-sve
mode we have custom lowering for 128-bit vector splats and later in
DAGCombiner::SimplifyVCastOp() we scalarized SPLAT because we have custom
lowering for SME. Later, we restored SPLAT opertion via performMulCombine().
Such dbg.assigns will occur if you write zero-sized memcpys (see
https://reviews.llvm.org/D146987#4240016).
Handle this in AssignmentTrackingAnalysis (back end) rather than
AssignmentTrackingPass (declare-to-assign) in case it is possible to reproduce
this as a result of optimisations.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D147435
This is the first change for FS-AFDO integration with CSSPGO. There are more patches coming.
With pseudo probes, we do not assign FS discriminators to any other instructions since we will be using only probes for profile correlation.
Also call instructions are excluded since their dwarf discriminators are used for other purposes, i.e, storing probe ids. Since they are not getting a FS discriminator, they will also be excluded from MIR profile loading. The corresponding changes will be in the subsequent patches.
Reviewed By: wenlei
Differential Revision: https://reviews.llvm.org/D147286
This broke Objective-C autorelease / retainAutoreleasedReturnValue, see
comments on the code review.
> This is a mitigation patch for
> https://bugs.chromium.org/p/llvm/issues/detail?id=30, where existing stack
> protection is skipped if a function is returned through by an unwinder rather
> than the normal call/return path. The recent patch D139254 added the ability to
> instrument a visible unwind path, at least in the IR case (I'm working on the
> SelectionDAG instrumentation too) but there are still invisible unwinds it
> can't reach.
>
> So this patch adds logic to DwarfEHPrepare that goes through a function,
> converting any call that might throw into an invoke to a simple resume cleanup,
> and adding cleanup clauses to existing landingpads that lack them. Obviously we
> don't really want to do this if it's wasted effort, so I also exposed
> requiresStackProtector from the actual StackProtector code to skip the extra
> paths if they won't be used.
>
> Changes:
> * Move test to AArch64 directory as it relies on target presence.
> * Re-add Dominator-tree maintenance. Accidentally cherry-picked wrong patch.
> * Skip adding paths on Windows EH functions.
>
> https://reviews.llvm.org/D143637
This reverts commit 2d690684f66fabc9ac6a2c70fcff3b31c9520794.
Verify the SlotIndexes analysis after a pass that claims to preserve it,
even if there are no further passes (apart from the verifier itself)
that would use the analysis.
Differential Revision: https://reviews.llvm.org/D129201
Similar to the existing SelectionDAG::SplitVector helper, this helper creates the EXTRACT_ELEMENT nodes for the LO/HI halves of the scalar source.
Differential Revision: https://reviews.llvm.org/D147264
It is expected that shuffles that we hoist through binops only have a single
vector operand, the other being undef/poison. The checks for
isDeInterleaveMaskOfFactor check that all the elements come from inside the
first vector, but with non-canonical shuffles the second operand could still
have a value. Add a quick check to make sure it is UndefValue as expected, to
make sure we don't run into problems with BinOpShuffles not using BinOps.
Fixes#61749
Differential Revision: https://reviews.llvm.org/D147306
This patch renames the `mroptr` option to `mxcoff-roptr` to indicate in the option itself that it is xcoff specific.
Reviewed By: hubert.reinterpretcast
Differential Revision: https://reviews.llvm.org/D147161
The elements in FragmentMap are big objects, use reference can get
better performance, as someone do in line 1912.
Differential Revision: https://reviews.llvm.org/D147126
Due to a change in the APIs used to determine what instructions can be outlined, the check for label outling was never hit. Instead, all labels were considered invisible, which is the opposite of the intended behavior and causes obscure crashes down the line. We now replicate the original behavior more closely, with explicit checks for known-good and known-bad instruction types.
Reviewed by: paquette
Differential Revision: https://reviews.llvm.org/D147178
Define and use a MachineOperand overload of isPlainlyKilled. This
improves codegen in a couple of tests because it catches the case where
MO does not kill Reg but another operand of the same instruction does.
Differential Revision: https://reviews.llvm.org/D147167
... when finding all memory uses for an address and make it a
parameter.
Now that we have avoided potentially exponential run time of
`FindAllMemoryUses` in D143893. it'd be beneficial to increase the
limit up from 20.
Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D143894
Change-Id: I3abdf40332ef65e9b2f819ac32ac60e4200ec51d
The counter of the number of instructions seen in `FindAllMemoryUses`
is reset after returning from a recursive invocation of
`FindAllMemoryUses` to the value it had before the call. In effect,
depending on the shape of the uses graph, the function may scan up to
`2^N-1` instructions where `N` is the scan limit
(`MaxMemoryUsesToScan`). This does not look intuitive or intended.
This patch changes the counting to just count the scanned
instructions, independent of the shape of the references.
Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D143893
Change-Id: I99f5de55e84843cf2fbea287d6ae4312fa196240
CodeGenPare may optimize memory access modes.
During such optimization, it might create a new instruction representing combined value.
Later, If the optimization failed, the generated value is not removed and remains a dead instruction.
Normally this won't be a problem as dead code will be eliminated later.
However, in this case (Issue 58538), the generated instruction may trigger an infinite loop.
The infinite loop involves `sinkCmpExpression`, where it tries to optimize the placeholder generated by us.
(See the test case detailed in the issue)
To fix this, we remove the unnecessary placeholder immediately when we abort the optimization.
`AddressingModeCombiner` will keep track of the placeholder, and remove it if it is an inserted placeholder and has no uses.
This patch fixes https://github.com/llvm/llvm-project/issues/58538, a test is also included.
Reviewed By: skatkov
Differential Revision: https://reviews.llvm.org/D147041
In this example:
```
$d14 = COPY killed $d18
$s0 = MI $s28
```
$s28 is a sub-register of $d14. However, $d18 does not have
sub-registers and thus cannot be forwarded. Previously, this resulted
in $noreg being substituted in place of the use of $s28, which later
led to an assertion failure.
Fixes https://github.com/llvm/llvm-project/issues/60908, a regression
that was introduced in D141747.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D146930
The patch customized lower vector type ISD::STRICT_FP_ROUND to RISCVISD::STRICT_FP_ROUND.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D147113
Given the intent of Split DWARF is to minimize .o file size it seems
like adequate signal that it's worth a minor tradeoff in .dwo size to
significantly reduce .o size (though it doesn't reduce linked executable
size - the cost is mostly in the static relocations resolved by the
linker).
MemLocFragmentFill uses an IntervalMap to track which bits of each variable are
stack-homed. Intervals with the same value (same stack location base address)
are automatically coalesced by the map. This patch changes the analysis to take
advantage of that and insert a new dbg loc after each def if any coalescing
took place. This results in some additional redundant defs (we insert a def,
then another that by definition shadows the previous one if any coalescing took
place) but they're all cleaned up thanks to the previous patch in this stack.
This reduces the total number of fragments created by
AssignmentTrackingAnalysis which reduces compile time because LiveDebugValues
computes SSA for every fragment it encounters. There's a geomean reduction in
instructions retired in a CTMark LTO-O3-g build of 0.3% with these two patches.
One small caveat is that this technique can produce partially overlapping
fragments (e.g. slice [0, 32) and slice [16, 64)), which we know
LiveDebugVariables doesn't really handle correctly. Used in combination with
instruction-referencing this isn't a problem, since LiveDebugVariables is
effectively side-stepped in instruction-referencing mode. Given this, the
coalescing is only enabled when instruction-referencing is enabled (but the
behaviour can be overriden using -debug-ata-coalesce-frags=<bool>).
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D146980
`removeRedundantDbgLocsUsingBackwardScan` removes redundant dbg loc definitions
by scanning backwards through contiguous sets of them (a "wedge"), removing
earlier (in IR order terms) defs for fragments of variables that are defined
later in the wedge.
In this patch we use a `Bitvector` for each variable to track which bits have
definitions to more accurately determine whether a loc def is redundant. This
patch increases compile time by itself, but reduces it when combined with the
follow-up patch.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D146978
Restructure AssignmentTrackingLowering::join to avoid a map copy in the case
where BB has more than one pred.
We only need to perform a copy of a pred LiveOut if there's exactly one
already-visited pred (Result = PredLiveOut). With more than one pred the result
is built by calling Result = join(std::move(Result), PredLiveOut) for each
subsequent pred, where join parameters are const &. i.e. with more than 1 pred
we can avoid copying by referencing the first two pred LiveOuts in the first
join and then using a move + reference for the rest.
This reduces compile time for CTMark LTO-O3-g builds.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D144732
Only calculate fragment overlaps for partially stack homed variables. This
filter is already applied to the rest of the analysis - this change simply
prevents some unnecessary work.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D145515
For Big Endian, the function `emitGlobalConstantLargeInt` tries to right shift `Realigned` by an amount `ExtraBitSize` in place. However, if the constant to emit has a bit width less than 64 and the bit width is not a multiple of 8, the shift amount will be greater than the bit width of `Realigned`, which causes assertion error described in issue [[ https://github.com/llvm/llvm-project/issues/59055 | issue #59055 ]].
This patch fixes the issue by avoiding right shift when bit width is under 64 to avoid the assertion error.
Reviewed By: Peter
Differential Revision: https://reviews.llvm.org/D138246
This reverts commit db6a979ae82410e42430e47afa488936ba8e3025.
Reland D102817 without any change. The previous revert was a mistake.
Differential Revision: https://reviews.llvm.org/D102817
This is a simple patch to make sure fast math flags are propagated through to
the newly created symmetric operations, which can help with later
simplifications.
Differential Revision: https://reviews.llvm.org/D146409
Add some special cases for UADDO to recover codegen after D146786.
Reviewed By: reames, liaolucy
Differential Revision: https://reviews.llvm.org/D146789
For add, if we match the constant edge case the add isn't used by
the compare so we shouldn't check for 2 users.
For sub, the compare is not a user of the sub so the math is
used if the sub has any users.
This regresses RISC-V which I will work on other patches for.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D146786
This DAG combine is correct on little endian targets but
is incorrect on big endian targets.
Add big endian code to correct it.
Differential revision: https://reviews.llvm.org/D146460
