In our default SelectionDAG where i32 isn't legal, the zext will become
and i64 AND and often get optimized out on its own. With i32 legal, we
need to turn it in into sext.w and rely on RISCVOptWInstrs to remove it.
Coerce the register bank based on the users of the G_LOAD or the
defining instruction for the G_STORE.
s64 on rv32 is handled by forcing the FPRB register bank.
The DWARF 5 specification says that:
> The name index must contain an entry for each debugging information
entry that
> defines a named [...] label [...].
The verifier currently verifies this, but the AsmPrinter does not add
entries for TAG_labels in debug_names. This patch addresses the issue by
ensuring we add labels in the accelerator tables once we have a fully
completed DIE for the TAG_label entry.
We also respect the spec as follows:
> DW_TAG_label debugging information entries without an address
attribute
> (DW_AT_low_pc, DW_AT_high_pc, DW_AT_ranges, or DW_AT_entry_pc) are
excluded.
The effect of this on the size of accelerator tables is minimal, as
TAG_labels are usually created by C/C++ labels (see example in test),
which are typically paired with "goto" statements.
Most (x86) swiftasync functions tend to use both SelectionDAGISel and
FastISel lowering:
* FastISel argument lowering can only handle C calling convention.
* FastISel fails mid-BB in a number of ways, including in simple `ret
void` instructions under certain circumstances.
This dance of SelectionDAG (argument) -> FastISel (some instructions) ->
SelectionDAG(remaining instructions) is lossy; in particular, Argument
information lowering is cleared after that first SelectionDAG run.
Since swiftasync functions rely heavily on proper Argument lowering for
debug information, this patch disables the use of FastISel in such
functions.
PR #71614 identified an issue in the lowering of v1f16 vector compares,
where the `v1i1 setcc` is expanded to `v1i16 setcc`, and the `v1i16
setcc` tries to be expanded to a `v2i16 setcc` which fails. For floating
point types we can let them scalarize instead though, generating a
`setcc f16` that can be lowered using normal fp16 lowering.
07a8ff4892b2a54f0bd5843f863bcffa7a258f1f added a special case combine
for v1 vselect to expand the predicate type to the same size as the fcmp
operands. This turns that off for float types, allowing them to
scalarize naturally, which hopefully fixes the issue by preventing the
v1i16 setcc, meaning it wont try to widen to larger vectors.
The codegen might not be optimal, but as far as I can tell everything
generated successfully, providing that no `v1i16 setcc v1f16`
instructions get generated.
AMDGPU target has faced the situation which can be illustrated with the
following testcase:
define void @dont_merge_cbranches(i32 %V) {
%divergent_cond = icmp ne i32 %V, 0
%uniform_cond = call i1 @uniform_result(i1 %divergent_cond)
br i1 %uniform_cond, label %bb2, label %exit, !prof !0
bb2:
br i1 %divergent_cond, label %bb3, label %exit
bb3:
call void @bar( )
br label %exit
exit:
ret void
}
!0 = !{!"branch_weights", i32 1, i32 100000}
SimplifyCFG merges branches on %uniform_cond and %divergent_cond which is undesirable because the first branch to bb2 is taken extremely rare and the second branch is expensive. The merged branch becomes as expensive as the second.
This patch prevents such merging if the branch to the second branch is unlikely to happen.
Recommits the changes from https://reviews.llvm.org/D148216.
Explicitly named globals are now matched literally, instead of emitting
a capture group for the name. This resolves#70047.
Metadata and annotations, on the other hand, are captured and matched
against by default, since their identifiers are not stable.
The reasons for revert (#63746) have been fixed:
The first issue, that of duplicated checkers, has already been resolved
in #70050.
This PR resolves the second issue listed in #63746, regarding the order
of named and unnamed globals. This is fixed by recording the index of
substrings containing global values, and sorting the checks according to
that index before emitting them. This results in global value checks
being emitted in the order they were seen instead of being grouped
separately.
Extends the existing code that performed something similar for SUBV_BROADCAST_LOAD, but this is just for cases where AVX2 targets loads full width 128-bit constant vectors but broadcasts the equivalent 256-bit constant vector
Fixes AVX2 case for Issue #70947
If there are function calls in the candidate loop and we have vectorized
variants available, try some wider VFs in case the conservative initial
maximum based on the widest types in the loop won't actually allow us
to make use of those function variants.
The compiler currently abends with an impossible reg-to-reg copy when
producing a negative zero FP immediate on RV32 with -Zdinx. This is
because we emit a negation that uses FP registers. Emit the right node
to produce correct code.
A constant value is unique in llvm context. InferAddressSpaces was
replacing its users in other functions as well. This leads to unexpected
behavior in our downstream use case after the pass.
InferAddressSpaces is a function passe, so it shall not modify functions
other than currently processed one.
Co-authored-by: Abhinav Gaba <abhinav.gaba@intel.com>
---------
Co-authored-by: Abhinav Gaba <abhinav.gaba@intel.com>
SGPR spill VGPRs are WWM registers so allow them to be allocated by
SIPreAllocateWWMRegs pass.
This intentionally prevents spilling of these VGPRs when enabled.
Add subrange tracking and handling for LiveIntervals during PHI
elimination.
This requires extending MachineBasicBlock::SplitCriticalEdge to also
update subrange intervals.
This PR fixes https://github.com/llvm/llvm-project/issues/55013 : the
max intrinsics is not generated for this simple loop case :
https://godbolt.org/z/hxz1xhMPh. This is caused by a ICMP not being
folded into a select, thus not generating the max intrinsics.
For the story :
Since LLVM 14, SCCP pass got smarter by folding sext into zext for
positive ranges : https://reviews.llvm.org/D81756. After this change,
InstCombine was sometimes unable to fold ICMP correctly as both of the
arguments pointed to mismatched zext/sext. To fix this, @rotateright
implemented this fix : https://reviews.llvm.org/D124419 that tries to
resolve the mismatch by knowing if the argument of a zext is positive
(in which case, it is like a sext) by using ValueTracking, however
ValueTracking is not smart enough to infer that the value is positive in
some cases. Recently, @nikic implemented #67982 which keeps the
information that a zext is non-negative. This PR simply uses this
information to do the folding accordingly.
TLDR : This PR uses the recent nneg tag on zext to fold the icmp
accordingly in instcombine.
This PR also contains test cases for sext/zext folding with InstCombine
as well as a x86 regression tests for the max/min case.
Add support for bf16x2 instructions such as setp, fneg, fabs, etc;
Fix the instructions that were not differentiated between sm_80 and
sm_90 support, such as fpround etc.
Add more bf16 test cases to ensure the correct behavior.
---------
Co-authored-by: shenhan03 <shenhan03@kuaishou.com>
This patch moves creating the middle VPBBs and an initial empty
vector loop region for the top-level loop to createInitialVPlan.
This consolidates code to create the initial VPlan skeleton and enables
adding other bits outside the main region during initial VPlan
construction. In particular, D150398 will add the exit check & branch to
the middle block.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D158333
When getSplatOp returns nullptr, the intrinsic cannot be scalarized.
This patch includes a test case that fixes a crash from trying to
scalarize the VPIntrinsic when getSplatOp returns nullptr.
This fixes https://github.com/llvm/llvm-project/issues/72034.
Fixes a bug introduced by
commit f95b2f1acf11 ("Reland [InstrProf][compiler-rt] Enable MC/DC
Support in LLVM Source-based Code Coverage (1/3)")
createDataVariable() needs to check that a data variable wasn't already
created before creating it. Previously, this was done inadvertantly in
getOrCreateRegionCounters(), which checked that the RegionCounters was
not created multiple times before creating the counter section and the
data variable. When the creation of the data variable was abstracted
into its own function (createDataVariable()), there was no corresponding
check. This was failing on a case in which an instrumented function was
being inlined into multiple functions and a duplicate data variable was
created, which led to a segfault in emitNameData(). Test case added
based on the repro that also ensures a single data variable was created
in this case.
After performing sink-and-fold over a COPY, the original instruction is
replaced with one that produces its output in the destination of the
copy. Its value is still available (in a hard register), so if there are
debug instructions which refer to the (now deleted) virtual register
they could be updated to refer to the hard register, in principle.
However, it's not clear how to do that, moreover in some cases the debug
instructions may need to be replicated proportionally to the number of
the COPY instructions replaced and in some extreme cases we can end up
with quadratic increase in the number of debug instructions, e.g:
int f(int);
void g(int x) {
int y = x + 1;
int t0 = y;
f(t0);
int t1 = y;
f(t1);
}
