Conflicting module flags leads to a proper error for regular LTO but a crash
(report_fatal_error) for ThinLTO. Switch to createStringError to fix the crash
and match regular LTO.
There are extra patterns that have for these three logic operations
that aren't covered in `SimplifyDemandedUseBits`. To avoid duplicating
the code, just use `analyzeKnownBitsFromAndXorOr` in
`SimplifyDemandedUseBits` to get full coverage.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D142429
`instrprof` currently does not set `__llvm_prf_vnds`'s alignment after creating it. The consequence is that the alignment is set to 16 later (c0f3ac1d00/llvm/lib/IR/DataLayout.cpp (L1019)). This can lead to undefined behaviour when we calculate `NumVNodes` in `lprofGetLoadModuleSignature` (c0f3ac1d00/compiler-rt/lib/profile/InstrProfilingMerge.c (L32)). The reason is that when the `__llvm_prf_vnds` array is 16 byte aligned, `__llvm_profile_end_vnodes() - __llvm_profile_begin_vnodes()` may not be a multiple of the size of ValueProfNode (which is 24, 20 on 32 bit targets).
This patch sets `__llvm_prf_vnds`'s alignment to its ABI alignment, which always divides its size. Then `__llvm_profile_end_vnodes() - __llvm_profile_begin_vnodes()` will be a multiple of `sizeof(ValueProfNode)`.
Reviewed By: w2yehia, MaskRay
Differential Revision: https://reviews.llvm.org/D144302
Prior to this patch, WPD was not acting on relative-vtables in C++. This
involves teaching WPD about these things:
- llvm.load.relative which is how relative-vtables are indexed (instead of GEP)
- dso_local_equivalent which is used in the vtable itself when taking the
offset between a virtual function and vtable
- Update llvm/test/ThinLTO/X86/devirt.ll to use opaque pointers and add
equivalent tests for RV
Differential Revision: https://reviews.llvm.org/D134320
Follow on to D144209 to support single implementation devirtualization
for Regular LTO when the vtable holds a function alias.
For now I have prevented other optimizations performed in regular LTO
that need to analyze the contents of the function target when the vtable
holds an alias, as I'm not sure they are always correct to perform in
that case.
Differential Revision: https://reviews.llvm.org/D144270
Added the condition 'TrueVal->getType()->isIntOrIntVectorTy' to a block of code
in foldSelectInstWithICmp which is only valid if the TrueVal is integer type.
This change was split off from D136861.
To query the maximum value for vscale, the LV queries the vscale_range
attribute or a TTI hook. To avoid having to reimplement the same behaviour
for multiple uses (such as in D142894), it makes sense to move this code
to a separate function.
The target hook prefersVectorizedAddressing() already exists to check with
target if address computations should be vectorized, so it seems like this
should be used in SLPVectorizer as well.
Reviewed By: ABataev, RKSimon
Differential Revision: https://reviews.llvm.org/D144128
The m_VScale() matcher is unusual in that it requires a DataLayout.
It is currently used to determine the size of the GEP type. However,
I believe it is sufficient to check for the canonical
<vscale x 1 x i8> form here -- I don't think there's a need to
recognize exotic variations like <vscale x 1 x i4> as a vscale
constant representation as well.
Differential Revision: https://reviews.llvm.org/D144566
This is motivated by patterns like !isfinite || zero. The AMDGPU math
libraries have a lot of patterns like this, and I'm trying to fix the
code to be more portable and less dependent on directly calling class
intrinsics.
I believe this is the first place where new is.fpclass calls are
introduced. There are more class-like compares that could be
recognized; this is a set I currently care about plus a few extras.
Keep the == 0 cases disabled for now. It depends on the denormal
mode. If we just check IEEE mode now, it will break my use case
without another patch I'm working on.
LSR can include Regs of AddRec SCEVs from different loops, which do not combine
well when added in Scalar Evolution. As they should never produce constant
differences so we can just guard against trying to create them.
Fixes#60927
Enabling assignment tracking without this patch, a significant amount of
additional compiler run time comes from the RemoveRedundantDbgInstrs call in
InstCombine. This patch reduces compiler run time by choosing better places to
call RemoveRedundantDbgInstrs.
In non-assignment-tracking builds, RemoveRedundantDbgInstrs is called by
InstCombine if LowerDbgDeclare makes a change (i.e. it is _sometimes_
called). In assignment tracking builds LowerDbgDeclare doesn't do anything. We
still need to clean up redundant intrinsics to avoid a large performance hit
due to the number of instructions, so the current approach is to have
InstCombine _always_ call RemoveRedundantDbgInstrs.
Instrumenting the compiler to run RemoveRedundantDbgInstrs after every pass and
dump the numbers and building CTMark/tramp3d-v4 indicates that SROA and
LoopVectorize give us a bigger bang (number removed) for buck (times pass is
run).
The compile time tracker reports that this patch reduces the number of
instructions retired building CTMark projects by an average of 1.1%.
Reviewed By: scott.linder
Differential Revision: https://reviews.llvm.org/D144483
SimplifyCFG currently drops !nontemporal metadata when sinking
common instructions. With this change, SimplifyCFG and similar
transforms will preserve !nontemporal metadata as long as it is
set on both original instructions.
Differential Revision: https://reviews.llvm.org/D144298
In order to allow targets to disable interleaving for scalable vectors, pass the entire VF's ElementCount to getMaxInterleaveFactor.
This is based off of the approach used here: 8d36708507
The plan would then be to disable interleaving on scalable VFs on RISC-V in a follow up patch.
See https://reviews.llvm.org/D143723#4132349
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D144474
With this patch freeze undef/poison will no longer be folded into a constant if it's used as a
vector operand in a shufflevector.
Differential Revision: https://reviews.llvm.org/D143593
No need to swap extractelements, which were not excluded from the list
during cost analysis. It leads to incorrect cost calculation and make
vector code more profitable than it is actually is.
When replacing return values with undef, we should also drop the
noundef attribute (and other UB implying attributes).
Differential Revision: https://reviews.llvm.org/D144461
The pair of div folds was just added with:
4966d8ebe1bbe5bd6a4d28
But as noted in the post-commit review, we don't actually need
the no-remainder requirement for an unsigned division (still
need the no-unsigned-wrap though):
https://alive2.llvm.org/ce/z/qHjK3Q
InstCombine currently performs a constant folding attempt as part
of the main InstCombine loop, before visiting the instruction.
However, each visit method will also attempt to simplify the
instruction, which will in turn constant fold it. (Additionally,
we also constant fold instructions before the main InstCombine loop
and use a constant folding IR builder, so this is doubly redundant.)
There is one place where InstCombine visit methods currently don't
call into simplification, and that's casts. To be conservative,
I've added an explicit constant folding call there (though it has
no impact on tests).
This makes for a mild compile-time improvement and in particular
mitigates the compile-time regression from enabling load
simplification in be88b5814d9efce131dbc0c8e288907e2e6c89be.
Differential Revision: https://reviews.llvm.org/D144369
foldAllocaCmp() checks whether the alloca is not captured (ignoring
the icmp). Replace the manual implementation of escape analysis
with CaptureTracking.
The primary practical difference is that CaptureTracking handles
nocapture arguments, while foldAllocaCmp() was using a hardcoded
list.
This is basically just the CaptureTracking refactoring from D120371
without the other changes.
If the scalar must be extracted and then used in the gather node,
instead we can emit shuffle instruction to avoid those extra
extractelements and vector-to-scalar and back data movement.
Part of D110978
Differential Revision: https://reviews.llvm.org/D141940
Suggested by @Ayal as follow-up improvement in D143864.
I was unable to find a case where this actually changes generated code,
but it is a unifying code to use common infrastructure.
This reverts commit f9599bbc7a3f831e1793a549d8a7a19265f3e504.
For some reason it caused us a huge compile time regression in downstream
workloads. Not sure whether the source of it is in upstream code ir not.
Temporarily reverting until investigated.
Differential Revision: https://reviews.llvm.org/D142330
[Originally committed as f6ddf7781471b71243fa3c3ae7c93073f95c7dff;
reverted in bbef38352fbade9e014ec97d5991da5dee306da7 due to test
breakage; now relanded with the Arm tests conditioned on
`arm-registered-target`]
The LowerTypeTests pass emits a jump table in the form of an
`inlineasm` IR node containing a string representation of some
assembly. It tests the target triple to see what architecture it
should be generating assembly for. But that's not good enough for
`Triple::thumb`, because the 32-bit PC-relative `b.w` branch
instruction isn't available in all supported architecture versions. In
particular, Armv6-M doesn't support that instruction (although the
similar Armv8-M Baseline does).
Most of this patch is concerned with working out whether the
compilation target is Armv6-M or not, which I'm doing by going through
all the functions in the module, retrieving a TargetTransformInfo for
each one, and querying it via a new method I've added to check its
SubtargetInfo. If any function's TTI indicates that it's targeting an
architecture supporting B.W, then we assume we're also allowed to use
B.W in the jump table.
The Armv6-M compatible jump table format requires a temporary
register, and therefore also has to use the stack in order to restore
that register.
Another consequence of this change is that jump tables on Arm/Thumb
are no longer always the same size. In particular, on an architecture
that supports Arm and Thumb-1 but not Thumb-2, the Arm and Thumb
tables are different sizes from //each other//. As a consequence,
``getJumpTableEntrySize`` can no longer base its answer on the target
triple's architecture: it has to take into account the decision that
``selectJumpTableArmEncoding`` made, which meant I had to move that
function to an earlier point in the code and store its answer in the
``LowerTypeTestsModule`` class.
Reviewed By: lenary
Differential Revision: https://reviews.llvm.org/D143576
There is no need to update the AlsoPack field when creating
VPReplicateRecipes. It can be easily computed based on the VP def-use
chains when it is needed.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D143864
InstCombine is supposed to be a superset of InstSimplify, but
failed to invoke load simplification.
Unfortunately, this causes a minor compile-time regression, which
will be mitigated in a future commit.
Since canonicalizeForInvariantConditionInjection is introduced the
in loop successor may be the second successor.
Reviewed By: mkazantsev
Differential Revision: https://reviews.llvm.org/D144361
1 << (cttz X) --> -X & X
https://alive2.llvm.org/ce/z/qv3E9e
This creates an extra use of the input value, so that's generally
not preferred, but there are advantages to this direction:
1. 'negate' and 'and' allow for better analysis than 'cttz'.
2. This is more likely to induce follow-on transforms (in the
example from issue #60801, we'll get the decrement pattern).
3. The more basic ALU ops are more likely to result in better
codegen across a variety of targets.
This won't solve the motivating bugs (see issue #60799) because
we do not recognize the redundant icmp+sel, and the x86 backend
may not have the pattern-matching to produce the optimal BMI
instructions.
Differential Revision: https://reviews.llvm.org/D144329
This fixes an infinite loop if isa<T>(II->getOperand(1)) is true.
Update Base at the top of the loop, before the continue.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D144292
At the moment, properlyDominates(A, A) can return true via
LocalComesBefore. Add an early exit to ensure it returns false if
A == B.
Note: no test has been added because the existing test suite covers this
case already with libc++ with assertions enabled.
Fixes https://github.com/llvm/llvm-project/issues/60850.
Given this transformation: X % Y -> X - (X / Y) * Y
This patch strips off the poison-generating flag of X / Y such as exact, because it may make the optimized form result poison whereas X % Y does not.
The issue was reported here: https://github.com/llvm/llvm-project/issues/60748
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D144333
