Before deleting extractelement instruction for vectorized GEP with
external users, need to check that all users vectorized before deleting
this extractelement.
We can try to vectorize long store sequences, if short ones were
unsuccessful because of the non-profitable vectorization. It should not
increase compile time significantly (stores are sorted already,
complexity is n x log n), but vectorize extra code.
Metric: size..text
Program size..text
results results0 diff
test-suite :: External/SPEC/CINT2006/400.perlbench/400.perlbench.test 1088012.00 1088236.00 0.0%
test-suite :: SingleSource/UnitTests/matrix-types-spec.test 480396.00 480476.00 0.0%
test-suite :: External/SPEC/CINT2017rate/525.x264_r/525.x264_r.test 664613.00 664661.00 0.0%
test-suite :: External/SPEC/CINT2017speed/625.x264_s/625.x264_s.test 664613.00 664661.00 0.0%
test-suite :: External/SPEC/CFP2017rate/510.parest_r/510.parest_r.test 2041105.00 2040961.00 -0.0%
test-suite :: MultiSource/Applications/JM/lencod/lencod.test 836563.00 836387.00 -0.0%
test-suite :: MultiSource/Benchmarks/7zip/7zip-benchmark.test 1035100.00 1032140.00 -0.3%
In all benchmarks extra code gets vectorized
Reviewers: RKSimon
Reviewed By: RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/88563
Addresses issue #88716.
Some function parameter names in the affected header files did not match
the parameter names in the definitions, or were listed in a different
order.
---------
Signed-off-by: Troy-Butler <squintik@outlook.com>
Hide the details of the one-off linked list used to implement the leader
lists by
wrapping them in iterators, and then use that to reimplement a number of
traversals
using standard algorithms and range-based for-loops.
No functional change intended.
Code expects `VectorOrPrimitiveTypeSizeInBits` compile time value,
which is not available for vscale.
In trivial case of the same type we need to do nothing.
Currently, the OptimizePointerDifference fold does not trigger when
working on the sub of two geps where one of the geps has multiple uses,
to avoid duplicating the offset arithmetic too much.
However, there are cases where performing it would still be
clearly profitable, e.g. test_sub_ptradd_multiuse.
This patch drops the one-use restriction using the same strategy we use
in GEP comparison folds: If there are multiple uses, we rewrite the GEP
to use the expanded offset arithmetic instead (effectively
canonicalizing it into ptradd representation).
Fixes https://github.com/llvm/llvm-project/issues/88231.
Extend LoopAccessAnalysis to support different strides and as a
consequence non-constant distances between dependences using SCEV to
reason about the direction of the dependence.
In multiple places, logic to rule out dependences using the stride has
been updated to only be used if StrideA == StrideB, i.e. there's a
common stride.
We now also may bail out at multiple places where we may have to set
FoundNonConstantDistanceDependence. This is done when we need to bail
out and the distance is not constant to preserve original behavior.
Fixes https://github.com/llvm/llvm-project/issues/87336
PR: https://github.com/llvm/llvm-project/pull/88039
These are already handled by canEvaluateShifted/getShiftedValue (one-use
only), and also in reassociateShiftAmtsOfTwoSameDirectionShifts (also
multi-use), so let's at least get rid of the *third* implementation...
Since all optimizations that use range metadata now also handle range attribute, this patch replaces writes of
range metadata for call instructions to range attributes.
The HWASan transform currently always uses x20 to pass the shadow base to hwasan_check_memaccess_shortgranules, even if the shadow base is a constant known at compile time (e.g., for Fuchsia, KHWASan, or via -hwasan-mapping-offset). This patch uses the fixed shadow variant of the hwasan_check_memaccess_shortgranules intrinsic (introduced in 365bddf634), allowing the shadow base to be materialized inside the memaccess callee.
We currently only support this optimization for AArch64; it is a no-op on other platforms due to lack of support for lowering the intrinsic.
Note: when a binary is instrumented with -hwasan-mapping-offset, it is necessary to specify HWASAN_OPTIONS=fixed_shadow_base=... (see ea991a11b2a3d2bfa545adbefb71cd17e8970a43) at runtime to ensure the shadow is mapped appropriately.
This patch does the following:
Adds the following functions:
- replaceDominatedUsesWithIf() that takes a callback.
- canReplacePointersIfEqual(...) returns true if the underlying object
is the same, and for null and const dereferencable pointer replacements.
- canReplacePointersIfEqualInUse(...) returns true for the above as well
as if the use is in icmp/ptrtoint or phi/selects feeding into them.
Updates GVN using the functions above so that the pointer replacements
are only made using the above API.
https://reviews.llvm.org/D143129
A logic issue arose when inlining via `CloneAndPruneFunctionInto`,
which, besides cloning, performs instruction simplification as well.
By the time a new cloned instruction is being simplified, phi-nodes
are not remapped yet as the whole CFG needs to be processed first.
As `VMap` state at this stage is incomplete, `threadCmpOverPHI` and
variants could lead to unsound optimizations. This issue has been
addressed by performing basic constant folding while cloning, and
postponing instruction simplification once phi-nodes are revisited.
Fixes: https://github.com/llvm/llvm-project/issues/87534.
Since `DivRemPairPass` runs after `ReassociatePass` in the optimization
pipeline, I decided to do this simplification in `InstCombine`.
Alive2: https://alive2.llvm.org/ce/z/JgsiqfFixes#76128.
We're replacing the select with the false value here, but it may be more
poisonous if m_Not contains poison elements. Fix this by introducing a
m_NotForbidPoison matcher and using it here.
Fixes https://github.com/llvm/llvm-project/issues/89500.
Swapping the operands of a select is not valid if one hand is more
poisonous that the other, because the negation zero contains poison
elements.
Fix this by adding an extra parameter to isKnownNegation() to forbid
poison elements.
I've implemented this using manual checks to avoid needing four variants
for the NeedsNSW/AllowPoison combinations. Maybe there is a better way
to do this...
Fixes https://github.com/llvm/llvm-project/issues/89669.
Almost NFC, instrumentation is as correct as it was before.
We need InstrumentationList grouped by origin instruction,
so we used stable_sort. However these objects already grouped
because we never interleave sequences of `insertShadowCheck`
of different instrunction.
Pointer sort has artifact that it was deppendent on allocator behavior,
so we could inserted checks in a different order.
There is no test, as I failed to reproduce this with `opt`. My guess
is that for reproducer we need to increase fragmentation in the
allocator.
This change adds the z/OS personality function to the list of known EH
personality functions. It enables removing of the EH data/labels if the
personality function is not invoked.
No need to try to vectorize single gather/buildvector with alternate
opcode graph, it is not profitable. In other cases, need to use last
instruction for inserting the vectorized code.
Ignore incoming values with constant false masks when trying to simplify
VPBlendRecipes.
As a follow-on optimization, we should also be able to drop all incoming
values with false masks by creating a new VPBlendRecipe with those
operands dropped.
PR: https://github.com/llvm/llvm-project/pull/89384
Ensure the getShuffleCost arguments/instruction args are populated - minor extension to #88743 to help improve shuffle costs for certain corner cases (e.g. shuffles of loads)
Currently, when inferring noundef, we only check that the return value
is not undef/poison. However, we fail to account for the possibility
that a poison-generating return attribute will convert the value to
poison, and then violate the noundef attribute, resulting in immediate
UB.
For the relevant return attributes (align, nonnull and range), check
whether we can trivially re-prove the relevant property, otherwise do
not infer noundef.
This fixes the FunctionAttrs side of
https://github.com/llvm/llvm-project/issues/88026.
When speculating a store based on a preceding load/store, we need
to ensure that the speculated store does not have a higher
alignment (which might only be guaranteed by the branch condition).
There are various ways in which this could be strengthened (we
could get or enforce the alignment), but for now just do the
simple check against the preceding load/store.
Fixes https://github.com/llvm/llvm-project/issues/89672.
Introduced transformNodes() function to perform transformation of the
nodes (cost-based, instruction count based, etc.).
Implemented transformation of consecutive loads + reverse order to
strided loads with stride -1, if profitable.
Reviewers: RKSimon, preames, topperc
Reviewed By: RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/88530
The compiler should not take into account the type of the cmp
instruction, otherwise it may treat the size incorrectly and it may lead
to incorrect codegen.
Another step towards cleaning up shuffles that have been split, often across bitcasts between SSE intrinsic.
Strip shuffles entirely if we fold to an identity shuffle.
Verify-uselistorder wants to take some input IR and verify that the
uselist order is stable after roundtripping to bitcode and assembly.
This is disrupted if the file is converted between the new and old debug
info formats after parsing - while there's no functional difference, the
change to the in-memory representation of the IR modifies the uselist.
This patch changes verify-uselistorder to not convert input files
between debug info formats by default, preventing changes from being
made to the file being checked. In addition, this patch makes it so that
when we _do_ print IR in the new debug info format to bitcode or
assembly, we delete any lingering debug intrinsic declarations, ensuring
that we don't write uselist entries for them.
