shylie/llvm-project
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity
llvm-project/llvm/test/Transforms/LoopVectorize/X86
History
Benjamin Maxwell f22a178b13
Reland "[LV] Support conditional scalar assignments of masked operations" (#180708) 
This patch extends the support added in #158088 to loops where the
assignment is non-speculatable (e.g. a conditional load or divide).

For example, the following loop can now be vectorized:

```
int simple_csa_int_load(
  int* a, int* b, int default_val, int N, int threshold)
{
  int result = default_val;
  for (int i = 0; i < N; ++i)
    if (a[i] > threshold)
      result = b[i];
  return result;
}
```

It does this by extending the recurrence matching from only looking for
selects, to include phis where all operands are the header phi, except
for one which can be an arbitrary value outside the recurrence.

---

Reverts llvm/llvm-project#180275 (original PR: #178862)

Additional type legalization for `ISD::VECTOR_FIND_LAST_ACTIVE` was
added in #180290, which should resolve the backend crashes on x86.
2026-02-10 09:57:48 +00:00
..
CostModel
[LV] Check Addr in getAddressAccessSCEV in terms of SCEV expressions. (#171204)
2025-12-19 22:05:27 +00:00
already-vectorized.ll
Revert "[PGO] Add llvm.loop.estimated_trip_count metadata" (#151585)
2025-07-31 15:56:31 -04:00
amdlibm-calls-finite.ll
…
amdlibm-calls.ll
…
avx1.ll
[LV][NFC] Remove undef from function return values (#163578)
2025-10-16 09:49:38 +01:00
avx512.ll
[LV] Remove use of llc from vectoriser tests (#154759)
2025-08-26 09:59:06 +01:00
cast-costs.ll
[VPlan] Handle non-free bitcasts in getCostForRecipeWithOpcode.
2026-01-02 18:04:13 +00:00
cleanup-runtime-checks.ll
[LV] Don't preserve LCSSA in SCEVExpander for runtime checks. (#159556)
2025-09-30 10:03:55 +00:00
conditional-scalar-assignment.ll
[LV] Separate runtime check cost from total overhead in profitability check (#176754)
2026-01-23 07:29:56 +00:00
consecutive-ptr-cg-bug.ll
[LV][NFC] Remove undef values in some test cases (#164401)
2025-11-05 09:18:02 +00:00
consecutive-ptr-uniforms.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
constant-fold.ll
[VPlan] Remove original loop blocks if dead. (#155497)
2025-10-01 16:53:59 +00:00
constant-vector-operand.ll
…
conversion-cost.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
cost-conditional-branches.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
cost-constant-known-via-scev.ll
Reland [VPlan] Simplify pow-of-2 (mul|udiv) -> (shl|lshr) (#174581)
2026-01-06 20:36:26 +00:00
cost-model-assert.ll
[LV][NFC] Remove undef values in some test cases (#164401)
2025-11-05 09:18:02 +00:00
cost-model-i386.ll
…
cost-model.ll
[VPlan] Fall back to legacy cost if operands may be force-scalarized.
2026-01-19 21:25:54 +00:00
divs-with-tail-folding.ll
[LV] Conservatively predicate SDiv/SRem (#170818)
2026-01-07 04:25:38 +00:00
drop-inbounds-flags-for-reverse-vector-pointer.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
drop-poison-generating-flags.ll
[IR] Replace alignment argument with attribute on masked intrinsics (#163802)
2025-10-20 08:50:09 +00:00
end-pointer-signed.ll
[VPlan] Use getSigned() for index in VectorEndPointer recipe (#174426)
2026-01-06 09:10:59 +01:00
ephemeral-recipes.ll
…
epilog-vectorization-inductions.ll
[LV] Teach m_One, m_ZeroInt patterns to look through broadcasts (#170159)
2026-01-07 10:35:08 +00:00
fixed-order-recurrence.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
float-induction-x86.ll
[VPlan] Directly unroll VectorPointerRecipe (#168886)
2025-12-15 10:54:06 +00:00
fminimumnum.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
fneg-cost.ll
…
fold-tail-low-trip-count.ll
[LV] Add tests for loops with low trip counts requiring tail-folding.
2025-11-20 18:42:12 +00:00
fp32_to_uint32-cost-model.ll
[LV] Remove use of llc from vectoriser tests (#154759)
2025-08-26 09:59:06 +01:00
fp64_to_uint32-cost-model.ll
[LV] Remove use of llc from vectoriser tests (#154759)
2025-08-26 09:59:06 +01:00
fp80-widest-type.ll
[LV][NFC] Remove undef from phi incoming values (#163762)
2025-10-21 10:49:27 +01:00
fp_to_sint8-cost-model.ll
…
funclet.ll
…
gather_scatter.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
gather-cost.ll
[LV] Check Addr in getAddressAccessSCEV in terms of SCEV expressions. (#171204)
2025-12-19 22:05:27 +00:00
gather-vs-interleave.ll
…
gcc-examples.ll
…
gep-use-outside-loop.ll
[IR] Replace alignment argument with attribute on masked intrinsics (#163802)
2025-10-20 08:50:09 +00:00
i1-reg-usage.ll
…
illegal-parallel-loop-uniform-write.ll
[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786)
2025-10-23 20:31:31 +08:00
imprecise-through-phis.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
induction-costs.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
induction-step.ll
[LV] Teach m_One, m_ZeroInt patterns to look through broadcasts (#170159)
2026-01-07 10:35:08 +00:00
int128_no_gather.ll
[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786)
2025-10-23 20:31:31 +08:00
interleave_short_tc.ll
…
interleave-cost.ll
[LV] Check Addr in getAddressAccessSCEV in terms of SCEV expressions. (#171204)
2025-12-19 22:05:27 +00:00
interleave-opaque-pointers.ll
[VPlan] Remove PtrIV::IsScalarAfterVectorization, use VPlan analysis. (#168289)
2025-11-20 18:58:25 +00:00
interleave-ptradd-with-replicated-operand.ll
[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670)
2025-10-19 18:49:05 +00:00
interleaved-accesses-hoist-load-across-store.ll
[LV] Check Addr in getAddressAccessSCEV in terms of SCEV expressions. (#171204)
2025-12-19 22:05:27 +00:00
interleaved-accesses-large-gap.ll
[LV][NFC] Remove undef values in some test cases (#164401)
2025-11-05 09:18:02 +00:00
interleaved-accesses-sink-store-across-load.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
interleaved-accesses-use-after-free.ll
…
interleaved-accesses-waw-dependency.ll
…
interleaving.ll
[VPlan] Remove original loop blocks if dead. (#155497)
2025-10-01 16:53:59 +00:00
intrinsiccost.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
invariant-load-gather.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
invariant-store-vectorization.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
iv-live-outs.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
libm-vector-calls-finite.ll
…
libm-vector-calls-VF2-VF8.ll
…
libm-vector-calls.ll
…
limit-vf-by-tripcount.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
lit.local.cfg
…
load-deref-pred.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
masked_load_store.ll
[VPlan] Extract reverse operation for reverse accesses (#146525)
2025-12-18 14:57:48 +00:00
masked-store-cost.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
max-mstore.ll
…
metadata-enable.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
min-trip-count-switch.ll
…
mul_slm_16bit.ll
…
multi-exit-cost.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
narrow-to-single-scalar.ll
[VPlan] Fix assert in store-user in narrowToSingleScalars (#167686)
2025-11-12 17:53:24 +00:00
no_fpmath_with_hotness.ll
…
no_fpmath.ll
…
no-vector.ll
…
nontemporal.ll
…
optsize.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
outer_loop_test1_no_explicit_vect_width.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
parallel-loops-after-reg2mem.ll
…
parallel-loops.ll
[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670)
2025-10-19 18:49:05 +00:00
pointer-runtime-checks-unprofitable.ll
…
powof2div.ll
…
pr23997.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
pr34438.ll
[VPlan] Remove original loop blocks if dead. (#155497)
2025-10-01 16:53:59 +00:00
pr35432.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
pr36524.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
pr39160.ll
[LV][NFC] Remove remaining uses of undef in tests (#169357)
2025-11-26 11:24:10 +00:00
pr42674.ll
…
pr47437.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
pr48340.ll
[IR] Replace alignment argument with attribute on masked intrinsics (#163802)
2025-10-20 08:50:09 +00:00
pr51366-sunk-instruction-used-outside-of-loop.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
pr54413-select-interleave-count-loop-with-cost-zero.ll
…
pr54634.ll
[IR] Replace alignment argument with attribute on masked intrinsics (#163802)
2025-10-20 08:50:09 +00:00
pr55096-scalarize-add.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
pr56319-vector-exit-cond-optimization-epilogue-vectorization.ll
[VPlan] Remove VPVectorPointer for part 0 after unrolling. (#149735)
2025-07-27 13:53:26 +01:00
pr72969.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
pr81872.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
pr109581-unused-blend.ll
[VPlan] Simplify Plan's entry in removeBranchOnConst. (#154510)
2025-09-18 19:25:05 +01:00
pr131359-dead-for-splice.ll
[VPlan] Simplify Plan's entry in removeBranchOnConst. (#154510)
2025-09-18 19:25:05 +01:00
pr141968-instsimplifyfolder.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
predicate-switch.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
predicated-instruction-cost.ll
[VPlan] Fall back to legacy cost model if PtrSCEV is nullptr.
2026-02-08 11:55:12 +00:00
predicated-udiv.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
propagate-metadata.ll
…
ptr-indvar-crash.ll
…
rauw-bug.ll
[LV][NFC] Remove undef from phi incoming values (#163762)
2025-10-21 10:49:27 +01:00
reduction-crash.ll
Reapply "[VPlan] Detect and create partial reductions in VPlan. (NFCI) (#167851)"
2026-02-01 16:27:27 +00:00
reduction-fastmath.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
reduction-small-size.ll
[VPlan] Include flags in VectorPointerRecipe::printRecipe (#169466)
2025-11-25 10:26:51 +00:00
redundant-vf2-cost.ll
…
reg-usage-debug.ll
…
reg-usage.ll
…
register-assumption.ll
…
replicate-recipe-with-only-first-lane-used.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
replicate-uniform-call.ll
Reapply "[VPlan] Remove legacy costing inside VPBlendRecipe::computeCost (#171846)" (#172261)
2025-12-16 06:30:54 +00:00
replicating-load-store-costs-max-bandwidth.ll
[VPlan] Match legacy behavior w.r.t. using pointer phis as scalar addrs.
2025-10-20 11:09:25 +01:00
replicating-load-store-costs.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
runtime-limit.ll
[LV][NFC] Remove undef from function return values (#163578)
2025-10-16 09:49:38 +01:00
scatter_crash.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
scev-checks-unprofitable.ll
[VPlan] Don't preserve LCSSA in expandSCEVs. (#165505)
2025-10-29 18:25:46 +00:00
slm-no-vectorize.ll
…
small-size.ll
Reapply "[LV] Use ExtractLane(LastActiveLane, V) live outs when tail-folding. (#149042)"
2025-11-26 20:03:55 +00:00
strided_load_cost.ll
[VPlan] Directly unroll VectorPointerRecipe (#168886)
2025-12-15 10:54:06 +00:00
struct-store.ll
…
svml-calls-finite.ll
…
svml-calls.ll
…
tail_folding_and_assume_safety.ll
[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786)
2025-10-23 20:31:31 +08:00
tail_loop_folding.ll
[VPlan] Sink recipes from the vector loop region in licm. (#168031)
2026-02-03 07:57:15 +00:00
transform-narrow-interleave-to-widen-memory-gaps.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
transform-narrow-interleave-to-widen-memory.ll
Revert "[VPlan] Run narrowInterleaveGroups during general VPlan optimizations. (#149706)"
2025-10-22 21:27:11 +01:00
tripcount.ll
…
uint64_to_fp64-cost-model.ll
…
uniform_load.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
uniform_mem_op.ll
Reland [VPlan] Simplify pow-of-2 (mul|udiv) -> (shl|lshr) (#174581)
2026-01-06 20:36:26 +00:00
uniform-phi.ll
…
uniformshift.ll
[SCEV] Handle non-constant start values in AddRec UDiv canonicalization. (#170474)
2025-12-03 21:13:11 +00:00
unroll_selection.ll
…
unroll-pm.ll
[LV][NFC] Remove undef from function return values (#163578)
2025-10-16 09:49:38 +01:00
unroll-small-loops.ll
…
veclib-calls.ll
…
vect.omp.force.ll
…
vect.omp.force.small-tc.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
vector_max_bandwidth.ll
…
vector_ptr_load_store.ll
…
vector-scalar-select-cost.ll
[VPlan] Remove VPWidenSelectRecipe, use VPWidenRecipe instead (NFCI). (#174234)
2026-01-05 22:33:37 +00:00
vectorization-remarks-loopid-dbg.ll
[LV] Remove use of llc from vectoriser tests (#154759)
2025-08-26 09:59:06 +01:00
vectorization-remarks-missed.ll
Reland "[LV] Support conditional scalar assignments of masked operations" (#180708)
2026-02-10 09:57:48 +00:00
vectorization-remarks-profitable.ll
[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786)
2025-10-23 20:31:31 +08:00
vectorization-remarks.ll
[LV] Remove use of llc from vectoriser tests (#154759)
2025-08-26 09:59:06 +01:00
vectorize-force-tail-with-evl.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
vectorize-interleaved-accesses-gap.ll
[IR] Replace alignment argument with attribute on masked intrinsics (#163802)
2025-10-20 08:50:09 +00:00
vectorize-only-for-real.ll
…
vplan-native-inner-loop-only.ll
[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670)
2025-10-19 18:49:05 +00:00
vplan-single-bit-ind-var-width-4.ll
[VPlan] Allow truncation for lanes in VPScalarIVStepsRecipe (#175268)
2026-01-10 10:34:46 -05:00
vplan-single-bit-ind-var.ll
[VPlan] Use unsigned integers for lane start indices (#175231)
2026-01-09 14:28:28 -08:00
vplan-vp-intrinsics.ll
[VPlan] Include flags in VectorPointerRecipe::printRecipe (#169466)
2025-11-25 10:26:51 +00:00
widened-value-used-as-scalar-and-first-lane.ll
[VPlan] Use DL index type consistently for GEPs (#169396)
2025-11-26 12:25:55 +00:00
x86_fp80-interleaved-access.ll
[LV][NFC] Remove undef from phi incoming values (#163762)
2025-10-21 10:49:27 +01:00
x86_fp80-vector-store.ll
[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670)
2025-10-19 18:49:05 +00:00
x86-interleaved-accesses-masked-group.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
x86-interleaved-store-accesses-with-gaps.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00
x86-pr39099.ll
…
x86-predication.ll
Reland [VPlan] Expand WidenInt inductions with nuw/nsw (#168354)
2025-11-17 13:44:25 +00:00