Mel Chen ba40a7bc2e

[LoopVectorize] Vectorize fixed-order recurrence with vscale x 1. (#142772) ...

When the fixed-order recurrence phi is live-out from the loop, the
vectorizer uses VPInstruction::ExtractPenultimateElement to extract the
penultimate element from the recurrence vector. However, this is not
feasible when the VF is vscale x 1, since vscale could be 1, making the
vector contain only one element.

This patch changes the behavior for vscale x 1 by extracting the last
element from the vector produced by splicing the recurrence phi and the
previous value. This ensures we can still determine the correct live-out
value of the recurrence phi.

2025-06-18 16:03:20 +08:00

..

bf16.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

blend-any-of-reduction-cost.ll

[VPlan] Also introduce broadcasts for live-ins used in vec preheader.

2025-03-11 21:19:14 +00:00

blocks-with-dead-instructions.ll

[VPlan] Mark VPFirstOrderRecurrencePHI as not reading/writing memory.

2025-06-15 22:00:47 +01:00

dead-ops-cost.ll

[VPlan] Add VPInstruction::StepVector and use it in VPWidenIntOrFpInductionRecipe (#129508)

2025-05-08 18:47:44 +08:00

defaults.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

divrem.ll

[VPlan] Generalize SCALAR-STEPS removal to any unroll factor.

2025-03-26 21:03:50 +00:00

evl-compatible-loops.ll

…

evl-iv-simplify.ll

[LV][EVL] Introduce the EVLIndVarSimplify Pass for EVL-vectorized loops (#131005)

2025-05-14 13:49:50 -07:00

f16.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

first-order-recurrence-scalable-vf1.ll

[LoopVectorize] Vectorize fixed-order recurrence with vscale x 1. (#142772)

2025-06-18 16:03:20 +08:00

fminimumnum.ll

[RISCV] Support vectorizing FMINIMUMNUM and FMAXIMUMNUM (#135727)

2025-04-27 19:10:02 +08:00

force-vect-msg.ll

[LoopVectorize][NFC] Disable output for tests that don't need it (#124747)

2025-01-29 08:09:50 +00:00

illegal-type.ll

[VPlan] Don't access canonical IV in VPWidenPointerInduction::execute.

2025-03-15 21:32:48 +00:00

induction-costs.ll

[VPlan] Add VPInstruction::StepVector and use it in VPWidenIntOrFpInductionRecipe (#129508)

2025-05-08 18:47:44 +08:00

inloop-reduction.ll

[LoopVectorizer] Prune VFs based on plan register pressure (#132190)

2025-05-19 13:27:17 +01:00

interleaved-accesses-zve32x.ll

…

interleaved-accesses.ll

[LV] Support scalable interleave groups for factors 3,5,6 and 7 (#141865)

2025-06-12 11:09:09 +01:00

interleaved-cost.ll

…

interleaved-masked-access.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

lit.local.cfg

…

lmul.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

low-trip-count.ll

[VPlan] Use InstSimplifyFolder instead of TargetFolder (#141222)

2025-05-28 11:00:14 +02:00

mask-index-type.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

masked_gather_scatter.ll

[VPlan] Add incoming values for all predecessor to ResumePHI (NFCI).

2025-02-09 11:20:20 +00:00

only-compute-cost-for-vplan-vfs.ll

…

ordered-reduction.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

partial-reduce-dot-product.ll

[RISCV][TTI] Allow partial reduce with mismatched extends (#143608)

2025-06-10 15:26:54 -07:00

pr87378-vpinstruction-or-drop-poison-generating-flags.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

pr88802.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

preserve-dbg-loc.ll

…

reg-usage-bf16.ll

[LV] Fix MVE regression from #132190 (#141736)

2025-06-16 09:58:03 +01:00

reg-usage-f16.ll

[LV] Fix MVE regression from #132190 (#141736)

2025-06-16 09:58:03 +01:00

reg-usage.ll

[LV] Fix MVE regression from #132190 (#141736)

2025-06-16 09:58:03 +01:00

remark-reductions.ll

[LV] Prevent query the computeCost() when VF=1 in emitInvalidCostRemarks(). (#117288)

2025-02-10 08:40:28 +08:00

riscv-interleaved.ll

[LoopVectorize][NFC] Disable output for tests that don't need it (#124747)

2025-01-29 08:09:50 +00:00

riscv-unroll.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

riscv-vector-reverse-output.ll

[LV] Add test cases for reverse accesses involving irregular types. nfc (#135139)

2025-04-14 14:17:39 +08:00

riscv-vector-reverse.ll

[RISCV][TTI] Refine reverse shuffle costing for high LMUL (#144155)

2025-06-16 10:20:09 -07:00

safe-dep-distance.ll

[LAA] Make sure MaxVF for Store-Load forward safe dep distances is pow2.

2025-04-12 20:05:37 +01:00

scalable-basics.ll

[VPlan] Remove no-op SCALAR-STEPS after unrolling. (#123655)

2025-03-25 12:57:24 +00:00

scalable-reductions.ll

[RISCV] Enable scalable loop vectorization for fmax/fmin reductions with f16/bf16 type for zvfhmin/zvfbfmin (#129629)

2025-03-04 16:49:24 +08:00

scalable-tailfold.ll

[VPlan] Add initial CFG simplification, removing BranchOnCond true. (#106748)

2025-04-04 15:44:26 +01:00

scalable-vf-hint.ll

…

select-cmp-reduction.ll

Reapply "[VPlan] Fold NOT into predicate of wide compares." (#130347)

2025-04-28 20:01:37 +01:00

select-invariant-cond-cost.ll

…

short-trip-count.ll

[VPlan] Use InstSimplifyFolder instead of TargetFolder (#141222)

2025-05-28 11:00:14 +02:00

strided-accesses.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

tail-folding-cost.ll

[LoopVectorize] Add cost of generating tail-folding mask to the loop (#130565)

2025-03-21 09:24:56 +00:00

truncate-to-minimal-bitwidth-cost.ll

[VPlan] Introduce constant folder in simplifyRecipe (#125365)

2025-05-20 14:16:01 +01:00

truncate-to-minimal-bitwidth-evl-crash.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

type-info-cache-evl-crash.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

uniform-load-store.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

unroll-in-loop-vectorizer.ll

…

veclib-function-calls.ll

…

vectorize-force-tail-with-evl-bin-unary-ops-args.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-call-intrinsics.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-cast-intrinsics.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-cond-reduction.ll

[VPlan] Add ReductionStartVector VPInstruction. (#142290)

2025-06-09 20:59:12 +01:00

vectorize-force-tail-with-evl-div.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-fixed-order-recurrence.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-gather-scatter.ll

…

vectorize-force-tail-with-evl-inloop-reduction.ll

[NFC][LLVM][Instrumentation][LoopVectorize] Regenerate test checks.

2025-06-05 11:38:30 +00:00

vectorize-force-tail-with-evl-interleave.ll

[VPlan] Expand VPWidenIntOrFpInductionRecipe into separate recipes (#118638)

2025-06-17 18:24:07 +01:00

vectorize-force-tail-with-evl-intermediate-store.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-iv32.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-known-no-overflow.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-masked-loadstore.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-no-masking.ll

…

vectorize-force-tail-with-evl-ordered-reduction.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-force-tail-with-evl-reduction-cost.ll

[LoopVectorize][NFC] Disable output for tests that don't need it (#124747)

2025-01-29 08:09:50 +00:00

vectorize-force-tail-with-evl-reduction.ll

[VPlan] Add ReductionStartVector VPInstruction. (#142290)

2025-06-09 20:59:12 +01:00

vectorize-force-tail-with-evl-reverse-load-store.ll

[NFC][LLVM][Instrumentation][LoopVectorize] Regenerate test checks.

2025-06-05 11:38:30 +00:00

vectorize-force-tail-with-evl-safe-dep-distance.ll

[NFC][LLVM][Instrumentation][LoopVectorize] Regenerate test checks.

2025-06-05 11:38:30 +00:00

vectorize-force-tail-with-evl-uniform-store.ll

[LV][EVL] Attach a new metadata on EVL vectorized loops (#131000)

2025-05-06 10:06:37 -07:00

vectorize-vp-intrinsics.ll

[NFC][LLVM][Instrumentation][LoopVectorize] Regenerate test checks.

2025-06-05 11:38:30 +00:00

vf-will-not-generate-any-vector-insts.ll

[VPlan] Add incoming values for all predecessor to ResumePHI (NFCI).

2025-02-09 11:20:20 +00:00

vplan-vp-call-intrinsics.ll

[VPlan] Introduce VPInstructionWithType, use instead of VPScalarCast(NFC) (#129706)

2025-04-10 22:30:40 +01:00

vplan-vp-cast-intrinsics.ll

[VPlan] Introduce VPInstructionWithType, use instead of VPScalarCast(NFC) (#129706)

2025-04-10 22:30:40 +01:00

vplan-vp-intrinsics-fixed-order-recurrence.ll

[VPlan] Use EMIT-SCALAR for single-scalar VPPhis (NFC).

2025-05-29 11:20:07 +01:00

vplan-vp-intrinsics-reduction.ll

[VPlan] Add ReductionStartVector VPInstruction. (#142290)

2025-06-09 20:59:12 +01:00

vplan-vp-intrinsics.ll

[VPlan] Introduce VPInstructionWithType, use instead of VPScalarCast(NFC) (#129706)

2025-04-10 22:30:40 +01:00

vplan-vp-select-intrinsics.ll

[VPlan] Use EMIT-SCALAR for single-scalar VPPhis (NFC).

2025-05-29 11:20:07 +01:00

zvl32b.ll

…