llvm-project

History

[VPlan] Don't apply predication discount to non-originally-predicated blocks (#160449 )

Split off from #158690. Currently if an instruction needs predicated due
to tail folding, it will also have a predicated discount applied to it
in multiple places.
This is likely inaccurate because we can expect a tail folded
instruction to be executed on every iteration bar the last.

This fixes it by checking if the instruction/block was originally
predicated, and in doing so prevents vectorization with tail folding
where we would have had to scalarize the memory op anyway.

On llvm-test-suite this causes 4 loops in total to no longer be
vectorized with -O3 on arm64-apple-darwin, and there's no observable
performance impact.

2025-11-10 12:10:40 +00:00

CostModel

[VPlan] Don't apply predication discount to non-originally-predicated blocks (#160449 )

2025-11-10 12:10:40 +00:00

already-vectorized.ll

Revert "[PGO] Add llvm.loop.estimated_trip_count metadata" (#151585 )

2025-07-31 15:56:31 -04:00

amdlibm-calls-finite.ll

…

amdlibm-calls.ll

…

avx1.ll

[LV][NFC] Remove undef from function return values (#163578 )

2025-10-16 09:49:38 +01:00

avx512.ll

[LV] Remove use of llc from vectoriser tests (#154759 )

2025-08-26 09:59:06 +01:00

cleanup-runtime-checks.ll

[LV] Don't preserve LCSSA in SCEVExpander for runtime checks. (#159556 )

2025-09-30 10:03:55 +00:00

consecutive-ptr-cg-bug.ll

[LV][NFC] Remove undef values in some test cases (#164401 )

2025-11-05 09:18:02 +00:00

consecutive-ptr-uniforms.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

constant-fold.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

constant-vector-operand.ll

…

conversion-cost.ll

[LV][NFC] Remove undef from function return values (#163578 )

2025-10-16 09:49:38 +01:00

cost-conditional-branches.ll

[VPlan] Handle single-scalar conds in VPWidenSelectRecipe. (#165506 )

2025-11-05 22:11:29 +00:00

cost-constant-known-via-scev.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

cost-model-assert.ll

[LV][NFC] Remove undef values in some test cases (#164401 )

2025-11-05 09:18:02 +00:00

cost-model-i386.ll

…

cost-model.ll

[VPlan] Extend licm to hoist assumes (#162636 )

2025-10-16 13:59:32 +00:00

divs-with-tail-folding.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

drop-inbounds-flags-for-reverse-vector-pointer.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

drop-poison-generating-flags.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

ephemeral-recipes.ll

…

epilog-vectorization-inductions.ll

[VPlan] Mark VPInstruction::Broadcast as not reading/writing memory.

2025-09-27 20:48:42 +01:00

fixed-order-recurrence.ll

[VPlan] Don't apply predication discount to non-originally-predicated blocks (#160449 )

2025-11-10 12:10:40 +00:00

float-induction-x86.ll

[LoopUnroll] Skip remainder loop guard if skip unrolled loop (#156549 )

2025-10-07 10:45:49 -04:00

fminimumnum.ll

[VPlan] Simplify Plan's entry in removeBranchOnConst. (#154510 )

2025-09-18 19:25:05 +01:00

fneg-cost.ll

…

fp32_to_uint32-cost-model.ll

[LV] Remove use of llc from vectoriser tests (#154759 )

2025-08-26 09:59:06 +01:00

fp64_to_uint32-cost-model.ll

[LV] Remove use of llc from vectoriser tests (#154759 )

2025-08-26 09:59:06 +01:00

fp80-widest-type.ll

[LV][NFC] Remove undef from phi incoming values (#163762 )

2025-10-21 10:49:27 +01:00

fp_to_sint8-cost-model.ll

…

funclet.ll

…

gather_scatter.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

gather-cost.ll

…

gather-vs-interleave.ll

…

gcc-examples.ll

…

gep-use-outside-loop.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

i1-reg-usage.ll

…

illegal-parallel-loop-uniform-write.ll

[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786 )

2025-10-23 20:31:31 +08:00

imprecise-through-phis.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

induction-costs.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

induction-step.ll

[VPlan] Mark VPInstruction::Broadcast as not reading/writing memory.

2025-09-27 20:48:42 +01:00

int128_no_gather.ll

[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786 )

2025-10-23 20:31:31 +08:00

interleave_short_tc.ll

…

interleave-cost.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

interleave-opaque-pointers.ll

…

interleave-ptradd-with-replicated-operand.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

interleaved-accesses-hoist-load-across-store.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

interleaved-accesses-large-gap.ll

[LV][NFC] Remove undef values in some test cases (#164401 )

2025-11-05 09:18:02 +00:00

interleaved-accesses-sink-store-across-load.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

interleaved-accesses-use-after-free.ll

…

interleaved-accesses-waw-dependency.ll

…

interleaving.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

intrinsiccost.ll

[VPlan] Create epilogue minimum iteration check in VPlan. (#157545 )

2025-09-25 07:13:38 +00:00

invariant-load-gather.ll

[VPlan] Handle single-scalar conds in VPWidenSelectRecipe. (#165506 )

2025-11-05 22:11:29 +00:00

invariant-store-vectorization.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

iv-live-outs.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

libm-vector-calls-finite.ll

…

libm-vector-calls-VF2-VF8.ll

…

libm-vector-calls.ll

…

limit-vf-by-tripcount.ll

[VPlan] Simplify branch-cond with getVectorTripCount (#155604 )

2025-11-10 10:43:37 +00:00

lit.local.cfg

…

load-deref-pred.ll

[VPlan] Simplify branch-cond with getVectorTripCount (#155604 )

2025-11-10 10:43:37 +00:00

masked_load_store.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

masked-store-cost.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

max-mstore.ll

…

metadata-enable.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

min-trip-count-switch.ll

…

mul_slm_16bit.ll

…

multi-exit-cost.ll

…

no_fpmath_with_hotness.ll

…

no_fpmath.ll

…

no-vector.ll

…

nontemporal.ll

…

optsize.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

outer_loop_test1_no_explicit_vect_width.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

parallel-loops-after-reg2mem.ll

…

parallel-loops.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

pointer-runtime-checks-unprofitable.ll

…

powof2div.ll

…

pr23997.ll

[VPlan] Create epilogue minimum iteration check in VPlan. (#157545 )

2025-09-25 07:13:38 +00:00

pr34438.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

pr35432.ll

Reapply "[SCEVExp] Add helper to clean up dead instructions after expansion. (#157308 )"

2025-09-09 09:47:41 +01:00

pr36524.ll

[VPlan] Simplify Plan's entry in removeBranchOnConst. (#154510 )

2025-09-18 19:25:05 +01:00

pr39160.ll

…

pr42674.ll

…

pr47437.ll

[VPlan] Create epilogue minimum iteration check in VPlan. (#157545 )

2025-09-25 07:13:38 +00:00

pr48340.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

pr51366-sunk-instruction-used-outside-of-loop.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

pr54413-select-interleave-count-loop-with-cost-zero.ll

…

pr54634.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

pr55096-scalarize-add.ll

…

pr56319-vector-exit-cond-optimization-epilogue-vectorization.ll

…

pr72969.ll

…

pr81872.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

pr109581-unused-blend.ll

[VPlan] Simplify Plan's entry in removeBranchOnConst. (#154510 )

2025-09-18 19:25:05 +01:00

pr131359-dead-for-splice.ll

[VPlan] Simplify Plan's entry in removeBranchOnConst. (#154510 )

2025-09-18 19:25:05 +01:00

pr141968-instsimplifyfolder.ll

[VPlan] Handle single-scalar conds in VPWidenSelectRecipe. (#165506 )

2025-11-05 22:11:29 +00:00

predicate-switch.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

predicated-instruction-cost.ll

…

propagate-metadata.ll

…

ptr-indvar-crash.ll

…

rauw-bug.ll

[LV][NFC] Remove undef from phi incoming values (#163762 )

2025-10-21 10:49:27 +01:00

reduction-crash.ll

…

reduction-fastmath.ll

[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786 )

2025-10-23 20:31:31 +08:00

reduction-small-size.ll

…

redundant-vf2-cost.ll

…

reg-usage-debug.ll

…

reg-usage.ll

…

replicate-recipe-with-only-first-lane-used.ll

[VPlan] Handle single-scalar conds in VPWidenSelectRecipe. (#165506 )

2025-11-05 22:11:29 +00:00

replicate-uniform-call.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

replicating-load-store-costs-max-bandwidth.ll

[VPlan] Match legacy behavior w.r.t. using pointer phis as scalar addrs.

2025-10-20 11:09:25 +01:00

replicating-load-store-costs.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

runtime-limit.ll

[LV][NFC] Remove undef from function return values (#163578 )

2025-10-16 09:49:38 +01:00

scatter_crash.ll

[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786 )

2025-10-23 20:31:31 +08:00

scev-checks-unprofitable.ll

[VPlan] Don't preserve LCSSA in expandSCEVs. (#165505 )

2025-10-29 18:25:46 +00:00

slm-no-vectorize.ll

…

small-size.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

strided_load_cost.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

struct-store.ll

…

svml-calls-finite.ll

…

svml-calls.ll

…

tail_folding_and_assume_safety.ll

[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786 )

2025-10-23 20:31:31 +08:00

tail_loop_folding.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

transform-narrow-interleave-to-widen-memory-gaps.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

transform-narrow-interleave-to-widen-memory.ll

Revert "[VPlan] Run narrowInterleaveGroups during general VPlan optimizations. (#149706 )"

2025-10-22 21:27:11 +01:00

tripcount.ll

…

uint64_to_fp64-cost-model.ll

…

uniform_load.ll

[VPlan] Remove original recipe after narrowing to single-scalar.

2025-10-31 04:38:16 +00:00

uniform_mem_op.ll

[VPlan] Remove original loop blocks if dead. (#155497 )

2025-10-01 16:53:59 +00:00

uniform-phi.ll

…

uniformshift.ll

…

unroll_selection.ll

…

unroll-pm.ll

[LV][NFC] Remove undef from function return values (#163578 )

2025-10-16 09:49:38 +01:00

unroll-small-loops.ll

…

veclib-calls.ll

…

vect.omp.force.ll

…

vect.omp.force.small-tc.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

vector_max_bandwidth.ll

…

vector_ptr_load_store.ll

…

vector-scalar-select-cost.ll

[VPlan] Handle single-scalar conds in VPWidenSelectRecipe. (#165506 )

2025-11-05 22:11:29 +00:00

vectorization-remarks-loopid-dbg.ll

[LV] Remove use of llc from vectoriser tests (#154759 )

2025-08-26 09:59:06 +01:00

vectorization-remarks-missed.ll

[LV] Emit all remarks for unvectorizable instructions (#153833 )

2025-08-18 18:04:53 +01:00

vectorization-remarks-profitable.ll

[test][Transforms] Remove unsafe-fp-math uses part 2 (NFC) (#164786 )

2025-10-23 20:31:31 +08:00

vectorization-remarks.ll

[LV] Remove use of llc from vectoriser tests (#154759 )

2025-08-26 09:59:06 +01:00

vectorize-force-tail-with-evl.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

vectorize-interleaved-accesses-gap.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

vectorize-only-for-real.ll

…

vplan-native-inner-loop-only.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

vplan-vp-intrinsics.ll

…

widened-value-used-as-scalar-and-first-lane.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

x86_fp80-interleaved-access.ll

[LV][NFC] Remove undef from phi incoming values (#163762 )

2025-10-21 10:49:27 +01:00

x86_fp80-vector-store.ll

[VPlan] Add VPInstruction to unpack vector values to scalars. (#155670 )

2025-10-19 18:49:05 +00:00

x86-interleaved-accesses-masked-group.ll

[LAA] Only use inbounds/nusw in isNoWrap if the GEP is dereferenced. (#161445 )

2025-11-04 17:08:12 +00:00

x86-interleaved-store-accesses-with-gaps.ll

[IR] Replace alignment argument with attribute on masked intrinsics (#163802 )

2025-10-20 08:50:09 +00:00

x86-pr39099.ll

…

x86-predication.ll

[VPlan] Handle single-scalar conds in VPWidenSelectRecipe. (#165506 )

2025-11-05 22:11:29 +00:00