David Sherwood 72f339de45

[LoopVectorize] Use predicated version of getSmallConstantMaxTripCount (#109928) ...

There are a number of places where we call getSmallConstantMaxTripCount
without passing a vector of predicates:

getSmallBestKnownTC
isIndvarOverflowCheckKnownFalse
computeMaxVF
isMoreProfitable

I've changed all of these to now pass in a predicate vector so that
we get the benefit of making better vectorisation choices when we
know the max trip count for loops that require SCEV predicate checks.

I've tried to add tests that cover all the cases affected by these
changes.

2024-10-11 10:10:15 +01:00

..

aarch64-predication.ll

[AArch64] Change the cost of vector insert/extract to 2

2023-07-28 21:26:50 +01:00

aarch64-unroll.ll

…

arbitrary-induction-step.ll

[VPlan] Implement unrolling as VPlan-to-VPlan transform. (#95842)

2024-09-21 19:47:37 +01:00

arm64-unroll.ll

…

backedge-overflow.ll

[test] Replace aarch64-*-{eabi,gnueabi}{,hf} with aarch64

2024-02-12 18:29:55 -08:00

blend-costs.ll

[VPlan] Implement VPBlendRecipe::computeCost.

2024-10-08 21:33:42 +01:00

call-costs.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

clamped-trip-count.ll

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

2024-09-10 10:41:35 +01:00

conditional-branches-cost.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

cost-no-valid-vplans-built.ll

[VPlan] Return std::nullopt early if plans are empty.

2024-06-27 12:25:59 +01:00

deterministic-type-shrinkage.ll

[VPlan] Implement unrolling as VPlan-to-VPlan transform. (#95842)

2024-09-21 19:47:37 +01:00

divs-with-scalable-vfs.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

eliminate-tail-predication.ll

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

2023-12-08 12:18:31 +00:00

epilog-vectorization-widen-inductions.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

extend-vectorization-factor-for-unprofitable-memops.ll

…

extractvalue-no-scalarization-required.ll

[VPlan] Generalize VPValue::isDefinedOutsideLoopRegions.

2024-09-20 15:34:00 +01:00

first-order-recurrence-fold-tail.ll

[VPlan] Introduce ResumePhi VPInstruction, use to create phi for FOR. (#94760)

2024-07-11 16:08:04 +01:00

first-order-recurrence.ll

Move several vector intrinsics out of experimental namespace (#88748)

2024-04-29 10:16:45 +01:00

fixed-order-recurrence.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

force-target-instruction-cost.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

gather-cost.ll

…

gather-do-not-vectorize-addressing.ll

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

2023-12-08 12:18:31 +00:00

i1-reg-usage.ll

…

induction-costs-sve.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

induction-costs.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

induction-trunc.ll

…

interleave_count_for_estimated_tc.ll

[LV] Remove loop trip count threshold for deciding whether to interleave a loop (#67725)

2024-02-05 17:23:58 -08:00

interleave_count_for_known_tc.ll

[LV] Remove loop trip count threshold for deciding whether to interleave a loop (#67725)

2024-02-05 17:23:58 -08:00

interleave-allocsize-not-equal-typesize.ll

[VPlan] Don't created GEP x, 0 for interleave group pointers.

2024-10-08 12:08:13 +01:00

interleaved_cost.ll

[LV] Assign cost to all interleave members when not interleaving.

2024-09-11 21:04:34 +01:00

interleaved-store-of-first-order-recurrence.ll

[VPlan] Don't created GEP x, 0 for interleave group pointers.

2024-10-08 12:08:13 +01:00

interleaved-vs-scalar.ll

[LV] Assign cost to all interleave members when not interleaving.

2024-09-11 21:04:34 +01:00

interleaving-load-store.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

interleaving-reduction.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

intrinsiccost.ll

TTI: Fix special casing vectorization costs of saturating add/sub (#97463)

2024-08-06 17:33:52 +04:00

invariant-replicate-region.ll

[LV] Move test requiring AArch64 to target subdir.

2024-09-21 12:54:59 +01:00

lit.local.cfg

…

loop-vectorization-factors.ll

[VPlan] Introduce ResumePhi VPInstruction, use to create phi for FOR. (#94760)

2024-07-11 16:08:04 +01:00

loopvectorize_pr33804_double.ll

[VPlan] Add a test for testing unused interleave recipes (#75026)

2023-12-14 21:16:11 +08:00

low_trip_count_predicates.ll

[LoopVectorize] Use predicated version of getSmallConstantMaxTripCount (#109928)

2024-10-11 10:10:15 +01:00

low_trip_memcheck_cost.ll

[LoopVectorize] Fix divide-by-zero bug (#80836) (#81721)

2024-02-14 16:07:51 +00:00

masked-call.ll

[LLVM][VPlan] Pick more optimal initial value for VPBlend. (#104019)

2024-08-30 13:30:23 +01:00

masked-op-cost.ll

[TTI] getCommonMaskedMemoryOpCost - use the target getMemoryOpCost/getCFInstrCost implementations.

2024-05-11 12:50:26 +01:00

max-vf-for-interleaved.ll

…

maximize-bandwidth-invalidate.ll

[test] Replace aarch64-*-eabi with aarch64

2023-06-29 09:06:00 +01:00

mul-simplification.ll

[VPlan] Simplify MUL operands at recipe construction.

2024-09-02 21:25:31 +01:00

neon-gather-cost.ll

…

no_vector_instructions.ll

…

nontemporal-load-store.ll

…

outer_loop_prefer_scalable.ll

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

2024-09-10 10:41:35 +01:00

outer_loop_test1_no_explicit_vect_width.ll

…

Oz-and-forced-vectorize.ll

…

pr31900.ll

…

pr33053.ll

…

pr36032.ll

…

pr46950-load-cast-context-crash.ll

…

pr56627.ll

…

pr60831-sve-inv-store-crash.ll

[VPlan] Add initial loop-invariant code motion transform. (#107894)

2024-09-20 11:22:03 +01:00

pr73894.ll

[VPlan] Remove VPPredInstPHIRecipes without users after region merging.

2024-07-20 13:21:32 +01:00

predication_costs.ll

[AArch64] Change the cost of vector insert/extract to 2

2023-07-28 21:26:50 +01:00

prefer-fixed-if-equal-to-scalable.ll

[LV][AArch64] Prefer Fixed over Scalable if cost-model is equal (Neoverse V2) (#95819)

2024-07-17 10:46:28 +01:00

reduction-recurrence-costs-sve.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

reduction-small-size.ll

…

reg-usage.ll

…

runtime-check-size-based-threshold.ll

…

runtime-check-trip-count-decisions.ll

[LoopVectorize] Enhance Vectorization decisions for predicate tail-folded loops with low trip counts (#69588)

2023-10-30 13:43:26 +00:00

scalable-alloca.ll

[VPlan] Port invalid cost remarks to VPlan. (#99322)

2024-07-27 12:52:12 +01:00

scalable-avoid-scalarization.ll

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

2024-09-10 10:41:35 +01:00

scalable-call.ll

[VPlan] Introduce VPWidenIntrinsicRecipe to separate from libcall. (#110486)

2024-10-08 22:37:20 +01:00

scalable-fp-ext-trunc-illegal-type.ll

[LV][AArch64] Don't query registers for illegal scalable vector elts (#109411)

2024-09-23 13:35:23 +01:00

scalable-predicate-instruction.ll

…

scalable-reduction-inloop-cond.ll

[LV] Prefer FLT_MIN/MAX for fmin/fmax reductions with ninf (#107141)

2024-09-03 12:21:54 -07:00

scalable-reductions-tf.ll

…

scalable-reductions.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

scalable-strict-fadd.ll

[VPlan] Don't created GEP x, 0 for interleave group pointers.

2024-10-08 12:08:13 +01:00

scalable-vectorization-cost-tuning.ll

…

scalable-vectorization.ll

…

scalable-vf-hint.ll

…

scalar_interleave.ll

…

scalarize-store-with-predication.ll

…

scatter-cost.ll

[AArch64] Increase scatter overhead on Neoverse-V2 (#101296)

2024-08-14 10:12:40 +05:30

sdiv-pow2.ll

…

select-costs.ll

[LoopVectorize] LLVM fails to vectorise loops with multi-bool varables (#89226)

2024-07-15 20:21:50 +01:00

smallest-and-widest-types.ll

…

sme-vectorize.ll

[AArch64] Consider streaming mode in TTI interfaces for vectorization. (#96305)

2024-06-24 11:06:16 +01:00

store-costs-sve.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

streaming-compatible-sve-no-maximize-bandwidth.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

streaming-vectorization.ll

[LLVM][TTI][SME] Allow optional auto-vectorisation for streaming functions. (#101679)

2024-08-05 11:25:44 +01:00

strict-fadd-cost.ll

[AArch64] Change the cost of vector insert/extract to 2

2023-07-28 21:26:50 +01:00

strict-fadd-vf1.ll

…

strict-fadd.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve2-histcnt-epilogue.ll

[LV] Vectorize histogram operations (#99851)

2024-09-27 13:08:55 +01:00

sve2-histcnt-no-scalar-interleave.ll

[LV] Vectorize histogram operations (#99851)

2024-09-27 13:08:55 +01:00

sve2-histcnt-outerloop-scevaddrec.ll

[LV] Vectorize histogram operations (#99851)

2024-09-27 13:08:55 +01:00

sve2-histcnt-too-many-deps.ll

[LV] Vectorize histogram operations (#99851)

2024-09-27 13:08:55 +01:00

sve2-histcnt-vplan.ll

[LV] Vectorize histogram operations (#99851)

2024-09-27 13:08:55 +01:00

sve2-histcnt.ll

[LV] Vectorize histogram operations (#99851)

2024-09-27 13:08:55 +01:00

sve-basic-vec.ll

…

sve-cond-inv-loads.ll

[VPlan] Add initial loop-invariant code motion transform. (#107894)

2024-09-20 11:22:03 +01:00

sve-epilog-vect-inloop-reductions.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-epilog-vect-reductions.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-epilog-vect-strict-reductions.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-epilog-vect-vscale-tune.ll

…

sve-epilog-vect.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-extract-last-veclane.ll

…

sve-fneg.ll

[VPlan] Model address separately. (#72164)

2024-01-01 19:51:15 +00:00

sve-gather-scatter-cost.ll

…

sve-gather-scatter.ll

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

2024-09-10 10:41:35 +01:00

sve-illegal-type.ll

[LoopVectorize][AArch64] Add limited support for scalable vectorisation of i1 types (#95920)

2024-06-25 15:04:24 +01:00

sve-inductions-unusual-types.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-inductions.ll

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

2024-09-10 10:41:35 +01:00

sve-interleaved-accesses.ll

[VPlan] Use pointer to member 0 as VPInterleaveRecipe's pointer arg. (#106431)

2024-10-06 22:53:13 +01:00

sve-interleaved-masked-accesses.ll

[VPlan] Use pointer to member 0 as VPInterleaveRecipe's pointer arg. (#106431)

2024-10-06 22:53:13 +01:00

sve-inv-loads.ll

[VPlan] Add initial loop-invariant code motion transform. (#107894)

2024-09-20 11:22:03 +01:00

sve-inv-store.ll

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

2023-12-08 18:30:30 +00:00

sve-invalid-costs.ll

[LoopVectorize] Fix build issue caused by #95920 (#96647)

2024-06-25 15:51:32 +01:00

sve-large-strides.ll

…

sve-live-out-pointer-induction.ll

[LV] Compute value of escaped induction based on the computed end value. (#110576)

2024-10-10 20:04:46 +01:00

sve-low-trip-count.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-masked-loadstore.ll

…

sve-multiexit.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-runtime-check-size-based-threshold.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-scalable-load-in-loop.ll

…

sve-select-cmp.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-strict-fadd-cost.ll

…

sve-tail-folding-cost.ll

…

sve-tail-folding-forced.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-tail-folding-option.ll

Prefer use of 0.0 over -0.0 for fadd reductions w/nsz (in IR) (#106770)

2024-09-03 09:16:37 -07:00

sve-tail-folding-optsize.ll

[LV] Remove unnecessary getRuntimeVF call when computing vector TC.

2024-06-12 14:35:37 +01:00

sve-tail-folding-overflow-checks.ll

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

2023-12-08 18:30:30 +00:00

sve-tail-folding-reductions.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

sve-tail-folding-unroll.ll

[VPlan] Run DCE after unrolling.

2024-09-22 22:08:46 +01:00

sve-tail-folding.ll

[VPlan] Add VPValue for VF, use it for VPWidenIntOrFpInductionRecipe. (#95305)

2024-09-10 10:41:35 +01:00

sve-type-conv.ll

…

sve-vector-reverse-mask4.ll

Move several vector intrinsics out of experimental namespace (#88748)

2024-04-29 10:16:45 +01:00

sve-vector-reverse.ll

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

2024-08-21 12:02:54 +02:00

sve-vfabi.ll

[VPlan] Initial modeling of VF * UF as VPValue. (#74761)

2023-12-08 18:30:30 +00:00

sve-widen-extractvalue.ll

[VPlan] Generalize VPValue::isDefinedOutsideLoopRegions.

2024-09-20 15:34:00 +01:00

sve-widen-gep.ll

[VPlan][NFC] Fix the value name of VECTOR_GEP (#107544)

2024-09-18 19:22:36 +08:00

sve-widen-phi.ll

[VPlan] Implement unrolling as VPlan-to-VPlan transform. (#95842)

2024-09-21 19:47:37 +01:00

synthesize-mask-for-call.ll

[VPlan] Model branch cond to enter scalar epilogue in VPlan. (#92651)

2024-07-05 10:08:42 +01:00

tail-fold-uniform-memops.ll

[VPlan] Add active-lane-mask as VPlan-to-VPlan transformation.

2023-09-25 13:34:45 +01:00

tail-folding-styles.ll

Move stepvector intrinsic out of experimental namespace (#98043)

2024-08-28 12:48:20 +01:00

type-shrinkage-insertelt.ll

[VPlan] Replace IR based truncateToMinimalBitwidths with VPlan version.

2023-12-02 16:12:38 +00:00

type-shrinkage-zext-costs.ll

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

2023-12-08 12:18:31 +00:00

uniform-args-call-variants.ll

[InstCombine] Replace all dominated uses of condition with constants (#105510)

2024-09-01 09:49:23 +08:00

unsafe-vf-hint-remark.ll

[AArch64] Change the cost of vector insert/extract to 2

2023-07-28 21:26:50 +01:00

veclib-calls-libsystem-darwin.ll

[aarch64] Add hyperbolic and arc trig intrinsic lowering (#98937)

2024-07-19 10:18:23 -04:00

veclib-function-calls.ll

[AArch64][Test] Update test variable names (NFC) (#110667)

2024-10-03 16:14:21 +01:00

veclib-intrinsic-calls.ll

[AArch64][Test] Update test variable names (NFC) (#110667)

2024-10-03 16:14:21 +01:00

vector-call-linear-args.ll

[LAA] Don't assume libcalls with output/input pointers can be vectorized (#108980)

2024-09-23 16:05:55 +01:00

vector-reverse-mask4.ll

[InstCombine] Remove some of the complexity-based canonicalization (#91185)

2024-08-21 12:02:54 +02:00

vector-reverse.ll

[VPlan] Model address separately. (#72164)

2024-01-01 19:51:15 +00:00

widen-call-with-intrinsic-or-libfunc.ll

[VPlan] Introduce VPWidenIntrinsicRecipe to separate from libcall. (#110486)

2024-10-08 22:37:20 +01:00

wider-VF-for-callinst.ll

[VPlan] Compute scalable VF in preheader for induction increment. (#74762)

2023-12-08 12:18:31 +00:00