Luke Lau 2566734adf
[RISCV] Handle freeze in vp.merge widening codegen prepare (#189346)
This fixes #189335. Previously we used to widen i1 vp.merges that were
used in AnyOf reduction patterns from the loop vectorizer to i8. This
improved codegen because RVV doesn't have support for tail undisturbed
mask instructions, so i1 vp.merges are expanded into a vid.v + compare
sequence. Widening it to i8 allows a single vmerge.vim to be used
instead.

The pattern used to look like this, where the loop vectorizer emitted a
freeze after the reduction:

    vector.body:
      ...
%3 = call <vscale x 4 x i1> @llvm.vp.merge.nxv4i1(<vscale x 4 x i1> %2,
<vscale x 4 x i1> splat (i1 true), <vscale x 4 x i1> %vec.phi, i32 %0)
      ...
      br i1 %5, label %middle.block, label %vector.body

for.cond.cleanup.loopexit: ; preds = %middle.block
      %6 = call i1 @llvm.vector.reduce.or.nxv4i1(<vscale x 4 x i1> %3)
      %7 = freeze i1 %6
      br label %for.cond.cleanup

But a recent change in InstCombiner now causes the freeze to be hoisted
in between the reduction and the vp.merge, breaking the existing
pattern:

    vector.body:
      ...
%3 = call <vscale x 4 x i1> @llvm.vp.merge.nxv4i1(<vscale x 4 x i1> %2,
<vscale x 4 x i1> splat (i1 true), <vscale x 4 x i1> %vec.phi, i32 %0)
      %.fr = freeze <vscale x 4 x i1> %3
      ...
      br i1 %5, label %middle.block, label %vector.body

    for.cond.cleanup.loopexit:
      %6 = call i1 @llvm.vector.reduce.or.nxv4i1(<vscale x 4 x i1> %.fr)
      br label %for.cond.cleanup

This patch teaches the transform to handle the freeze. We need to do the
change when visiting the freeze instruction otherwise we invalidate the
make_early_inc_range iterator.
2026-03-30 13:44:09 +00:00
..