Luke Lau
7ef77eb998
[LV] Support scalable interleave groups for factors 3,5,6 and 7 ( #141865 )
...
Currently the loop vectorizer can only vectorize interleave groups for
power-of-2 factors at scalable VFs by recursively interleaving
[de]interleave2 intrinsics.
However after https://github.com/llvm/llvm-project/pull/124825 and
#139893 , we now have [de]interleave intrinsics for all factors up to 8,
which is enough to support all types of segmented loads and stores on
RISC-V.
Now that the interleaved access pass has been taught to lower these in
#139373 and #141512 , this patch teaches the loop vectorizer to emit
these intrinsics for factors up to 8, which enables scalable
vectorization for non-power-of-2 factors.
As far as I'm aware, no in-tree target will vectorize a scalable
interelave group above factor 8 because the maximum interleave factor is
capped at 4 on AArch64 and 8 on RISC-V, and the
`-max-interleave-group-factor` CLI option defaults to 8, so the
recursive [de]interleaving code has been removed for now.
Factors of 3 with scalable VFs are also turned off in AArch64 since
there's no lowering for [de]interleave3 just yet either.
2025-06-12 11:09:09 +01:00
..
2025-04-17 11:50:49 +01:00
2024-09-21 19:47:37 +01:00
2025-01-06 10:26:51 +00:00
2025-05-14 14:56:35 +01:00
2025-05-28 11:00:14 +02:00
2025-05-08 18:47:44 +08:00
2025-05-02 15:42:21 +01:00
2025-05-28 11:00:14 +02:00
2025-05-08 18:47:44 +08:00
2025-03-25 12:57:24 +00:00
2025-04-28 20:01:37 +01:00
2025-06-05 11:38:30 +00:00
2025-06-09 20:59:12 +01:00
2025-03-26 21:03:50 +00:00
2025-03-26 21:03:50 +00:00
2025-03-25 12:57:24 +00:00
2025-04-29 09:18:07 +08:00
2025-04-04 15:44:26 +01:00
2024-11-21 11:21:12 +00:00
2025-03-26 21:03:50 +00:00
2025-05-30 13:25:55 +02:00
2025-03-26 21:03:50 +00:00
2025-03-27 21:15:32 +00:00
2025-06-05 11:38:30 +00:00
2025-06-02 17:40:50 +01:00
2025-04-04 15:44:26 +01:00
2025-03-26 21:03:50 +00:00
2025-03-26 21:03:50 +00:00
2025-03-04 08:43:08 +00:00
2025-02-09 11:20:20 +00:00
2025-01-02 14:31:36 +00:00
2025-03-25 12:57:24 +00:00
2024-09-11 21:04:34 +01:00
2025-03-26 21:03:50 +00:00
2025-03-26 21:03:50 +00:00
2025-02-09 11:20:20 +00:00
2024-11-21 21:11:21 +00:00
2025-01-27 21:35:36 +00:00
2025-05-28 11:00:14 +02:00
2025-01-14 22:07:38 +00:00
2025-04-04 15:44:26 +01:00
2025-03-05 18:19:39 +00:00
2025-04-28 20:01:37 +01:00
2025-05-14 14:56:35 +01:00
2025-04-28 20:01:37 +01:00
2024-11-19 08:55:39 +00:00
2025-02-03 17:01:02 +00:00
2025-02-28 12:56:12 +00:00
2024-11-20 09:33:39 +00:00
2025-01-29 08:09:50 +00:00
2025-05-22 07:59:57 +03:00
2025-05-28 11:00:14 +02:00
2024-12-16 11:55:48 +08:00
2024-11-06 11:53:33 +00:00
2025-03-25 12:57:24 +00:00
2025-06-09 20:59:12 +01:00
2025-05-01 16:06:37 +01:00
2025-04-11 14:31:08 +01:00
2025-05-19 13:27:17 +01:00
2025-04-23 13:19:18 +01:00
2025-03-26 21:03:50 +00:00
2025-05-28 11:00:14 +02:00
2025-04-04 15:44:26 +01:00
2025-01-20 15:05:15 +00:00
2025-05-24 19:17:16 +01:00
2025-06-02 17:40:50 +01:00
2024-07-27 12:52:12 +01:00
2025-05-08 18:47:44 +08:00
2024-11-21 11:21:12 +00:00
2025-03-25 12:57:24 +00:00
2025-03-25 12:57:24 +00:00
2024-09-22 22:08:46 +01:00
2025-06-05 11:38:30 +00:00
2025-02-17 09:51:35 +00:00
2025-01-29 08:09:50 +00:00
2025-01-29 08:09:50 +00:00
2024-11-29 17:41:05 +00:00
2024-11-28 15:07:36 +08:00
2025-06-08 22:34:07 +01:00
2025-04-28 20:01:37 +01:00
2025-04-17 13:26:15 +01:00
2025-05-09 11:19:12 +01:00
2025-03-25 12:57:24 +00:00
2025-01-29 08:09:50 +00:00
2025-02-27 17:43:24 +00:00
2025-02-17 09:51:35 +00:00
2025-03-31 12:40:41 +01:00
2024-12-05 14:36:40 +01:00
2025-01-29 08:09:50 +00:00
2025-03-31 12:40:41 +01:00
2025-06-07 21:49:54 +01:00
2025-04-23 19:06:38 +01:00
2024-11-21 11:21:12 +00:00
2025-03-31 12:40:41 +01:00
2025-03-26 21:03:50 +00:00
2025-03-31 12:40:41 +01:00
2025-03-31 12:40:41 +01:00
2024-11-20 09:33:39 +00:00
2025-03-31 12:40:41 +01:00
2025-04-07 14:09:43 +01:00
2025-03-31 12:40:41 +01:00
2024-11-05 22:45:56 +01:00
2025-03-31 12:40:41 +01:00
2024-11-06 11:53:33 +00:00
2025-05-08 18:47:44 +08:00
2025-03-31 12:40:41 +01:00
2025-01-21 21:37:51 +00:00
2025-06-12 11:09:09 +01:00
2025-06-12 11:09:09 +01:00
2024-09-20 11:22:03 +01:00
2025-03-31 12:40:41 +01:00
2024-11-21 11:21:12 +00:00
2025-03-26 21:03:50 +00:00
2024-11-21 11:21:12 +00:00
2025-03-31 12:40:41 +01:00
2025-04-25 09:47:47 +01:00
2025-04-28 20:01:37 +01:00
2025-05-29 11:15:16 +08:00
2025-03-21 09:24:56 +00:00
2025-05-27 16:07:56 +01:00
2024-09-03 09:16:37 -07:00
2025-04-04 15:44:26 +01:00
2025-04-04 15:44:26 +01:00
2025-04-04 15:44:26 +01:00
2025-04-04 15:44:26 +01:00
2025-04-28 20:01:37 +01:00
2025-03-31 12:40:41 +01:00
2025-05-28 12:42:41 +00:00
2025-03-13 07:16:04 -07:00
2025-06-05 11:38:30 +00:00
2025-03-28 21:48:59 +00:00
2025-06-07 21:49:54 +01:00
2025-04-28 20:01:37 +01:00
2025-04-04 15:44:26 +01:00
2025-03-26 21:03:50 +00:00
2025-03-29 20:14:10 +00:00
2025-05-09 11:19:12 +01:00
2025-05-28 11:00:14 +02:00
2025-04-06 22:14:24 +01:00
2025-03-26 21:03:50 +00:00
2025-03-25 20:57:10 +00:00
2024-11-06 11:53:33 +00:00
2025-01-29 16:56:47 +01:00
2024-12-05 14:36:40 +01:00
2024-10-24 17:53:12 -07:00
2025-04-22 09:33:52 +01:00
2025-04-22 09:33:52 +01:00
2024-09-23 16:05:55 +01:00
2025-05-28 11:00:14 +02:00
2024-11-06 11:53:33 +00:00
2025-06-09 20:59:12 +01:00
2025-06-07 21:49:54 +01:00
2024-11-21 11:21:12 +00:00