Luke Lau
7ef77eb998
[LV] Support scalable interleave groups for factors 3,5,6 and 7 ( #141865 )
...
Currently the loop vectorizer can only vectorize interleave groups for
power-of-2 factors at scalable VFs by recursively interleaving
[de]interleave2 intrinsics.
However after https://github.com/llvm/llvm-project/pull/124825 and
#139893 , we now have [de]interleave intrinsics for all factors up to 8,
which is enough to support all types of segmented loads and stores on
RISC-V.
Now that the interleaved access pass has been taught to lower these in
#139373 and #141512 , this patch teaches the loop vectorizer to emit
these intrinsics for factors up to 8, which enables scalable
vectorization for non-power-of-2 factors.
As far as I'm aware, no in-tree target will vectorize a scalable
interelave group above factor 8 because the maximum interleave factor is
capped at 4 on AArch64 and 8 on RISC-V, and the
`-max-interleave-group-factor` CLI option defaults to 8, so the
recursive [de]interleaving code has been removed for now.
Factors of 3 with scalable VFs are also turned off in AArch64 since
there's no lowering for [de]interleave3 just yet either.
2025-06-12 11:09:09 +01:00
..
2025-06-12 11:09:09 +01:00
2025-03-04 17:19:06 -06:00
2025-05-29 11:15:16 +08:00
2025-01-14 22:07:38 +00:00
2025-06-02 17:40:50 +01:00
2025-06-09 20:59:12 +01:00
2025-06-12 11:09:09 +01:00
2025-05-20 14:16:01 +01:00
2023-05-17 17:03:15 +02:00
2025-03-25 12:57:24 +00:00
2025-06-09 20:59:12 +01:00
2023-05-17 17:03:15 +02:00
2024-11-21 08:06:56 +00:00
2025-04-04 15:44:26 +01:00
2024-11-06 11:53:33 +00:00
2025-02-27 17:43:24 +00:00
2024-12-29 19:05:08 +00:00
2024-11-06 11:53:33 +00:00
2025-03-26 21:03:50 +00:00
2024-09-04 16:28:39 +05:30
2023-06-12 13:10:22 +02:00
2025-04-14 17:11:13 +02:00
2025-05-20 14:16:01 +01:00
2025-05-28 11:00:14 +02:00
2024-09-19 09:41:25 +01:00
2025-03-25 12:57:24 +00:00
2025-04-29 21:40:42 +01:00
2025-01-23 22:29:30 +00:00
2025-04-04 15:44:26 +01:00
2025-05-20 14:16:01 +01:00
2025-04-28 20:01:37 +01:00
2025-04-21 18:36:43 +01:00
2025-03-25 12:57:24 +00:00
2023-10-28 17:47:46 +08:00
2023-10-24 20:27:39 -07:00
2025-04-04 15:34:45 +02:00
2025-03-26 21:03:50 +00:00
2025-04-04 15:44:26 +01:00
2025-01-30 10:37:00 +00:00
2025-06-09 20:59:12 +01:00
2025-04-28 20:01:37 +01:00
2025-03-25 12:57:24 +00:00
2025-03-25 12:57:24 +00:00
2025-01-14 22:07:38 +00:00
2025-03-26 21:03:50 +00:00
2025-01-29 08:09:50 +00:00
2025-01-29 08:09:50 +00:00
2025-01-29 08:09:50 +00:00
2025-03-25 12:57:24 +00:00
2024-12-05 14:36:40 +01:00
2025-06-07 21:49:54 +01:00
2025-03-25 12:57:24 +00:00
2025-03-25 12:57:24 +00:00
2025-03-26 21:03:50 +00:00
2025-03-25 12:57:24 +00:00
2024-07-21 14:03:40 +01:00
2025-06-09 20:59:12 +01:00
2025-05-20 14:16:01 +01:00
2024-07-05 10:08:42 +01:00
2023-04-17 13:08:15 +01:00
2025-04-21 18:36:43 +01:00
2025-03-25 12:57:24 +00:00
2025-04-14 17:11:13 +02:00
2025-03-25 12:57:24 +00:00
2024-08-27 11:55:47 +01:00
2024-12-05 14:36:40 +01:00
2025-03-14 15:50:49 +00:00
2025-05-27 16:07:56 +01:00
2024-11-21 08:06:56 +00:00
2025-05-14 14:56:35 +01:00
2025-02-27 17:43:24 +00:00
2025-04-05 16:22:47 +01:00
2024-11-06 11:53:33 +00:00
2025-06-09 20:59:12 +01:00
2025-06-03 18:13:51 +01:00
2024-11-21 08:06:56 +00:00
2025-03-25 12:57:24 +00:00
2024-01-31 13:31:28 +00:00
2025-03-26 21:03:50 +00:00
2025-03-25 12:57:24 +00:00
2025-04-02 17:23:00 +01:00
2025-06-09 20:59:12 +01:00
2025-03-25 12:57:24 +00:00
2025-06-07 21:49:54 +01:00
2025-03-26 21:03:50 +00:00
2025-01-29 16:56:47 +01:00
2025-02-09 11:20:20 +00:00
2025-03-25 12:57:24 +00:00
2023-12-05 14:09:36 +01:00
2023-08-08 18:10:23 -04:00
2024-07-01 09:26:01 +02:00
2025-04-14 17:11:13 +02:00
2024-06-14 17:18:20 -04:00
2025-03-25 12:57:24 +00:00
2025-03-11 21:19:14 +00:00
2025-02-26 13:57:51 +00:00
2025-04-02 10:26:48 +01:00
2025-03-26 21:03:50 +00:00
2024-12-12 16:48:31 +08:00
2025-06-03 18:13:51 +01:00
2025-03-26 21:03:50 +00:00
2025-03-25 12:57:24 +00:00
2024-12-10 20:09:18 +08:00
2025-03-26 21:03:50 +00:00
2025-06-03 18:13:51 +01:00
2024-02-05 11:57:34 +01:00
2025-04-28 20:01:37 +01:00
2025-01-27 21:35:36 +00:00
2025-06-06 21:18:41 +01:00
2025-04-21 18:36:43 +01:00
2025-04-28 20:01:37 +01:00
2024-12-05 14:36:40 +01:00
2025-01-29 08:09:50 +00:00
2025-02-20 17:01:49 +01:00
2025-03-27 20:29:37 +09:00
2025-04-04 15:44:26 +01:00
2023-10-28 17:47:46 +08:00
2025-04-14 21:51:15 +02:00
2025-06-01 22:39:37 +01:00
2025-04-28 11:16:36 +01:00
2023-06-20 13:17:28 -04:00
2024-12-18 09:25:45 +00:00
2024-12-18 09:25:45 +00:00
2024-12-18 19:06:34 +01:00
2025-02-28 12:56:12 +00:00
2025-03-25 12:57:24 +00:00
2025-05-14 14:56:35 +01:00
2024-08-11 20:38:36 +02:00
2025-03-25 12:57:24 +00:00
2024-12-18 19:06:34 +01:00
2025-01-29 08:09:50 +00:00
2024-07-05 21:41:31 +01:00
2024-11-21 08:06:56 +00:00
2025-03-25 12:57:24 +00:00
2025-01-29 08:09:50 +00:00
2025-03-25 12:57:24 +00:00
2025-01-29 08:09:50 +00:00
2025-03-25 12:57:24 +00:00
2025-04-04 15:44:26 +01:00
2024-10-02 10:28:54 +01:00
2025-03-09 15:05:35 +00:00
2024-12-16 11:55:48 +08:00
2024-11-06 11:53:33 +00:00
2024-11-06 11:53:33 +00:00
2025-04-28 20:01:37 +01:00
2024-11-06 11:53:33 +00:00
2025-02-17 16:40:37 +01:00
2025-05-14 14:56:35 +01:00
2025-03-16 11:37:58 +00:00
2025-03-26 21:03:50 +00:00
2025-04-28 20:01:37 +01:00
2025-03-26 21:03:50 +00:00
2025-03-25 12:57:24 +00:00
2024-11-07 14:09:45 +01:00
2024-07-07 20:08:01 +01:00
2025-01-29 16:56:47 +01:00
2025-03-25 12:57:24 +00:00
2024-11-21 08:06:56 +00:00
2024-09-14 21:21:55 +01:00
2025-03-25 12:57:24 +00:00
2024-07-01 09:52:56 -07:00
2023-12-05 14:09:36 +01:00
2025-03-25 12:57:24 +00:00
2024-11-06 11:53:33 +00:00
2025-03-23 09:06:01 +00:00
2024-11-06 11:53:33 +00:00
2025-04-17 11:50:49 +01:00
2025-04-04 15:44:26 +01:00
2025-02-09 11:20:20 +00:00
2025-03-25 12:57:24 +00:00
2025-04-04 15:44:26 +01:00
2025-05-14 14:56:35 +01:00
2024-10-10 20:04:46 +01:00
2025-02-15 21:54:16 +01:00
2025-02-15 21:54:16 +01:00
2025-04-29 09:47:38 +01:00
2025-05-08 11:49:54 +01:00
2025-03-26 21:03:50 +00:00
2025-03-15 21:32:48 +00:00
2024-12-27 10:37:21 +08:00
2025-03-25 12:57:24 +00:00
2025-02-09 11:20:20 +00:00
2025-04-14 17:11:13 +02:00
2025-01-29 16:56:47 +01:00
2025-05-29 11:15:16 +08:00
2025-04-14 17:11:13 +02:00
2025-05-29 11:15:16 +08:00
2025-03-26 21:03:50 +00:00
2024-11-21 08:06:56 +00:00
2025-04-14 17:11:13 +02:00
2025-03-11 21:19:14 +00:00
2025-03-25 12:57:24 +00:00
2025-05-14 14:56:35 +01:00
2024-09-19 09:41:25 +01:00
2025-03-25 12:57:24 +00:00
2025-03-26 21:03:50 +00:00
2024-01-16 21:54:58 -08:00
2025-03-25 12:57:24 +00:00
2024-01-16 21:54:58 -08:00
2025-03-25 12:57:24 +00:00
2025-04-02 17:22:40 +01:00
2025-03-25 12:57:24 +00:00
2025-04-02 17:22:40 +01:00
2025-03-25 12:57:24 +00:00
2025-01-12 20:10:28 +00:00
2025-02-26 13:57:51 +00:00
2024-12-29 19:05:08 +00:00
2025-05-08 18:47:44 +08:00
2025-03-25 12:57:24 +00:00
2024-11-21 11:21:12 +00:00
2025-06-05 11:38:30 +00:00
2024-09-20 11:22:03 +01:00
2025-01-29 08:09:50 +00:00
2025-03-26 21:03:50 +00:00
2024-11-06 11:53:33 +00:00
2025-01-29 08:09:50 +00:00
2025-03-25 12:57:24 +00:00
2023-10-17 03:49:39 +08:00
2025-03-25 12:57:24 +00:00
2025-04-21 18:36:43 +01:00
2025-04-21 18:36:43 +01:00
2025-04-28 20:01:37 +01:00
2025-06-08 22:34:07 +01:00
2025-04-28 20:01:37 +01:00
2025-04-04 15:44:26 +01:00
2025-06-08 22:34:07 +01:00
2025-06-08 22:34:07 +01:00
2025-06-08 22:34:07 +01:00
2025-06-08 22:34:07 +01:00
2025-06-08 22:34:07 +01:00
2025-03-25 12:57:24 +00:00
2025-04-28 20:01:37 +01:00
2024-10-17 16:50:59 +01:00
2025-05-16 21:12:39 +01:00
2025-04-28 20:01:37 +01:00
2025-05-14 14:56:35 +01:00
2025-05-28 11:00:14 +02:00
2023-08-03 00:37:19 -07:00
2025-04-04 15:44:26 +01:00
2025-04-04 15:44:26 +01:00
2025-03-26 21:03:50 +00:00
2025-04-21 18:36:43 +01:00
2025-04-04 15:44:26 +01:00
2025-05-14 14:56:35 +01:00
2025-04-04 15:44:26 +01:00
2025-04-04 15:44:26 +01:00
2025-04-24 08:37:49 +01:00
2025-04-21 18:36:43 +01:00
2025-03-25 12:57:24 +00:00
2025-03-25 12:57:24 +00:00
2024-12-05 14:36:40 +01:00
2025-03-25 12:57:24 +00:00
2025-03-25 12:57:24 +00:00
2025-06-07 21:49:54 +01:00
2025-01-29 08:09:50 +00:00
2025-05-02 15:42:21 +01:00
2025-05-02 15:42:21 +01:00
2025-05-02 15:42:21 +01:00
2025-05-02 15:42:21 +01:00
2025-05-02 15:42:21 +01:00
2024-08-21 12:02:54 +02:00
2025-05-22 08:16:15 +01:00
2025-03-26 21:03:50 +00:00
2023-10-28 17:47:46 +08:00
2024-08-19 12:40:20 +01:00
2024-12-11 21:11:05 +00:00
2025-03-25 12:57:24 +00:00
2024-10-16 06:10:19 +01:00
2025-03-25 12:57:24 +00:00
2025-03-25 12:57:24 +00:00
2025-01-29 08:09:50 +00:00
2024-09-19 18:50:10 +01:00
2024-12-05 14:36:40 +01:00
2025-03-25 12:57:24 +00:00
2025-05-28 11:00:14 +02:00
2025-05-28 11:00:14 +02:00
2025-05-28 11:00:14 +02:00
2023-04-25 21:47:14 +01:00
2024-09-06 11:20:14 +02:00
2025-03-25 12:57:24 +00:00
2025-05-28 11:00:14 +02:00
2025-04-18 21:52:05 +01:00
2025-06-07 21:49:54 +01:00
2024-04-04 18:30:17 -04:00
2024-07-07 20:08:01 +01:00
2025-06-07 21:49:54 +01:00
2025-03-10 20:28:56 +00:00
2025-06-07 21:49:54 +01:00
2025-06-07 21:49:54 +01:00
2025-05-08 20:56:48 +01:00
2025-06-09 20:59:12 +01:00
2025-06-07 21:49:54 +01:00
2025-06-07 21:49:54 +01:00
2025-06-07 21:49:54 +01:00
2025-05-27 16:07:56 +01:00
2024-11-06 11:53:33 +00:00
2025-03-19 21:35:15 +00:00
2025-02-26 13:57:51 +00:00
2025-05-27 16:07:56 +01:00
2025-05-24 17:32:05 +01:00
2025-05-01 01:25:41 +08:00