Sander de Smalen
3157758190
[LV] Handle partial sub-reductions with sub in middle block. ( #178919 ) ...
Sub-reductions can be implemented in two ways:
(1) negate the operand in the vector loop (the default way).
(2) subtract the reduced value from the init value in the middle block.
Note that both ways keep the reduction itself as an 'add' reduction,
which is necessary because only llvm.vector.partial.reduce.add exists.
The ISD nodes for partial reductions don't support folding the
sub/negation into its operands because the following is not a valid
transformation:
```
sub(0, mul(ext(a), ext(b)))
-> mul(ext(a), ext(sub(0, b)))
```
It can therefore be better to choose option (2) such that the partial
reduction is always positive (starting at '0') and to do a final
subtract in the middle block.
For AArch64 there are no dot-product instructions that can
do a `partial.reduce.sub(acc, mul(ext(a), ext(b)))` operation.
I'm not sure if such instructions exist for other targets.
(If so then we may want to make this decision a target option)
This PR also increases the AArch64 cost of a partial sub-reduction
when this exists in an 'add-sub' reduction chain.
Fixes https://github.com/llvm/llvm-project/issues/178703
2026-02-10 11:00:32 +00:00
2025-09-18 19:25:05 +01:00
2025-12-16 06:30:54 +00:00
2025-11-13 06:12:40 -10:00
2025-11-26 12:25:55 +00:00
2026-01-12 17:12:35 -08:00
2026-01-07 10:35:08 +00:00
2026-01-30 16:12:30 +00:00
2026-02-10 09:57:48 +00:00
2025-11-26 12:25:55 +00:00
2026-01-07 10:35:08 +00:00
2025-10-20 08:50:09 +00:00
2026-01-25 23:09:58 +00:00
2026-01-23 07:29:56 +00:00
2026-01-06 20:36:26 +00:00
2025-11-26 12:25:55 +00:00
2026-01-20 14:19:11 +00:00
2025-11-26 12:25:55 +00:00
2026-01-07 11:32:43 +00:00
2025-11-10 17:03:14 +00:00
2025-11-26 12:25:55 +00:00
2026-01-14 14:59:18 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 15:41:26 +08:00
2025-11-26 12:25:55 +00:00
2025-11-26 12:25:55 +00:00
2025-11-26 12:25:55 +00:00
2025-11-26 12:25:55 +00:00
2025-11-26 12:25:55 +00:00
2026-01-20 12:15:50 +00:00
2026-02-09 13:20:44 +00:00
2025-11-26 16:36:54 +00:00
2025-12-19 22:05:27 +00:00
2026-01-06 20:36:26 +00:00
2026-01-14 22:19:11 +00:00
2025-06-02 17:40:50 +01:00
2026-01-06 20:36:26 +00:00
2026-02-03 07:57:15 +00:00
2025-09-18 19:25:05 +01:00
2025-09-18 19:25:05 +01:00
2025-11-26 16:36:54 +00:00
2025-08-21 21:40:47 +01:00
2026-02-05 17:55:49 +00:00
2026-02-05 17:55:49 +00:00
2025-11-26 12:25:55 +00:00
2026-01-06 20:36:26 +00:00
2026-02-03 07:57:15 +00:00
2025-11-26 12:25:55 +00:00
2026-01-06 20:36:26 +00:00
2025-09-25 07:13:38 +00:00
2026-01-06 20:36:26 +00:00
2025-12-11 21:41:47 +00:00
2026-01-06 20:36:26 +00:00
2025-11-05 22:11:29 +00:00
2026-01-06 20:36:26 +00:00
2025-04-28 20:01:37 +01:00
2025-10-23 11:18:55 +00:00
2025-11-25 05:58:41 +00:00
2026-02-09 15:32:04 +00:00
2025-10-06 22:24:02 +01:00
2026-02-05 17:55:49 +00:00
2026-01-27 21:37:21 +02:00
2026-01-07 10:35:08 +00:00
2026-02-09 22:10:13 +00:00
2025-11-17 13:44:25 +00:00
2026-01-12 16:46:25 +00:00
2026-02-10 11:00:32 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-02-03 07:57:15 +00:00
2026-02-03 07:57:15 +00:00
2026-02-03 21:07:02 +00:00
2026-01-30 20:31:32 +00:00
2025-11-26 12:25:55 +00:00
2025-11-26 12:25:55 +00:00
2026-02-10 11:00:32 +00:00
2026-02-10 11:00:32 +00:00
2026-02-09 23:14:26 +00:00
2026-01-06 20:36:26 +00:00
2025-10-23 20:31:31 +08:00
2025-10-21 10:49:27 +01:00
2026-01-07 10:35:08 +00:00
2025-10-01 16:53:59 +00:00
2025-11-26 12:25:55 +00:00
2025-10-06 15:32:13 +01:00
2026-01-30 16:12:30 +00:00
2025-08-05 09:42:55 +01:00
2026-02-03 07:57:15 +00:00
2026-01-06 20:36:26 +00:00
2025-11-08 21:59:38 +00:00
2026-01-21 16:37:12 +00:00
2025-11-26 12:25:55 +00:00
2026-01-23 07:29:56 +00:00
2025-07-27 13:53:26 +01:00
2026-01-06 20:36:26 +00:00
2025-10-22 15:21:27 +01:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2025-09-09 10:46:12 +01:00
2025-11-26 16:36:54 +00:00
2026-01-18 19:55:46 +00:00
2026-01-26 21:32:50 +00:00
2026-01-13 20:14:15 +00:00
2026-01-13 20:14:15 +00:00
2025-04-17 13:26:15 +01:00
2026-01-30 16:12:30 +00:00
2025-11-21 08:14:46 +00:00
2026-02-03 07:57:15 +00:00
2025-12-15 10:54:06 +00:00
2025-09-25 07:13:38 +00:00
2025-11-21 08:14:46 +00:00
2025-11-25 10:26:51 +00:00
2025-12-15 10:54:06 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2025-09-04 19:31:30 +01:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2025-06-19 13:24:39 +01:00
2025-11-26 12:25:55 +00:00
2026-01-06 20:36:26 +00:00
2025-12-15 10:54:06 +00:00
2026-01-07 10:35:08 +00:00
2025-11-17 13:44:25 +00:00
2026-01-06 20:36:26 +00:00
2025-11-17 13:44:25 +00:00
2026-01-06 20:36:26 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2025-10-20 08:50:09 +00:00
2025-05-29 11:15:16 +08:00
2026-01-06 20:36:26 +00:00
2026-01-06 15:41:26 +08:00
2026-01-06 20:36:26 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-07 10:35:08 +00:00
2025-12-18 14:57:48 +00:00
2025-12-08 15:46:02 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2026-01-06 20:36:26 +00:00
2025-11-25 10:26:51 +00:00
2025-10-20 08:50:09 +00:00
2026-01-06 20:36:26 +00:00
2025-10-22 21:27:11 +01:00
2025-11-26 12:25:55 +00:00
2025-11-25 05:58:41 +00:00
2025-09-18 19:25:05 +01:00
2025-11-26 12:25:55 +00:00
2025-10-22 21:27:11 +01:00
2026-01-06 20:36:26 +00:00
2025-11-26 12:25:55 +00:00
2025-11-05 20:57:52 +00:00
2026-01-06 20:36:26 +00:00
2025-10-19 18:49:05 +00:00
2025-10-22 21:27:11 +01:00
2025-10-19 18:49:05 +00:00
2026-01-06 20:36:26 +00:00
2025-12-15 10:54:06 +00:00
2025-06-17 11:07:43 +01:00
2026-01-09 18:08:16 +00:00
2025-11-26 12:25:55 +00:00
2025-12-18 14:57:48 +00:00
2026-01-17 21:59:50 +00:00
2026-01-14 07:45:44 +00:00
2025-11-25 10:26:51 +00:00
2026-01-16 13:59:51 +08:00
2026-01-06 20:36:26 +00:00
3157758190