Allows shuffle to fold constant vectors that have already been lowered to constant pool - shuffle combining can then constant fold this.
Noticed while triaging #79100
The enc/dec of promoted AMX-TILE instructions have been supported in
https://github.com/llvm/llvm-project/pull/76210.
This patch support lowering for promoted AMX-TILE instructions and
integrate test to existing tests.
Printing the raw symbol is useful in inline asm (e.g. getting the C++
mangled name, referencing a symbol in a custom way while ensuring it is
not optimized out even if internal). Similar constraints are available
in other targets (e.g. "S" for aarch64/riscv, "Cs" for m68k).
```
namespace ns { extern int var, a[4]; }
void foo() {
asm(".pushsection .xxx,\"aw\"; .dc.a %p0; .popsection" :: "Ws"(&ns::var));
asm(".reloc ., BFD_RELOC_NONE, %p0" :: "Ws"(&ns::a[3]));
}
```
Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105576
X86 doesn't have a BITREVERSE instruction, so if we're working with a casted boolean vector, we're better off shuffling the vector instead if we have PSHUFB (SSSE3 or later)
Fixes#77459
This is the logical equivalent for #76710 for APInt and uses the same
naming scheme.
Converted existing users through:
`git grep -l "cast<ConstantSDNode>\(.*\).*getAPIntValueValue" | xargs
sed -E -i
's/cast<ConstantSDNode>\((.*)\)->getAPIntValue/\1->getAsAPIntVal/'`
This follows on from #76708, allowing
`cast<ConstantSDNode>(N)->getZExtValue()` to be replaced with just
`N->getAsZextVal();`
Introduced via `git grep -l "cast<ConstantSDNode>\(.*\).*getZExtValue" |
xargs sed -E -i
's/cast<ConstantSDNode>\((.*)\)->getZExtValue/\1->getAsZExtVal/'` and
then using `git clang-format` on the result.
After d6bb96e677759375b2bea00115918b2cb6552f5b, calling
getPrefLoopAlignment without passing in a pointer to a MachineLoop
causes a segmentation fault. This conflicts with the API in
TargetLoweringBase where the default MachineLoop pointer passed is
nullptr. This patch fixes this by checking if the pointer points to
something before enabling the optional functionality.
Some shuffles with `bf16` as element type were running into a
`llvm_unreachable`. Key to reproducing was to chain two shuffles.
```llvm
define <2 x bfloat> @shuffle_chained_v32bf16_v2bf16(<32 x bfloat> %a) {
%s = shufflevector <32 x bfloat> %a, <32 x bfloat> zeroinitializer, <32 x i32> <i32 0, i32 16, i32 1, i32 17, i32 2, i32 18, i32 3, i32 19, i32 4, i32 20, i32 5, i32 21, i32 6, i32 22, i32 7, i32 23, i32 8, i32 24, i32 9, i32 25, i32 10, i32 26, i32 11, i32 27, i32 12, i32 28, i32 13, i32 29, i32 14, i32 30, i32 15, i32 31>
%s2 = shufflevector <32 x bfloat> %s, <32 x bfloat> zeroinitializer, <2 x i32> <i32 0, i32 1>
ret <2 x bfloat> %s2
}
```
This was hitting this UNREACHABLE:
```
Not a valid 512-bit x86 vector type!
UNREACHABLE executed at /home/benoit/iree/third_party/llvm-project/llvm/lib/Target/X86/X86ISelLowering.cpp:17124!
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0. Program arguments: /home/benoit/mlir-build/bin/llc -mtriple=x86_64-unknown-linux-gnu -mattr=+avx512f,+avx512vl,+avx512bw,+avx512bf16
1. Running pass 'Function Pass Manager' on module '<stdin>'.
2. Running pass 'X86 DAG->DAG Instruction Selection' on function '@shuffle_chained_v32bf16_v2bf16'
```
If SHUF128 is accessing only the upper half of a vector source that is a concatenation/insert_subvector then try to access the subvector directly and adjust the element mask accordingly.
This helper function shortens examples like
`cast<ConstantSDNode>(Node->getOperand(1))->getZExtValue();` to
`Node->getConstantOperandVal(1);`.
Implemented with:
`git grep -l
"cast<ConstantSDNode>\(.*->getOperand\(.*\)\)->getZExtValue\(\)" | xargs
sed -E -i
's/cast<ConstantSDNode>\((.*)->getOperand\((.*)\)\)->getZExtValue\(\)/\1->getConstantOperandVal(\2)/`
and `git grep -l
"cast<ConstantSDNode>\(.*\.getOperand\(.*\)\)->getZExtValue\(\)" | xargs
sed -E -i
's/cast<ConstantSDNode>\((.*)\.getOperand\((.*)\)\)->getZExtValue\(\)/\1.getConstantOperandVal(\2)/'`.
With a couple of simple manual fixes needed. Result then processed by
`git clang-format`.
By looking at whether a global is large instead of looking at the code
model.
This also fixes references to large data in the small code model.
We now always fold any 32-bit offset into the addressing mode with the
large code model since it uses 64-bit relocations.
Return ConstantPoolSDNode instead of const ConstantPoolSDNode - doesn't affect the accessors at all and makes it easier to use result in calls expecting a SDNode.
We only iterate over the original target constant/undef width, so keep the alternative data in its original form.
This should help if we try to merge constant data in the future.
This essentially reverts https://reviews.llvm.org/D140593. Somewhere
along the line we properly fixed the medium code model to assume
functions are small, so now we get a 32-bit movl as desired.
This will result in larger atomic operations getting expanded to
`__atomic_*` libcalls via AtomicExpandPass, which matches what Clang
already does in the frontend.
Rename to canonicalizeShuffleWithOp and begin adding SHUFFLE(UNARYOP(X),UNARYOP(Y)) -> UNARYOP(SHUFFLE(X,Y)) fold support.
This is only kicking in after legalization, so targets that expand bit counts are still duplicating but it helps with a few initial cases.
I'm investigating adding support for extensions/conversions as well, but this is a first step.
Create a new constant pool entry directly instead of going via a BUILD_VECTOR node, which makes constant pool reuse more difficult.
Helps with some regressions in #73509
This option enables an experimental lowering for unordered atomics I worked
on a few years back. It never reached production quality, and hasn't been
worked on in years. So let's rip it out.
This wasn't a crazy idea, but I hit some stumbling block which prevented me
from pushing it across the finish line. From the look of 027aa27, that
change description is probably a good summary. I don't remember the
details any longer.
