We currently accept label arguments to inline asm calls. This support
predates both blockaddresses and callbr and is only covered by one X86
test. Remove it in favor of callbr (or at least blockaddress, though
that cannot guarantee correct codegen, just like using block labels
directly can't).
I didn't bother implementing bitcode upgrade support for this, but I can
add it if desired.
Linkers do not currently support PT_GNU_RELRO for SHF_X86_64_LARGE
sections; that would require the linker to emit more than one
PT_GNU_RELRO because large sections are discontiguous by design,
and most ELF dynamic loaders do not support that (bionic appears to
support it but glibc/musl/FreeBSD/NetBSD/OpenBSD appear not to). With
current linkers these sections will end up in .ldata which results
in silently disabling RELRO. Therefore, disable SHF_X86_64_LARGE for
RELRO sections. If this ever gets supported by downstream components
in the future we could add an opt-in flag for moving these sections
to .ldata.rel.ro which would trigger the creation of a second
PT_GNU_RELRO.
Reviewers: MaskRay, aeubanks
Reviewed By: aeubanks
Pull Request: https://github.com/llvm/llvm-project/pull/137742
The code below the removed check looks generic enough to support
arbitrary integer widths. This change helps 32-bit targets avoid
expensive expansion/libcalls in the case of zero input.
Pull Request: https://github.com/llvm/llvm-project/pull/137197
We are adding new test cases to see the transformation impact due to #134979.
These are similar to previous commit #688c3ffb057a87b86c6c1e77040418adf511efbb.
Handling struct with two member and different vector factor.
---------
Co-authored-by: Rohit Aggarwal <Rohit.Aggarwal@amd.com>
Suppress EGPR/NDD instructions for relocations to avoid APX relocation
types emitted. This is to keep backward compatibility with old version
of linkers without APX support. The use case is to try APX features with
LLVM + old built-in linker on RHEL9 OS which is expected to be EOL in
2032.
If there are APX relocation types, the old version of linkers would
raise "unsupported relocation type" error. Example:
```
$ llvm-mc -filetype=obj -o got.o -triple=x86_64-unknown-linux got.s
$ ld got.o -o got.exe
ld: got.o: unsupported relocation type 0x2b
...
$ cat got.s
...
movq foo@GOTPCREL(%rip), %r16
$ llvm-objdump -dr got.o
...
1: d5 48 8b 05 00 00 00 00 movq (%rip), %r16
0000000000000005: R_X86_64_CODE_4_GOTPCRELX foo-0x4
```
Currently shouldReduceLoadWidth is very relaxed about when loads can be
split to avoid extractions from the original full width load - resulting
in many cases where the number of memory operations notably increases,
replacing the cost of a extract_subvector for additional loads.
This patch adjusts the 256/512-bit vector load splitting metric to
detect cases where ANY use of the full width load is be used directly -
in which case we will now reuse that load for smaller types, unless we'd
need to extract an upper subvector / integer element - i.e. we now
correctly treat (extract_subvector cst, 0) as free.
We retain the existing logic of never splitting loads if all uses are
extract+stores but we improve this by peeking through bitcasts while
looking for extract_subvector/store chains.
This required a number of fixes - shouldReduceLoadWidth now needs to
peek through bitcasts UP the use-chain to find final users (limited to
hasOneUse cases to reduce complexity). It also exposed an issue in
isTargetCanonicalConstantNode which assumed that a load of vector
constant data would always extract, which is no longer the case.
Current implementation tries to fold the operand before
rematerialization because it can reduce one register usage. But if there
is a physical register available we can still rematerialize it without
causing high register pressure.
This patch do this check to find the better choice. Then we can produce
xorps %xmm1, %xmm1
ucomiss %xmm1, %xmm0
instead of
ucomiss LCPI0_1(%rip), %xmm0
This reverts commit 7ae75851b2e1570662261c97c13cfc65357c283d.
There is a problem with peephole optimization for CCMP instruction. See
the example below:
C source code:
```
if (a > 2 || (b && (a == 2))) { … }
```
MIR before peephole optimization:
```
TEST8rr %21:gr8, %21:gr8, implicit-def $eflags // b
CCMP32ri %30:gr32, 2, 0, 5, implicit-def $eflags, implicit $eflags // a == 2
CCMP32ri %30:gr32, 3, 0, 5, implicit-def $eflags, implicit $eflags // a > 2 (transformed to a < 3)
JCC_1 %bb.6, 2, implicit $eflags
JMP_1 %bb.3
```
Inputs:
```
a = 1, b = 0.
```
With the inputs above, the expected behavior is to jump to %bb.6 BB.
After TEST8rr instruction being executed with b(%21) == 0, the ZF bit is
set to 1 in eflags, so the eflags doesn't satisfy SCC condition in the
following CCMP32ri instruction (for a==2 condition) which skips compare
a(%30) with 2 and set flags in its payload to 0x202 (ZF = 0). The eflags
satisfies the SCC condition in the 2nd CCMP32ri instruction which
compares a(%30) with 3. It sets CF to 1 in eflags and the JCC
instruction jumps to %bb.6 BB.
But after adding CCMP support, peephole optimization eliminates the 2nd
CCMP32ri instruction and updates the condition of JCC instruction to
"BE" from "B". With the same inputs, JCC instruction falls through to
the next instruction. It's not expected and the 2nd CCMP32ri should not
be eliminated.
```
TEST8rr %21:gr8, %21:gr8, implicit-def $eflags // b
CCMP32ri %30:gr32, 2, 0, 5, implicit-def $eflags, implicit $eflags // a == 2
JCC_1 %bb.6, 6, implicit $eflags
JMP_1 %bb.3
```
The problem is that `SrcBC = peekThroughBitcasts(Src)` doesn't ensure
single use chain. It results in the situation when a cast may have
multiple users and instead of replacing a load we introduce a new one.
The situation is worsened by the fact that we've replaced the token from
the original load and its correct memory order now is not guaranteed.
512-bit X86ISD::VPERMI nodes handle the lower/upper 256-bits separately - so if we don't demand the upper half elements, we can just use the 256-bit variant.
addr matching was the only gatekeeper for starting selecting G_LOAD
and G_STORE using SDAG patterns.
* Introduce a complex renderer gi_addr for addr. In this patch only
the existing functionality has been implemented. The renderer's name is
the same as in SDAG: selectAddr. Apparently the type of
GIComplexOperandMatcher doesn't matter as RISCV also uses s32 for
both 64 and 32 bit pointers.
* X86SelectAddress now is used for both: pattern matching and manual
selection. As a result it accumulates all the code that previously was
distributed among different selection functions.
* Replace getLoadStoreOp with getPtrLoadStoreOp in Load/Store
selector as GlobalISel matcher or emitter can't map the pointer type
into i32/i64 types used in SDAG patterns for pointers. So the load and
store selection of pointers is still manual. getLoadStoreOp is still
present because it is used in G_FCONSTANT lowering that requires extra
efforts to select it using SDAG patterns.
* Since truncating stores are not supported, we custom legalize them by
matching types of store and MMO.
* Introduce a constant pool flag in X86AddressMode because otherwise
we need to introduce a GlobalISel copy for X86ISelAddressMode.
* Also please notice in the tests that GlobalISel prefers to fold memory
operands immediately comparing to SDAG. The reason is that GlobalISel
doesn't have target hooks in GIM_CheckIsSafeToFold. Or maybe another
check on profitability is required along with safety check that is
currently not present.
Propagation to poison in function `SDValue
SelectionDAG::getNode(unsigned Opcode, const SDLoc &DL, EVT VT,SDValue
N1, SDValue N2, const SDNodeFlags Flags) ` if one of the input is
poison.
The patch also revert the test cases
llvm/test/CodeGen/X86/pr119158.ll
llvm/test/CodeGen/X86/half.ll
which are mentioned in
https://github.com/llvm/llvm-project/pull/125883#discussion_r2021390919
---------
Co-authored-by: Amy Kwan <amy.kwan1@ibm.com>
Fold an AND or OR of two NaN SETCC nodes into a single SETCC where
possible. This optimization already exists in InstCombine but adding in
here as well can allow for additional folding if more logical operations
are exposed.
#135597 didn't correctly fix the issue of binops with an undef element
from only one operand - only reporting the common undef elements could
incorrectly recognise splats where the (binop X, undef) fold might
actually be different - we need to ensure both operands have the same
demanded undefs for certainty.
Fixes#135917
The 8 and 16 bit LEA instruction support was added by PR #122102, and we
have to update creating index register negations accordingly. The issue
is exposed with APX NDD instructions.
#134602 demonstrated an issue where an AND node always had at least one demanded UNDEF element in either operand, and incorrectly reported this an all-undef result - despite the other element being 0 (so would correctly fold to 0).
This fix only assumes a binops splats element is undefined if both operands are undef.
Fixes#134602
`weightCalcHelper` is responsible for adding hints to MRI. Prior to this
PR, we fell back on register ID as the last tie breaker for sorting
hints. However, there is an opportunity to add an additional sorting
characteristic: whether or not a register is a callee-saved-register.
I thought of this idea because I saw that `AllocationOrder::create`
calls `RegisterClassInfo::getOrder`, which returns a list of registers
such that the registers which alias callee-saved-registers come last.
From this, I conclude that the register allocator prefers an order such
that callee-saved-registers are allocated after
non-callee-saved-registers to avoid having to spill the CSR.
This sorting characteristic occurs only as a tie breaker to the Weight
calculation. This is a good idea since the weight calculation is pretty
complex and I'm sure it is a pretty stable metric. I think its pretty
reasonable to agree that whether a register is callee-saved or not is a
better tie breaker than register ID. I think this is evident by the test
diff, since the changes all seem to have no impact or improve the
register allocation.
Stop just replacing 2*UNPCK+INSERT_SUBVECTOR with 2*INSERT_SUBVECTOR+UNPCK
Currently limited to sub-64-bit element cases until we've accounted for the remaining regressions from some build_vector style patterns.
We already did this for constant cases, this patch generalizes the existing fold to attempt to extract the splat from either operand of a ADD node for the gather/scatter index value
This cleanup should also make it easier to add support for splitting vXi32 indices on x86_64 64-bit pointer targets in the future as well.
Noticed while reviewing #134979
These nodes are effectively free, so we should only concatenate if the
inner nodes will concatenate together.
This also exposed a regression in canonicalizeShuffleWithOp that failed
to realize it could potentially merge shuffles with a CONCAT_VECTORS
node.
