DAGCombiner can fold a chain of INSERT_VECTOR_ELT into a vector AND/OR
operation. This patch adds protection to avoid that we end up making the
vector more poisonous by freezing the source vector when the elements
that should be set to 0/-1 may be poison in the source vector.
The patch also fixes a bug in SimplifyDemandedVectorElts for
MUL/MULHU/MULHS/AND that could result in making the vector more
poisonous. Problem was that we skipped demanding elements from Op0 that
were known to be zero in Op1. But that could result in elements being
simplified into poison when simplifying Op0, and then the result would
be poison and not zero after the MUL/MULHU/MULHS/AND. The solution is to
defensively make sure that we demand all the elements originally
demanded also when simplifying Op0.
This bugs were found when analysing the miscompiles in
https://github.com/llvm/llvm-project/issues/179448
Main culprit in #179448 seems to have been the bug in DAGCombiner. The
bug in SimplifyDemandedVectorElts surfaced when fixing the DAGCombiner,
as that fix typically introduce the (AND (FREEZE x), y) pattern that
wasn't handled correctly in SimplifyDemandedVectorElts.
Also fixes#180409.
Also fixes#176682.
This PR fixes a big-endian regression in `ForwardStoreValueToDirectLoad`
where the wrong subvector was being extracted. In big-endian, memory
offset 0 corresponds to the high bits, so the extraction index needs to
be adjusted.
As suggested by @KennethHilmersson, calculate the extraction index as
the difference between the number of elements in the intermediate vector
and the load vector when in big-endian mode.
Special thanks to Kenneth Hilmersson for providing the fix logic and the
ARM regression test.
https://github.com/llvm/llvm-project/pull/172523#issuecomment-3878065191https://github.com/llvm/llvm-project/pull/172523#issuecomment-3879575092
Similar to other portings created by @aidenboom154. No specific test
coverage as there are no MIR->MIR tests that exercise this pass. Going
with other naming conventions, I renamed WaitInsert to
X86InsertX87WaitLegacy
This lowers the splitting as:
```
any_active(hi_mask)
? (find_last_active(hi_mask) + lo_mask.getVectorElementCount())
: find_last_active(lo_mask)
```
And trivially lowers `<1 x i1>` scalarization to returning zero. Which
is a natural result of the splitting (and the lack of a sentinel
"none-active" result value).
The lowerings likely can be improved. This patch is for completeness.
Should fix:
https://github.com/llvm/llvm-project/pull/178862#issuecomment-3862310334Fixes#180212
If the scalar integer selection sources are freely transferable to the
FPU, then splat to create an allbits select condition and create a
vector select instead
This patch adds initial support for AMD Zen 6 architecture (znver6):
- Added znver6 CPU target recognition in Clang and LLVM
- Updated compiler-rt CPU model detection for znver6
- Added znver6 to target parser and host CPU detection
- Added znver6 to various optimizer tests
znver6 features: FP16, AVXVNNIINT8, AVXNECONVERT, AVXIFMA (without BMM).
The llvm.cond.loop intrinsic is semantically equivalent to a conditional
branch conditioned on ``pred`` to a basic block consisting only of an
unconditional branch to itself. Unlike such a branch, it is guaranteed
to use specific instructions. This allows an interrupt handler or
other introspection mechanism to straightforwardly detect whether
the program is currently spinning in the infinite loop and possibly
terminate the program if so. The intent is that this intrinsic may
be used as a more efficient alternative to a conditional branch to
a call to ``llvm.trap`` in circumstances where the loop detection
is guaranteed to be present. This construct has been experimentally
determined to be executed more efficiently (when the branch is not taken)
than a conditional branch to a trap instruction on AMD and older Intel
microarchitectures, and is also more code size efficient by avoiding the
need to emit a trap instruction and possibly a long branch instruction.
On i386 and x86_64, the infinite loop is guaranteed to consist of a short
conditional branch instruction that branches to itself. Specifically,
the first byte of the instruction will be between 0x70 and 0x7F, and
the second byte will be 0xFE.
Part of this RFC:
https://discourse.llvm.org/t/rfc-optimizing-conditional-traps/89456
Reviewers: arsenm, RKSimon, fmayer, vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/177686
https://github.com/llvm/llvm-project/pull/178333 updates the memprof
pass to annotate string literal section prefix.
The StaticDataProfileInfo.cpp provides an analysis pass to reconcile
global variable hotness. It's used by StaticDataAnnotator and AsmPrinter
to look up global variable hotness.
This PR updates the analysis pass to compute the hotness of string
literals.
* When both data access profiles and pgo counters provide a hotness
attribute, use the hotter one.
* Otherwise, use the hotness attribute that's available.
Implementation-wise, the option `AnnotateStringLiteralSectionPrefix` is
moved from MemProf (a transform pass) to StaticDataProfileInfo (an
Analysis pass). Otherwise, there might be errors like caught by CI. Note
https://github.com/llvm/llvm-project/pull/178336#issuecomment-3808537817
is an edited message, and its history shows the intermediate failures
like below. ~My understanding is~ Preliminary LLM study (:)) shows that
the error manifests in PowerPC but not X86 due to cmake variable
differences.
```
FAILED: unittests/Target/PowerPC/PowerPCTests
...
>>> referenced by CommandLine.h:1437 (/home/gha/actions-runner/_work/llvm-project/llvm-project/llvm/include/llvm/Support/CommandLine.h:1437)
>>> StaticDataProfileInfo.cpp.o:(llvm::StaticDataProfileInfo::getConstantSectionPrefix(llvm::Constant const*, llvm::ProfileSummaryInfo const*) const) in archive lib/libLLVMAnalysis.a
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
```
This patch adds -filetype=null to the llc RUN lines in pr142937.ll, to
ensure that the test does not attempt to write files to the local
directory.
Follow-up to #179249.
This patch fixes a regression introduced by PR #175022, where
a freeze was introduced with the following transformation:
ext(freeze(load(x))) -> freeze(extload(x))
If a new extend is introduced afterwards we then have
ext(freeze(extload(x)))
which doesn't get picked up by existing DAG combines due to
the freeze getting in the way.
Previously, the DAG combiner did not optimize exact signed division by a
power-of-two constant divisor for integer types exceeding the size of
division supported by the target architecture (e.g., i128 on x86-64).
However, such an optimization was expected by the division expansion
logic, leading to unsupported division operations making it to
instruction selection.
This commit addresses this issue by making an exception to the existing
exclusion of signed division with the exact flag for the aforementioned
operations. That is, the DAG combiner will now optimize exact signed
division if the divisor is a power-of-two constant and the integer type
exceeds the size of division supported by the target architecture.
---------
Signed-off-by: Steffen Holst Larsen <HolstLarsen.Steffen@amd.com>
combineVectorSizedSetCCEquality attempts to convert equality comparisons
of larger-than-legal scalar integers to PTEST/PMOVMSKB vector
comparisons.
However, combineSetCC has a number of other folds with more complex
icmp_eq/ne patterns that work with big integers (including bit test and
reduction patterns) that don't get a change to match as
combineVectorSizedSetCCEquality is run first, and the other folds are
then more difficult to match from PTEST/PMOVMSKB nodes.
This patch moves the combineVectorSizedSetCCEquality fold later to give
other icmp_eq/ne folds a chance to run first.
Port x86-cleanup-local-dynamic-tls to the new pass manager.
I moved LDTLSCleanup to a new
llvm/lib/Target/X86/X86CleanupLocalDynamicTLS.cpp file and renamed it to
X86CleanupLocalDynamicTLSPass. Then I renamed the legacy pass, and
adding a NewPM wrapper around the impl.
No test coverage added for now as there are no MIR->MIR tests exercising
this pass and we do not have enough ported to run any end to end tests.
Convert "denormal-fp-math" and "denormal-fp-math-f32" into a first
class denormal_fpenv attribute. Previously the query for the effective
denormal mode involved two string attribute queries with parsing. I'm
introducing more uses of this, so it makes sense to convert this
to a more efficient encoding. The old representation was also awkward
since it was split across two separate attributes. The new encoding
just stores the default and float modes as bitfields, largely avoiding
the need to consider if the other mode is set.
The syntax in the common cases looks like this:
`denormal_fpenv(preservesign,preservesign)`
`denormal_fpenv(float: preservesign,preservesign)`
`denormal_fpenv(dynamic,dynamic float: preservesign,preservesign)`
I wasn't sure about reusing the float type name instead of adding a
new keyword. It's parsed as a type but only accepts float. I'm also
debating switching the name to subnormal to match the current
preferred IEEE terminology (also used by nofpclass and other
contexts).
This has a behavior change when using the command flag debug
options to set the denormal mode. The behavior of the flag
ignored functions with an explicit attribute set, per
the default and f32 version. Now that these are one attribute,
the flag logic can't distinguish which of the two components
were explicitly set on the function. Only one test appeared to
rely on this behavior, so I just avoided using the flags in it.
This also does not perform all the code cleanups this enables.
In particular the attributor handling could be cleaned up.
I also guessed at how to support this in MLIR. I followed
MemoryEffects as a reference; it appears bitfields are expanded
into arguments to attributes, so the representation there is
a bit uglier with the 2 2-element fields flattened into 4 arguments.
If a EXPAND node mask is just the lowest bits, then we can replace it
with a more general SELECT node, which can be cheaper and potentially
allow predication.
Fixes#179008
The vgf2p8affineqb instruction performs an affine transformation on each
byte and then XORs the result with an 8-bit immediate operand. When this
instruction is followed by a standalone XOR with a splatted constant,
LLVM currently generates extra instructions instead of folding the
constant into the instruction's immediate.
This PR adds a DAG combine optimization that detects the pattern
vgf2p8affineqb(x, m, imm8) ^ C where C is a splatted 8-bit constant and
transforms it to vgf2p8affineqb(x, m, imm8 ^ C), eliminating the
unnecessary XOR instruction.
- The optimization runs during the combine phase after type legalization
- Handles XOR with the constant on either side (commutative)
- Only applies when the GFNI instruction has a single use to avoid
de-optimization
- Validates that the XOR operand is a splatted 8-bit constant before
folding
- Includes test coverage for positive cases and negative cases
(multi-use, non-splat constant, variable XOR)
CTTZ can be lowered using GF2P8AFFINEQB if we isolate the lowest set bit
(or is zero) and then use GF2P8AFFINEQB to perform a look up
With CTTZ, CTLZ can be lowered as CTTZ(BITREVERSE())
As discussed on #110308
106549a436/src/pnpbios.c (L70)
has code that looks like
```
extern int X;
unsigned f(void) { return (unsigned)&X - 0xf0000; }
```
which compiles (-O1 or above, -fno-pic) to
```
.code16
leal X-983040, %eax # R_386_16 relocation, [-65535,65535]
```
in 16-bit mode. This is rejected by both GNU Assembler and LLVM
integrated assembler (after #176827).
Restrict offset folding in 16-bit mode to the range [-65535, 65535].
This patch adds post legalize combiner for X86 Target.
Use case for OptNone Combine: I am curious about OptNone usecase,
specifically when we are using -On on command line and no opt on
specific function.
The PUSH2/POP2/PPX instructions for APX require updates to the Microsoft
Windows OS x64 calling convention documented at
https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170
due to lack of suitable unwinder opcodes that can support APX
PUSH2/POP2/PPX.
The PR request disables this support by default for code robustness;
workloads that choose to explicitly enable this support can change the
default behavior by explicitly specifying the flag options that enable
this support e.g. for experimentation or code paths that do not need
unwinder support.
TLI.isBinOp recognises some opcodes that have multiple results,
including UADDO etc.
In most cases we currently just bail if a binop has multiple results,
but shuffle combining was missing the check and its pretty trivial to
add handling in this case.
I've added add/sub-overflow opcodes to verifyNode to help catch these
cases in the future - IIRC there was a plan to autogen these, but there
isn't anything at the moment.
Fixes#179112
