This function gets called for vectors and ISD::SELECT_CC was never
intended to support vectors. Some updates were made to support
it when this function started getting used for vectors.
Overall, using separate ISD::SETCC and ISD::SELECT looks like an
improvement even for scalar.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D149481
This is stricter than the default "ieee", and should probably be the
default. This patch leaves the default alone. I can change this in a
future patch.
There are non-reversible transforms I would like to perform which are
legal under IEEE denormal handling, but illegal with flushing zero
behavior. Namely, conversions between llvm.is.fpclass and fcmp with
zeroes.
Under "ieee" handling, it is legal to translate between
llvm.is.fpclass(x, fcZero) and fcmp x, 0.
Under "preserve-sign" handling, it is legal to translate between
llvm.is.fpclass(x, fcSubnormal|fcZero) and fcmp x, 0.
I would like to compile and distribute some math library functions in
a mode where it's callable from code with and without denormals
enabled, which requires not changing the compares with denormals or
zeroes.
If an IEEE function transforms an llvm.is.fpclass call into an fcmp 0,
it is no longer possible to call the function from code with denormals
enabled, or write an optimization to move the function into a denormal
flushing mode. For the original function, if x was a denormal, the
class would evaluate to false. If the function compiled with denormal
handling was converted to or called from a preserve-sign function, the
fcmp now evaluates to true.
This could also be of use for strictfp handling, where code may be
changing the denormal mode.
Alternative name could be "unknown".
Replaces the old AMDGPU custom inlining logic with more conservative
logic which tries to permit inlining for callees with dynamic handling
and avoids inlining other mismatched modes.
The previous patch (D148980) didn't set the InstrIdxForVirtReg correctly
in genAlternativeDpCodeSequence(). It causes vnni lit test failure when
LLVM_ENABLE_EXPENSIVE_CHECKS is on.
"vpmaddwd + vpaddd" can be combined to vpdpwssd and the latency is
reduced after combination. However when vpdpwssd is in a critical path
the combination get less ILP. It happens when vpdpwssd is in a loop, the
vpmaddwd can be executed in parallel in multi-iterations while vpdpwssd
has data dependency for each iterations. If vpaddd is in a critical path
while vpmaddwd is not, it is profitable to split vpdpwssd into "vpmaddwd
+ vpaddd".
This patch is based on the machine combiner framework to acheive decision
on "vpmaddwd + vpaddd" combination. The typical example code is as
below.
```
__m256i foo(int cnt, __m256i c, __m256i b, __m256i *p) {
for (int i = 0; i < cnt; ++i) {
__m256i a = p[i];
__m256i m = _mm256_madd_epi16 (b, a);
c = _mm256_add_epi32(m, c);
}
return c;
}
```
Differential Revision: https://reviews.llvm.org/D148980
These functions where missing support but are used enough that it
makes sense to track them.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D148963
Replacement for D144903
If we're concatenating freeze(undef) subvector ops with multiple uses then we can't treat them as a wider freeze(undef), but we can replace them with a zero subvector, which is cheap on AVX
Differential Revision: https://reviews.llvm.org/D149249
426db6b4eb2e9298598 added the build_vector(undef,freeze(undef)) -> freeze(undef) fold, but failed to account for cases where the scalar freeze(undef) had multiple uses, in those cases we can only only safely fold to a zero vector
https://alive2.llvm.org/ce/z/87jG8K
This fixes [issue 62242](https://github.com/llvm/llvm-project/issues/62242)
This code block can potentially swap the order of V1 and V2 in Ops and therefore
also in the unpck instruction generated.
```
SDValue &Op = Ops[Elt & 1];
if (M < NumElts && (Op.isUndef() || Op == V1))
Op = V1;
else if (NumElts <= M && (Op.isUndef() || Op == V2)) {
Op = V2;
NormM -= NumElts;
} else
return SDValue();
```
But the permute mask is calculated assuming the first operand being V1 and
second V2, therefore causing a mis-compile.
First check if the input operands are swapped, and then calculate the permute
mask based on that.
Differential Revision: https://reviews.llvm.org/D148843
Similar to the original VPERMILPSri -> VSHUFPSrri mapping added in D143787, replacing VPERMILPDri -> VSHUFPDrri should never be any slower and saves an encoding byte.
The sibling VPERMILPDmi -> VPSHUFDmi mapping is trickier as we need the same shuffle mask in every lane (and it needs to be adjusted) - I haven't attempted that yet but we can investigate it in the future if there's interest.
Fixes#61060
Differential Revision: https://reviews.llvm.org/D148999
This previously existed for `C == 0`, but is mostly beneficial for any
`C`.
There is a slight codesize cost as we get more imm32 (as opposed to
imm8) constants in some cases. But the benefit is was get less imm16
constants (LCP stalls) and save instructions in some vec -> scalar
codegen.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D148594
PSHUFD/PSHUFLW/PSHUFHW can act as a vector move / folded load, notably helping simplify pre-AVX cases in particular.
This is a much milder alternative to refactoring canonicalizeShuffleWithBinOps to support SSE shifts nodes.
Some applications make heavy use of the crc32 operation (e.g., as part
of a hash function), so having a FastISel path avoids fallbacks to
SelectionDAG and improves compile times, in our case by ~1.5%.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D148023
The pass uses ReachingDefAnalysis which has no information about
instructions in dead blocks.
So do not process them.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D148329
The placement is currently wrong in the presence of function entry related
instrumentations (prefixdata, -fpatchable-function-entry=, -fsanitize=kcfi,
etc).
Both Linux and LiteOS for all OpenHarmony targets use emulated TLS
~~~
Huawei RRI, OS Lab
Reviewed By: DavidSpickett, jrtc27, MaskRay
Differential Revision: https://reviews.llvm.org/D145224
We often repeat the vector operand in TESTPS(X,X)/TESTPD(X,X) for anyof comparisons - ensure we still only demand the sign bits when the TESTP is the only user of that operand
`lowerShuffleAsUNPCKAndPermute` requires the shuffle mask element to be
in the same lane in both the input and output vectors. This prevents it from
matching certain patterns for example in [GHI
61964](https://github.com/llvm/llvm-project/issues/61964). Removing the lane
requirement fixes the issue.
The change I'm targeting is in the test llvm/test/CodeGen/X86/pr61964.ll. The
codegen has improved notably with this patch. Otherwise, looks like some
broadcast instructions are replaced with unpck and perm. To check if there's
any other performance change, I ran llvm-test-suite benchmarks from the
SingleSource, MultiSource, and MicroBenchmarks directories:
```
Tests: 2665
Short Running: 2009 (filtered out)
Same hash: 140 (filtered out)
In Blacklist: 513 (filtered out)
Remaining: 3
Metric: exec_time
Program exec_time
lhs rhs diff
test-suite :: MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000.test 1.64 1.64 0.1%
test-suite :: SingleSource/Benchmarks/Adobe-C++/loop_unroll.test 1.06 1.06 0.0%
test-suite :: MultiSource/Applications/JM/lencod/lencod.test 5.25 5.25 0.0%
Geomean difference nan nan 0.0%
exec_time
l/r lhs rhs diff
count 3.000000 3.000000 3.000000
mean 2.648300 2.649100 0.000462
std 2.269035 2.268849 0.000415
min 1.055500 1.055900 0.000095
25% 1.349300 1.350250 0.000237
50% 1.643100 1.644600 0.000379
75% 3.444700 3.445700 0.000646
max 5.246300 5.246800 0.000913
```
The patch only hits three cases and the result is neutral. (The 513 blacklisted
benchmarks are the ones under MicroBenchmarks, which `--filter-hash` does
not work and I manually verified their code did not change).
Differential Revision: https://reviews.llvm.org/D147668
We can probably be more aggressive with TESTPS/TESTPD (instead of relying on the SimplifyMultipleUseDemandedBits call) - I've updated an existing TODO to suggest this for now.
Part of Issue #60007
This shows up in the wild, notably as a regression in D127115 .
Depends on D147821
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D147827
We already allow multiple uses when calling from combineSext/combineZext - adding hasOneUse seems to have been a copy+paste from some of the similar AVX512 vselect folds.
Fixes#62014
