This patch updates several functions in LLVM's IR generation code to accept
an IRBuilder object as an argument, rather than an Instruction that indicates
the insertion point for new instructions.
This change is necessary to handle sophisticated -Ofast optimization cases
from D148558 where it's unclear which instructions should be used as the
insertion point for new operations.
Differential Revision: https://reviews.llvm.org/D148703
Update collectLoopUniforms to identify uniform pointers using
Legal::isUniform. This is more powerful and brings pointer
classification here in sync with setCostBasedWideningDecision
which uses isUniformMemOp. The existing mis-match in reasoning
can causes crashes due to D134460, which is fixed by this patch.
Fixes https://github.com/llvm/llvm-project/issues/60831.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D150991
This is pretty straight forward in the basic form. I did need to move the slideup matching earlier, but that looks generally profitable on it's own.
As follow ups, I plan to explore the v(f)slide1down variants, and see what I can do to canonicalize the shuffle then insert pattern (see _inverse tests at the end of the vslide1up.ll test).
Differential Revision: https://reviews.llvm.org/D151468
Create SchedWrites, WriteRes for reduction instructions that
are SEW specific. Future patches can use these resources
to customize the behavior of these resources depending on SEW.
Differential Revision: https://reviews.llvm.org/D151470
This fold is buggy if the constant adjustment overflows.
Additionally, since we now canonicalize to min/max intrinsics,
the constants picked here don't actually matter, as long as SPF
still recognizes the pattern.
Fixes https://github.com/llvm/llvm-project/issues/62088.
As raised on Issue #62602 - the IceLake scheduler model is still mainly a copy of the SkylakeServer model. This initial commit just fixes the ROB/MicroOpBufferSize to match the size reported on WikiChip/Agner, further fixes to follow in later commits.
As reported on https://reviews.llvm.org/D150375#4367861 and
following, this change causes PDT invalidation issues. Revert
it and dependent commits.
This reverts commit 0524534d5220da5ecb2cd424a46520184d2be366.
This reverts commit ced90d1ff64a89a13479a37a3b17a411a3259f9f.
This reverts commit 9f992cc9350a7f7072a6dbf018ea07142ea7a7ed.
This reverts commit 1b1232047e83b69561fd64b9547cb0a0d374473a.
lowerBuildVectorAsBroadcast will not broadcast splat constants in all cases, resulting in a lot of situations where a full width vector load that has failed to fold but is loading splat constant values could use a broadcast load instruction just as cheaply, and save constant pool space.
This patch adds four new tests for upcoming functionality in LLVM:
* complex-deinterleaving-add-mull-fixed-contract.ll
* complex-deinterleaving-add-mull-scalable-contract.ll
* complex-deinterleaving-add-mull-fixed-fast.ll
* complex-deinterleaving-add-mull-scalable-fast.ll.
These tests were generated from the IR of vectorizable loops, which were
compiled from C++ code using different optimization flags in Clang. Each pair
of tests corresponds to Neon and SVE architectures, respectively, and
each pair contains tests compiled with -Ofast and -O3 -ffp-contract=fast
-ffinite-math-only optimization flags.
The tests were stripped of nnan and ninf flags as they have no impact on the
output.
The primary objective of these tests is to show the various sequences of
complex computations that may be encountered and to demonstrate the ability
of ComplexDeinterleaving to support any ordering.
Depends on D147451
Differential Revision: https://reviews.llvm.org/D148550
Fixes issue introduced by 0f8e0f4228805cbecce13dcfadef4c48a4f0f4cd where SimplifyDemandedBits could crash when trying to extract fp data from broadcasted constants
We try to fold constant computeKnownBits() with context for return
instructions only. Otherwise, we rely on SimplifyDemandedBits() to
fold instructions with constant known bits.
The presence of this special fold for returns is dangerous, because
it makes our tests lie about what works and what doesn't. Tests are
usually written by returning the result we're interested in, but
will go through this separate code path that is not used for anything
else. This patch removes the special fold.
This primarily regresses patterns of the style "assume(x); return x".
The responsibility of handling such patterns lies with passes like
EarlyCSE/GVN anyway, which will do this reliably, and not just for
returns.
Differential Revision: https://reviews.llvm.org/D151099
This results in improved codegen for half/bf16 libcalls on soft ABIs
Adds a RISCVSubtarget helper method for determining if a soft FP ABI is
being targeted (future bf16 related patches make use of this).
Differential Revision: https://reviews.llvm.org/D151434
This moves memintrinsic interceptors (memcpy/memmove/memset) into a new
file sanitizer_common_interceptors_memintrinsics.inc.
This is in preparation of redefining builtins, however, we must be
careful to not redefine builtins in TUs that define interceptors of the
same name.
In all cases except for MSan, memintrinsic interceptors were moved to a
new TU $tool_interceptors_memintrinsics.cpp. In the case of MSan, it
turns out this is not yet necessary (as shown by the later patch
introducing memcpy tests).
NFC.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D151552
.Imm can get lexed as a real, but a real doesn't equal to .Imm, e.g.,
2.5 or .123e+8. We should report error for it rather than silently ignore.
Reviewed By: skan
Differential Revision: https://reviews.llvm.org/D151652
Previously we only handled instructions with merge ops that were
also masked. This patch supports instructions with merge ops that
aren't masked, like FMA.
I'm only folding into a TU vmerge for now. Supporting TA vmerge
shouldn't be much more work, but we need to make sure we get the
policy operand for the result correct. And of course we need more
tests.
Reviewed By: fakepaper56, frasercrmck
Differential Revision: https://reviews.llvm.org/D151596
Follow-up to da54bd230a.
* Add dep to _cg and _sema targets only to CodeGen and Sema,
like with the other Basic clang_tablegen()s
* Make tablegen_headers depend on arm_sme_draft_spec_subject_to_change
so that the header gets installed
Extend test coverage for and conditions where the first operand implies
the second. The update includes the select form of AND and a few cases
where one of the AND operands gets simplified before solving.
The last use was removed by:
commit d5b840131223f2ffef4e48ca769ad1eb7bb1869a
Author: Philip Reames <preames@rivosinc.com>
Date: Thu May 11 08:10:49 2023 -0700
The corresponding function definitions were removed by:
commit 14e773500e036de57ed0ca4af6fddc1f8b6767d8
Author: Reid Kleckner <rnk@google.com>
Date: Fri Oct 9 23:34:53 2015 +0000
The last use was removed by:
commit e760ec2a01fba0d90e2fea33cf99b75baa2c2a1c
Author: Arnold Schwaighofer <aschwaighofer@apple.com>
Date: Thu Feb 11 08:25:31 2021 -0800
The declaration was added without a corresponding function definition
by:
commit 217fd59cc5849f1afdbb8a74807a80b23f2da227
Author: Lang Hames <lhames@gmail.com>
Date: Thu Jan 12 23:17:46 2023 -0800
The declaration was added without a corresponding function definition
by:
commit cb84e4827e43921659e75509dfb42ebf56c50502
Author: Lang Hames <lhames@gmail.com>
Date: Wed Mar 25 13:07:00 2020 -0700
The declaration was added without a corresponding function definition
by:
commit d8e077e2caebc1415fb7af1714dd436adf99b6bf
Author: Greg Clayton <gclayton@fb.com>
Date: Mon Mar 6 15:57:17 2023 -0800
The declaration was added without a corresponding function definition
by:
commit 4c52fb1a5ee20846627d16e38f5dec08c08f8884
Author: Vladislav Dzhidzhoev <vdzhidzhoev@accesssoftek.com>
Date: Mon Aug 15 21:51:13 2022 +0300
The corresponding function definition was removed by:
commit 85d6a16c46616336c52792e7a490e70ea8b6461e
Author: Tim Northover <tnorthover@apple.com>
Date: Fri Apr 4 09:03:09 2014 +0000
Previously, getConstantOffset could return an APInt with a different
bitwidth than the input pointers. For example, we might be loading an
opaque 64-bit pointer, but stripAndAccumulateInBoundsConstantOffsets
might give a 32-bit offset.
This was OK in most cases because in gatherChains, we casted the APInt
back to the original ASPtrBits.
But it was not OK when considering selects. We'd call getConstantOffset
twice and compare the resulting APInt's, which might not have the same
bit width.
This fixes that. Now getConstantOffset always returns offsets with the
correct width, so we don't need the hack of casting it in gatherChains,
and it works correctly when we're handling selects.
Differential Revision: https://reviews.llvm.org/D151640
lowerBuildVectorAsBroadcast will not broadcast splat constants in all cases, resulting in a lot of situations where a full width vector load that has failed to fold but is loading splat constant values could use a broadcast load instruction just as cheaply, and save constant pool space.
NOTE: SSE3 targets can use MOVDDUP but not all SSE era CPUs can perform this as cheaply as a vector load, we will need to add scheduler model checks if we want to pursue this.
This reverts commit 754f3ae65518331b7175d7a9b4a124523ebe6eac.
Unfortunately the change can cause regressions due to dropping flags
from instructions (like nuw,nsw,inbounds), prevent further optimizations
depending on those flags.
A simple example is the IR below, where `inbounds` is dropped with the
patch and the phase-ordering test added in 7c91d82ab912fae8b.
define i1 @test(ptr %base, i64 noundef %len, ptr %p2) {
bb:
%gep = getelementptr inbounds i32, ptr %base, i64 %len
%c.1 = icmp uge ptr %p2, %base
%c.2 = icmp ult ptr %p2, %gep
%select = select i1 %c.1, i1 %c.2, i1 false
ret i1 %select
}
For more discussion, see D149404.
As discussed in D151436, it's safe to do this as a simple shift (as is
done in LegalizeDAG.cpp) rather than needing a libcall. The added test
cases for RISC-V previously just triggered an assertion.
Codegen for bfloat_to_double will be slightly improved by D151434.
Differential Revision: https://reviews.llvm.org/D151563
