Almost all of the other SVE LLVM IR intrinsics take i32 values
for lane indices or other immediates. We should bring the bfloat
intrinsics in line with that. It will also make it easier to
add support for the SVE2.1 float intrinsics in future, since
they reuse the same underlying instruction classes.
I've maintained backwards compatibility with the old i64 variants
and used the autoupgrade mechanism.
Differential Revision: https://reviews.llvm.org/D138788
We already collect all instructions that need to be promoted. The
custom isInstInList() implementation could provide incorrect
results if a new use of the original pointer was introduced as
part of promotion. This probably cannot happen with normal code,
because of the pointer capture, but it can happen with a null
pointer.
Fixes https://github.com/llvm/llvm-project/issues/59324.
Add the default unoptimized implementation implementation
of hlfir.assign. It relies on the runtime for array assignment
and always makes a temp of the right hand side for arrays.
Assignment optimization will be done when all HLFIR pieces are in place
and aliasing analysis is available.
Differential Revision: https://reviews.llvm.org/D139426
We've exploited test data class instructions introduced in ISA 3.0.
This change unifies the scalar intrinsics into ppc_test_data_class
and add support for 128-bit precision float values using xststdcqp.
Vector versions of the intrinsic can't be unified because they return
vector int instead of int.
Reviewed By: shchenz
Differential Revision: https://reviews.llvm.org/D138105
Preserves current behavior (always select Exact if known, otherwise select Constant Max).
This is the final preparation step before letting each particular computation way to decide
how exactly it should be computed. Functional improvement is coming shortly as follow-up.
Differential Revision: https://reviews.llvm.org/D139402
Reviewed By: nikic, fhahn
s_endpgm is a special SOPP instruction in that its operand is optional
and if it is not present then we don't want to print a space after the
mnemonic.
Previously this was handled by defaulting real_name to the mnemonic with
a trailing space, and having s_endpgm override it to be the mnemonic
with no trailing space.
This patch implements a different approach where the separator between
Mnemonic and AsmOperands defaults to a space, but s_endpgm overrides it
to be the empty string.
Differential Revision: https://reviews.llvm.org/D139412
This reverts commit 7883e5b061bdbbe8bee5f479ebe911db5045b7e9.
The original commit was reverted that it didn't update test files after D136263
landed. The recommit fixed those.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D139509
Correct the pseudo atomic instruction size for branch
relaxation and branch folding passes.
Inspired by D118175, D118009 and D117970.
Depends on D138481
Reviewed By: SixWeining, gonglingqin, xen0n
Differential Revision: https://reviews.llvm.org/D138469
We were only allowing these med3 patterns if the operands were known
to not be NaN, but we should also allow it if the calls to max/min
have the `nnan` or `fast` flags.
Differential Revision: https://reviews.llvm.org/D139506
Let Propeller use specialized IDs for basic blocks, instead of MBB number.
This allows optimizations not just prior to asm-printer, but throughout the entire codegen.
This patch only implements the functionality under the new `LLVM_BB_ADDR_MAP` version, but the old version is still being used. A later patch will change the used version.
####Background
Today Propeller uses machine basic block (MBB) numbers, which already exist, to map native assembly to machine IR. This is done as follows.
- Basic block addresses are captured and dumped into the `LLVM_BB_ADDR_MAP` section just before the AsmPrinter pass which writes out object files. This ensures that we have a mapping that is close to assembly.
- Profiling mapping works by taking a virtual address of an instruction and looking up the `LLVM_BB_ADDR_MAP` section to find the MBB number it corresponds to.
- While this works well today, we need to do better when we scale Propeller to target other Machine IR optimizations like spill code optimization. Register allocation happens earlier in the Machine IR pipeline and we need an annotation mechanism that is valid at that point.
- The current scheme will not work in this scenario because the MBB number of a particular basic block is not fixed and changes over the course of codegen (via renumbering, adding, and removing the basic blocks).
- In other words, the volatile MBB numbers do not provide a one-to-one correspondence throughout the lifetime of Machine IR. Profile annotation using MBB numbers is restricted to a fixed point; only valid at the exact point where it was dumped.
- Further, the object file can only be dumped before AsmPrinter and cannot be dumped at an arbitrary point in the Machine IR pass pipeline. Hence, MBB numbers are not suitable and we need something else.
####Solution
We propose using fixed unique incremental MBB IDs for basic blocks instead of volatile MBB numbers. These IDs are assigned upon the creation of machine basic blocks. We modify `MachineFunction::CreateMachineBasicBlock` to assign the fixed ID to every newly created basic block. It assigns `MachineFunction::NextMBBID` to the MBB ID and then increments it, which ensures having unique IDs.
To ensure correct profile attribution, multiple equivalent compilations must generate the same Propeller IDs. This is guaranteed as long as the MachineFunction passes run in the same order. Since the `NextBBID` variable is scoped to `MachineFunction`, interleaving of codegen for different functions won't cause any inconsistencies.
The new encoding is generated under the new version number 2 and we keep backward-compatibility with older versions.
####Impact on Size of the `LLVM_BB_ADDR_MAP` Section
Emitting the Propeller ID results in a 23% increase in the size of the `LLVM_BB_ADDR_MAP` section for the clang binary.
Reviewed By: tmsriram
Differential Revision: https://reviews.llvm.org/D100808
The Zfhmin subset only has FLH, FSH, FMV.X.H, FMV.H.X, FCVT.S.H, and FCVT.H.S.
If the D extension is present, the FCVT.D.H and FCVT.H.D instructions are also included.
Since most instructions are not included for Zfhmin, so most operations are promoted.
The patch primarily about making f16 a legal type.
RISC-V ISA info:
https://wiki.riscv.org/display/HOME/Recently+Ratified+Extensions
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D139391
Adding a switch to drop profile symbol list during merging AutoFDO profiles. This is needed to minimize the impact on default profiles when the profile symbol list is enabled for the source input profiles. The symbol list is quite large and could potentially slow down the compiler.
Reviewed By: davidxl, wenlei
Differential Revision: https://reviews.llvm.org/D139486
The patch made VectorLegalizer expand ISD::VP_FSHL and ISD::VP_FSHR to
achieve the codegen.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D138379
LValuePathSerializationHelper's type properly
Close https://github.com/llvm/llvm-project/issues/58716.
Tested with libcxx's modules build.
When we read the type of
LValuePathSerializationHelper, we didn't read the correct type. We read
the element type as its name suggests. But the problem here is that it
looks like that both the usage and serialization use its type as the
top level type. So here is the mismatch.
Actually, the type of LValuePathSerializationHelper is never used after
Deserialization without the assertion. So it doesn't matter for the
release users. And this patch shouldn't change the behavior too.
Reviewed By: erichkeane
Differential Revision: https://reviews.llvm.org/D139406
I'll be making changes to the ObjC implementation (bug fix) in a later
change, and I noticed that this test isn't being tested at all. Fix that up
here separately so further changes are easier to differentiate.
Reviewed By: #lld-macho, thakis, int3
Differential Revision: https://reviews.llvm.org/D139396
This just adds a test. With CSE of single block ops, and other
previously landed changes, this works at HEAD. Just adding a test that
triggered this line of work that I missed adding.
Differential Revision: https://reviews.llvm.org/D139385
Bionic's sys/user.h declares a "struct user". The header tends to be
included, and when it is, it conflicts with "namespace user". Rename
user to User.
Differential Revision: https://reviews.llvm.org/D139380
This covers `SDot`, `SUDot`, and `UDot`. The `*AccSat` version will be
added in a follow-up revision.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D139242
Since changes to account for LMUL in scheduler model existed over patches, we had to keep
both LMUL specific and all LMUL classes around. Now that only the LMUL specific
classes are used, we can remove the old ones.
It is likley that subtargets act differently for a vector load store instructions based on the LMUL.
This patch creates seperate SchedRead, SchedWrite, WriteRes, ReadAdvance for each relevant LMUL.
Differential Revision: https://reviews.llvm.org/D137429
It is likley that subtargets act differently for vector fixed-point arithmetic instructions
based on the LMUL. This patch creates seperate SchedRead, SchedWrite, WriteRes, ReadAdvance
for each relevant LMUL.
Differential Revision: https://reviews.llvm.org/D137428
Update the sort order to make sure that comesBefore is never used from
conditional facts, which are instructions but may use DFS numbers from
different blocks.
This fixes a crash in the added test on some platforms.
We need a scratch GPR to increment the base pointer for each subsequent
register. We currently reuse the input GPR for the base pointer without
declaring it as a Def of the pseudo.
We can't add it as a Def of the pseudo at creation time because it doesn't
get register allocated. This was tried in D109405.
Seems the only choice we have is to scavenge the GPR. This patch
moves the expansion to eliminateFrameIndex where we can create
virtual registers that will be scavenged. This also eliminates the
extra operand for passing vlenb from frame lowering to expand pseudos.
I need to do more testing on real world code, but wanted to get this
up for early review.
I hope this will fix the issue reported in D123394, but I haven't
checked yet.
Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D139169
Make the types consistent with each other, and return size_t instead of a long int.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D126121
