I've kept the legalForCartesianProduct call, but this requires us to maintain 32-bit/64-bit integer lists - we might want to just use legalIf and perform the type pair set matching manually.
I've kept the legalForCartesianProduct call, but this requires us to maintain 32-bit/64-bit integer lists - we might want to just use legalIf and perform the type pair set matching manually.
Replace the legacy G_MUL legalizer, this handles all scalar promotion and vector clamping, however we still need to add custom legalization for many vector multiplies.
This patch adds logic for determining RegisterBank size to RegisterBankInfo, which allows accounting for the HwMode of the target. Individual RegisterBanks cannot be constructed with HwMode information as construction is generated by TableGen, but a RegisterBankInfo subclass can provide the HwMode as a constructor argument. The HwMode is used to select the appropriate RegisterBank size from an array relating sizes to RegisterBanks.
Targets simply need to provide the HwMode argument to the <target>GenRegisterBankInfo constructor. The RISC-V RegisterBankInfo constructor has been updated accordingly (plus an unused argument removed).
Reviewed By: simoncook, craig.topper
Differential Revision: https://reviews.llvm.org/D76007
Replace the legacy legalizers for G_ADD/G_SUB with a pattern that should work for the full mixture of scalar/vector types we need to support for most opcodes.
If we can decide on this as a suitable pattern we can reuse it for the other op types, but G_ADD/G_SUB are the only ones that have decent legalizer test coverage at this stage.
Differential Revision: https://reviews.llvm.org/D151878
The X86SchedSapphireRapids.td file is automatically generated by
schedtool (D130897). Most of instruction's scheduling information
is from SapphireRapids tpt/lat data provided by intel doc. Some data
is from measured ADL-P data in uops.info. The rest instruction's
scheduling information is from skylake server schedule model in order
to get a relative complete model.
Reviewed By: LuoYuanke
Differential Revision: https://reviews.llvm.org/D141485
BOLT may add an extra MCOperand as annotation, making the number of operands in
MCInst different from MCInstrDesc information. A recent change broke our use as
the last operand is no longer a cond code.
Partially revert 287dd0142c08caad5b32628f3842dfd5d5c59d03 to rely on MCInstrDesc
to access CondCode operand.
Reviewed By: skan
Differential Revision: https://reviews.llvm.org/D151930
Sometimes an developer would like to have more control over cmov vs branch. We have unpredictable metadata in LLVM IR, but currently it is ignored by X86 backend. Propagate this metadata and avoid cmov->branch conversion in X86CmovConversion for cmov with this metadata.
Example:
```
int MaxIndex(int n, int *a) {
int t = 0;
for (int i = 1; i < n; i++) {
// cmov is converted to branch by X86CmovConversion
if (a[i] > a[t]) t = i;
}
return t;
}
int MaxIndex2(int n, int *a) {
int t = 0;
for (int i = 1; i < n; i++) {
// cmov is preserved
if (__builtin_unpredictable(a[i] > a[t])) t = i;
}
return t;
}
```
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D118118
Adds a dynamic stack alignment to functions under the interrupt call
convention on x86-32. This fixes the issue where the stack can be
misaligned on entry, since x86-32 makes no guarantees about the stack
pointer position when the interrupt service routine is called.
The alignment is done by overriding X86RegisterInfo::shouldRealignStack,
and by setting the correct alignment in X86FrameLowering::calculateMaxStackAlign.
This forces the interrupt handler to be dynamically aligned, generating
the appropriate `and` instruction in the prologue and `lea` in the
epilogue. The `no-realign-stack` attribute can be used as an opt-out.
Fixes#26851
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D151400
In cases where a broadcast op is loading from a constant entry wider than the broadcast element, we were incorrectly printing the entire entry and not just the lower bits referenced by the broadcast.
We might be broadcasting the lower portion of a wider constant pool entry, ensure we truncate the full width of the constant back to the broadcast width.
(This also exposes an issue with how asm comments print these constants - I'll address that in a followup).
Fixes#63030
As raised on Issue #62602 - the IceLake scheduler model is still mainly a copy of the SkylakeServer model. This initial commit just fixes the ROB/MicroOpBufferSize to match the size reported on WikiChip/Agner, further fixes to follow in later commits.
lowerBuildVectorAsBroadcast will not broadcast splat constants in all cases, resulting in a lot of situations where a full width vector load that has failed to fold but is loading splat constant values could use a broadcast load instruction just as cheaply, and save constant pool space.
Fixes issue introduced by 0f8e0f4228805cbecce13dcfadef4c48a4f0f4cd where SimplifyDemandedBits could crash when trying to extract fp data from broadcasted constants
.Imm can get lexed as a real, but a real doesn't equal to .Imm, e.g.,
2.5 or .123e+8. We should report error for it rather than silently ignore.
Reviewed By: skan
Differential Revision: https://reviews.llvm.org/D151652
lowerBuildVectorAsBroadcast will not broadcast splat constants in all cases, resulting in a lot of situations where a full width vector load that has failed to fold but is loading splat constant values could use a broadcast load instruction just as cheaply, and save constant pool space.
NOTE: SSE3 targets can use MOVDDUP but not all SSE era CPUs can perform this as cheaply as a vector load, we will need to add scheduler model checks if we want to pursue this.
D150436 changed the order for INC*r, but unintentionally remove 'isConvertibleToThreeAddress = 1' for INC*r_alt.
This patch also update wrong comment X86InstrArithmetic.td, and move class ITy to InstrInfo.td for it's a general class.
Reviewed By: skan
Differential Revision: https://reviews.llvm.org/D151527
Noticed in D150143/D150526 - we currently create scalar Constant values using the broadcast instruction width, which might be wider than the original build vector width, making it tricky to recognise the original constant bits data.
If we have widened the broadcast value, its much more useful for asm comments if we create a ConstantVector with the original element data, add that to the constant-pool and load that with the same (wider) broadcast instruction.
When avx512 is available the lhs operand of select instruction can be
folded with mask instruction, while the rhs operand can't. This patch is
to commute the lhs and rhs of the select instruction to create the
opportunity of folding.
Differential Revision: https://reviews.llvm.org/D151535
Note that the builders are protected by is64Bit().
More fine-grained availibility checks.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D150790
Without frame pointers, the locations of variables on the stack are emitted
relative to the stack pointer (via the stack pointer being the value of
DW_AT_frame_base on the subprogram). If a call modifies the stack pointer
this results in the locations being wrong and the debugger displaying the
wrong values for variables.
By using DW_OP_call_frame_cfa in these situations the emitted location for
the variable will automatically handle changes in the stack pointer
(provided LLVM is emitting the correct CFI directives elsewhere, of course).
The CFA needs to be adjusted for the size of the stack frame (including the
return address) to allow the variable locations themselves to remain
unchanged by this patch.
Certain LLDB features cannot cope with DW_OP_call_frame_cfa, so this change
is heuristically limited to the cases where it's necessary for correctness
to minimize the fallout there.
Reviewed By: #debug-info, scott.linder, jryans, jmorse
Differential Revision: https://reviews.llvm.org/D143463
There are two motivations.
`-fno-pic -fstack-protector -mstack-protector-guard=global` created
`__stack_chk_guard` is referenced directly on all ELF OSes except FreeBSD.
This patch allows referencing the symbol indirectly with
-fno-direct-access-external-data.
Some Linux kernel folks want
`-fno-pic -fstack-protector -mstack-protector-guard-reg=gs -mstack-protector-guard-symbol=__stack_chk_guard`
created `__stack_chk_guard` to be referenced directly, avoiding
R_X86_64_REX_GOTPCRELX (even if the relocation may be optimized out by the linker).
https://github.com/llvm/llvm-project/issues/60116
Why they need this isn't so clear to me.
---
Add module flag "direct-access-external-data" and set the dso_local property of
the stack protector symbol. The module flag can benefit other LLVMCodeGen
synthesized symbols that are not represented in LLVM IR.
Nowadays, with `-fno-pic` being uncommon, ideally we should set
"direct-access-external-data" when it is true. However, doing so would require
~90 clang/test tests to be updated, which are too much.
As a compromise, we set "direct-access-external-data" only when it's different
from the implied default value.
Reviewed By: nickdesaulniers
Differential Revision: https://reviews.llvm.org/D150841
This patch analyzes AVX512 instructions for full vector width folded loads from the constant pool and attempts to determine if it can be replaced with a smaller broadcast folded variant. Typically the broadcast opportunities were missed by type-width mismatches or mulituse limitations which have been removed in later passes.
As well as introducing broadcast fold tables (which can hopefully be extended/automated in the future), this also handles mismatches in the AND/ANDN/OR/XOR/TERNLOG type-widths, catching additional missed opportunities.
This is patch is pulled from the ongoing work based on D150143, but without removing the existing DAG constant broadcast lowering code - this patch is currently a late stage cleanup only.
The intention is to add additional broadcast/extension handling of constants in future patches, but it turned out that AVX512 broadcast handling was the easiest to start with.
Differential Revision: https://reviews.llvm.org/D150526
