Also, move the XformToShuffleWithZero and combineCarryDiamond folds later after some of the more basic canonicalizations/combines (such as this) have had a chance to occur
Fixes the v8i1-masks.ll regression from D127115
Currently both calling conventions preserve registers that are used to
store a return value. This causes the returned value to be lost:
define i32 @bar() {
%1 = call preserve_mostcc i32 @foo()
ret i32 %1
}
define preserve_mostcc i32 @foo() {
ret i32 2
; preserve_mostcc will restore %rax,
; whatever it was before the call.
}
This contradicts the current documentation (preserve_allcc "behaves
identical to the `C` calling conventions on how arguments and return
values are passed") and also breaks [[clang::preserve_most]].
This change makes CSRs be preserved iff they are not used to store a
return value (e.g. %rax for scalars, {%rax:%rdx} for __int128, %xmm0
for double). For void functions no additional registers are
preserved, i.e. the behaviour is backward compatible with existing
code.
Differential Revision: https://reviews.llvm.org/D141020
This shows the failure to merge to and(ext(x),and(c1,ext(c2))) if the outer and has already been folded to a clear shuffle mask
Similar to the v8i1-masks.ll from regression D127115
Was previously buggy to assume that NOT'd Value was always an
instruction. If the NOT'd value is not an Instruction, we should just
return as its either a constant, in which can we will re-run the logic
after constant-folding, or its a type we can't evaluate anyways.
This is a follow up to: `D140939`
Reviewed By: pengfei, RKSimon
Differential Revision: https://reviews.llvm.org/D142339
Now, when llc encounters the case that contains a lot of
extract_vector_elt and a BUILD_VECTOR, it will replace these to
vector_shuffle to decrease the size of code, the actions are done in
createBuildVecShuffle in DAGCombiner.cpp, but now the code cannot handle
the case that the size of source vector reg is more than twice the dest
size.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D139685
The baseline `allowsMemoryAccess()` is wrong for X86.
It assumes that aligned memory operations are always allowed,
but that is not true.
For example, We can not perform a 32-byte aligned non-temporal load
of a 32-byte vector, without AVX2 that is, yet `allowsMemoryAccess()`
will say it is allowed, so we may end up merging non-temporal loads,
only to split them up to legalize them, and here we go again.
NOTE: the test changes here are superfluous. The main effect is that without this change,
in D141777, we'd get stuck endlessly merging and splitting non-temporal stores.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D141776
Previous logic was buggy and erroneously asserted that I->operand(0) must
be the RMW instruction. This change fixes that and makes it so that the
RMW instruction can be used in operand 0 or 1.
Also update the tests to explicitly test RMW as operand 0/1 (no change
to codegen).
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D142166
This change adjusts the cost modeling used when the target does not have a schedule model with individual instruction latencies. After this change, we use the default latency information available from TargetSchedule. The default latency information essentially ends up treating most instructions as latency 1, with a few "expensive" ones getting a higher cost.
Previously, we unconditionally applied the first legal pattern - without any consideration of profitability. As a result, this change both prevents some patterns being applied, and changes which patterns are exercised. (i.e. previously the first pattern was applied, afterwards, maybe the second one is because the first wasn't profitable.)
The motivation here is two fold.
First, this brings the default behavior in line with the behavior when -mcpu or -mtune is specified. This improves test coverage, and generally makes it less likely we will have bad surprises when providing more information to the compiler.
Second, this enables some reassociation for ILP by default. Despite being unconditionally enabled, the prior code tended to "reassociate" repeatedly through an entire chain and simply moving the first operand to the end. The result was still a serial chain, just a different one. With this change, one of the intermediate transforms is unprofitable and we end up with a partially flattened tree.
Note that the resulting code diffs show significant room for improvement in the basic algorithm. I am intentionally excluding those from this patch.
For the test diffs, I don't seen any concerning regressions. I took a fairly close look at the RISCV ones, but only skimmed the x86 (particularly vector x86) changes.
Differential Revision: https://reviews.llvm.org/D141017
The original check was trying to avoid checking UndefMask itself,
and deduce it via simpler means, but checking `NonZeroMask`
does not, e.g., check `ZeroMask`.
Fixes https://github.com/llvm/llvm-project/issues/60168
Follow-up for 4ece50737d5385fb80cfa23f5297d1111f8eed39 (D142027).
Assignment Tracking Analysis now always runs and is skipped internally if
assignment tracking is disabled. Update these tests to expect to see the
pass run.
Buildbot failure: https://lab.llvm.org/buildbot/#/builders/216/builds/16085
A lit test used overaligned i8, apparently due to an old copy-paste
error, intending to specify i32 alignment.
Change the datalayout string to use naturally aligned i8.
Add a flag state (and a MIR key) to MachineFunctions indicating whether they
contain instruction referencing debug-info or not. Whether DBG_VALUEs or
DBG_INSTR_REFs are used needs to be determined by LiveDebugValues at least, and
using the current optimisation level as a proxy is proving unreliable.
Test updates are purely adding the flag to tests, in a couple of cases it
involves separating out VarLocBasedLDV/InstrRefBasedLDV tests into separate
files, as they can no longer share the same input.
Differential Revision: https://reviews.llvm.org/D141387
This is an alternative to D141074 to fix the problem by adjusting
the precision control dynamically.
This isn't quite complete yet. I want to support fadd with an load
folded into it too. That's the code we will usually generate.
Posting for early review so we can do some testing of this solution.
Differential Revision: https://reviews.llvm.org/D142178
This reverts commit a6e3027db7ebe6863e44bafcfeaacc16bdc88a3f.
Chrome and Halide are both reporting issues with importing builtins.
Maybe the better direction is to manually adjust FPCW for the inline
sequence on Windows.
Add {e,r}flags, {g,f}s.base registers so they can be referenced in cfi
directives,. They are not otherwise useable in any instructions,
but can be implicitly pushed to the stack like with pushf for
{e,r}flags.
Differential Revision: https://reviews.llvm.org/D141879
This transofrm loses information that can be useful for other transforms.
Reviewed By: lebedev.ri
Differential Revision: https://reviews.llvm.org/D141883
The code below currently prints less accurate values only on Windows 32-bit. On Windows, the default precision control on x87 is only 53-bit, and FADD triggers rounding with that precision, so the final result may be less accurate. This revision avoids less accurate conversions by using library calls instead.
```
int main() {
int64_t n = 0b0000000000111111111111111111111111011111111111111111111111111111;
printf("%lld, %.0f, %.0f", n, (float)n, (float)(uint64_t)n);
return 0;
}
```
Reviewed By: craig.topper, lebedev.ri
Differential Revision: https://reviews.llvm.org/D141074
Issue #58168 describes the difficulty diagnosing stack size issues
identified by -Wframe-larger-than. For simple code, its easy to
understand the stack layout and where space is being allocated, but in
more complex programs, where code may be heavily inlined, unrolled, and
have duplicated code paths, it is no longer easy to manually inspect the
source program and understand where stack space can be attributed.
This patch implements a machine function pass that emits remarks with a
textual representation of stack slots, and also outputs any available
debug information to map source variables to those slots.
The new behavior can be used by adding `-Rpass-analysis=stack-frame-layout`
to the compiler invocation. Like other remarks the diagnostic
information can be saved to a file in a machine readable format by
adding -fsave-optimzation-record.
Fixes: #58168
Reviewed By: nickdesaulniers, thegameg
Differential Revision: https://reviews.llvm.org/D135488
As per the test case from Steven Johnson in https://reviews.llvm.org/rGf8d9097168b7#1165311
we can indeed encounter such shuffles, that produce all-undef after folding,
before something else manages to optimize them away.
IR is now always parsed in opaque pointer mode, unless
-opaque-pointers=0 is explicitly given. There is no automatic
detection of typed pointers anymore.
The -opaque-pointers=0 option is added to any remaining IR tests
that haven't been migrated yet.
Differential Revision: https://reviews.llvm.org/D141912
Let Propeller use specialized IDs for basic blocks, instead of MBB number.
This allows optimizations not just prior to asm-printer, but throughout the entire codegen.
This patch only implements the functionality under the new `LLVM_BB_ADDR_MAP` version, but the old version is still being used. A later patch will change the used version.
####Background
Today Propeller uses machine basic block (MBB) numbers, which already exist, to map native assembly to machine IR. This is done as follows.
- Basic block addresses are captured and dumped into the `LLVM_BB_ADDR_MAP` section just before the AsmPrinter pass which writes out object files. This ensures that we have a mapping that is close to assembly.
- Profiling mapping works by taking a virtual address of an instruction and looking up the `LLVM_BB_ADDR_MAP` section to find the MBB number it corresponds to.
- While this works well today, we need to do better when we scale Propeller to target other Machine IR optimizations like spill code optimization. Register allocation happens earlier in the Machine IR pipeline and we need an annotation mechanism that is valid at that point.
- The current scheme will not work in this scenario because the MBB number of a particular basic block is not fixed and changes over the course of codegen (via renumbering, adding, and removing the basic blocks).
- In other words, the volatile MBB numbers do not provide a one-to-one correspondence throughout the lifetime of Machine IR. Profile annotation using MBB numbers is restricted to a fixed point; only valid at the exact point where it was dumped.
- Further, the object file can only be dumped before AsmPrinter and cannot be dumped at an arbitrary point in the Machine IR pass pipeline. Hence, MBB numbers are not suitable and we need something else.
####Solution
We propose using fixed unique incremental MBB IDs for basic blocks instead of volatile MBB numbers. These IDs are assigned upon the creation of machine basic blocks. We modify `MachineFunction::CreateMachineBasicBlock` to assign the fixed ID to every newly created basic block. It assigns `MachineFunction::NextMBBID` to the MBB ID and then increments it, which ensures having unique IDs.
To ensure correct profile attribution, multiple equivalent compilations must generate the same Propeller IDs. This is guaranteed as long as the MachineFunction passes run in the same order. Since the `NextBBID` variable is scoped to `MachineFunction`, interleaving of codegen for different functions won't cause any inconsistencies.
The new encoding is generated under the new version number 2 and we keep backward-compatibility with older versions.
####Impact on Size of the `LLVM_BB_ADDR_MAP` Section
Emitting the Propeller ID results in a 23% increase in the size of the `LLVM_BB_ADDR_MAP` section for the clang binary.
Reviewed By: tmsriram
Differential Revision: https://reviews.llvm.org/D100808
This is essentially expanding on the optimizations added on: D120199
but applies the optimization to cases where the bit being changed /
tested is not am IMM but is a provable power of 2.
The only case currently added for cases like:
`__atomic_fetch_xor(p, 1 << c, __ATOMIC_RELAXED) & (1 << c)`
Which instead of using a `cmpxchg` loop can be done with `btcl; setcc; shl`.
There are still a variety of missed cases that could/should be
addressed in the future. This commit documents many of those
cases with Todos.
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D140939
I.e for blsi match (and (sub 0, x), x) but we currently miss valid
patterns like (and (and (sub 0, x), y), x).
Reviewed By: pengfei
Differential Revision: https://reviews.llvm.org/D141178
Delete mention of the llvm.eh.begincatch/llvm.eh.endcatch intrinsics,
and remove them from a few remaining test-cases. These intrinsics were
from a previous attempt at implementing Windows exception-handling,
but were removed from LLVM in 2015.
Also mention that dynamic exception specifications ("throw filters")
were removed from the spec in C++17.
Since there is no mask instruction for vXi1 with avx512f in X86 target.
Folding select for vXi1 doesn't help to reduce instructions.
Differential Revision: https://reviews.llvm.org/D141782
This patch is to fix regression of D122875. X86 has fpext instructions
supporting rmb form, which takes advantage of fpext(fplat(X)) than
splat(fpext(X)).
Reviewed By: RKSimon, skan
Differential Revision: https://reviews.llvm.org/D141657
As noted in https://reviews.llvm.org/D141778#inline-1369900,
we fail to produce splat shuffles from certain sequences
of shuffles, that may have non-shuffles in the middle of seq.
There is a big pitfail to avoid here: just because `isSplatValue()`
says that all demanded elements are splat, we can't pick any random
one of them, because some of them could be undef! We must ignore those!
