Add LLVM Context to getOptimalMemOpType and findOptimalMemOpLowering. So
that we can use EVT::getVectorVT to generate EVT type in
getOptimalMemOpType.
Related to [#146673](https://github.com/llvm/llvm-project/pull/146673).
Anatoly Trosinenko found that when hasSideEffect was set to 0 in the
definition of LOADgotAUTH, MultiSource/Benchmarks/Ptrdist/ks/ks test
from llvm-test-suite started to crash. The issue was traced down to
MachineLICM pass placing LOADgotAUTH right after an unrelated copy to
x16 like rewriting this code:
````
bb.0:
renamable $x16 = COPY renamable $x12
B %bb.1
bb.1:
...
/* use $x16 */
...
renamable $x20 = LOADgotAUTH target-flags(aarch64-got) @some_variable, implicit-def dead $x16, implicit-def dead $x17, implicit-def dead $nzcv
/* use $x20 */
...
````
like the following:
````
bb.0:
renamable $x16 = COPY renamable $x12
renamable $x20 = LOADgotAUTH target-flags(aarch64-got) @some_variable, implicit-def dead $x16, implicit-def dead $x17, implicit-def dead $nzcv
B %bb.1
bb.1:
...
/* use $x16 */
...
/* use $x20 */
...
```
The issue was caused by inconsistent logic between implicit and explicit
operand definitions, where the implicit side was incorrectly skipping
checking RUDefs for dead operands, leading to RuledOut not being set
for the X16 operand.
Because there isn't really a semantic difference between implicit and
explicit operands at this point, let's remove the isImplicit check and
adjust the logic to do the same thing in both cases:
- For implicit operands, we now check and update RUDefs in the same way
as explicit operands.
- For explicit operands, we now allow dead operands to be skipped.
Reviewers: arsenm, s-barannikov, atrosinenko
Reviewed By: arsenm, s-barannikov
Pull Request: https://github.com/llvm/llvm-project/pull/147624
As noted in post commit review, the API change here was not required.
I'd apparently confused myself when teasing apart patches from my
development branch.
Callsite offsets will help map addresses to the right position in the
basic block (before or after a callsite).
This PR also bumps the BBAddrMap version to 3.
The encoding/decoding ability is already pushed upstream
8d7a8fcc3ab9f6d4c4a7e4312876fe94ed3d6c4f.
Introduces saturated truncate instructions to Global ISel:
G_TRUNC_SSAT_S, G_TRUNC_SSAT_U, G_TRUNC_USAT_U. These were previously
introduced to SDAG to reduce redundant code.
The patch only initially introduces the instruction, a later patch will
follow to add combines and legalization for each instruction.
For the fixed vector cases, we already support this, but the
deinterleave intrinsic cases (primary used by scalable vectors) didn't.
Supporting it requires plumbing through the Factor separately from the
extracts, as there can now be fewer extracts than the Factor. Note that
the fixed vector path handles this slightly differently - it uses the
shuffle and indices scheme to achieve the same thing.
Previously we had a table of entries for every Libcall for
the comparison to use against an integer 0 if it was a soft
float compare function. This was only relevant to a handful of
opcodes, so it was wasteful. Now that we can distinguish the
abstract libcall for the compare with the concrete implementation,
we can just directly hardcode the comparison against the libcall
impl without this configuration system.
Emit a context error and delete the instruction. This
allows removing the AMDGPU hack where some atomic libcalls
are falsely added. NVPTX also later copied the same hack,
so remove it there too.
For now just emit the generic error, which is not good. It's
missing any useful context information (despite taking the instruction).
It's also confusing in the failed atomicrmw case, since it's reporting
failure at the intermediate failed cmpxchg instead of the original
atomicrmw.
Use `emitValueToAlignment` as the section does not contain code.
`emitCodeAlignment` would lead to ALIGN relocations on RISC-V and
LoongArch with linker relaxation.
In addition, change the alignment to wordsize, sufficient for the
runtime requirement (`XRayFunctionSledIndex`).
Related to #147322
llvm/llvm-project#147560 changed when the legacy SelectionDAG pass needs
TargetTransformInfoWrapperPass to always require it (rather than only
when assertions are enabled). `SelectionDAGISelLegacy::getAnalysisUsage`
was not updated in that PR, which was causing crashes on
assertions-disabled builds, which are hard to track down.
This makes the required update, which should avoid crashes being seen on
some buildbots and by some users.
This PR takes the work previously done by @pawan-nirpal-031 on X86 in
#106370, and makes it available in common code. This should enable all
targets to use `__builtin_canonicalize` for all `f(16|32|64|128)` data
types.
Canonicalization is implemented here as multiplication by `1.0`, as
suggested in [the
docs](https://llvm.org/docs/LangRef.html#llvm-canonicalize-intrinsic).
This patch fixes:
llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp:3575:44: error:
missing field 'Cases' initializer
[-Werror,-Wmissing-field-initializers]
This PR provides more information to debuggers and analysis tools on
Windows. It adds `S_LABEL32` symbols for each target BB of each jump
table. This allows debuggers to insert symbolic labels when
disassembling code. `S_LABEL32` symbol records indicate that a location
is definitely code, and can optionally associate a string label with the
code location. BBs generated for jump tables may or may not have string
labels, so it is acceptable for the "name" field within `S_LABEL32`
symbols to be an empty string.
More importantly, this PR allows Windows analysis tools, such as those
that generate hot-patches for the Windows kernel, to use these labels to
distinguish code basic blocks from data blocks. Microsoft's analysis
tools (similar to Bolt) rely on being able to identify all code blocks,
so that the tools can traverse all instructions and verify that
important requirements for hot-patching are met.
This PR has no effect on code generation. It only affects the CodeView
symbols that are emitted into OBJ files, which the linker then
repackages into PDB files.
Currently the default exception handling type is scattered
across the backends in MCAsmInfo constructors. Allow this
to be computed from the triple so the IR can centrally determine
the set of ABI calls.
Manually submitting, closes#147225
DAGCombiner can already constant fold build vectors of constants/undefs
to a new vector type, but it has to be incredibly careful after
legalization to not affect a target's canonicalized constants.
This patch proposes we move the implementation inside SelectionDAG to
make it easier for targets to manually use the constant folding whenever
it deems it safe to do so.
I've also altered the method to take the BuildVectorSDNode input
directly and consistently use the same SDLoc.
The register coalescer collects the copies forward, the stale comment
wasn't addressed back then by:
75961ecc1a.
This change updates the coment and also adds a similar one to the other
branch.
ISD::ABDS can be used if the signed subtraction will not overwrap (this
is an extension to handle cases where the NSW flag has been lost)
ISD::ABDU can be used if both operands have at least 1 zero sign bit.
Fixes#147049
This is a GISel equivalent of #130665, preventing a double-rounding
issue in sitofp/uitofp by scalarizing i64->f32 converts. Most of the
changes are made in the ActionDefinitionsBuilder for G_SITOFP/G_UITOFP.
Because it is legal to convert i64->f16 itofp without double-rounding,
but not a fpround f64->f16, that variant is lowered to build the two
extends.
When checking whether it is profitable to hoist an instruction, the pass
may override a target's ruling because it assumes that all COPY
instructions are cheap, and that may not be the case for all
micro-architectures (especially for when copying between different
register classes).
On AArch64 there's 0% difference in performance in LLVM's test-suite
with this change. Additionally, very few tests were affected which shows
how it is not so useful to keep it.
x86 performance is slightly better (but maybe that's just noise) for an
A/B comparison consisting of five iterations on LLVM's test suite (Ryzen
5950X on Ubuntu):
```
$ ./utils/compare.py build-a/results* vs build-b/results* --lhs-name base --rhs-name patch --absolute-diff
Tests: 3341
Metric: exec_time
Program exec_time
base patch diff
LoopVector...meChecks4PointersDBeforeA/1000 824613.68 825394.06 780.38
LoopVector...timeChecks4PointersDBeforeA/32 18763.60 19486.02 722.42
LCALS/Subs...test:BM_MAT_X_MAT_LAMBDA/44217 37109.92 37572.52 462.60
LoopVector...ntimeChecks4PointersDAfterA/32 14211.35 14562.14 350.79
LoopVector...timeChecks4PointersDEqualsA/32 14221.44 14562.85 341.40
LoopVector...intersAllDisjointIncreasing/32 14222.73 14562.20 339.47
LoopVector...intersAllDisjointDecreasing/32 14223.85 14563.17 339.32
LoopVector...nLoopFrom_uint32_t_To_uint8_t_ 739.60 807.45 67.86
harris/har...est:BENCHMARK_HARRIS/2048/2048 15953.77 15998.94 45.17
LoopVector...nLoopFrom_uint8_t_To_uint16_t_ 301.94 331.21 29.27
LCALS/Subs...Raw.test:BM_DISC_ORD_RAW/44217 616.35 637.13 20.78
LCALS/Subs...Raw.test:BM_MAT_X_MAT_RAW/5001 3814.95 3833.70 18.75
LCALS/Subs...Raw.test:BM_HYDRO_2D_RAW/44217 812.98 830.64 17.66
LCALS/Subs...test:BM_IMP_HYDRO_2D_RAW/44217 811.26 828.13 16.87
ImageProce...ENCHMARK_BILATERAL_FILTER/64/4 714.77 726.23 11.46
exec_time
l/r base patch diff
count 3341.000000 3341.000000 3341.000000
mean 903.866450 899.732349 -4.134101
std 20635.900959 20565.289417 115.346928
min 0.000000 0.000000 -3380.455787
25% 0.000000 0.000000 0.000000
50% 0.000000 0.000000 0.000000
75% 1.806500 1.836397 0.000100
max 824613.680801 825394.062500 780.381699
```
RFC on discourse:
https://discourse.llvm.org/t/rfc-debug-info-for-coroutine-suspension-locations-take-2/86606
With this commit, we add `DILabel` debug infos to the resume points of a
coroutine. Those labels can be used by debugging scripts to figure out
the exact line and column at which a coroutine was suspended by looking
up current `__coro_index` value inside the coroutines frame, and then
searching for the corresponding label inside the coroutine's resume
function.
The DWARF information generated for such a label looks like:
```
0x00000f71: DW_TAG_label
DW_AT_name ("__coro_resume_1")
DW_AT_decl_file ("generator-example.cpp")
DW_AT_decl_line (5)
DW_AT_decl_column (3)
DW_AT_artificial (true)
DW_AT_LLVM_coro_suspend_idx (0x01)
DW_AT_low_pc (0x00000000000019be)
```
The labels can be mapped to their corresponding `__coro_idx` values
either via their naming convention `__coro_resume_<N>` or using the new
`DW_AT_LLVM_coro_suspend_idx` attribute. In gdb, those line numebrs can
be looked up using `info line -function my_coroutine -label
__coro_resume_1`. LLDB unfortunately does not understand DW_TAG_label
debug information, yet.
Given this is an artificial compiler-generated label, I did apply the
DW_AT_artificial tag to it. The DWARFv5 standard only allows that tag on
type and variable definitions, but this is a natural extension and was
also blessed in the RFC on discourse.
Also, this commit adds `DW_AT_decl_column` to labels, not only for
coroutines but also for normal C and C++ labels. While not strictly
necessary, I am doing so now because it would be harder to do so later
without breaking the binary LLVM-IR format
Drive-by fixes: While reading the existing test cases to understand how
to write my own test case, I did a couple of small typo fixes and
comment improvements
This reverts commit 8ac7210b7f0ad49ae7809bf6a9faf2f7433384b0.
This breaks the building the AArch64 backend, e.g. see
https://github.com/llvm/llvm-project/pull/144947
Revert to unbreak the build.
Also reverts follow-up commits 1e76f012db3ccfaa05e238812e572b5b6d12c17e.
If a kernel is known to be executing only a single lane, IR
UniformityAnalysis will take note of that (via
GCNTTIImpl::hasBranchDivergence) and report that all values are uniform.
SelectionDAG's built-in divergence tracking should do the same.
This patch is part of a series that adds origin-tracking to the debugify
source location coverage checks, allowing us to report symbolized stack
traces of the point where missing source locations appear.
This patch adds the logic for collecting stack traces in DebugLoc
instances. We do not symbolize the stack traces in this patch - that
only happens when we decide to actually print them, which will be the
responsibility of debugify. The collection happens in the constructor of
a DebugLoc that has neither a valid location nor an annotation; we also
collect an extra stack trace every time we call setDebugLoc, as
sometimes the more interesting point is not where the DebugLoc was
constructed, but where it was applied to an instruction. This takes the
form of a getCopied() method on DebugLoc, which is the identity function
in normal builds, but adds an extra stack trace in origin-tracking
builds.
When generating SDAG for a getelementptr with a vector result, we were
previously generating splats for each scalar operand. This essentially
has the effect of aggressively vectorizing the sequence, and leaving it
later combines to scalarize if profitable.
Instead, we can keep the accumulating address as a scalar for as long as
the prefix of operands allows before lazily converting to vector on the
first vector operand. This both better fits hardware which frequently
has a scalar base on the scatter/gather instructions, and reduces the
addressing cost even when not as otherwise we end up with a scalar to
vector domain crossing for each scalar operand.
Note that constant splat offsets are treated as scalar for the above,
and only variable offsets can force a conversion to vector.
---------
Co-authored-by: Craig Topper <craig.topper@sifive.com>
