try_emplace can default-construct values, so we do not need to do so
on our own. Plus, try_emplace(Key) is much simpler/shorter than
insert({Key, LongValueType()}).
Widening the mask and padding with zeros doesn't work for scalable
vectors. Using VL produces less code for fixed vectors.
Similar was recently done for MLOAD.
This adds support for lowering deinterleave and interleave intrinsics
for factors 3 5 and 7 into target specific memory intrinsics.
Notably this doesn't add support for handling higher factors constructed
from interleaving interleave intrinsics, e.g. factor 6 from interleave3
+ interleave2.
I initially tried this but it became very complex very quickly. For
example, because there's now multiple factors involved
interleaveLeafValues is no longer symmetric between interleaving and
deinterleaving. There's then also two ways of representing a factor 6
deinterleave: It can both be done as either 1 deinterleave3 and 3
deinterleave2s OR 1 deinterleave2 and 3 deinterleave3s.
I'm not sure the complexity of supporting arbitrary factors is warranted
given how we only need to support a small number of factors currently:
SVE only needs factors 2,3,4 whilst RVV only needs 2,3,4,5,6,7,8.
My preference would be to just add a interleave6 and deinterleave6
intrinsic to avoid all this ambiguity, but I'll defer this discussion to
a later patch.
We can use the offset from the original store instead of creating
a new undef offset.
We didn't check if the offset was undef already so we really shouldn't
drop it if it isn't.
Padding the mask using 0 elements doesn't work for scalable vectors. Use
VP_LOAD and change the VL instead.
This fixes crash for Zve32x. Test file was split since i64 isn't a valid
element type for Zve32x.
Fixes#140198.
add `GenericFloatingPointPredicateUtils` in order to generalize
effects of floating point comparisons on `KnownFPClass` for both IR and
MIR.
---------
Co-authored-by: Matt Arsenault <arsenm2@gmail.com>
It was using the full size of the vector as the SrcBitWidth. This patch
changes the code to split G_ASSERT_ZEXT away from the others (G_INTTOPTR
/ G_PTRTOINT / G_ZEXT / G_TRUNC) which are simpler, and make the code
match the SDAG equivalent.
This is the bare minimum to get the intrinsic to compile for AMDGPU,
and it's not optimal. We need to follow along closer with the existing
G_FMINNUM/G_FMAXNUM with custom lowering to handle the IEEE=0 case
better.
Just re-use the existing lowering for the old semantics for
G_FMINNUM/G_FMAXNUM. This does not change G_FMINNUM/G_FMAXNUM's
treatment,
nor try to handle the general expansion without an underlying min/max
variant (or with G_FMINIMUM/G_FMAXIMUM).
try_emplace can default-construct values, so we do not need to do so
on our own. Plus, try_emplace(Key) is much shorter than
insert(std::make_pair(Key, Value()).
## Purpose
This patch prepares the llvm/CodeGen library for public interface
annotations in support of an LLVM Windows DLL (shared library) build,
tracked in #109483. The purpose of this patch is to make the upcoming
codemod of this library more straight-forward. It is not expected to
impact any functionality.
The `LLVM_ABI` annotations will be added in a subsequent patch. These
changes are required to build with visibility annotations using Clang
and gcc on Linux/Darwin/etc; Windows DLL can build fine without them.
## Overview
This PR does four things in preparation for adding `LLVM_ABI`
annotations to llvm/CodeGen:
1. Explicitly include `Machine.h` and `Function.h` headers from
`MachinePassManager.cpp` so that `Function` and `Machine` types are
available for the instantiations of `InnerAnalysisManagerProxy`. Without
this change, Clang only will only export one of the templates after
visibility annotations are added to them. Unclear if this is a Clang bug
or expected behavior, but this change avoids the issue and should be
harmless.
2. Refactor the definition of `MachineFunctionAnalysisManager` to its
own header file. Without this change, it is not possible to add
visibility annotations to the declaration with causing gcc to produce
`-Wattribute` warnings.
3. Remove the redundant specialization of the
`DominatorTreeBase<MachineBasicBlock, false>::addRoot` method. The
specialization is the same as implemented in `DominatorTreeBase` so
should be unnecessary. Without this change, it is not possible to
annotate the subsequent instantiations of `DominatorTreeBase` in the
header file without gcc producing `-Wattribute` warnings. Mark
unspecialized `addRoot` as `inline` to match the removed specialized
version.
4. Move the explicit instantiations of the `GenericDomTreeUpdater`
template earlier in the header file. These need to appear before being
used in the `MachineDomTreeUpdater` class definition or gcc will produce
warnings once visibility annotations are added.
## Background
The LLVM Windows DLL effort is tracked in #109483. Additional context is
provided in [this
discourse](https://discourse.llvm.org/t/psa-annotating-llvm-public-interface/85307).
Clang and gcc handle visibility attributes on explicit template
instantiations a bit differently; gcc is pickier and generates
`-Wattribute` warnings when an explicit instantiation with a visibility
annotation appears after the type has already appeared in the
translation unit. These warnings can be avoided by moving explicit
template instantiations so they always appear first.
## Validation
Local builds and tests to validate cross-platform compatibility. This
included llvm, clang, and lldb on the following configurations:
- Windows with MSVC
- Windows with Clang
- Linux with GCC
- Linux with Clang
- Darwin with Clang
This forwards the KnownBits / SignBits from the scalar operand through
to the vector elements. The value is implicitly truncated if it is
larger than the vector element size. The change in
GISelValueTracking::computeNumSignBits allows scalable sign-bits using a
single element demanded elts, and brings it in line with the code in
ValueTracking and SDAG (and GISelValueTracking::getKnownBits). That was
the main motivation for adding this opcode, to prevent scalable vector
asserts.
Prevent LowerFunnelShift from creating an invalid ISD::FSHR when
lowering "ISD::FSHL X, Y, 0". Such inputs are rare because it's a NOP
that DAGCombiner will optimise away. However, we should not rely on this
and so this PR mirrors the same optimisation.
Ensure LowerFunnelShift normalises constant shift amounts because isel
rules expect them to be in the range [0, src bit length).
NOTE: To simiplify testing, this PR also adds a command line option to
disable the DAG combiner (-combiner-disabled).
branch-folder hoists common instructions from TBB and FBB into their
pred. Without this patch it achieves this by splicing the instructions from TBB
and deleting the common ones in FBB. That moves the debug locations and debug
instructions from TBB into the pred without modification, which is not
ideal. The merged instructions should get merged debug locations for debugging
and PGO purposes, which is handled in this patch. Debug instructions also need
to be handled differently. That'll come in another patch. This issue was found
by @omern1.
The load not being included in the chain meant that it could materialize
after a `@llvm.lifetime.end` annotation on the pointer. This could
result in miscompiles if the stack slot is reused for another value.
Fixes https://github.com/llvm/llvm-project/issues/140491
Of the 128-bits of buffer descriptor only 48 bits are address bits, so
following the discussion on https://discourse.llvm.org/t/clarifiying-the-semantics-of-ptrtoint/83987/54,
the logic conclusion is to set the index width to 48 bits instead of
the current value of 128.
Most of the test changes are mechanical datalayout updates, but there
is one actual change: the ptrmask test now uses .i48 instead of .i128
and I had to update SelectionDAGBuilder to correctly extend the mask.
Reviewed By: krzysz00
Pull Request: https://github.com/llvm/llvm-project/pull/139419
Added APInt::clearBits(unsigned loBit, unsigned hiBit) that clears bits within a certain range.
Fixes#136550
---------
Co-authored-by: Simon Pilgrim <llvm-dev@redking.me.uk>
With the large number of G_ opcodes now present, this debug information
emitted for checking the global isel legalizer rules are valid is quite
verbose and runs even when gisel is not being used. This makes it
especially verbose when running a single pass with -debug.
This patch puts it behind a verbose debug option so that most people
don't need to worry about it.
Moves `MIRPRinter` and `MIPrinter` fields into a new `MFPrintState`
struct. Turns methods into independent functions.
- This slightly simplifies the code.
- In the future this will allow to pass the state around to other printing functions
without the need to expose classes in the public headers.
The MS bit-field packing ABI depends on the storage size of the type of
being placed in the bit-field. This PR addresses a number of cases in
llvm where the storage type has lead to suboptimal packing.
std::tie simplifies the lexicographical comparison while making the
code a little more consistent within MIRPrinter.cpp as we have a very
similar comparison functor in MIRPrinter::convertCalledGlobals, about
30 lines below the code this patch touches.
Fold (srl (lop x, (shl (zext y), c1)), c1) -> (lop (srl x, c1), (zext y)) where c1 <= leadingzeros(zext(y)).
This is equivalent of existing fold chain (srl (shl (zext y), c1), c1) -> (and (zext y), mask) -> (zext y), but logical op in the middle prevents it from combining.
Profit : Allow to reduce the number of instructions.
Original commit: #138290 / bbc5221
Previously reverted due to conflict in LIT test. Mainline changed
default version of load instruction to untyped version by this #137698 .
Updated test uses `ld.param.b64` instead of `ld.param.u64`.
Function optimizeBlock may do optimizations on a block for multiple
times. In the first iteration of the loop, MemoryInst1 may generate a
sunk instruction and store it into SunkAddrs. In the second iteration of
the loop, MemoryInst2 may use the same address and then it can reuse the
sunk instruction stored in SunkAddrs, but MemoryInst2 may be before
MemoryInst1 and the corresponding sunk instruction. In order to avoid
use before def error, we need to find appropriate insert position for the
sunk instruction.
Fixes#138208.
