This is a conservative workaround for broken liveness tracking of
SUBREG_TO_REG to speculatively fix all targets. The current reported
failures are on X86 only, but this issue should appear for all targets
that use SUBREG_TO_REG. The next minimally correct refinement would be
to disallow only implicit defs.
The coalescer now introduces implicit-defs of the super register to
track the dependency on other subregisters. If we see such an implicit
operand, we cannot simply treat the subregister def as the result
operand in case downstream users depend on the implicitly defined
parts. Really target implementations should be considering the
implicit defs and trying to interpret them appropriately (maybe with
some generic helpers). The full implicit def could possibly be
reported as the move result, rather than the subregister def but that
requires additional work.
Hopefully fixes#64060 as well.
This needs to be applied to the release branch.
https://reviews.llvm.org/D156346
Currently coalescing with SUBREG_TO_REG introduces an invisible load
bearing undef. There is liveness for the super register not
represented in the MIR.
This is part 1 of a fix for regressions that appeared after
b7836d856206ec39509d42529f958c920368166b. The allocator started
recognizing undef-def subregister MOVs as copies. Since there was no
representation for the dependency on the high bits, different undef
segments of the super register ended up disconnected and downstream
users ended up observing different undefs than they did previously.
This does not yet fix the regression. The isCopyInstr handling needs
to start handling implicit-defs on any instruction.
I wanted to include an end to end IR test since the actual failure
only appeared with an interaction between the coalescer and the
allocator. It's a bit bigger than I'd like but I'm having a bit of
trouble reducing it to something which definitely shows a diff that's
meaningful.
The same problem likely exists everywhere trying to do anything with
SUBREG_TO_REG. I don't understand how this managed to be broken for so
long.
This needs to be applied to the release branch.
https://reviews.llvm.org/D156345
Introduce isDesirableCastOp() which determines whether IR builder
and constant folding should produce constant expressions for a
given cast type. This mirrors what we do for binary operators.
Mark zext/sext as undesirable, which prevents most creations of such
constant expressions. This is still somewhat incomplete and there
are a few more places that can create zext/sext expressions.
This is part of the work for
https://discourse.llvm.org/t/rfc-remove-most-constant-expressions/63179.
The reason for the odd result in the constantexpr-fneg.c test is
that initially the "a[]" global is created with an [0 x i32] type,
at which point the icmp expression cannot be folded. Later it is
replaced with an [1 x i32] global and the icmp gets folded away.
But at that point we no longer fold the zext.
If this was coalescing a def of a subregister with a def of the super
register, it was introducing a redundant super-register def and
marking the subregister def as dead.
Resulting in something like:
dead $eax = MOVr0, implicit-def $rax, implicit-def $rax
Avoid this by checking if the new instruction already has the super
def, so we end up with this instead:
dead $eax = MOVr0, implicit-def $rax
The dead flag looks suspicious to me, seems like it's easy to buggily
interpret dead def of subreg and a non-dead def of an aliasing
register. It seems to be intentional though.
https://reviews.llvm.org/D156343
Permit an implicit-def of a virtual register when rematerializing if
it defines a super register of a subregister def. The
rematerialization pre-legality check should really have been checking
the implicit operands, but that should be fixed separately.
https://reviews.llvm.org/D156331
When performing a masked load of an unpacked SVE vector type, i.e.
nxv8i8, followed by a zero- or sign-extend to an illegal wide type
such as nxv8i32 we typically end up with a combination of an
extending masked load and pair(s) of uunpklo/hi or sunpklo/hi
instructions. For example, see test @masked_sload_8i8_8i32 in file
CodeGen/AArch64/sve-masked-ldst-sext.ll
where
%aval = call <vscale x 8 x i8> @llvm.masked.load.nxv8i8(...
%aext = sext <vscale x 8 x i8> %aval to <vscale x 8 x i32>
gets lowered to
ld1sb { z1.h }, ...
sunpklo z0.s, z1.h
sunpkhi z1.s, z1.h
Currently the cost for the 'sext' operation in the example above is
1, whereas this patch changes it to 2 to reflect the pair of
instructions required. Similarly, when doing a masked load of a
nxv8i8 and extending to nxv8i64 the cost is changed to 6 to reflect
the 6 unpacks required.
Not sure how to produce a test that demonstrates the problem
today. The coalescer would have to introduce a verifier caught SSA
violation, like multiple defs of a virtual register. I'm not sure what
would do that now, but an upcoming patch will.
https://reviews.llvm.org/D156271
This reverts commit 6403287eff71a3d6f6c862346d6ed3f0f000eb70.
This is failing on all but 1 of Linaro's flang builders.
CMake Error at /home/tcwg-buildbot/worker/clang-aarch64-full-2stage/llvm/flang-rt/unittests/CMakeLists.txt:37 (message):
Target llvm_gtest not found.
This change adds two related DAG combines which together will take a
left-reduce scalar add tree of an explode_vector, and will incrementally
form a vector reduction of the vector prefix. If the entire vector is
reduced, the result will be a reduction over the entire vector.
Profitability wise, this relies on vredsum being cheaper than a pair of
extracts and scalar add. Given vredsum is linear in LMUL, and the
vslidedown required for the extract is *also* linear in LMUL, this is
clearly true at higher index values. At N=2, it's a bit questionable,
but I think the vredsum form is probably a better canonical form
anyways.
Note that this only matches left reduces. This happens to be the
motivating example I have (from spec2017 x264). This approach could be
generalized to handle right reduces without much effort, and could be
generalized to handle any reduce whose tree starts with adjacent
elements if desired. The approach fails for a reduce such as (A+C)+(B+D)
because we can't find a root to start the reduce with without scanning
the entire associative add expression. We could maybe explore using
masked reduces for the root node, but that seems of questionable
profitability. (As in, worth questioning - I haven't explored in any
detail.)
This is covering up a deficiency in SLP. If SLP encounters the scalar
form of reduce_or(A) + reduce_sum(a) where a is some common
vectorizeable tree, SLP will sometimes fail to revisit one of the
reductions after vectorizing the other. Fixing this in SLP is hard, and
there's no good reason not to handle the easy cases in the backend.
Another option here would be to do this in VectorCombine or generic DAG.
I chose not to as the profitability of the non-legal typed prefix cases
is very target dependent. I think this makes sense as a starting point,
even if we move it elsewhere later.
This is currently restructed only to add reduces, but obviously makes
sense for any associative reduction operator. Once this is approved, I
plan to extend it in this manner. I'm simply staging work in case we
decide to go in another direction.
Without this patch, we pass G.getEndianness() as one of the parameters
to DWARFContext::create. The problem is that G.getEndianness() is of:
enum endianness {big, little, native};
whereas DWARFContext::create is expecting "bool isLittleEndian". That
is, we are relying on an implicit conversion to convert big and little
to false and true, respectively.
When we migrate llvm::support::endianness to std::endian in future, we
can no longer rely on an implicit conversion because std::endian is
declared with "enum class". Even if we could, the conversion would
not be guaranteed to work because, for example, libcxx defines:
enum class endian {
little = 0xDEAD,
big = 0xFACE,
:
where big and little are not boolean values.
This patch fixes the problem by properly converting G.getEndianness()
to a boolean value.
Without this patch, we pass Endian as one of the parameters to the
constructor of DataExtractor. The problem is that Endian is of:
enum endianness {big, little, native};
whereas the constructor is expecting "bool IsLittleEndian". That is,
we are relying on an implicit conversion to convert big and little to
false and true, respectively.
When we migrate llvm::support::endianness to std::endian in future, we
can no longer rely on an implicit conversion because std::endian is
declared with "enum class". Even if we could, the conversion would
not be guaranteed to work because, for example, libcxx defines:
enum class endian {
little = 0xDEAD,
big = 0xFACE,
:
where big and little are not boolean values.
This patch fixes the problem by properly converting Endian to a
boolean value.
- Add tests for computeOverflowFor*Sub functions
- extend the computeOverflowForSignedSub/computeOverflowForUnsignedSub
implementations with ConstantRange (#37109)
We could maybe extend this by allowing the lowest subop to have multiple uses and extract the lowest subvector result of the concatenated op, but let's just get the fix in first.
Fixes#67333
When custom install names and rpaths setups are used they may not
work in the build tree as-is (namely when using absolute paths for
install names in order to avoid rpath juggling in downstream projects).
Add a flag for opting out of this behaviour.
See: https://reviews.llvm.org/D42463
Without this, various CodeGen tests fail because a
RISCV::FCVT_D_W[_IN32X] machine node is created without the rounding
mode operand.
The relevant PR was committed as bf94ba39b65d1212ea84d5783b393280e1ce7478
This is a follow-up to #67555, performing the same fix for the other
instructions that had this issue:
* fcvt.d.w
* fcvt.d.wu
* fcvt.s.h
* fcvt.d.h
As before, we stick to the 'rne' default because this gives maximum
compatibility with older LLVM and GNU tools when disassembling. I've
also double checked disassembling fp-default-rounding-mode.s with GNU
objdump to re-confirm it uses frm=0b000 for these instructions.
This avoids some redundant spills of subranges, and avoids a compile failure.
This greatly reduces the numbers of spills in a loop.
The main range is not informative when multiple instructions are needed to fully define
a register. A common scenario is a lowered reg_sequence where every subregister
is sequentially defined, but each def changes the main range's value number. If
we look at specific lanes at the use index, we can see the value is actually the
same.
In this testcase, there are a large number of materialized 64-bit constant defs
which are hoisted outside of the loop by MachineLICM. These are feeding REG_SEQUENCES,
which is not considered rematerializable inside the loop. After coalescing, the split
constant defs produce main ranges with an apparent phi def. There's no phi def if you look
at each individual subrange, and only half of the register is really redefined to a constant.
Fixes: SWDEV-380865
https://reviews.llvm.org/D147079
SplitKit creates questionably formed bundles of copies
when it needs to copy a subset of live lanes and can't do
it with a single subregister index. These are merely marked
as part of a bundle, and don't start with a BUNDLE instruction.
Queries for the slot index would give the first copy in the
bundle, and we need to inspect the operands of all the other
bundled copies.
Also fix and simplify detection of read lane subsets. This causes
some RISCV test regressions, but these look like accidentally beneficial
splits. I don't see a subrange based reason to perform these splits.
Avoids some really ugly regressions in a future patch.
https://reviews.llvm.org/D146859
https://reviews.llvm.org/D152789 added an `exit` op before each
`unreachable`. This means we never get to the `trap` instruction.
This change limits the insertion of `exit` instructions to the cases
where `unreachable` is not lowered to `trap`. Trap itself is changed to
be emitted as `trap; exit;` to convey to `ptxas` that it exits the CFG.
This will make it possible to add visibility attributes to these
variables. This also fixes some type mismatches between the
declaration and the definition.
Reviewed By: bogner, huangjd
Differential Revision: https://reviews.llvm.org/D156599
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Object\MachOUniversalWriter.cpp(352) : error C2220: the following warning is treated as an error
C:\buildbot\mlir-x64-windows-ninja\llvm-project\llvm\lib\Object\MachOUniversalWriter.cpp(352) : warning C4715: 'llvm::object::writeUniversalBinaryToStream': not all control paths return a value
Xcode `lipo` seems to support a non-documented `-fat64` option that
creates Universal Mach-O archives using 64 bit versions of the
`fat_arch` header, which allows offsets larger than 32 bits to be
specified.
Modify `llvm-lipo` to support the same flag, and use the value of the
flag to use either 32 bits or 64 bits Mach-O headers.
The Mach-O universal writer allows specifying a new option to write
these 64 bits headers. The default is still using 32 bits.
`dsymutil` implemented support for a similar flag in
https://reviews.llvm.org/D146879.
This seems to be an issue common to both GCC and LLVM. There are various
RISC-V FCVT instructions where the frm field makes no difference to the
output as the result is always exact (e.g. fcvt.d.s, fcvt.s.h,
fcvt.d.h). As with GCC, we always generate a form of these fcvt
instructions where frm=0b000. However, the ISA manual _doesn't_ state
that frm values are invalid, and we should ensure we can accept them.
This patch does so by adding the frm field to fcvt.d.s and adding an
InstAlias so that if no frm is specified, it defaults to rne (0b000).
This patch just corrects fcvt.d.s in order to allow the approach to be
reviewed, before applying it to the other affected instructions.
I haven't added tests to llvm/test/MC/Disassembler/RISCV, because it
doesn't seem necessary to test there in addition to our usual round-trip
tests in llvm/test/MC/RISCV. But feedback is welcome.
Recently added tests ensure that the default `rne` rounding mode is
printed as desired.
See discourse thread https://discourse.llvm.org/t/rfc-support-cmake-option-to-control-link-type-built-for-flang-runtime-libraries/71602/18 for full details.
Flang-rt is the new library target for the flang runtime libraries. It builds the Flang-rt library (which contains the sources of FortranRuntime and FortranDecimal) and the Fortran_main library. See documentation in this patch for detailed description (flang-rt/docs/GettingStarted.md).
This patch aims to:
- integrate Flang's runtime into existing llvm infrasturcture so that Flang's runtime can be built similarly to other runtimes via the runtimes target or via the llvm target as an enabled runtime
- decouple the FortranDecimal library sources that were used by both compiler and runtime so that different build configurations can be applied for compiler vs runtime
- add support for running flang-rt testsuites, which were created by migrating relevant tests from `flang/test` and `flang/unittest` to `flang-rt/test` and `flang-rt/unittest`, using a new `check-flang-rt` target.
- provide documentation on how to build and use the new FlangRT runtime
Reviewed By: DanielCChen
Differential Revision: https://reviews.llvm.org/D154869
Fixes https://github.com/llvm/llvm-project/issues/65227
LLVMGetOrdering previously did not support Fence instructions, and
calling it on a fence would lead to a bad cast as it
assumed a load/store, or an AtomicRMWInst. This would either read a
garbage memory order, or assertion
LLVMIsAtomicSingleThread did not support either Fence instructions,
loads, or stores, and would similarly lead to a bad cast.
It happened to work out since the relevant types all have their synch
scope ID at the same offset, but it still should be fixed
These cases are now fixed for the C API, and tests for these
instructions are added. The echo test utility now also supports cloning
Fence instructions, which it did not previously
-----
From what I can tell, there's no unified API to pull
`getOrdering`/`getSyncScopeID` from, and instead requires casting to
individual types: if there is a better way of handling this I can switch
to that
The newly added tests check which rounding mode is used by default, and
that it is printed when aliases are disabled but not otherwise.
These tests will be used in #67555.
* Remove if its sole use is to support an unnecessary ptr-to-ptr bitcast
(remove the bitcast as well)
* Replace with use of other APIs.
NFC opaque pointer cleanup effort.
The AArch64StorePairSuppress pass prevents the creation of STP under some
heuristics. Unfortunately it often prevents the creation of STP in cases where
it is obviously beneficial, and it doesn't match my understanding of
scheduling/cpu pipelining to prevent the creation of STP. From some
benchmarking, even on an in-order cpu where the scheduling is most important I
don't see it giving better results. In general the lower instruction count for
STP would be expected to give a slightly better cycle count.
As the pass specifically mentions the cyclone cpu, this patch adds a target
feature for FeatureStorePairSuppress, enabled for all the non-Arm cpus. This
has the effect of disabling it for all Arm cpus.
Differential Revision: https://reviews.llvm.org/D134646
This patch replaces any_cast with llvm::any_cast. This in turn allows
us to gracefully switch to std::any in future by forwarding llvm::Any
and llvm::any_cast to:
using Any = std::any;
template <class T> T *any_cast(Any *Value) {
return std::any_cast<T>(Value);
}
respectively.
Without this patch, it's ambiguous whether any_cast refers to
std::any_cast or llvm::any_cast.
As an added bonus, this patch makes it easier to mechanically replace
llvm::any_cast with std::any_cast without affecting other occurrences of
any_cast (e.g. in libcxx).
The goal of the class is to be an (almost) drop in replacement for
SmallVector and std::vector when those are presized and filled later, as
it happens in SourceManager and ASTReader.
By doing so, sparsely accessed PagedVector can profit from reduced
memory footprint.
