Follow-up to #140494
`shouldForceRelocation` is conservative and produces redundant
relocations.
For example, RISCVAsmBackend::ForceRelocs (introduced to support mixed
relax/norelax code) leads to redundant relocations in the following
example adapted from #77436
```
.option norelax
j label
// For assembly input, RISCVAsmParser::ParseInstruction sets ForceRelocs (https://reviews.llvm.org/D46423).
// For direct object emission, RISCVELFStreamer sets ForceRelocs (#77436)
.option relax
call foo // linker-relaxable
.option norelax
j label // redundant relocation due to ForceRelocs
.option relax
label:
```
Root problem: The `isSymbolRefDifferenceFullyResolvedImpl` condition in
MCAssembler::evaluateFixup does not check whether two locations are
separated by a fragment whose size can be indeterminate due to linker
instruction (e.g. MCDataFragment with relaxation, or MCAlignFragment
due to indeterminate start offst).
This patch
* Updates the fragment walk code in
`attemptToFoldSymbolOffsetDifference` to treat MCRelaxableFragment
(for --riscv-asm-relax-branches) as fixed size after finishLayout.
* Adds a condition in `addReloc` to complement
`isSymbolRefDifferenceFullyResolvedImpl`.
* Removes the no longer needed `shouldForceRelocation`.
This fragment walk code path handles nicely handles
mixed relax/norelax case from
https://discourse.llvm.org/t/possible-problem-related-to-subtarget-usage/75283
and allows us to remove `MCSubtargetInfo` argument (#73721) as a follow-up.
This fragment walk code should be avoided in the absence of
linker-relaxable fragments within the current section.
Adjust two bolt/test/RISCV tests (#141310)
Pull Request: https://github.com/llvm/llvm-project/pull/140692
This is a follow on to 9b4de7 which handles the fixed vector cases. In
retrospect, this is simple enough if probably should have just been part
of the original commit, but oh well.
Vector shift word or double requires a shift amount vector of 31 or 63
which is too big for splat immediate and requires a multi-instruction
sequence. However the PPC instructions only use 5 or 6 bits of the shift
amount vector elements so an all ones mask, which we can generate
efficiently, works.
Make it easier for splat/element-equivalent detection by ensuring
constant splats contain no undefs.
Integer constants are limited to rematerializable zeros/ones values to
avoid unnecessary scalar_to_vector(int) -> load conversions - we can
relax this later if useful
Fix for a bug found by the AMD fuzzing project.
The legaliser would originally try to widen a small vector such as `<4 x
i1>` to a single `i16` during the legalisation of bitshifts, as it was
not originally written with consideration for vector operands. This
patch simply adds a guard to prohibit this transformation and allow
other legalisation transformations to step in.
The use of `LowerToPredicatedOp` here seems like a mistake as
`LowerToPredicatedOp` turns the SDValue passed to it into the desired
predicated node by copying over operands (and adding a predicate). This
results in two odd things here, the BITCASTs created and passed to
`LowerToPredicatedOp` are not used, only the operands of those bitcasts
are taken. Secondly, when a shuffle vector node is passed directly to
`LowerToPredicatedOp` to create a `REVD_MERGE_PASSTHRU` node an invalid
REV node is created as REV only takes one vector operand, but both
operands from the shuffle vector are copied to the new REV node. This is
not an issue in practice as the extra operand is ignored.
These issues were found by the verification added in #140472.
Part of #140472.
Note: Test changes only result in the vxf64 lowering matching the vxi64
lowering.
Note: This relands #140615 adding a ".count" suffix to the non-".all"
variants.
Our current intrinsic support for barrier intrinsics is confusing and
incomplete, with multiple intrinsics mapping to the same instruction and
intrinsic names not clearly conveying intrinsic semantics. Further, we
lack support for some variants. This change unifies the IR
representation to a single consistently named set of intrinsics.
- llvm.nvvm.barrier.cta.sync.aligned.all(i32)
- llvm.nvvm.barrier.cta.sync.aligned.count(i32, i32)
- llvm.nvvm.barrier.cta.arrive.aligned.count(i32, i32)
- llvm.nvvm.barrier.cta.sync.all(i32)
- llvm.nvvm.barrier.cta.sync.count(i32, i32)
- llvm.nvvm.barrier.cta.arrive.count(i32, i32)
The following Auto-Upgrade rules are used to maintain compatibility with
IR using the legacy intrinsics:
* llvm.nvvm.barrier0 --> llvm.nvvm.barrier.cta.sync.aligned.all(0)
* llvm.nvvm.barrier.n --> llvm.nvvm.barrier.cta.sync.aligned.all(x)
* llvm.nvvm.bar.sync --> llvm.nvvm.barrier.cta.sync.aligned.all(x)
* llvm.nvvm.barrier --> llvm.nvvm.barrier.cta.sync.aligned.count(x, y)
* llvm.nvvm.barrier.sync --> llvm.nvvm.barrier.cta.sync.all(x)
* llvm.nvvm.barrier.sync.cnt --> llvm.nvvm.barrier.cta.sync.count(x, y)
This makes little difference in the final output, as we manage to form this
after these are lowered to the _ieee operations. This does result in fewer steps
in the DAG, and helps prepare for changing the handling of minnum/maxnum.
- The double trap extension requires `mtval2' register, so add Zicsr as
required extension
Signed-off-by: Jerry Zhang Jian <jerry.zhangjian@sifive.com>
The semantics of the PARTIAL_REDUCE_SMLA with i32 result element, and i8
sources corresponds to vqdot. Analogously PARTIAL_REDUCE_UMLA
corresponds to vqdotu. There is currently no vqdotsu equivalent.
This patch is a starting place. We can extend this quite a bit more, and
I plan to take a look at the fixed vector lowering, the TTI hook to
drive loop vectorizer, and to try to integrate the reduction based
lowering I'd added for zvqdotq into this flow.
Widening the mask and padding with zeros doesn't work for scalable
vectors. Using VL produces less code for fixed vectors.
Similar was recently done for MLOAD.
On an AVX1-only target, if both operands of a 256-bit shuffle can be freely split (concatenations or splats), then prefer to lower as 128-bit shuffles using splitAndLowerShuffle
Some hardware (for example, certain AVR chips) have peripheral registers
mapped to the data space address 0. Although a volatile load/store on
`ptr null` already generates expected code, the wording in the LangRef
makes operations on null seem like undefined behavior in all cases. This
commit adds a comment that, for volatile operations, it may be defined
behavior to access the address null, if the architecture permits it. The
intended use case is MMIO registers with hard-coded addresses that
include bit-value 0. A simple CodeGen test is included for AVR, as an
architecture known to have this quirk, that does `load volatile` and
`store volatile` to `ptr null`, expecting to generate `lds <reg>, 0` and
`sts 0, <reg>`.
See [this
thread](https://rust-lang.zulipchat.com/#narrow/channel/213817-t-lang/topic/Adding.20the.20possibility.20of.20volatile.20access.20to.20address.200)
and [the
RFC](https://discourse.llvm.org/t/rfc-volatile-access-to-non-dereferenceable-memory-may-be-well-defined/86303)
for discussion and context.
The `[G]CSRXCHG` instruction must not use R0 or R1 as the `rj` operand,
as encoding `rj` as 0 or 1 will be interpreted as `[G]CSRRD` OR
`[G]CSRWR`, respectively, rather than `[G]CSRXCHG`.
This patch introduces a new register class `GPRNoR0R1` and updates the
`[G]CSRXCHG` instruction definition to use it for the `rj` operand,
ensuring the register allocator avoids assigning R0 or R1.
Fixes#140842
As with the other update scripts this takes the output of
-passes=print<gisel-value-tracking> and inserts the results into an
existing mir file. This means that the input is a lot like
update_analysis_test_checks.py, and the output needs to insert into a
mir file similarly to update_mir_test_checks.py. The code used to do the
inserting has been moved to common, to allow it to be reused. Otherwise
it tries to reuse the existing infrastructure, and
update_givaluetracking_test_checks is kept relatively short.
This adds support for lowering deinterleave and interleave intrinsics
for factors 3 5 and 7 into target specific memory intrinsics.
Notably this doesn't add support for handling higher factors constructed
from interleaving interleave intrinsics, e.g. factor 6 from interleave3
+ interleave2.
I initially tried this but it became very complex very quickly. For
example, because there's now multiple factors involved
interleaveLeafValues is no longer symmetric between interleaving and
deinterleaving. There's then also two ways of representing a factor 6
deinterleave: It can both be done as either 1 deinterleave3 and 3
deinterleave2s OR 1 deinterleave2 and 3 deinterleave3s.
I'm not sure the complexity of supporting arbitrary factors is warranted
given how we only need to support a small number of factors currently:
SVE only needs factors 2,3,4 whilst RVV only needs 2,3,4,5,6,7,8.
My preference would be to just add a interleave6 and deinterleave6
intrinsic to avoid all this ambiguity, but I'll defer this discussion to
a later patch.
Not adding them was leading to a crash when trying to expand these
pseudo instructions.
I've also fixed the register class types for the Xqcibi instructions in
these pseudo instructions which was incorrect and was exposed by the
machine verifier while running the test case added in this patch.
Fixes#140697
This fixes an issue in the waves/EU range calculation wherein, if the
`amdgpu-waves-per-eu` attribute exists and is valid, the entire
attribute may be spuriously and completely ignored if workgroup sizes
and LDS usage restrict the maximum achievable occupancy below the
subtarget maximum. In such cases, we should still honor the requested
minimum number of waves/EU, even if the requested maximum is higher than
the actually achievable maximum (but still within subtarget
specification).
As such, the added unit test `empty_at_least_2_lds_limited`'s waves/EU
range should be [2,4] after this patch, when it is currently [1,4] (i.e,
as if `amdgpu-waves-per-eu` was not specified at all).
Before e377dc4 the default maximum waves/EU was always set to the
subtarget maximum, trivially avoiding the issue.
Padding the mask using 0 elements doesn't work for scalable vectors. Use
VP_LOAD and change the VL instead.
This fixes crash for Zve32x. Test file was split since i64 isn't a valid
element type for Zve32x.
Fixes#140198.
It was using the full size of the vector as the SrcBitWidth. This patch
changes the code to split G_ASSERT_ZEXT away from the others (G_INTTOPTR
/ G_PTRTOINT / G_ZEXT / G_TRUNC) which are simpler, and make the code
match the SDAG equivalent.
These classes are redundant, as the untyped "Int" classes can be used
for all float operations. This change is intended to be as minimal as
possible and leaves the many potential simplifications and refactors
this exposes as future work.
Make sure to cover all the scalable types which are legal, plus
splitting. Make sure to cover all instructions. Not duplicating
vx testing at this time.
