[Recommit of e88ba6d975d887ca001cae30bfa0c53d91165148]
According to the specification in
https://github.com/ARM-software/acle/pull/309 this adds the intrinsics
void_svadd_za16_vg1x2_f16(uint32_t slice, svfloat16x2_t zn)
__arm_streaming __arm_inout("za");
void_svadd_za16_vg1x4_f16(uint32_t slice, svfloat16x4_t zn)
__arm_streaming __arm_inout("za");
void_svsub_za16_vg1x2_f16(uint32_t slice, svfloat16x2_t zn)
__arm_streaming __arm_inout("za");
void_svsub_za16_vg1x4_f16(uint32_t slice, svfloat16x4_t zn)
__arm_streaming __arm_inout("za");
as well as the corresponding `bf16` variants.
For all of the following reductions:
vector.reduce.or
vector.reduce.and
vector.reduce.xor
vector.reduce.add
vector.reduce.mul
vector.reduce.umin
vector.reduce.umax
vector.reduce.smin
vector.reduce.smax
vector.reduce.fmin
vector.reduce.fmax
if the input operand is the result of a vector.reverse then we can
perform a reduction on the vector.reverse input instead since the answer
is the same. If the reassociation is permitted we can also do the same
folds for these:
vector.reduce.fadd
vector.reduce.fmul
Instead of widening e.g. i8 cttz(x) to i16 cttz(x | 0x100), use the more
optimizable form cttz_zero_undef(x | 0x100) since the widened operand is
definitely not zero.
See the test file. At head, this crashes with
```
assertion failed at llvm/lib/Support/APInt.cpp:492 in
uint64_t llvm::APInt::extractBitsAsZExtValue(unsigned int, unsigned int) const:
bitPosition < BitWidth && (numBits + bitPosition) <= BitWidth &&
"Illegal bit extraction"
```
After internal intrinsic 'spv_switch' is processed we need to delete
G_BLOCK_ADDR instructions that were generated to keep track of the
corresponding basic blocks. If we just delete G_BLOCK_ADDR instructions
with BlockAddress operands, this leaves their BasicBlock counterparts in
a "address taken" status. This would make AsmPrinter to generate a
series of unneeded labels of a `"Address of block that was removed by
CodeGen"` kind. This PR is to ensure that we don't have a dangling
BlockAddress constants by zapping the BlockAddress nodes, and only after
that proceed with erasing G_BLOCK_ADDR instructions.
See also https://github.com/llvm/llvm-project/pull/87823 for more
details.
This PR generation of argument types of internal intrinsic functions
`spv_const_composite` and `spv_track_constant`, so that composite
constants of ConstantVector type preserve their correct type in
transformation passes and can be successfully used further by LLVM
intrinsic functions.
The added test case serves two purposes: it is to check the above
mentioned fix and to demonstrate that a call to __builtin_alloca() maps
to instructions from SPV_INTEL_variable_length_array when this extension
is available.
This PR is to ensure that internal intrinsic functions for PHI's operand
are inserted at the correct positions and don't break rules of
instruction domination and PHI nodes grouping at top of basic block.
The only tricky point here is PlaceSafepoints has an awful hack where
it's creating a legacy PassManager inside it's runImpl, which was not
propagating the incoming TLI. This means there's an implicit bug fix,
where PlaceSafepoints would have been treating too many calls as
builtins.
I'm trying to delete the default constructor altogether, but this seems
to be more difficult.
The LHS and RHS are of SmallString and StringRef, respectively. We
can safely use operator==(StringRef, SringRef) with one implicit
conversion from SmallString to StringRef.
The 32 bits arguments and returns on LA64 are always sign extended to
i64. So we should be taking this into account around libcalls.
Reviewed By: heiher, SixWeining
Pull Request: https://github.com/llvm/llvm-project/pull/92375
If an ifunc has local linkage, do not add it into ref edges and mark its
referencer (a function or global variable) not eligible for import. An
ifunc doesn't have summary and ThinLTO cannot promote it. Importing the
referencer may cause linkage errors.
To reference a similar fix, https://reviews.llvm.org/D158961 marks
callers of local ifunc not eligible for import to fix
https://github.com/llvm/llvm-project/issues/58740
In late 2021, both Intel and AMD finally documented that every
AVX-capable CPU has always been guaranteed to execute aligned 16-byte
loads/stores atomically, and further, guaranteed that all future CPUs
with AVX will do so as well.
Therefore, we may use normal SSE 128-bit load/store instructions to
implement atomics, if AVX is enabled.
Per AMD64 Architecture Programmer's manual, 7.3.2 Access Atomicity:
> Processors that report [AVX] extend the atomicity for cacheable,
> naturally-aligned single loads or stores from a quadword to a double
> quadword.
Per Intel's SDM:
> Processors that enumerate support for Intel(R) AVX guarantee that the
> 16-byte memory operations performed by the following instructions will
> always be carried out atomically:
> - MOVAPD, MOVAPS, and MOVDQA.
> - VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128.
> - VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded with
> EVEX.128 and k0 (masking disabled).
This was also confirmed to be true for Zhaoxin CPUs with AVX, in
https://gcc.gnu.org/PR104688
This patch adds comments for writeMemProf{V0,V1,V2} in a
version-specific manner. The mostly repetitive nature of the comments
is somewhat unfortunate but intentional to make it easy to retire
older versions.
Without this patch, the comment just before writeMemProf documents the
Version1 format, which is very confusing.
LLVM is built with C++17, where left shift of any negative value is still UB.
Detected with UBSan on the buildbot.
This reverts commit 0647d1035cb208195e002b38089b82004b6f7b92.
This is valid as long as the sign of the wrap flag doesn't differ from
the sign of the `pred`.
Proofs: https://alive2.llvm.org/ce/z/35NsrR
NB: The online Alive2 hasn't been updated with `trunc nuw/nsw`
support, so the proofs must be reproduced locally.
Closes#87935
`(icmp ule/ult (add nuw X, Y), C)` implies both `(icmp ule/ult X, C)` and
`(icmp ule/ult Y, C)`. We can use this to deduce leading zeros in `X`/`Y`.
`(icmp uge/ugt (sub nuw X, Y), C)` implies `(icmp uge/uge X, C)` . We
can use this to deduce leading ones in `X`.
Proofs: https://alive2.llvm.org/ce/z/sc5k22Closes#87180
This patch groups the three Memprof data structures into a struct
named IndexedMemProfData and teaches InstrProfWriter to use it. This
way, we can pass IndexedMemProfData to writeMemProf and its helpers
instead of individual data structures.
As a follow-up, we can use the new struct in MemProfReader also. That
in turn allows loadInput in llvm-profdata to move the MemProf data
into the writer context, saving a few seconds for a large MemProf
profile.
I accidentally left out the code to transfer sret attributes to entry
thunks, so values weren't being passed in the right registers, and the
sret pointer wasn't returned in the correct register.
Fixes#90229
This allows us to handle cases where the constant has already been type legalized behind a bitcast
Despite calling ComputeKnownBits I'm not seeing any notable change in compile time.
Fixes#88935
Toggling reduction by-ref broke when multiple reduction clauses were
used. Decisions made for the by-ref status for later clauses could then
invalidate decisions for earlier clauses. For example,
```
reduction(+:scalar,scalar2) reduction(+:array)
```
The first clause would choose by value reduction and generate by-value
reduction regions, but then after this the second clause would force
by-ref to support the array argument. But by the time the second clause
is processed, the first clause has already had the wrong kind of
reduction regions generated.
This is solved by toggling whether a variable should be reduced by
reference per variable. In the above example, this allows only `array`
to be reduced by ref.
The znver3/4 scheduler models have previously associated the LoopMicroOpBufferSize with the maximum size of their op caches, and when this led to quadratic complexity issues this were reduced to a value of 512 uops, based mainly on compilation time and not its effectiveness on runtime performance.
From a runtime performance POV, a large LoopMicroOpBufferSize leads to a higher number of loop unrolls, meaning the cpu has to rely on the frontend decode rate (4 ins/cy max) for much longer to fill the op cache before looping begins and we make use of the faster op cache rate (8/9 ops/cy).
This patch proposes we instead cap the size of the LoopMicroOpBufferSize based off the maximum rate from the op cache (znver3 = 8op/cy, znver4 = 9op/cy) and the branch misprediction penalty from the opcache (~12cy) as a estimate of the useful number of ops we can unroll a loop by before mispredictions are likely to cause stalls. This isn't a perfect metric, but does try to be closer to the spirit of how we use LoopMicroOpBufferSize in the compiler vs the size of a similar naming buffer in the cpu.
llvm-project/llvm/lib/Transforms/Utils/DemoteRegToStack.cpp:54:23:
error: comparison of integers of different signs: 'int' and 'unsigned int' [-Werror,-Wsign-compare]
for (int i = 0; i < CBI->getNumSuccessors(); i++) {
~ ^ ~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
