Rather than creating a dedicated ClangIR test file, the original test file for
this intrinsic is effectively reused:
* clang/test/CodeGen/AArch64/neon-intrinsics.c
“Effectively” meaning that the corresponding test is moved (rather than
literally reused) to a new file within the original AArch64 builtins test
directory:
* clang/test/CodeGen/AArch64/neon/intrinsics.c
This is necessary to avoid lowering unsupported examples from intrinsics.c with
`-fclangir`. The new file will eventually replace the original one once all
builtins from it can be lowered via ClangIR.
To facilitate test re-use, new LIT "feature" is added so that CIR tests can be
run conditionally, e.g. the following will only run when `CLANG_ENABLE_CIR` is
set:
```C
// RUN: %if cir %{%clang_cc1 ... %}
```
This sort of substitutions are documented in [2].
REFERENCES:
[1] https://developer.arm.com/architectures/instruction-sets/intrinsics/#f:@navigationhierarchiessimdisa=[Neon]&q=vceqzd_s64
[2] https://llvm.org/docs/TestingGuide.html#substitutions
This patch adds support in Clang for the RPRFM instruction, by adding
the following intrinsics:
```
void __pldx_range(unsigned int *access_kind*, unsigned int retention_policy,
signed int length*, unsigned int count, signed int stride,
size_t reuse distance, void const *addr);
void __pld_range(unsigned int access_kind*, unsigned int retention_policy,
uint64_t metadata, void const *addr);
```
The `__ARM_PREFETCH_RANGE` macro can be used to test whether these
intrinsics are implemented. If the RPRFM instruction is not available, this
instruction is a NOP.
This implements the following ACLE proposal:
https://github.com/ARM-software/acle/pull/423
This is the last of the generic instructions created from MVE
intrinsics. It was a little more awkward than the others due to it
taking a Type as one of the arguments. This creates a new function to
create the intrinsic we need.
Proposed in [this ACLE
proposal](https://github.com/ARM-software/acle/pull/409), this PR
implements widening FMMLA intrinsics.
- F16 to F32
- MF8 to F32
- MF8 to F16
Additional changes:
- IsOverloadCvt flag renamed to IsOverloadFirstandLast for clarity, as
the name implies conversion. Implementation remains unchanged.
Add support for the following new AArch64 Neon intrinsics:
```
float16x8_t vmmlaq_f16_mf8_fpm(float16x8_t, mfloat8x16_t, mfloat8x16_t, fpm_t);
float32x4_t vmmlaq_f32_mf8_fpm(float32x4_t, mfloat8x16_t, mfloat8x16_t, fpm_t);
```
This is the first step in removing some NEON reduction intrinsics that
duplicate the behaviour of their llvm.vector.reduce counterpart.
NOTE: The i8/i16 variants differ in that the NEON versions return an i32
result. However, this looks more about making their code generation
convenient with SelectionDAG disgarding the extra bits. This is only
relevant for the next phase because the Clang usage always truncate
their result, making llvm.vector.reduce a drop in replacement.
`UnqualPtrTy` didn't always match `llvm::PointerType::getUnqual`:
sometimes it returned a pointer that is not in address space 0 (notably
for SPIRV).
Since `UnqualPtrTy` was used as the "generic" or "default" pointer type,
this patch renames it to `DefaultPtrTy` to avoid confusion with LLVM's
`PointerType::getUnqual`.
Builtins for reading the streaming vector length are canonicalised to
use the aarch64.sme.cntsd intrinisic and a multiply, i.e.
- cntsb -> cntsd * 8
- cntsh -> cntsd * 4
- cntsw -> cntsd * 2
This patch also removes the LLVM intrinsics for cnts[b,h,w], and adds
patterns to improve codegen when cntsd is multiplied by a constant.
Use .i16.f16 intrinsic formats for intrinsics like vcvth_s16_f16.
Avoids issues with incorrect saturation that arise when using .i32.f16
formats for the same conversions.
Fixes https://github.com/llvm/llvm-project/issues/154343.
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
This option is confusingly named. What it actually controls is whether,
under the default of `-ffloat16-excess-precision=standard`, it is
beneficial for performance to perform calculations on float (without
intermediate rounding) or not. For `-ffloat16-excess-precision=none` the
LLVM `half` type will always be used, and all backends are expected to
legalize it correctly.
This is handled by the instcombine added in #147930; there is no need
for any clang-specific folding. NFC as all clang tests for
`__arm_in_streaming_mode()` used -O1, which applies the LLVM
instcombines.
FMV priority is the returned value of a polymorphic function. On RISC-V
and X86 targets a 32-bit value is enough. On AArch64 we currently need
64 bits and we will soon exceed that. APInt seems to be a suitable
replacement for uint64_t, presumably with minimal compile time overhead.
It allows bit manipulation, comparison and variable bit width.
Adds support for __sys Clang builtin for AArch64
__sys is a long existing MSVC intrinsic used to manage caches, tlbs, etc
by writing to system registers:
* It takes a macro-generated constant and uses it to form the AArch64 SYS instruction which is MSR with op0=1. The macro drops op0 and expects the implementation to hardcode it to 1 in the encoding.
* Volume use is in systems code (kernels, hypervisors, boot environments, firmware)
* Has an unused return value due to MSVC cut/paste error
Implementation:
* Clang builtin, sharing code with Read/WriteStatusReg
* Hardcodes the op0=1
* Explicitly returns 0
* Code-format change from clang-format
* Unittests included
* Not limited to MSVC-environment as its generally useful and neutral
This marks ffloor as legal providing that armv8 and neon is present (or
fullfp16 for the fp16 instructions). The existing arm_neon_vrintm
intrinsics are auto-upgraded to llvm.floor.
If this is OK I will update the other vrint intrinsics.
CreateVScale took a scaling parameter that had a single use outside of
IRBuilder with all other callers having to create a redundant
ConstantInt. To work round this some code perferred to use
CreateIntrinsic directly.
This patch simplifies CreateVScale to return a call to the llvm.vscale()
intrinsic and nothing more. As well as simplifying the existing call
sites I've also migrated the uses of CreateIntrinsic.
Whilst IRBuilder used CreateVScale's scaling parameter as part of the
implementations of CreateElementCount and CreateTypeSize, I have
follow-on work to switch them to the NUW varaiety and thus they would
stop using CreateVScale's scaling as well. To prepare for this I have
moved the multiplication and constant folding into the implementations
of CreateElementCount and CreateTypeSize.
As a final step I have replaced some callers of CreateVScale with
CreateElementCount where it's clear from the code they wanted the
latter.
This patch adds fp8 variants to existing intrinsics, whose operation
doesn't depend on arguments being a specific type.
It also changes mfloat8 type representation in memory from `i8` to
`<1xi8>`
Most callers want a constant index. Instead of making every caller
create a ConstantInt, we can do it in IRBuilder. This is similar to
createInsertElement/createExtractElement.
This is an expensive header, only include it where needed. Move some
functions out of line to achieve that.
This reduces time to build clang by ~0.5% in terms of instructions
retired.
Currently arm_neon.h emits C-style casts to do vector type casts. This
relies on implicit conversion between vector types to be enabled, which
is currently deprecated behaviour and soon will disappear. To ensure
NEON code will keep working afterwards, this patch changes all this
vector type casts into bitcasts.
Co-authored-by: Momchil Velikov <momchil.velikov@arm.com>
clang/lib/CodeGen/CGBuiltin.cpp is over 1MB long (>23k LoC), and can
take minutes to recompile (depending on compiler and host system) when
modified, and 5 seconds for clangd to update for every edit. Splitting
this file was discussed in this thread:
https://discourse.llvm.org/t/splitting-clang-s-cgbuiltin-cpp-over-23k-lines-long-takes-1min-to-compile/
and the idea has received a number of +1 votes, hence this change.
