This patch essentially re-lands
https://github.com/llvm/llvm-project/pull/114293 with the following
fixups
- `nosve2-aes` should disable the backend feature `FeatureSVEAES` such
that the set of existing instructions that this removes is unchanged.
- FMV dependencies now use the autogenerated `ExtensionDepencies`
structure (since https://github.com/llvm/llvm-project/pull/113281) so we
do not require the change to `AArch64FMV.td`.
This re-applies #96164 after revert in #102434.
Support the following relocations and assembly operators:
- `R_AARCH64_AUTH_ADR_GOT_PAGE` (`:got_auth:` for `adrp`)
- `R_AARCH64_AUTH_LD64_GOT_LO12_NC` (`:got_auth_lo12:` for `ldr`)
- `R_AARCH64_AUTH_GOT_ADD_LO12_NC` (`:got_auth_lo12:` for `add`)
`LOADgotAUTH` pseudo-instruction is introduced which is later expanded to
actual instruction sequence like the following.
```
adrp x16, :got_auth:sym
add x16, x16, :got_auth_lo12:sym
ldr x0, [x16]
autia x0, x16
```
If a resign is requested, like below, `LOADgotPAC` pseudo is used, and GOT
load is lowered similarly to `LOADgotAUTH`.
```
@var = global i32 0
define ptr @resign_globalvar() {
ret ptr ptrauth (ptr @var, i32 3, i64 43)
}
```
If FPAC bit is not set and auth instruction is emitted, a check+trap sequence
similar to one used for `AUT` pseudo is emitted to ensure auth success.
Both SelectionDAG and GlobalISel are suppported.
For FastISel, we fall back to SelectionDAG.
Tests starting with 'ptrauth-' have corresponding variants w/o this prefix.
See also specification
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#appendix-signed-got
As part of FEAT_PAuthLR, a new DWARF Frame Instruction was introduced,
`DW_CFA_AARCH64_negate_ra_state_with_pc`. This instructs Libunwind that
the PC has been used with the signing instruction. This change includes
three commits
- Libunwind support for the newly introduced DWARF Instruction
- CodeGen Support for the DWARF Instructions
- Reversing the changes made in #96377. Due to
`DW_CFA_AARCH64_negate_ra_state_with_pc`'s requirements to be placed
immediately after the signing instruction, this would mean the CFI
Instruction location was not consistent with the generated location when
not using FEAT_PAuthLR. The commit reverses the changes and makes the
location consistent across the different branch protection options.
While this does have a code size effect, this is a negligible one.
For the ABI information, see here:
853286c7ab/aadwarf64/aadwarf64.rst (id23)
…uctions (#112726)
This patch adds the assembly/disassembly for the following instructions:
CBB<cc>, CBH<cc>,
CB<cc>(immediate), CB<cc>(register)
CBBLE, CBBLO, CBBLS, CBBLT
CBHLE, CBHLO, CBHLS, CBHLT
CBGE, CBHS, CBLE, CBLS (immediate)
CBLE, CBLO, CBLS, CBLT(register)
According to [1]
[1]https://developer.arm.com/documentation/ddi0602
Co-authored-by: Momchil Velikov momchil.velikov@arm.com
Co-authored-by: Spencer Abson spencer.abson@arm.com
This patch was reverted(git commit 83c6e2f8f4d3) and is being submitted
again with the fix for buildbot failure in:
https://lab.llvm.org/buildbot/#/builders/25/builds/3493
The fix was to replaced a shift left of a possibly negative value with a
multiplication in DecodePCRelLabel9.
Because int64_t ImmVal is signed it needed to replace:
(ImmVal << 2)
with :
(ImmVal * 4)
Add support for the following Armv9.6-A memory systems extensions:
FEAT_LSUI - Unprivileged Load Store
FEAT_OCCMO - Outer Cacheable Cache Maintenance Operation
FEAT_PCDPHINT - Producer-Consumer Data Placement Hints
FEAT_SRMASK - Bitwise System Register Write Masks
as documented here:
https://developer.arm.com/documentation/109697/2024_09/Feature-descriptions/The-Armv9-6-architecture-extension
Co-authored-by: Jonathan Thackray <jonathan.thackray@arm.com>
---------
Co-authored-by: Jonathan Thackray <jonathan.thackray@arm.com>
This reverts commit dc84337f7b5bb2447e30f3364ebc863e9e04b8be.
Reversting because the sanitizer fails with the following error
llvm/lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp:502:56:
runtime error: left shift of negative value -256
Revert #106625 and fix attempt #106804"
There is another issue in
https://lab.llvm.org/buildbot/#/builders/169/builds/2690
directive-cpu-err.s
and the fix like #106804 fixed the overflow but fails CHECKs.
This reverts commit 10affaf894a72bee9b84ada77dc943b1bb03d02e.
This reverts commit 24977395592fb3a47d0356b6e9e6d25358a521c5.
The commit ff3f3a54e2d1 made it possible to enable transitively implied
features when parsing assembler directives. For example enabling sve2
also enables sve.
This patch allows disabling features which depend on each other. For
example disabling sve also disables sve2.
S.substr(N) is simpler than S.slice(N, StringRef::npos) and
S.slice(N, S.size()). Also, substr is probably better recognizable
than slice thanks to std::string_view::substr.
Implement FEAT_SME_B16B16 to enable ZA-targeting non-widening SME
BFloat16 instructions. Remove the now redundant FEAT_B16B16 which has
been replaced by FEAT_SVE_B16B16 and FEAT_SME_B16B16 (this commit), see
https://github.com/llvm/llvm-project/pull/101480/ for the details and
reasoning of this change to LLVM.
FEAT_SME_B16B16 is documented under the latest Armv9.4 feature
documentation:
https://developer.arm.com/documentation/109697/0100/Feature-descriptions/The-Armv9-4-architecture-extensio
- Changes to Clang AArch64 frontend
- Change target guard of SME2 ZA-targeting non-widening BFloat16
intrinsics to 'sme-b16b16'
- Changes to LLVM AArch64 backend
- llvm/lib/Target/AArch64/AArch64Features.td
- Create FeatureSMEB16B16, which implies FeatureSME2 and
FeatureSVEB16B16
- Remove FeatureB16B16
- Fix description of FeatureSVEB16B16
- llvm/lib/Target/AArch64/AArch64InstrInfo.td
- Create HasSMEB16B16 predicate
- llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td
- Change predictication of SME2 ZA-targeting non-widening BFloat16
instructions to new HasSMEB16B16
- llvm/lib/Target/AArch64/AArch64.td
- Add HasSMEB16B16 to SME2Unsupported (FEAT_SME_B16B16 implies
FEAT_SME2)
- llvm/lib/AArch64/AsmParser/AArch64AsmParser.cpp
- Remove flag 'b16b16' mapping to removed FeatureB16B16
- Add flag 'sme-b16b16' mapping to new FeatureSMEB16B16
- Changes to LLVM unit tests
- llvm/unittests/TargetParser/TargetParserTest.cpp
- Add new sme-b16b16 flag to existing target parser tests
- Add tests for the sme-b16b16 dependencies:
- 'sme-b16b16' should enable 'sme2', 'sve-b16b16'. - Remove 'b16b16'
from bf16 dependency test
- Added MC tests
- llvm/test/MC/AArch64/SME2p1
- To ensure that ZA-targeting multi-vector non-widening BFloat16
instructions are enabled by +sme-b16b16, and that this feature is
removed by +nosme-b61b6.
- Modidified tests
- All CodeGen, Semantic, and MC tests that are effected by the removal
of 'b16b16', have been modified to supply and/or expect 'sme-b16b16'
where appropriate.
This patch adds FeatureSVEB16B16 to the AArch64 backend in order to
represent the new behavior of FEAT_SVE_B16B16 (as described in the
latest [Armv9.4 extensions
documentation](https://developer.arm.com/documentation/109697/0100/Feature-descriptions/The-Armv9-4-architecture-extension?lang=en#md461-the-armv94-architecture-extension__FEAT_SVE_B16B16))
as well as a 'sve-b16b16' flag to enable it.
The predication of non-widening SVE BFloat16 instructions has changed to
require this feature, instead of the previously required and
soon-to-be-removed FeatureB16B16 which is enabled by the 'b16b16' flag.
Therefore, this change weakens the 'b16b16' flag in favour of
'sve-b16b16'. Existing tests that are effected by this have been
modified to use and/or expect 'sve-b16b16', and new tests have been
added to verify the behavior and implementation of 'sve-b16b16'.
This patch is in response to the response to the following changes.
The architecture features previously enabled by FEAT_SVE_B16B16 have
been relaxed such that it now implements:
- With FEAT_SVE2 : SVE non-widening BFloat16 instructions in
Non-streaming SVE mode
- With FEAT_SME2: SVE non-widening BFloat16 instructions when the
PE is in Streaming SVE mode and SME
Z-targeting multi-vector non-widening BFloat16 instructions.
- **It no longer implements** SME ZA-targeting non-widening
BFloat16 instructions.
The SME ZA-targeting non-widening BFloat16 instructions are implemented
by the new FEAT_SME_B16B16, **this patch does not change how this
architecture feature is enabled** ('+b16b16+sme2'). Only those that are
implemented by FEAT_SVE_B16B16 have been changed to require 'sve-b16b16'
instead of 'b16b16'.
New flags must be created to represent FEAT_SVE_B16B16 and
FEAT_SME_B16B16:
- 'sve-b16b16' enables the updated FEAT_SVE_B16B16 (described
here)
- 'sme-b16b16' will enable the new FEAT_SME_B16B16
- **This patch includes 'sve-b16b16' only**
A future patch will add 'sme-b16b16', SME ZA-targeting non-widening
BFloat16 instructions would then be guarded by '+sme-b16b16+sme2', and
'b16b16' can be removed.
Depends on #96158 and #96159
Support the following relocations and assembly operators:
- `R_AARCH64_AUTH_ADR_GOT_PAGE` (`:got_auth:` for `adrp`)
- `R_AARCH64_AUTH_LD64_GOT_LO12_NC` (`:got_auth_lo12:` for `ldr`)
- `R_AARCH64_AUTH_GOT_ADD_LO12_NC` (`:got_auth_lo12:` for `add`)
`LOADgotAUTH` pseudo-instruction is introduced which is later expanded
to actual instruction sequence like the following.
```
adrp x16, :got_auth:sym
add x16, x16, :got_auth_lo12:sym
ldr x0, [x16]
autia x0, x16
```
If a resign is requested, like below, `LOADgotPAC` pseudo is used, and
GOT load is lowered similarly to `LOADgotAUTH`.
```
@var = global i32 0
define ptr @resign_globalvar() {
ret ptr ptrauth (ptr @var, i32 3, i64 43)
}
```
Both SelectionDAG and GlobalISel are suppported. For FastISel, we fall
back to SelectionDAG.
Tests starting with 'ptrauth-' have corresponding variants w/o this
prefix.
See also specification
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst#appendix-signed-got
This patch removes FEAT_FPMR from list of available of architecture
features, instead enabling FMPR register by default.
Additionally dependencies between architectural features are added and
fixed.
7dc20ab introduced an extra COPY when spilling and filling a PNR
register, which can't be elided as the input (PNR predicate) and output
(PPR predicate) register classes differ. The patch adds a new register
class that covers both PPR and PNR so that STR_PXI and LDR_PXI can
take either of them, removing the need for the copy.
An immediate integer operand not prefixed with # can also be followed
by "lsl". Parse "lsl".
In the wild, edk2 ArmPkg/Include/AsmMacroIoLibV8.h has
`movz Reg, (Val) >> 16, lsl #16`.
Note: our support for paren expression not prefixed with # is not very
good. For example, `adds x3, x4, (1024>>0), lsl 12` fails to be parsed.
Shift and extend modifiers are parsed as separate operands. When a
symbol operand of a branch instruction has such a "bad" name,
AArch64AsmParser will report an error.
```
% cat a.c
void lsl(); void lsr(); void asr(); void ror(); void uxtb(); void sxtx();
void foo() { lsl(); asr(); asr(); ror(); uxtb(); sxtx(); }
% clang --target=aarch64 -c -save-temps a.c
a.s:15:8: error: expected #imm after shift specifier
bl lsl
^
a.s:16:8: error: expected #imm after shift specifier
bl asr
^
a.s:17:8: error: expected #imm after shift specifier
bl asr
^
a.s:18:8: error: expected #imm after shift specifier
bl ror
^
a.s:19:5: error: expected label or encodable integer pc offset
bl uxtb
^
a.s:20:5: error: expected label or encodable integer pc offset
bl sxtx
^
```
In contrast, gas correctly parses these instructions.
Fix#79729 by parsing shift/extend modifier after an immediate
value/register
Currently there are several bits of code in the AArch64 driver which
attempt to enforce dependencies between optional features in the -march=
and -mcpu= options. However, these are based on the list of feature
names being enabled/disabled, so they have a lot of logic to consider
the order in which features were turned on and off, which doesn't scale
well as dependency chains get longer.
This patch moves the code handling these dependencies to TargetParser,
and changes them to use a Bitset of enabled features. This makes it easy
to check which features are enabled, and is converted back to a list of
LLVM feature names once all of the command-line options are parsed.
The motivating example for this was the -mcpu=cortex-r82+nofp option.
Previously, the code handling the dependency between the fp16 and
fp16fml extensions did not consider the nofp modifier, so it added
+fullfp16 to the feature list. This should have been disabled by the
+nofp modifier, and also the backend did follow the dependency between
fullfp16 and fp, resulting in fp being turned back on in the backend.
Most of the dependencies added to AArch64TargetParser.h weren't known
about by clang before, I built that list by checking what the backend
thinks the dependencies between SubtargetFeatures are.
This implements assembly support for the Memory Systems Extensions
introduced as part of the Armv9.5-A architecture version.
The changes include:
* New subtarget feature for FEAT_TLBIW.
* New system registers for FEAT_HDBSS:
* HDBSSBR_EL2 and HDBSSPROD_EL2.
* New system registers for FEAT_HACDBS:
* HACDBSBR_EL2 and HACDBSCONS_EL2.
* New TLBI instructions for FEAT_TLBIW:
* VMALLWS2E1(nXS), VMALLWS2E1IS(nXS) and VMALLWS2E1OS(nXS).
* New system register for FEAT_FGWTE3:
* FGWTE3_EL3.
Add assembly/disassembly support for the new PAuthLR instructions
introduced in Armv9.5-A:
- AUTIASPPC/AUTIBSPPC
- PACIASPPC/PACIBSPPC
- PACNBIASPPC/PACNBIBSPPC
- RETAASPPC/RETABSPPC
- PACM
Documentation for these instructions can be found here:
https://developer.arm.com/documentation/ddi0602/2023-09/Base-Instructions/
This introduces assembly support for the Checked Pointer Arithmetic
Extension (FEAT_CPA), annouced as part of the Armv9.5-A architecture
version.
The changes include:
* New subtarget feature for FEAT_CPA
* New scalar instruction for pointer arithmetic
* ADDPT, SUBPT, MADDPT, and MSUBPT
* New SVE instructions for pointer arithmetic
* ADDPT (vectors, predicated), ADDPT (vectors, unpredicated)
* SUBPT (vectors, predicated), SUBPT (vectors, unpredicated)
* MADPT and MLAPT
* New ID_AA64ISAR3_EL1 system register
Mode details about the extension can be found at:
* https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-a-profile-architecture-developments-2023
* https://developer.arm.com/documentation/ddi0602/2023-09/
Co-authored-by: Rodolfo Wottrich <rodolfo.wottrich@arm.com>
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
This introduces assembly support for the Checked Pointer Arithmetic
Extension (FEAT_CPA), annouced as part of the Armv9.5-A architecture
version.
The changes include:
* New subtarget feature for FEAT_CPA
* New scalar instruction for pointer arithmetic
* ADDPT, SUBPT, MADDPT, and MSUBPT
* New SVE instructions for pointer arithmetic
* ADDPT (vectors, predicated), ADDPT (vectors, unpredicated)
* SUBPT (vectors, predicated), SUBPT (vectors, unpredicated)
* MADPT and MLAPT
* New ID_AA64ISAR3_EL1 system register
Mode details about the extension can be found at:
* https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-a-profile-architecture-developments-2023
* https://developer.arm.com/documentation/ddi0602/2023-09/
Co-authored-by: Rodolfo Wottrich <rodolfo.wottrich@arm.com>
