Add support for the following Armv9.6-A memory systems extensions:
FEAT_LSUI - Unprivileged Load Store
FEAT_OCCMO - Outer Cacheable Cache Maintenance Operation
FEAT_PCDPHINT - Producer-Consumer Data Placement Hints
FEAT_SRMASK - Bitwise System Register Write Masks
as documented here:
https://developer.arm.com/documentation/109697/2024_09/Feature-descriptions/The-Armv9-6-architecture-extension
Co-authored-by: Jonathan Thackray <jonathan.thackray@arm.com>
---------
Co-authored-by: Jonathan Thackray <jonathan.thackray@arm.com>
This feature provided CPM_IOACC_CTL_EL3, a lone system register that has
been carried over since the original ARM64 implementation, where it was
the only processor-specific register in a long list of architectural
sysregs. We don't need it here.
It's been used as a generic processor-specific sysreg in tests, but the
functionality they target is now better covered in other more exhaustive
tests.
This patch removes FEAT_FPMR from list of available of architecture
features, instead enabling FMPR register by default.
Additionally dependencies between architectural features are added and
fixed.
Most of the floating-point instructions are already gated on the
fp-armv8 subtarget feature (or some other feature), but most of the load
and store instructions, and one move instruction, were not.
I found this list of instructions with a script which consumes the
output of llvm-tblgen --dump-json, looking for instructions which have
an FPR operand but no predicate. That script now finds zero
instructions.
This only affects assembly, not codegen, because the floating-point
types and registers are already not marked as legal when the FPU is
disabled, so it is impossible for any of these to be selected.
This implements assembly support for the Memory Systems Extensions
introduced as part of the Armv9.5-A architecture version.
The changes include:
* New subtarget feature for FEAT_TLBIW.
* New system registers for FEAT_HDBSS:
* HDBSSBR_EL2 and HDBSSPROD_EL2.
* New system registers for FEAT_HACDBS:
* HACDBSBR_EL2 and HACDBSCONS_EL2.
* New TLBI instructions for FEAT_TLBIW:
* VMALLWS2E1(nXS), VMALLWS2E1IS(nXS) and VMALLWS2E1OS(nXS).
* New system register for FEAT_FGWTE3:
* FGWTE3_EL3.
FEAT_ATS1A adds three new AT system instruction aliases. This feature is
optional from v8.9a/v9.4a. FEAT_ATS1A is a very late addition to the
2022 A-profile VMSA extension, and has not yet been added to the public
docs available on developer.arm.com
These AT instructions are added without a command-line flag or feature,
because it is system-instruction only, and FEAT_S1PIE also has no
command-line flag.
Differential Revision: https://reviews.llvm.org/D146962
This implements support for two new 2022 A-profile extensions:
- FEAT_CHK - Check Feature Status Extension
- FEAT_GCS - Guarded Control Stacks
FEAT_CHK is mandatory from armv8.0-a, but is in the hint space so
there's no clang command-line flag for it, and we only print the hint as
`chkfeat x16` at v8.9a and above, to be compatible when using a
non-integrated assembler that might not yet know about the extension.
FEAT_GCS is optional from armv9.4-a onwards. It is enabled using `+gcs`
in a clang `-march=` or `-mcpu=` option string, or using a
`.arch_extension gcs` assembly directive.
This patch includes changes by Ties Stuij, Tomas Matheson, and Keith
Walker.
Differential Revision: https://reviews.llvm.org/D145563
This adds support for the missing `PIRE0_EL12` system register, part of
v8.9-A/v9.4-A's Permission Indirection Extension.
Patch by Son Tuan Vu.
Reviewed By: tmatheson
Differential Revision: https://reviews.llvm.org/D141400
The `CNTPCTSS_EL0` and `CNTVCTSS_EL0` system registers, part of
Armv8.6-A's Enhanced Counter Virtualization extension (FEAT_ECV), are
described as read-only in the Arm ARM. This updates their implementation
to match the spec.
Original patch by Simon Tatham.
Reviewed By: lenary
Differential Revision: https://reviews.llvm.org/D141398
The encodings of these 2 RME MEC instructions are
incorrect and need swapping:
- DC CIPAE
- DC CIGDPAE
The correct encoding is:
Operation op0 op1 CRn CRm op2
DC CIPAE, Xt 0b01 0b100 0b0111 0b1110 0b000
DC CIGDPAE, Xt 0b01 0b100 0b0111 0b1110 0b111
Differential Revision: https://reviews.llvm.org/D141689
This adds support for the missing Non-maskable Interrupts (FEAT_NMI)
feature from armv8.8-A, which consists of the `ALLINT` pstate register.
This is a second iteration of the patch from D131389, building on top of
the D139925 changes that enable better support for `msr (immediate)`
instructions that take 1-bit immediates.
Contributors:
* David Candler
* Tomas Matheson
* Sam Elliott
Reviewed By: lenary, tmatheson
Differential Revision: https://reviews.llvm.org/D140216
This adds support for the new PM pstate system register introduced by
the v9.4-A Exception-based Event Profiling extension (FEAT_EBEP).
The new PM pstate register takes a 1-bit immediate and requires
different values to be specified for the higher bits of the Crm field.
To enable that, this patch creates an explicit separation between the
pstate system registers that take 4-bit and 1-bit immediate operands,
allowing each entry to specify the value for the 3 high bits of Crm.
This also updates other pstate registers to correctly accept 4-bit
immediates, matching their decoding specification from the Arm ARM.
These include: `PAN`, `UAO`, `DIT` and `SSBS`.
More information about this extension and the new register can be found
at:
* https://developer.arm.com/documentation/ddi0601/2022-09/AArch64-Registers/PM--PMU-Exception-Mask
Contributors:
* Lucas Prates
* Sam Elliott
Reviewed By: lenary
Differential Revision: https://reviews.llvm.org/D139925
This feature adds upstream support for FEAT_RASv2 and FEAT_PFAR. Both
are system-register-only, but FEAT_RAS is behind the command-line
extension "+ras", so FEAT_RASv2 is behind "+rasv2".
This patch includes support for ID_AA64MMFR4_EL1. This is an ID system
register so it is not behind any feature flags.
Differential Revision: https://reviews.llvm.org/D139936
Virtual Memory System Architecture (VMSA)
This is part of the 2022 A-Profile Architecture extensions and adds support for
the following:
- Translation Hardening Extension (FEAT_THE)
- 128-bit Page Table Descriptors (FEAT_D128)
- 56-bit Virtual Address (FEAT_LVA3)
- Support for 128-bit System Registers (FEAT_SYSREG128)
- System Instructions that can take 128-bit inputs (FEAT_SYSINSTR128)
- 128-bit Atomic Instructions (FEAT_LSE128)
- Permission Indirection Extension (FEAT_S1PIE, FEAT_S2PIE)
- Permission Overlay Extension (FEAT_S1POE, FEAT_S2POE)
- Memory Attribute Index Enhancement (FEAT_AIE)
New instructions added:
- FEAT_SYSREG128 adds MRRS and MSRR.
- FEAT_SYSINSTR128 adds the SYSP instruction and TLBIP aliases.
- FEAT_LSE128 adds LDCLRP, LDSET, and SWPP instructions.
- FEAT_THE adds the set of RCW* instructions.
Specs for individual instructions can be found here:
https://developer.arm.com/documentation/ddi0602/2022-09/Base-Instructions/
Contributors:
Keith Walker
Lucas Prates
Sam Elliott
Son Tuan Vu
Tomas Matheson
Differential Revision: https://reviews.llvm.org/D138920
The following system registers have been missing upstream:
- ID_DFR1_EL1
- AMCG1IDR_EL0 (present when FEAT_AMUv1p1 implemented - optional from v8.6-a)
- HAFGRTR_EL2 (present when FEAT_AMUv1 and FEAT_FGT are implemented)
With regards to HAFGRTR_EL2, this is only present when you have both
extensions. As FEAT_FGT is part of a later architecture, we group it
with those registers. In all honesty, this is a good example of the
kinds of place where just enabling all system registers all the time
would be easiest.
Differential Revision: https://reviews.llvm.org/D138553
This adds support for the 2022 Debug and PMU extensions that are part of
the v8.9-A and v9.4-A architecture versions. This includes:
* New archtecture extension for the v9.4-A Instrumentation Extension
(FEAT_ITE), including 'trcit' instruction and system registers
* New system registers for:
* 2022 Debug features (FEAT_Debugv8p9)
* 2022 Performance Monitors Extension features (FEAT_PMUv3p9)
* PMU Snapshot extension (FEAT_PMUv3_SS)
* PMU Fixed-function instruction counter (FEAT_PMUv3_ICNTR)
* System Performance Monitors Extension (FEAT_SPMU)
* Synchornous-exception-based event profiling (FEAT_SEBEP)
* Fine Grained Traps Extension (FEAT_FGT2)
* SPE Data Source filtering (FEAT_SPE_FDS)
More information on the new extensions can be found on:
* https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-a-profile-architecture-2022
* https://developer.arm.com/downloads/-/exploration-tools
Changes by Son Tuan Vu, Sam Elliott and me.
Reviewed By: tmatheson
Differential Revision: https://reviews.llvm.org/D138556
This patch implements the 2022 Architecture General Data-Processing Instructions
They include:
Common Short Sequence Compression (CSSC) instructions
- scalar comparison instructions
SMAX, SMIN, UMAX, UMIN (32/64 bits) with or without immediate
- ABS (absolute), CNT (count non-zero bits), CTZ (count trailing zeroes)
- command-line options for CSSC
Associated with these instructions in the documentation is the Range Prefetch
Memory (RPRFM) instruction, which signals to the memory system that data memory
accesses from a specified range of addresses are likely to occur in the near
future. The instruction lies in hint space, and is made unconditional.
Specs for the individual instructions can be found here:
https://developer.arm.com/documentation/ddi0602/2022-09/Base-Instructions/
contributors to this patch:
- Cullen Rhodes
- Son Tuan Vu
- Mark Murray
- Tomas Matheson
- Sam Elliott
- Ties Stuij
Reviewed By: lenary
Differential Revision: https://reviews.llvm.org/D138488
This is used by disassemblers: `llvm-mc -disassemble -mattr=` and `llvm-objdump --mattr=`.
The main use case is for llvm-objdump to disassemble all known instructions
(D128030).
In user-facing tools, "all" is intentionally not supported in producers:
integrated assembler (`.arch_extension all`), clang -march (`-march=armv9.3a+all`).
Due to the code structure, `llvm-mc -mattr=+all` `llc -mattr=+all` are not
rejected (they are internal tool). Add `llvm/test/CodeGen/AArch64/mattr-all.ll`
to catch behavior changes.
AArch64SysReg::SysReg::haveFeatures: check `FeatureAll` to print
`AArch64SysReg::SysReg::AltName` for some system registers (e.g. `ERRIDR_EL1, RNDR`).
AArch64.td: add `AssemblerPredicateWithAll` to additionally test `FeatureAll`.
Change all `AssemblerPredicate` (except `UseNegativeImmediates`) to `AssemblerPredicateWithAll`.
utils/TableGen/{DecoderEmitter,SubtargetFeatureInfo}.cpp: support arbitrarily
nested all_of, any_of, and not.
Note: A predicate supports all_of, any_of, and not. For a target (though
currently not for AArch64) an encoding may be disassembled differently with
different target features.
Note: AArch64MCCodeEmitter::computeAvailableFeatures is not available to
the disassembler.
Reviewed By: peter.smith, lenary
Differential Revision: https://reviews.llvm.org/D128029
The Arm architecture reference manual (ARM DDI 0487H.a section
D13.5.12) lists every field in the register as RO, and does not list
an MSR instruction that writes it. So we should be defining it as an
ROSysReg, not an RWSysReg.
Reviewed By: vhscampos
Differential Revision: https://reviews.llvm.org/D123111
The Armv8-R.64 architecture defines numbered MPU region registers with
indices 1-15, not 0-15. So there's no such register as PRBAR0_EL2 or
PRLAR0_EL1 (for example). The encodings that they would occupy are
used for the unnumbered PRBAR_ELn and PRLAR_ELn registers.
Reviewed By: labrinea
Differential Revision: https://reviews.llvm.org/D117755
This change introduces subtarget features to predicate certain
instructions and system registers that are available only on
'A' profile targets. Those features are not present when
targeting a generic CPU, which is the default processor.
In other words the generic CPU now means the intersection of
'A' and 'R' profiles. To maintain backwards compatibility we
enable the features that correspond to -march=armv8-a when the
architecture is not explicitly specified on the command line.
References: https://developer.arm.com/documentation/ddi0600/latest
Differential Revision: https://reviews.llvm.org/D110065
Following on from an earlier patch that introduced support for -mtune
for AArch64 backends, this patch splits out the tuning features
from the processor features. This gives us the ability to enable
architectural feature set A for a given processor with "-mcpu=A"
and define the set of tuning features B with "-mtune=B".
It's quite difficult to write a test that proves we select the
right features according to the tuning attribute because most
of these relate to scheduling. I have created a test here:
CodeGen/AArch64/misched-fusion-addr-tune.ll
that demonstrates the different scheduling choices based upon
the tuning.
Differential Revision: https://reviews.llvm.org/D111551
FeaturePMU was created in AArch64 to accommodate one missing system
register, PMMIR_EL1, in commit ffcd7698aea7bcbb2b4edffc484793e1ff47b85d.
However, the Performance Monitors extension already had a target
feature, which is called FeaturePerfMon. Therefore, FeaturePMU is
redundant.
This patch removes FeaturePMU and merges its contents into
FeaturePerfMon.
Reviewed By: dnsampaio
Differential Revision: https://reviews.llvm.org/D109246
This patch adds the new system registers introduced in SME:
- ID_AA64SMFR0_EL1 (ro) SME feature identifier.
- SMCR_ELx (r/w) streaming mode control register for configuring
effective SVE Streaming SVE Vector length when the PE is in
Streaming SVE mode.
- SVCR (r/w) streaming vector control register, visible at all
exception levels. Provides access to PSTATE.SM and PSTATE.ZA
using MSR and MRS instructions.
- SMPRI_EL1 (r/w) streaming mode execution priority register.
- SMPRIMAP_EL2 (r/w) streaming mode priority mapping register.
- SMIDR_EL1 (ro) streaming mode identification register.
- TPIDR2_EL0 (r/w) for use by SME software to manage per-thread
SME context.
- MPAMSM_EL1 (r/w) MPAM (v8.4) streaming mode register, for
labelling memory accesses performed in streaming mode.
Also added in this patch are the SME mode change instructions.
Three MSR immediate instructions are implemented to set or clear
PSTATE.SM, PSTATE.ZA, or both respectively:
- MSR SVCRSM, #<imm1>
- MSR SVCRZA, #<imm1>
- MSR SVCRSMZA, #<imm1>
The following smstart/smstop aliases are also implemented for
convenience:
smstart -> MSR SVCRSMZA, #1
smstart sm -> MSR SVCRSM, #1
smstart za -> MSR SVCRZA, #1
smstop -> MSR SVCRSMZA, #0
smstop sm -> MSR SVCRSM, #0
smstop za -> MSR SVCRZA, #0
The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2021-06
Reviewed By: david-arm
Differential Revision: https://reviews.llvm.org/D105576
This adds support for Armv9-A's Realm Management Extension, including
three new system registers - MFAR_EL3, GPCCR_EL3 and GPTBR_EL3 - and
four new TLBI instructions.
The reference for the Realm Management Extension can be found at: https://developer.arm.com/documentation/ddi0615/aa.
Based on patches by Victor Campos.
Reviewed By: dmgreen
Differential Revision: https://reviews.llvm.org/D104773
The Arm Architecture Reference Manual says that the SystemHintOp_BTI
opcode is prefered when CRm:op2 matches 0100:xx0, but llvm-mc
currently accepts 0100:xxx, which isn't right.
Differential Revision: https://reviews.llvm.org/D102415
This is an addition to the existing Statistical Profiling extension, which
introduces an extra system register that is enabled by the new 'spe-eef'
subtarget feature.
Patch written by Simon Tatham.
Reviewed By: ostannard
Differential Revision: https://reviews.llvm.org/D92391
This introduces asm support for the Branch Record Buffer extension, through
the new 'brbe' subtarget feature. It consists of a new set of system registers
that enable the handling of branch records.
Patch written by Simon Tatham.
Reviewed By: ostannard
Differential Revision: https://reviews.llvm.org/D92389
This adds support for the v8.7-A LD64B/ST64B Accelerator extension
through a subtarget feature called "ls64". It adds four 64-byte
load/store instructions with an operand in the new GPR64x8 register
class, and one system register that's part of the same extension.
Based on patches written by Simon Tatham.
Reviewed By: ostannard
Differential Revision: https://reviews.llvm.org/D91775
This introduces support for the v8.7-A architecture through a new
subtarget feature called "v8.7a". It adds two new "WFET" and "WFIT"
instructions, the nXS limited-TLB-maintenance qualifier for DSB and TLBI
instructions, a new CPU id register, ID_AA64ISAR2_EL1, and the new
HCRX_EL2 system register.
Based on patches written by Simon Tatham and Victor Campos.
Reviewed By: ostannard
Differential Revision: https://reviews.llvm.org/D91772
Some use cases (e.g. kernel devs) have strict requirements to only enable
features available with -march=armv8-a, e.g. no armv8.1-a. Enabling RAS 1.1 in
all AArch64 means they can consider to support it.
Bear in mind that the first versions of the Armv8 architecture still do not
support RAS 1.1. This patch only lets devs write code with the user-friendly
register mnemonic instead of the ugly generic S<op0>_<op1>_<Cn>_<Cm>_<op2>.
They still need to place runtime checks to make sure that the CPU to run on
supports RAS 1.1.
Differential Revision: https://reviews.llvm.org/D90594
This adds support for -mcpu=cortex-r82. Some more information about this
core can be found here:
https://www.arm.com/products/silicon-ip-cpu/cortex-r/cortex-r82
One note about the system register: that is a bit of a refactoring because of
small differences between v8.4-A AArch64 and v8-R AArch64.
This is based on patches from Mark Murray and Mikhail Maltsev.
Differential Revision: https://reviews.llvm.org/D88660
Summary:
AArch64's system register ERXTS_EL1 is present in the backend as a
component of the Arm Reliability, Availability and Serviceability (RAS)
extension. However, it has been removed from the specification before
its final release.
This patch removes the register.
Reviewers: SjoerdMeijer, DavidSpickett
Reviewed By: DavidSpickett
Subscribers: DavidSpickett, kristof.beyls, hiraditya, danielkiss, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D79007
Summary:
This patch upstreams support for the ARMv8.6A Enhanced Counter Virtualization
(ECV) extension, which adds 6 new system registers.
See ARMv8.6-ECV in the Arm Architecture Reference Manual Armv8 for more
information.
Reviewers: t.p.northover, rengolin, SjoerdMeijer, pcc, ab, chill
Reviewed By: SjoerdMeijer
Subscribers: LukeGeeson, ostannard, kristof.beyls, hiraditya, danielkiss, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D77094
Summary:
This patch upstreams support for the ARMv8.6A Fine Grain Traps (FGT)
extension, which adds 5 new system registers.
See ARMv8.6-FGT in the Arm Architecture Reference Manual Armv8 for more
information.
Reviewers: t.p.northover, rengolin, SjoerdMeijer, ab, momchil.velikov
Reviewed By: SjoerdMeijer
Subscribers: LukeGeeson, ostannard, kristof.beyls, hiraditya, danielkiss, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D76991
Summary:
This patch upstreams v8.6A activity monitors virtualization
assembler support, which consists of 32 new system
registers (two groups, each with 16 numbered registers).
See ARMv8.6-AMU in the Arm Architecture Reference Manual Armv8 for more
information.
Reviewers: t.p.northover, rengolin, SjoerdMeijer, ab, john.brawn, ostannard
Reviewed By: ostannard
Subscribers: LukeGeeson, dnsampaio, ostannard, kristof.beyls, hiraditya, danielkiss, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D76998
For context, the proposed RISC-V bit manipulation extension has a subset
of instructions which require one of two SubtargetFeatures to be
enabled, 'zbb' or 'zbp', and there is no defined feature which both of
these can imply to use as a constraint either (see comments in D65649).
AssemblerPredicates allow multiple SubtargetFeatures to be declared in
the "AssemblerCondString" field, separated by commas, and this means
that the two features must both be enabled. There is no equivalent to
say that _either_ feature X or feature Y must be enabled, short of
creating a dummy SubtargetFeature for this purpose and having features X
and Y imply the new feature.
To solve the case where X or Y is needed without adding a new feature,
and to better match a typical TableGen style, this replaces the existing
"AssemblerCondString" with a dag "AssemblerCondDag" which represents the
same information. Two operators are defined for use with
AssemblerCondDag, "all_of", which matches the current behaviour, and
"any_of", which adds the new proposed ORing features functionality.
This was originally proposed in the RFC at
http://lists.llvm.org/pipermail/llvm-dev/2020-February/139138.html
Changes to all current backends are mechanical to support the replaced
functionality, and are NFCI.
At this stage, it is illegal to combine features with ands and ors in a
single AssemblerCondDag. I suspect this case is sufficiently rare that
adding more complex changes to support it are unnecessary.
Differential Revision: https://reviews.llvm.org/D74338
