This change introduces OPC_Scope opcode, whose only purpose is to record
a continuation point to resume at if a subsequent opcode fails.
Each OPC_Scope pushes an entry onto the scope stack; an entry is popped
if an opcode in the scope fails.
Previously, we recorded this information on several opcodes, it has been
removed. A series of such opcodes often referred to the same
continuation point; this information is now recorded in one place,
reducing table sizes in most cases. Average reduction is 1.1%, some
table observe up to 7% reduction in size.
The new behavior of those opcodes is "check or leave scope". If we're in
the outermost scope (scope stack is empty), they act as "check or fail".
There is one opcode, OPC_FilterValueOrSkip that behaves like the old
OPC_FilterValue. It is special because it acts as a case of a switch
statement and has nothing to do with scopes. (If a case fails, we should
try the next case instead of leaving the current scope.)
The name is most interesting and if you really need the number you can
use the name to find the entry in the enum or use the first field of the
table row.
There are two classes of operands that DecoderEmitter cannot currently
handle:
1. Operands that do not participate in instruction encoding.
2. Operands whose encoding contains only 1s and 0s.
Because of this, targets developed various workarounds. Some targets
insert missing operands after an instruction has been (incompletely)
decoded, other take into account the missing operands when printing the
instruction. Some targets do neither of that and fail to correctly
disassemble some instructions.
This patch makes it possible to decode both classes of operands and
allows to remove existing workarounds.
For the case of operand with no contribution to instruction encoding,
one should now add `bits<0> OpName` field to instruction encoding
record. This will make DecoderEmitter generate a call to the decoder
function specified by the operand's DecoderMethod. The function has a
signature different from the usual one and looks like this:
```
static DecodeStatus DecodeImm42Operand(MCInst &Inst, const MCDisassembler *Decoder) {
Inst.addOperand(MCOperand::createImm(42));
return DecodeStatus::Success;
}
```
Notably, encoding bits are not passed to it (since there are none).
There is nothing special about the second case, the operand bits are
passed as usual. The difference is that before this change, the function
was not called if all the bits of the operand were known (no '?' in the
operand encoding).
There are two options controlling the behavior. Passing an option
enables the old behavior. They exist to allow smooth transition to the
new behavior. They are temporary (yeah, I know) and will be removed once
all targets migrate, possibly giving some more time to downstream
targets.
Subsequent patches in the stack enable the new behavior on some in-tree
targets.
As noted in #153256, TableGen is generating reserved names for
RuntimeLibcalls, which resulted in a build failure for Arm64EC since
`vcruntime.h` defines `__security_check_cookie` as a macro.
To avoid using reserved names, all impl names will now be prefixed with
`Impl_`.
`NumLibcallImpls` was lifted out as a `constexpr size_t` instead of
being an enum field.
While I was churning the dependent code, I also removed the TODO to move
the impl enum into its own namespace and use an `enum class`: I
experimented with using an `enum class` and adding a namespace, but we
decided it was too verbose so it was dropped.
This change adds an option to specialize decoders per bitwidth, which
can help reduce the (compiled) code size of the decoder code.
**Current state**:
Currently, the code generated by the decoder emitter consists of two key
functions: `decodeInstruction` which is the entry point into the
generated code and `decodeToMCInst` which is invoked when a decode op is
reached while traversing through the decoder table. Both functions are
templated on `InsnType` which is the raw instruction bits that are
supplied to `decodeInstruction`.
Several backends call `decodeInstruction` with different `InsnType`
types, leading to several template instantiations of these functions in
the final code. As an example, AMDGPU instantiates this function with
type `DecoderUInt128` type for decoding 96/128-bit instructions,
`uint64_t` for decoding 64-bit instructions, and `uint32_t` for decoding
32-bit instructions. Since there is just one `decodeToMCInst` in the
generated code, it has code that handles decoding for *all* instruction
sizes. However, the decoders emitted for different instructions sizes
rarely have any intersection with each other. That means, in the AMDGPU
case, the instantiation with InsnType == DecoderUInt128 has decoder code
for 32/64-bit instructions that is *never exercised*. Conversely, the
instantiation with InsnType == uint64_t has decoder code for
128/96/32-bit instructions that is never exercised. This leads to
unnecessary dead code in the generated disassembler binary (that the
compiler cannot eliminate by itself).
**New state**:
With this change, we introduce an option
`specialize-decoders-per-bitwidth`. Under this mode, the DecoderEmitter
will generate several versions of `decodeToMCInst` function, one for
each bitwidth. The code is still templated, but will require backends to
specify, for each `InsnType` used, the bitwidth of the instruction that
the type is used to represent using a type-trait `InsnBitWidth`. This
will enable the templated code to choose the right variant of
`decodeToMCInst`. Under this mode, a particular instantiation will only
end up instantiating a single variant of `decodeToMCInst` generated and
that will include only those decoders that are applicable to a single
bitwidth, resulting in elimination of the code duplication through
instantiation and a reduction in code size.
Additionally, under this mode, decoders are uniqued only within a given
bitwidth (as opposed to across all bitwidths without this option), so
the decoder index values assigned are smaller, and consume less bytes in
their ULEB128 encoding. As a result, the generated decoder tables can
also reduce in size.
Adopt this feature for the AMDGPU and RISCV backend. In a release build,
this results in a net 55% reduction in the .text size of
libLLVMAMDGPUDisassembler.so and a 5% reduction in the .rodata size. For
RISCV, which today uses a single `uint64_t` type, this results in a 3.7%
increase in code size (expected as we instantiate the code 3 times now).
Actual measured sizes are as follows:
```
Baseline commit: 72c04bb882ad70230bce309c3013d9cc2c99e9a7
Configuration: Ubuntu clang version 18.1.3, release build with asserts disabled.
AMDGPU Before After Change
======================================================
.text 612327 275607 55% reduction
.rodata 369728 351336 5% reduction
RISCV:
======================================================
.text 47407 49187 3.7% increase
.rodata 35768 35839 0.1% increase
```
If a custom operand has MIOperandInfo with >= 2 sub-operands, it is
required that either the operand or its sub-operands have a decoder
method (depending on usage). Require this for single sub-operand
operands as well, since there is no good reason not to.
There are no changes in the generated files.
It can never be reached. It could be reached if we emitted an opcode
that could fall outside the outermost scope, but emission of all such
opcodes is guarded by `!isOutermostScope()`.
That also means we never add fixups to the outermost scope, so avoid
pushing an entry for it onto the stack.
We were sizing the table appropriately for the number of LibcallImpls,
but many of those have identical names which were pushing up the
collision count unnecessarily. This ends up decreasing the table size
slightly, and makes it a bit faster.
BM_LookupRuntimeLibcallByNameRandomCalls improves by ~25% and
BM_LookupRuntimeLibcallByNameSampleData by ~5%.
As a secondary change, align the table size up to the next
power of 2. This makes the table larger than before, but improves
the sample data benchmark by an additional 5%.
This is meant as the inverse of getNamedOperandIdx and returns the
OpName for a given operand index for a given opcode.
---------
Co-authored-by: Matt Arsenault <Matthew.Arsenault@amd.com>
Change `InstructionEncoding` to use `Size` field to derive the BitWidth
for fixed length instructions as opposed to the number of bits in the
`Inst` field. For some backends, `Inst` has more bits than `Size`, but
`Size` is the true size of the instruction.
Also add validation that `Inst` has at least `Size * 8` bits and any
bits in `Inst` beyond that are either 0 or unset.
Verified no change in generated *GenDisassembler.inc files before/after.
So we can see the changes in table sizes after making changes to
DecoderEmitter by simply running `grep DecoderTable`.
Also, remove an unnecessary terminating 0 from the end of the tables.
See the added test. Before this change the decoder would first read
the second byte, despite the fact that there are 1-byte instructions
that could match:
```
/* 0 */ MCD::OPC_ExtractField, 8, 8, // Inst{15-8} ...
/* 3 */ MCD::OPC_FilterValue, 0, 4, 0, // Skip to: 11
/* 7 */ MCD::OPC_Decode, 186, 2, 0, // Opcode: I16_0, DecodeIdx: 0
/* 11 */ MCD::OPC_FilterValue, 1, 4, 0, // Skip to: 19
/* 15 */ MCD::OPC_Decode, 187, 2, 0, // Opcode: I16_1, DecodeIdx: 0
/* 19 */ MCD::OPC_FilterValue, 2, 4, 0, // Skip to: 27
/* 23 */ MCD::OPC_Decode, 188, 2, 0, // Opcode: I16_2, DecodeIdx: 0
/* 27 */ MCD::OPC_ExtractField, 0, 1, // Inst{0} ...
/* 30 */ MCD::OPC_FilterValue, 0, 4, 0, // Skip to: 38
/* 34 */ MCD::OPC_Decode, 189, 2, 1, // Opcode: I8_0, DecodeIdx: 1
/* 38 */ MCD::OPC_FilterValueOrFail, 1,
/* 40 */ MCD::OPC_Decode, 190, 2, 1, // Opcode: I8_1, DecodeIdx: 1
/* 44 */ MCD::OPC_Fail,
```
There are no changes in the generated files. The only in-tree target
that uses variable length decoder is M68k, which is free of decoding
conflicts that could result in the decoder doing OOB access.
This also fixes misaligned "Decoding Conflict" dump,
prettified example output is shown in the second test.
Previously, HW mode name was appended to decoder namespace name when
enumerating encodings, and then emitTable appended the bit width to it
to form the final table name. Let's do this all in one place.
A nice side effect is that this allows us to avoid having to deal with
std::string.
The changes in the tests are caused by the different order of tables.
Also starts pruning out these calls if the exception model is
forced to none.
I worked backwards from the logic in addPassesToHandleExceptions
and the pass content. There appears to be some tolerance
for mixing and matching exception modes inside of a single module.
As far as I can tell _Unwind_CallPersonality is only relevant for
wasm, so just add it there.
As usual, the arm64ec case makes things difficult and is
missing test coverage. The set of calls in list form is necessary
to use foreach for the duplication, but in every other context a
dag is more convenient. You cannot use foreach over a dag, and I
haven't found a way to flatten a dag into a list.
This removes the last manual setLibcallImpl call in generic code.
`NO_FIXED_SEGMENTS_SENTINEL` has a value that is actually a valid field
encoding and so it cannot be used as a sentinel.
Replace the sentinel with a new member variable, `VariableFC`, that
contains the value previously stored in `FilterChooserMap` with
`NO_FIXED_SEGMENTS_SENTINEL` key.
* Print filter stack in non-reversed order.
* Print encoding name to the right of encoding bits to deal with
alignment issues.
* Use the correct bit width when printing encoding bits.
Example of old output:
```
01000100........
01000...........
0100............
................
tADDhirr 000000000000000001000100________
tADDrSP 000000000000000001000100_1101___
tADDspr 0000000000000000010001001____101
```
New output:
```
................
0100............
01000...........
01000100........
01000100________ tADDhirr
01000100_1101___ tADDrSP
010001001____101 tADDspr
```
…210)"
This reverts commit 9a14b1d254a43dc0d4445c3ffa3d393bca007ba3.
Revert "RuntimeLibcalls: Return StringRef for libcall names (#153209)"
This reverts commit cb1228fbd535b8f9fe78505a15292b0ba23b17de.
Revert "TableGen: Emit statically generated hash table for runtime
libcalls (#150192)"
This reverts commit 769a9058c8d04fc920994f6a5bbb03c8a4fbcd05.
Reverted three changes because of a CMake error while building llvm-nm
as reported in the following PR:
https://github.com/llvm/llvm-project/pull/150192#issuecomment-3192223073
Avoids strlen when constructing the returned StringRef. We were emitting
these in the libcall name lookup anyway, so split out the offsets for
general use.
Currently emitted as a separate table, not sure if it would be better
to change the string offset table to store pairs of offset and width
instead.
a96121089b9c94e08c6632f91f2dffc73c0ffa28 reverted a change
to use a binary search on the string name table because it
was too slow. This replaces it with a static string hash
table based on the known set of libcall names. Microbenchmarking
shows this is similarly fast to using DenseMap. It's possibly
slightly slower than using StringSet, though these aren't an
exact comparison. This also saves on the one time use construction
of the map, so it could be better in practice.
This search isn't simple set check, since it does find the
range of possible matches with the same name. There's also
an additional check for whether the current target supports
the name. The runtime constructed set doesn't require this,
since it only adds the symbols live for the target.
Followed algorithm from this post
http://0x80.pl/notesen/2023-04-30-lookup-in-strings.html
I'm also thinking the 2 special case global symbols should
just be added to RuntimeLibcalls. There are also other global
references emitted in the backend that aren't tracked; we probably
should just use this as a centralized database for all compiler
selected symbols.
error: integer literal is too large to be represented
in a signed integer type, interpreting as unsigned
[-Werror,-Wimplicitly-unsigned-literal]
Introduced in #153391
The numbers are stored in an `uint8_t` array, either directly
or via `GIMT_Encode2/4/8` macros
(see `getEncodedEmitStr` and `emitEncodingMacrosDef`).
Passing negative numbers only adds confusion and results in a warning
for the `INT64_MIN` value (C++ parses `-9223372036854775808` as two
tokens: a unary minus and an unsigned integer, triggered in #151687).
BTW, parenthesize parameters in `GIMT_Encode2/4/8` macros
and avoid unnecessary casts.
This updates everywhere we emit/check an SME routines to use
RuntimeLibcalls to get the function name and calling convention.
Note: RuntimeLibcallEmitter had some issues with emitting non-unique
variable names for sets of libcalls, so I tweaked the output to avoid
the need for variables.
This a follow on to https://github.com/llvm/llvm-project/pull/152219 to
reduce both code and frame size of `Intrinsic::getAttributes` further.
Currently, this function consists of several switch cases (one per
unique argument attributes) that populates the local `AS` array with all
non-empty argument attributes for that intrinsic by calling
`getIntrinsicArgAttributeSet`. This change makes this code table driven
and implements `Intrinsic::getAttributes` without any switch cases,
which reduces the code size of this function on all platforms and in
addition reduces the frame size by a factor of 10 on Windows.
This is achieved by:
1. Emitting table `ArgAttrIdTable` containing a concatenated list of
`<ArgNo, AttrID>` entries across all unique arguments.
2. Emitting table `ArgAttributesInfoTable` (indexed by unique
arguments-ID) to store the starting index and number of non-empty arg
attributes.
3. Reserving unique function-ID 255 to indicate that the intrinsic has
no function attributes (to replace `HasFnAttr` setup in each switch
case).
4. Using a simple table lookup and for loop to build the list of
argument and function attributes for a given intrinsic.
Experimental data shows that with release builds and assertions
disabled, this change reduces the code size for GCC and Clang builds on
Linux by ~9KB for a modest (80/152 byte) increase in frame size. For
Windows, it reduces the code size by 20KB and frame size from 4736 bytes
to 461 bytes which is 10x reduction. Actual data is as follows:
```
Current trunk:
Compiler gcc-13.3.0 clang-18.1.3 MSVC 19.43.34810.0
code size 0x35a9 0x370c 0x5581
frame size 0x120 0x118 0x1280
table driven Intrinsic::getAttributes:
code size 0xcfb 0xcd0 0x1cf
frame size 0x1b8 0x188 0x1A0
Total savings (code + data) 9212 bytes 9790 bytes 20119 bytes
```
Total savings above accounts for the additional data size for the 2 new
tables, which in this experiment was: `ArgAttributesInfoTable` = 314
bytes and `ArgAttrIdTable` = 888 bytes. Coupled with the earlier
https://github.com/llvm/llvm-project/pull/152219, this achieves a 46x
reduction in frame size for this function in Windows release builds.
This change fixes a stack size regression that got introduced in
0de0354aa8.
That change did 2 independent things:
1. Uniquify argument and function attributes separately so that we
generate a smaller number of unique sets as opposed to uniquifying them
together. This is beneficial for code size.
2. Eliminate the fixed size array `AS` and `NumAttrs` variable and
instead build the returned AttribteList in each case using an
initializer list.
The second part seems to have caused a regression in the stack size
usage of this function for Windows. This change essentially undoes part
2 and reinstates the use of the fixed size array `AS` which fixes this
stack size regression. The actual measured stack frame size for this
function before/after this change is as follows:
```
Current trunk data for release build (x86_64 builds for Linux, x86 build for Windows):
Compiler gcc-13.3.0 clang-18.1.3 MSVC 19.43.34810.0
DLLVM_ENABLE_ASSERTIONS=OFF 0x120 0x110 0x54B0
DLLVM_ENABLE_ASSERTIONS=ON 0x2880 0x110 0x5250
After applying the fix:
Compiler gcc-13.3.0 clang-18.1.3 MSVC 19.43.34810.0
DLLVM_ENABLE_ASSERTIONS=OFF 0x120 0x118 0x1240h
DLLVM_ENABLE_ASSERTIONS=ON 0x120 0x118 0x1240h
```
Note that for Windows builds with assertions disabled, the stack frame
size for this function reduces from 21680 to 4672 which is a 4.6x
reduction. Stack frame size for GCC build with assertions also improved
and clang builds are unimpacted. The speculation is that clang and gcc
is able to reuse the stack space across these switch cases better with
existing code, but MSVC is not, and re-introducing the `AS` variable
forces all cases to use the same local variable, addressing the stack
space regression.
Hack in the default setting so it's consistently generated like
the other cases. Maintain a list of targets where this applies.
The alternative would require new infrastructure to sort the system
library initialization in some way.
I wanted the unhandled target case to be treated as a fatal
error, but it turns out there's a hack in IRSymtab using
RuntimeLibcalls, which will fail out in many tests that
do not have a triple set. Many of the failures are simply
running llvm-as with no triple, which probably should not
depend on knowing an accurate set of calls.
This patch removes all uses of %T from within LLVM tests. %T has been
deprecated for about seven years and use is not advised given it is not
unique per test and can thus lead to races. The goal of this is to
eventually remove support for %T from lit.
Use direct table lookup instead of a switch over all opcodes.
In my Release+Asserts build this reduced the code size for
AMDGPU::getNamedOperandIdx from 11422 to 80 bytes, and the total size of
all its tables (including the jump table for the switch) from 243564 to
155020 bytes.
There are currently no ways to add names to dag
operators other than when defining them. Furthermore a !con operation as
well as some others, drops the operator names.
This patch propagates the name from the LHS dag
for !con and adds a way to get and set the operator name for a dag
(!getdagopname, !setdagopname).
---------
Co-authored-by: Nemanja Ivanovic <nemanja@synopsys.com>
In addition to checking the predicate from the CompressPat, also check
the destination instruction. This prevents creating bad instructions if
CompressPat isn't a proper subset of the destination instruction. This
prevents mistakes that we can't catch at compile time.
We are able to verify RegisterClass hierarchy at compile time so don't
have to check the destination register class.
I've added comments for the operand names to make auditing easier.
Instead of using the Record from the instruction definition, use the
Record specified in the CompressPat DAG. This will allow us to use
Records that are subsets of both the source and destination.
I want to use this to merge the C_*_HINT instructions back into their
regular non-HINT versions, but prevent those encodings from being part
of compress/uncompress. For example, we will use GPRNoX0 for the C_ADDI
CompressPat while both C_ADDI and ADDI will use GPR in their instruction
definitions.
To do this I've recorded the original DAG Record in the OperandMap using
a struct in the union to represent the 3 fields needed for an Operand.
Previously we stored TiedOpIdx outside the union, but only used it for
Operand.
There is a verification hole here where we don't have any way to check
that an immediate predicate is a subset of an instruction predicate at
tablegen time. Prior to #148660 we had this hole in one direction, but
that patch made it in two directions. I'm not sure if this patch makes
it any worse. Now we're using what is in the CompressPat where before we
were using whatever was in the instructions and ignoring the predicate
in the CompressPat.
Additionally, add sentinel values <Enum>::First_ and <Enum>::Last_ to
each one of those enums.
This will allow using `enum_seq_inclusive` to generate the list of
enum-typed values of any generated scoped (non-bitmask) enum.
- This implementation is adapted from SDAG
X86TargetLowering::LowerGET_ROUNDING.
- llvm.set.rounding will be added later because it involves MXCSR
updates currently unsupported.
Make `AppendZero` a class member instead of an argument to
`GetOrAddStringOffset` to reflect the intended usage that for a given
`StringToOffsetTable`, all strings must use the same value of
`AppendZero`.
Modify `EmitStringTableDef` to drop the `Indent` argument as its always
set to `""`, and to fail if it's called for a table with
non-null-terminated strings.
Introduces saturated truncate instructions to Global ISel:
G_TRUNC_SSAT_S, G_TRUNC_SSAT_U, G_TRUNC_USAT_U. These were previously
introduced to SDAG to reduce redundant code.
The patch only initially introduces the instruction, a later patch will
follow to add combines and legalization for each instruction.
Allow associating a non-default CallingConv with a set of library
functions, and applying a default for a SystemLibrary.
I also wanted to be able to apply a default calling conv
to a RuntimeLibcallImpl, but that turned out to be annoying
so leave it for later.
