Previously, HW mode name was appended to decoder namespace name when
enumerating encodings, and then emitTable appended the bit width to it
to form the final table name. Let's do this all in one place.
A nice side effect is that this allows us to avoid having to deal with
std::string.
The changes in the tests are caused by the different order of tables.
Also starts pruning out these calls if the exception model is
forced to none.
I worked backwards from the logic in addPassesToHandleExceptions
and the pass content. There appears to be some tolerance
for mixing and matching exception modes inside of a single module.
As far as I can tell _Unwind_CallPersonality is only relevant for
wasm, so just add it there.
As usual, the arm64ec case makes things difficult and is
missing test coverage. The set of calls in list form is necessary
to use foreach for the duplication, but in every other context a
dag is more convenient. You cannot use foreach over a dag, and I
haven't found a way to flatten a dag into a list.
This removes the last manual setLibcallImpl call in generic code.
`NO_FIXED_SEGMENTS_SENTINEL` has a value that is actually a valid field
encoding and so it cannot be used as a sentinel.
Replace the sentinel with a new member variable, `VariableFC`, that
contains the value previously stored in `FilterChooserMap` with
`NO_FIXED_SEGMENTS_SENTINEL` key.
* Print filter stack in non-reversed order.
* Print encoding name to the right of encoding bits to deal with
alignment issues.
* Use the correct bit width when printing encoding bits.
Example of old output:
```
01000100........
01000...........
0100............
................
tADDhirr 000000000000000001000100________
tADDrSP 000000000000000001000100_1101___
tADDspr 0000000000000000010001001____101
```
New output:
```
................
0100............
01000...........
01000100........
01000100________ tADDhirr
01000100_1101___ tADDrSP
010001001____101 tADDspr
```
…210)"
This reverts commit 9a14b1d254a43dc0d4445c3ffa3d393bca007ba3.
Revert "RuntimeLibcalls: Return StringRef for libcall names (#153209)"
This reverts commit cb1228fbd535b8f9fe78505a15292b0ba23b17de.
Revert "TableGen: Emit statically generated hash table for runtime
libcalls (#150192)"
This reverts commit 769a9058c8d04fc920994f6a5bbb03c8a4fbcd05.
Reverted three changes because of a CMake error while building llvm-nm
as reported in the following PR:
https://github.com/llvm/llvm-project/pull/150192#issuecomment-3192223073
Avoids strlen when constructing the returned StringRef. We were emitting
these in the libcall name lookup anyway, so split out the offsets for
general use.
Currently emitted as a separate table, not sure if it would be better
to change the string offset table to store pairs of offset and width
instead.
a96121089b9c94e08c6632f91f2dffc73c0ffa28 reverted a change
to use a binary search on the string name table because it
was too slow. This replaces it with a static string hash
table based on the known set of libcall names. Microbenchmarking
shows this is similarly fast to using DenseMap. It's possibly
slightly slower than using StringSet, though these aren't an
exact comparison. This also saves on the one time use construction
of the map, so it could be better in practice.
This search isn't simple set check, since it does find the
range of possible matches with the same name. There's also
an additional check for whether the current target supports
the name. The runtime constructed set doesn't require this,
since it only adds the symbols live for the target.
Followed algorithm from this post
http://0x80.pl/notesen/2023-04-30-lookup-in-strings.html
I'm also thinking the 2 special case global symbols should
just be added to RuntimeLibcalls. There are also other global
references emitted in the backend that aren't tracked; we probably
should just use this as a centralized database for all compiler
selected symbols.
error: integer literal is too large to be represented
in a signed integer type, interpreting as unsigned
[-Werror,-Wimplicitly-unsigned-literal]
Introduced in #153391
The numbers are stored in an `uint8_t` array, either directly
or via `GIMT_Encode2/4/8` macros
(see `getEncodedEmitStr` and `emitEncodingMacrosDef`).
Passing negative numbers only adds confusion and results in a warning
for the `INT64_MIN` value (C++ parses `-9223372036854775808` as two
tokens: a unary minus and an unsigned integer, triggered in #151687).
BTW, parenthesize parameters in `GIMT_Encode2/4/8` macros
and avoid unnecessary casts.
This updates everywhere we emit/check an SME routines to use
RuntimeLibcalls to get the function name and calling convention.
Note: RuntimeLibcallEmitter had some issues with emitting non-unique
variable names for sets of libcalls, so I tweaked the output to avoid
the need for variables.
This a follow on to https://github.com/llvm/llvm-project/pull/152219 to
reduce both code and frame size of `Intrinsic::getAttributes` further.
Currently, this function consists of several switch cases (one per
unique argument attributes) that populates the local `AS` array with all
non-empty argument attributes for that intrinsic by calling
`getIntrinsicArgAttributeSet`. This change makes this code table driven
and implements `Intrinsic::getAttributes` without any switch cases,
which reduces the code size of this function on all platforms and in
addition reduces the frame size by a factor of 10 on Windows.
This is achieved by:
1. Emitting table `ArgAttrIdTable` containing a concatenated list of
`<ArgNo, AttrID>` entries across all unique arguments.
2. Emitting table `ArgAttributesInfoTable` (indexed by unique
arguments-ID) to store the starting index and number of non-empty arg
attributes.
3. Reserving unique function-ID 255 to indicate that the intrinsic has
no function attributes (to replace `HasFnAttr` setup in each switch
case).
4. Using a simple table lookup and for loop to build the list of
argument and function attributes for a given intrinsic.
Experimental data shows that with release builds and assertions
disabled, this change reduces the code size for GCC and Clang builds on
Linux by ~9KB for a modest (80/152 byte) increase in frame size. For
Windows, it reduces the code size by 20KB and frame size from 4736 bytes
to 461 bytes which is 10x reduction. Actual data is as follows:
```
Current trunk:
Compiler gcc-13.3.0 clang-18.1.3 MSVC 19.43.34810.0
code size 0x35a9 0x370c 0x5581
frame size 0x120 0x118 0x1280
table driven Intrinsic::getAttributes:
code size 0xcfb 0xcd0 0x1cf
frame size 0x1b8 0x188 0x1A0
Total savings (code + data) 9212 bytes 9790 bytes 20119 bytes
```
Total savings above accounts for the additional data size for the 2 new
tables, which in this experiment was: `ArgAttributesInfoTable` = 314
bytes and `ArgAttrIdTable` = 888 bytes. Coupled with the earlier
https://github.com/llvm/llvm-project/pull/152219, this achieves a 46x
reduction in frame size for this function in Windows release builds.
This change fixes a stack size regression that got introduced in
0de0354aa8.
That change did 2 independent things:
1. Uniquify argument and function attributes separately so that we
generate a smaller number of unique sets as opposed to uniquifying them
together. This is beneficial for code size.
2. Eliminate the fixed size array `AS` and `NumAttrs` variable and
instead build the returned AttribteList in each case using an
initializer list.
The second part seems to have caused a regression in the stack size
usage of this function for Windows. This change essentially undoes part
2 and reinstates the use of the fixed size array `AS` which fixes this
stack size regression. The actual measured stack frame size for this
function before/after this change is as follows:
```
Current trunk data for release build (x86_64 builds for Linux, x86 build for Windows):
Compiler gcc-13.3.0 clang-18.1.3 MSVC 19.43.34810.0
DLLVM_ENABLE_ASSERTIONS=OFF 0x120 0x110 0x54B0
DLLVM_ENABLE_ASSERTIONS=ON 0x2880 0x110 0x5250
After applying the fix:
Compiler gcc-13.3.0 clang-18.1.3 MSVC 19.43.34810.0
DLLVM_ENABLE_ASSERTIONS=OFF 0x120 0x118 0x1240h
DLLVM_ENABLE_ASSERTIONS=ON 0x120 0x118 0x1240h
```
Note that for Windows builds with assertions disabled, the stack frame
size for this function reduces from 21680 to 4672 which is a 4.6x
reduction. Stack frame size for GCC build with assertions also improved
and clang builds are unimpacted. The speculation is that clang and gcc
is able to reuse the stack space across these switch cases better with
existing code, but MSVC is not, and re-introducing the `AS` variable
forces all cases to use the same local variable, addressing the stack
space regression.
Hack in the default setting so it's consistently generated like
the other cases. Maintain a list of targets where this applies.
The alternative would require new infrastructure to sort the system
library initialization in some way.
I wanted the unhandled target case to be treated as a fatal
error, but it turns out there's a hack in IRSymtab using
RuntimeLibcalls, which will fail out in many tests that
do not have a triple set. Many of the failures are simply
running llvm-as with no triple, which probably should not
depend on knowing an accurate set of calls.
This patch removes all uses of %T from within LLVM tests. %T has been
deprecated for about seven years and use is not advised given it is not
unique per test and can thus lead to races. The goal of this is to
eventually remove support for %T from lit.
Use direct table lookup instead of a switch over all opcodes.
In my Release+Asserts build this reduced the code size for
AMDGPU::getNamedOperandIdx from 11422 to 80 bytes, and the total size of
all its tables (including the jump table for the switch) from 243564 to
155020 bytes.
There are currently no ways to add names to dag
operators other than when defining them. Furthermore a !con operation as
well as some others, drops the operator names.
This patch propagates the name from the LHS dag
for !con and adds a way to get and set the operator name for a dag
(!getdagopname, !setdagopname).
---------
Co-authored-by: Nemanja Ivanovic <nemanja@synopsys.com>
In addition to checking the predicate from the CompressPat, also check
the destination instruction. This prevents creating bad instructions if
CompressPat isn't a proper subset of the destination instruction. This
prevents mistakes that we can't catch at compile time.
We are able to verify RegisterClass hierarchy at compile time so don't
have to check the destination register class.
I've added comments for the operand names to make auditing easier.
Instead of using the Record from the instruction definition, use the
Record specified in the CompressPat DAG. This will allow us to use
Records that are subsets of both the source and destination.
I want to use this to merge the C_*_HINT instructions back into their
regular non-HINT versions, but prevent those encodings from being part
of compress/uncompress. For example, we will use GPRNoX0 for the C_ADDI
CompressPat while both C_ADDI and ADDI will use GPR in their instruction
definitions.
To do this I've recorded the original DAG Record in the OperandMap using
a struct in the union to represent the 3 fields needed for an Operand.
Previously we stored TiedOpIdx outside the union, but only used it for
Operand.
There is a verification hole here where we don't have any way to check
that an immediate predicate is a subset of an instruction predicate at
tablegen time. Prior to #148660 we had this hole in one direction, but
that patch made it in two directions. I'm not sure if this patch makes
it any worse. Now we're using what is in the CompressPat where before we
were using whatever was in the instructions and ignoring the predicate
in the CompressPat.
Additionally, add sentinel values <Enum>::First_ and <Enum>::Last_ to
each one of those enums.
This will allow using `enum_seq_inclusive` to generate the list of
enum-typed values of any generated scoped (non-bitmask) enum.
- This implementation is adapted from SDAG
X86TargetLowering::LowerGET_ROUNDING.
- llvm.set.rounding will be added later because it involves MXCSR
updates currently unsupported.
Make `AppendZero` a class member instead of an argument to
`GetOrAddStringOffset` to reflect the intended usage that for a given
`StringToOffsetTable`, all strings must use the same value of
`AppendZero`.
Modify `EmitStringTableDef` to drop the `Indent` argument as its always
set to `""`, and to fail if it's called for a table with
non-null-terminated strings.
Introduces saturated truncate instructions to Global ISel:
G_TRUNC_SSAT_S, G_TRUNC_SSAT_U, G_TRUNC_USAT_U. These were previously
introduced to SDAG to reduce redundant code.
The patch only initially introduces the instruction, a later patch will
follow to add combines and legalization for each instruction.
Allow associating a non-default CallingConv with a set of library
functions, and applying a default for a SystemLibrary.
I also wanted to be able to apply a default calling conv
to a RuntimeLibcallImpl, but that turned out to be annoying
so leave it for later.
Add a way to define a SystemLibrary for a complete set of
libcalls, subdivided by a predicate based on the triple.
Libraries can be defined using dag set operations, and the
prior default set can be subtracted from and added to (though
I think eventually all targets should move to explicit opt-ins.
We're still doing things like reporting ppcf128 libcalls as
available dy default on all targets).
Start migrating some of the easier targets to only use the new
system. Targets that don't define a SystemLibrary are still
manually mutating a table set to the old defaults.
Add a convenience wrapper struct for the `bit_value_t` enum type to host
various constructors, query, and printing support. Also refactor related
code in several places. In `getBitsField`, use `llvm::append_range` and
`SmallVector::append()` and eliminate manual loops. Eliminate
`emitNameWithID` and instead use the `operator <<` that does the same
thing as this function. Have `BitValue::getValue()` (replacement for
`Value`) return std::optional<> instead of -1 for unset bits. Terminate
with a fatal error when a decoding conflict is encountered.
Work towards separating the ABI existence of libcalls vs. the
lowering selection. Set libcall selection through enums, rather
than through raw string names.
Replace RuntimeLibcalls.def with a tablegenerated version. This
is in preparation for splitting RuntimeLibcalls into two components.
For now match the existing functionality.
Dynamic relocations are expensive on ELF/Linux platforms because they
are applied in userspace on process startup. Therefore, it is worth
optimizing them to make PIE and PIC dylib builds faster. In +asserts
builds (non-NDEBUG), nikic identified these schedule class name string
pointers as the leading source of dynamic relocations. [1]
This change uses llvm::StringTable and the StringToOffsetTable TableGen
helper to turn the string pointers into 32-bit offsets into a separate
character array.
The number of dynamic relocations is reduced by ~60%:
❯ llvm-readelf --dyn-relocations lib/libLLVM.so | wc -l
381376 # before
155156 # after
The test suite time is modestly affected, but I'm running on a shared
noisy workstation VM with a ton of cores:
https://gist.github.com/rnk/f38882c2fe2e63d0eb58b8fffeab69de
Testing Time: 100.88s # before
Testing Time: 78.50s. # after
Testing Time: 96.25s. # before again
I haven't used any fancy hyperfine/denoising tools, but I think the
result is clearly visible and we should ship it.
[1] https://gist.github.com/nikic/554f0a544ca15d5219788f1030f78c5a
Add option `use-fn-table-in-decode-to-mcinst` to use a table of function
pointers instead of a switch case in the generated `decodeToMCInst`
function.
When the number of switch cases in this function is large, the generated
code takes a long time to compile in release builds. Using a table of
function pointers instead improves the compile time significantly (~3x
speedup in compiling the code in a downstream target). This option will
allow targets to opt into this mode if they desire for better build
times.
Tested with `check-llvm-mc` with the option enabled by default.
This patch adds an even more aggressive short-circuit on `!and` and
`!or` that completely avoids the evaluation of RHS operand until short
circuiting decisions are made.
Previously PatLeafs could not be imported, causing the following
warnings to be emitted when running tblgen with
`-warn-on-skipped-patterns:`
```
/work/clean/llvm/lib/Target/AArch64/AArch64InstrInfo.td:2631:1: warning: Skipped pattern: Src pattern child has unsupported predicate
def : Pat<(i64 (mul top32Zero:$Rn, top32Zero:$Rm)),
^
```
These changes allow the patterns to now be imported successfully.
In "get<lang>DirectiveName(Kind, Version)", return the spelling that
corresponds to Version, and in "get<lang>DirectiveKindAndVersions(Name)"
return the pair {Kind, VersionRange}, where VersionRange contains the
minimum and the maximum versions that allow "Name" as a spelling. This
applies to clauses as well. In general it applies to classes that have
spellings (defined via TableGen class "Spelling").
Given a Kind and a Version, getting the corresponding spelling requires
a runtime search (which can fail in a general case). To avoid generating
the search function inline, a small additional component of
llvm/Frontent was added: LLVMFrontendDirective. The corresponding header
file also defines C++ classes "Spelling" and "VersionRange", which are
used in TableGen/DirectiveEmitter as well.
For background information see
https://discourse.llvm.org/t/rfc-alternative-spellings-of-openmp-directives/85507
Print DecodeIdx associated with Decode MCD ops in the generated decoder
tables. This can help in debugging decode failures by first mapping the
Op -> DecodeIdx and then inspecting the code in `decodeToMCInst`
associated with that DecodeIdx.
Use the spellings in the generated clause parser. The functions
`get<lang>ClauseKind` and `get<lang>ClauseName` are not yet updated.
The definitions of both clauses and directives now take a list of
"Spelling"s instead of a single string. For example
```
def ACCC_Copyin : Clause<[Spelling<"copyin">,
Spelling<"present_or_copyin">,
Spelling<"pcopyin">]> { ... }
```
A "Spelling" is a versioned string, defaulting to "all versions".
For background information see
https://discourse.llvm.org/t/rfc-alternative-spellings-of-openmp-directives/85507
The code in DirectiveEmitter that generates clause parsers sorted clause
names to ensure that longer names were tried before shorter ones, in
cases where a shorter name may be a prefix of a longer one. This matters
in the strict Fortran source format, since whitespace is ignored there.
This sorting did not take into account clause aliases, which are just
alternative names. These extra names were not protected in the same way,
and were just appended immediately after the primary name.
This patch generates a list of pairs Record+Name, where a given record
can appear multiple times with different names. Sort that list and use
it to generate parsers for each record. What used to be
```
("fred" || "f") >> construct<SomeClause>{} ||
"foo" << construct<OtherClause>{}
```
is now
```
"fred" >> construct<SomeClause>{} ||
"foo" >> construct<OtherClause>{} ||
"f" >> construct<SomeClause>{}
```
Introduce `OrFail` variants for all MCD Decoder Ops that have
`NumToSKip` encoded with them. This is intended to capture the common
case of jumps to the end of the decoder table which has a `OP_Fail` at
the end. Using the `OrFail` variants of these ops avoid encoding the
`NumToSkip` jump offset for these cases, resulting in a reduction in the
size of the decoder tables (from 5 - 17%). Additionally, for the AArch64
target, the table size reduces enough to switch to using 2-byte
`NumToSkip` encoding instead of existing 3-bytes, resulting in a net 30%
reduction in the size of the decoder table.
The total reduction in the size of the decoder tables for different
targets is as follows (computed using the following command: `for i in
*.inc; do echo -n ``basename $i: ``; grep "MCD::OPC_Fail," $i | awk
'{sum += $2} END { print sum}'; done`)
```
Target Old Size New Size % Reduction
================================================
AArch64 153268 106987 30.20
AMDGPU 412056 340856 17.28
ARC 5061 4605 9.01
ARM 73831 60847 17.59
AVR 1306 1158 11.33
BPF 1927 1795 6.85
CSKY 8692 6922 20.36
Hexagon 41965 34759 17.17
Lanai 982 924 5.91
LoongArch 21629 20035 7.37
M68k 13461 11689 13.16
MSP430 3716 3384 8.93
Mips 31415 25771 17.97
PPC 28931 24771 14.38
RISCV 34800 28352 18.53
Sparc 7432 6236 16.09
SystemZ 32248 29716 7.85
VE 42873 36923 13.88
XCore 2316 2196 5.18
Xtensa 3443 2793 18.88
```
The class "ClauseVal" actually represents a definition of an enumeration
value, and in itself it is not bound to any clause. Rename it to EnumVal
and add a comment clarifying how it's translated into an actual enum
definition in the generated source code.
There is no change in functionality.
Without this patch the !subst in the test drops the name "$frag" from
(one_frag:$frag ...) and returns:
```
(set FPR32_NEW:$dst, (one_frag FPR32_NEW:$a, FPR32_NEW:$b))
```
