Commit 1b531d54f623 (#74203) removed the usage of unique_ptrs of arrays
in favour of using vectors, but inadvertently increased peak memory
usage by removing the ability to deallocate vector memory that was no
longer needed mid-LDV.
In that same review, it was pointed out that `FuncValueTable` typedef
could be removed, since it was "just a vector".
This commit addresses both issues by making `FuncValueTable` a real data
structure, capable of mapping BBs to ValueTables and able to free
ValueTables as needed.
This reduces peak memory usage in the compiler by 10% in the benchmarks
flagged by the original review.
As a consequence, we had to remove a handful of instances of the
"declare-then-initialize" antipattern in unittests, as the
FuncValueTable class is no longer default-constructible.
These are usually difficult to reason about, and they were being used to
pass raw pointers around with array semantic (i.e., we were using
operator [] on raw pointers). To put it in InstrRef terminology: we were
passing a pointer to a ValueTable but using it as if it were a
FuncValueTable.
These could have easily been SmallVectors, which now allow us to have
reference semantics in some places, as well as simpler initialization.
In the future, we can use even more pass-by-reference with some extra
changes in the code.
This patch replaces uses of StringRef::{starts,ends}with with
StringRef::{starts,ends}_with for consistency with
std::{string,string_view}::{starts,ends}_with in C++20.
I'm planning to deprecate and eventually remove
StringRef::{starts,ends}with.
The functions should not compare results of
getExtendedSizeInBits(), i.e TypeSize variables with plain integer values,
but create a fixed TypeSize object so the correct operator can be used.
It seems TypeSize is currently broken in the sense that:
TypeSize::Fixed(4) + TypeSize::Scalable(4) => TypeSize::Fixed(8)
without failing its assert that explicitly tests for this case:
assert(LHS.Scalable == RHS.Scalable && ...);
The reason this fails is that `Scalable` is a static method of class
TypeSize,
and LHS and RHS are both objects of class TypeSize. So this is
evaluating
if the pointer to the function Scalable == the pointer to the function
Scalable,
which is always true because LHS and RHS have the same class.
This patch fixes the issue by renaming `TypeSize::Scalable` ->
`TypeSize::getScalable`, as well as `TypeSize::Fixed` to
`TypeSize::getFixed`,
so that it no longer clashes with the variable in
FixedOrScalableQuantity.
The new methods now also better match the coding standard, which
specifies that:
* Variable names should be nouns (as they represent state)
* Function names should be verb phrases (as they represent actions)
FileCheck currently compiles a regular expression of the form
`Prefix1|Prefix2|...` and uses it to find the next prefix in the input.
If we had a fast regex implementation, this would be a useful thing to
do, as the regex implementation would be able to match multiple prefixes
more efficiently than a naive approach. However, with our actual regex
implementation, finding the prefixes basically becomes O(InputLen *
RegexLen * LargeConstantFactor), which is a lot worse than a simple
string search.
Replace the regex with StringRef::find(), and keeping track of the next
position of each prefix. There are various ways this could be improved
on, but it's already significantly faster that the previous approach.
For me, this improves check-llvm time from 138.5s to 132.5s, so by
around 4-5%.
For vector-interleaved-load-i16-stride-7.ll in particular, test time
drops from 5s to 2.5s.
The current isScalable function requires a user to call isVector before
hand in order to avoid an assertion failure in the case that the LLT is
not a vector.
This patch addds helper functions that allow a user to query whether the
LLT is fixed or scalable, not wanting an assertion failure in the case
that the LLT was never a vector in the first place.
As alluded to in #20571, it would be nice if we could mutate operand
lists of MachineInstr's more safely. Add an insert method that together
with removeOperand allows for easier splicing of operands.
Splitting this patch off early to get feedback; I need to either:
- mutate an INLINEASM{_BR} MachinInstr's MachineOperands from being
registers (physical or virtual) to memory
(MachineOperandType::MO_FrameIndex). These are not 1:1 operand
replacements, but N:M operand replacements. i.e. we need to
update 2 MachineOperands into the middle of the operand list to 5 (at
least for x86_64).
- copy, modify, write a new MachineInstr which has its relevant operands
replaced.
Either approaches are hazarded by existing references to either the
operands being moved, or the instruction being removed+replaced. For my
purposes in regalloc, either seem to work for me, so hopefully reviewers
can help me determine which approach is preferable. The second would
involve no new methods on MachineInstr.
One question I had while looking at this was: "why does MachineInstr
have BOTH a NumOperands member AND a MCInstrDesc member that itself has
a NumOperands member? How many operands can a MachineInstr have? Do I
need to update BOTH (keeping them in sync)?" FWICT, only "variadic"
MachineInstrs have MCInstrDesc with NumOperands (of the MCInstrDesc) set
to zero. If the MCInstrDesc's NumOperands is non-zero, then the
NumOperands
on the MachineInstr itself cannot exceed this value (IIUC) else an
assert will
be triggered.
For most non-psuedo instructions (or at least non-varidic instructions),
insert is less likely to be useful.
To run the newly added unittest:
$ pushd llvm/build; ninja CodeGenTests; popd
$ ./llvm/build/unittests/CodeGen/CodeGenTests \
--gtest_filter=MachineInstrTest.SpliceOperands
This is meant to mirror `MCInst::insert`.
This is an attempt at rebooting https://reviews.llvm.org/D28990
I've included AutoUpgrade changes to modify the data layout to satisfy the compatible layout check. But this does mean alloca, loads, stores, etc in old IR will automatically get this new alignment.
This should fix PR46320.
Reviewed By: echristo, rnk, tmgross
Differential Revision: https://reviews.llvm.org/D86310
The legalizer currently generates lots of G_AND artifacts.
For example between boolean uses and defs there is always a G_AND with a mask of 1, but when the target uses ZeroOrOneBooleanContents, this is unnecessary.
Currently these artifacts have to be removed using post-legalize combines.
Omitting these artifacts at their source in the artifact combiner has a few advantages:
- We know that the emitted G_AND is very likely to be useless, so our KnownBits call is likely worth it.
- The G_AND and G_CONSTANT can interrupt e.g. G_UADDE/... sequences generated during legalization of wide adds which makes it harder to detect these sequences in the instruction selector (e.g. useful to prevent unnecessary reloading of AArch64 NZCV register).
- This cleans up a lot of legalizer output and even improves compilation-times.
AArch64 CTMark geomean: `O0` -5.6% size..text; `O0` and `O3` ~-0.9% compilation-time (instruction count).
Since this introduces KnownBits into code-paths used by `O0`, I reduced the default recursion depth.
This doesn't seem to make a difference in CTMark, but should prevent excessive recursive calls in the worst case.
Reviewed By: aemerson
Differential Revision: https://reviews.llvm.org/D159140
This will make it easy for callers to see issues with and fix up calls
to createTargetMachine after a future change to the params of
TargetMachine.
This matches other nearby enums.
For downstream users, this should be a fairly straightforward
replacement,
e.g. s/CodeGenOpt::Aggressive/CodeGenOptLevel::Aggressive
or s/CGFT_/CodeGenFileType::
There are missing include and using in TextStubTests and
AsmPrinterDwarfTest and they causes build failures when using vanilla
GoogleTest v1.14.0. This patch fixes this issue.
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Reland2: temporarily disabled call to "--linker llvm" for tls-variable.test
and location-expression.test as it does not work properly on bigendian
architecture.
Differential Revision: https://reviews.llvm.org/D153268
This reverts commit 0229dd0626b2538c78ebbd2b1bd44c31fbf1cdec.
This introduces two test failures on s390x.
tools/dsymutil/X86/location-expression.test:
warning: cann't load line table.
note: while processing CU1
/builddir/build/BUILD/llvm-18.0.0.src/test/tools/dsymutil/X86/location-expression.test:20:10: error: CHECK: expected string not found in input
# CHECK: DW_AT_name{{.*}}"CU1"
^
<stdin>:34:32: note: scanning from here
0x0000000b: DW_TAG_compile_unit [1] *
^
<stdin>:37:2: note: possible intended match here
DW_AT_name [DW_FORM_strp] ( .debug_str[0x09000000] = )
^
tools/dsymutil/X86/tls-variable.test:
warning: cann't load line table.
note: while processing CU1
/builddir/build/BUILD/llvm-18.0.0.src/test/tools/dsymutil/X86/tls-variable.test:19:10: error: CHECK: expected string not found in input
# CHECK: DW_AT_name{{.*}}"CU1"
^
<stdin>:26:32: note: scanning from here
0x0000000b: DW_TAG_compile_unit
^
<stdin>:29:2: note: possible intended match here
DW_AT_name ()
^
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Differential Revision: https://reviews.llvm.org/D153268
This patch is extracted from D96035, it adds support for the existing
DWARFLinker functionality. What is not supported yet:
1. Types deduplication(--odr mode).
2. Modules deduplication.
3. Generation of index tables.
Differential Revision: https://reviews.llvm.org/D153268
Introduced the convergent equivalent of the existing G_INTRINSIC opcodes:
- G_INTRINSIC_CONVERGENT
- G_INTRINSIC_CONVERGENT_W_SIDE_EFFECTS
Out of the targets that currently have some support for GlobalISel, the patch
assumes that the convergent intrinsics only relevant to SPIRV and AMDGPU.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D154766
Re-landing the code that was reverted because of the buildbot failure
in https://lab.llvm.org/buildbot#builders/9/builds/27319.
Original commit message
======================
The class `ResourceSegments` is used to keep track of the intervals
that represent resource usage of a list of instructions that are
being scheduled by the machine scheduler.
The collection is made of intervals that are closed on the left and
open on the right (represented by the standard notation `[a, b)`).
These collections of intervals can be extended by `add`ing new
intervals accordingly while scheduling a basic block.
Unit tests are added to verify the possible configurations of
intervals, and the relative possibility of scheduling a new
instruction in these configurations. Specifically, the methods
`getFirstAvailableAtFromBottom` and `getFirstAvailableAtFromTop` are
tested to make sure that both bottom-up and top-down scheduling work
when tracking resource usage across the basic block with
`ResourceSegments`.
Note that the scheduler tracks resource usage with two methods:
1. counters (via `std::vector<unsigned> ReservedCycles;`);
2. intervals (via `std::map<unsigned, ResourceSegments> ReservedResourceSegments;`).
This patch can be considered a NFC test for existing scheduling models
because the tracking system that uses intervals is turned off by
default (field `bit EnableIntervals = false;` in the tablegen class
`SchedMachineModel`).
Reviewed By: andreadb
Differential Revision: https://reviews.llvm.org/D150312
Reverted because it produces the following builbot failure at https://lab.llvm.org/buildbot#builders/9/builds/27319:
/b/ml-opt-rel-x86-64-b1/llvm-project/llvm/unittests/CodeGen/SchedBoundary.cpp: In member function ‘virtual void ResourceSegments_getFirstAvailableAtFromBottom_empty_Test::TestBody()’:
/b/ml-opt-rel-x86-64-b1/llvm-project/llvm/unittests/CodeGen/SchedBoundary.cpp:395:31: error: call of overloaded ‘ResourceSegments(<brace-enclosed initializer list>)’ is ambiguous
395 | auto X = ResourceSegments({});
| ^
This reverts commit dc312f0331309692e8d6e06e93b3492b6a40989f.
The class `ResourceSegments` is used to keep track of the intervals
that represent resource usage of a list of instructions that are
being scheduled by the machine scheduler.
The collection is made of intervals that are closed on the left and
open on the right (represented by the standard notation `[a, b)`).
These collections of intervals can be extended by `add`ing new
intervals accordingly while scheduling a basic block.
Unit tests are added to verify the possible configurations of
intervals, and the relative possibility of scheduling a new
instruction in these configurations. Specifically, the methods
`getFirstAvailableAtFromBottom` and `getFirstAvailableAtFromTop` are
tested to make sure that both bottom-up and top-down scheduling work
when tracking resource usage across the basic block with
`ResourceSegments`.
Note that the scheduler tracks resource usage with two methods:
1. counters (via `std::vector<unsigned> ReservedCycles;`);
2. intervals (via `std::map<unsigned, ResourceSegments> ReservedResourceSegments;`).
This patch can be considered a NFC test for existing scheduling models
because the tracking system that uses intervals is turned off by
default (field `bit EnableIntervals = false;` in the tablegen class
`SchedMachineModel`).
Reviewed By: andreadb
Differential Revision: https://reviews.llvm.org/D150312
Need to finalize the DIBuilder to avoid leak sanitizer errors
like this:
Direct leak of 48 byte(s) in 1 object(s) allocated from:
#0 0x55c99ea1761d in operator new(unsigned long)
#1 0x55c9a518ae49 in operator new
#2 0x55c9a518ae49 in llvm::MDTuple::getImpl(...)
#3 0x55c9a4f1b1ec in getTemporary
#4 0x55c9a4f1b1ec in llvm::DIBuilder::createFunction(...)
Make sure we do not crash in rfindDebugLoc when starting at
instr_rend(). Solution is to see it as we start one MI before the
first MI, so we can start searching forward at instr_begin()
instead.
This behavior is similar to how findPrevDebugLoc(instr_end()) works.
Differential Revision: https://reviews.llvm.org/D150577
- Add some unittests for the findDebugLoc, rfindDebugLoc,
findPrevDebugLoc and rfindPrevDebugLoc helpers in MachineBasicBlock.
- Clean up code comments and code formatting related to the functions
mentioned above.
This was extracted as a pre-commit to D150577, adn some of the tests
are commented out since they would crash/assert in a rather
uncontrolled way.
For always poison shifts, any KnownBits return value is valid.
Currently we return unknown, but returning zero is generally more
profitable. We had some code in ValueTracking that tried to do this,
but was actually dead code.
Differential Revision: https://reviews.llvm.org/D150648
Previously, `CCState::AllocateStack` always allocated stack space by increasing
offsets. For targets with stack growing up (away from zero) it is more
convenient to allocate arguments by decreasing offsets, so that the first
argument is at the top of the stack. This is important when calling a function
with variable number of arguments: the callee does not know the size of the
stack, but must be able to access "fixed" arguments. For that to work, the
"fixed" arguments should have fixed offsets relative to the stack top, i.e. the
variadic arguments area should be at the stack bottom (at lowest addresses).
The in-tree target with stack growing up is AMDGPU, but it allocates
arguments by increasing addresses. It does not support variadic arguments.
A drive-by change is to promote stack size/offset to 64-bit integer.
This is what MachineFrameInfo expects.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D149575
This reduces dependencies on `llvm-tblgen` so much.
`CodeGenTypes` depends on `Support` at the moment.
Be careful to append deps on this, since Targets' tablegens
depend on this.
Depends on D149024
Differential Revision: https://reviews.llvm.org/D148769
This is rework of;
- rG13e77db2df94 (r328395; MVT)
Since `LowLevelType.h` has been restored to `CodeGen`, `MachinveValueType.h`
can be restored as well.
Depends on D148767
Differential Revision: https://reviews.llvm.org/D149024
This is rework of;
- D30046 (LLT)
Since I have introduced `llvm-min-tblgen` as D146352, `llvm-tblgen`
may depend on `CodeGen`.
`LowLevlType.h` originally belonged to `CodeGen`. Almost all userse are
still under `CodeGen` or `Target`. I think `CodeGen` is the right place
to put `LowLevelType.h`.
`MachineValueType.h` may be moved as well. (later, D149024)
I have made many modules depend on `CodeGen`. It is consistent but
inefficient. It will be split out later, D148769
Besides, I had to isolate MVT and LLT in modmap, since
`llvm::PredicateInfo` clashes between `TableGen/CodeGenSchedule.h`
and `Transforms/Utils/PredicateInfo.h`.
(I think better to introduce namespace llvm::TableGen)
Depends on D145937, D146352, and D148768.
Differential Revision: https://reviews.llvm.org/D148767
This shrinks MCInstrDesc (and hence the whole TargetInsts table) because
we can store a 16-bit offset value to access the operands info, instead
of a pointer. This also reduces the number of relocs that need to be
applied when LLVM is compiled as position-independent code.
Differential Revision: https://reviews.llvm.org/D142219
This shrinks MCInstrDesc (and hence the whole TargetInsts table) because
we can store a 16-bit offset value to access the implicit operands,
instead of a pointer. This also reduces the number of relocs that need
to be applied when LLVM is compiled as position-independent code.
Differential Revision: https://reviews.llvm.org/D142218
These checks show optimized instructions if an operand is known to be
(partially) zero.
Change-Id: Ie2f6d0d3ee9d5b279d1f4c1dd0787492e39cc77a
Differential Revision: https://reviews.llvm.org/D140208
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
Basically NFC: A TEST/TEST_F/etc that bails out early (usually because
setup failed or some other runtime condition wasn't met) generally
should use GTEST_SKIP() to report its status correctly, unless it
takes steps to report another status (e.g., FAIL()).
I did see a handful of tests show up as SKIPPED after this change,
which is not unexpected. The status seemed appropriate in all the new
cases.
Combine the implicit uses and defs lists into a single list of uses
followed by defs. Instead of 0-terminating the list, store the number
of uses and defs. This avoids having to scan the whole list to find the
length and removes one pointer from MCInstrDesc (although it does not
get any smaller due to alignment issues).
Remove the old accessor methods getImplicitUses, getNumImplicitUses,
getImplicitDefs and getNumImplicitDefs as all clients are using the new
implicit_uses and implicit_defs.
Differential Revision: https://reviews.llvm.org/D142216
Avoid copying MCInstrDesc instances because a future patch will change
them to find their implicit operands and operand info array based on
their own "this" pointer, so it will only work for MCInstrDescs in the
TargetInsts table, not for a copy of an MCInstrDesc at a different
address.
Differential Revision: https://reviews.llvm.org/D142214
Add support to the MI matching of vector splats for patterns that
consist of `G_CONCAT_VECTORS` of smaller splats with the same constant
value. With this, we would consider the following pseudo-MIR to be a splat:
```
%0 = G_[F]CONSTANT [...]
%1 = G_BUILD_VECTOR %0, %0, ..., %0
%2 = G_CONCAT_VECTORS %1, %1, ..., %1
```
Since it uses recursion for matching splats, it could match pretty
complicated patterns with all sorts of combinations of `G_BUILD_VECTOR`
and `G_CONCAT_VECTORS` (e.g. a `G_CONCAT_VECTORS` with
a `G_BUILD_VECTOR_TRUNC` and another `G_CONCAT_VECTORS` as operands),
and it should also look through copies etc.
This should make it easier to match complex immediates for certain
instructions on AMDGPU, where for instance a <8 x s16> will be split
before instruction selection into a `G_CONCAT_VECTORS` of <2 x s16>
splats.
Differential Revision: https://reviews.llvm.org/D141902
generation code from DWARFLinker. It adds command line option:
--build-accelerator [none,DWARF]
Build accelerator tables(default: none)
=none - Do not build accelerators
=DWARF - Build accelerator tables according to the resulting DWARF version
DWARFv4: .debug_pubnames and .debug_pubtypes
DWARFv5: .debug_names
Differential Revision: https://reviews.llvm.org/D139638
