1406 Commits

Author SHA1 Message Date
Kazu Hirata
32aa35b504 Drop empty string literals from static_assert (NFC)
Identified with modernize-unary-static-assert.
2022-09-03 11:17:47 -07:00
Kazu Hirata
89f1433225 Use llvm::lower_bound (NFC) 2022-09-03 11:17:37 -07:00
Kazu Hirata
a0c7ca8ad4 [BOLT] Use range-based for loops (NFC)
LLVM Coding Standards discourage for_each unless callable objects
already exist.
2022-09-03 11:17:32 -07:00
Kazu Hirata
20f0f15a40 Use StringRef::contains (NFC) 2022-08-28 23:29:02 -07:00
Amir Ayupov
f119a2483d [BOLT][NFC] Use llvm::any_of
Replace the imperative pattern of the following kind
```
bool IsTrue = false;
for (Element : Range) {
  if (Condition(Element)) {
    IsTrue = true;
    break;
  }
}
```
with functional style `llvm::any_of`:
```
bool IsTrue = llvm::any_of(Range, [&](Element) {
  return Condition(Element);
});
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132276
2022-08-27 21:36:15 -07:00
Amir Ayupov
62a034cc14 [BOLT][UTILS] Stash including untracked in nfc-check-setup
The command to detect whether the stash is needed is `git status --porcelain`
which includes untracked files by default. We want to stash untracked files
as well as they may affect compilation (LLVM CMake checks that all source files
should be included in CMakeLists).

Update the stash command to include untracked files as well.

Reviewed By: #bolt, rafauler

Differential Revision: https://reviews.llvm.org/D132610
2022-08-26 19:12:41 -07:00
Alexander Yermolovich
1ee74064e0 [BOLT][DWARF] Fix updating CU that has no entry in .debug_addr
We were trying to process .debug_addr for CU that doesn't have it. This resulted
in assert. Example came from GCC that also doesn't use DW_OP_addrx in
DW_FORM_exprloc.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D132422
2022-08-25 17:03:11 -07:00
John Ericson
34fe6ddce1 Revert "[CMake] Avoid LLVM_BINARY_DIR when other more specific variable are better-suited"
This reverts commit ad8c34bc3089d847a09bb740f7a58c96073e0959.
2022-08-25 11:13:46 -04:00
Denis Revunov
6040415ef9 [BOLT][AArch64] Handle references to the middle of Constant Islands
Fix BinaryContext::handleAddressRef to properly detect references to
other function's Constant islands.

Revieved By: rafauler, yota9

Differential Revision: https://reviews.llvm.org/D132376
2022-08-25 04:32:35 -04:00
Fabian Parzefall
9b6e7861ae [BOLT] Track fragment info for all split fragments
To generate all symbols correctly, it is necessary to record the address
of each fragment. This patch moves the address info for the main and
cold fragments from BinaryFunction to FunctionFragment, where this data
is recorded for all fragments.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132051
2022-08-24 18:07:09 -07:00
Fabian Parzefall
07f63b0ac5 [BOLT] Allocate FunctionFragment on heap
This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-specific information.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132050
2022-08-24 18:06:08 -07:00
Fabian Parzefall
d5c03def24 [BOLT] Towards FunctionLayout const-correctness
A const-qualified reference to function layout allows accessing
non-const qualified basic blocks on a const-qualified function. This
patch adds or removes const-qualifiers where necessary to indicate where
basic blocks are used in a non-const manner.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132049
2022-08-24 16:32:33 -07:00
Fabian Parzefall
f24c299e7d Revert "[BOLT] Towards FunctionLayout const-correctness"
This reverts commit 587d2653420d75ef10f30bd612d86f1e08fe9ea7.
2022-08-24 10:51:38 -07:00
Fabian Parzefall
5065134aa0 Revert "[BOLT] Allocate FunctionFragment on heap"
This reverts commit 101344af1af82d1633c773b718788eaa813d7f79.
2022-08-24 10:51:36 -07:00
Fabian Parzefall
6304e38281 Revert "[BOLT] Track fragment info for all split fragments"
This reverts commit 7e254818e49454a53bd00e3737007025b62d0f79.
2022-08-24 10:51:19 -07:00
Fabian Parzefall
7e254818e4 [BOLT] Track fragment info for all split fragments
To generate all symbols correctly, it is necessary to record the address
of each fragment. This patch moves the address info for the main and
cold fragments from BinaryFunction to FunctionFragment, where this data
is recorded for all fragments.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132051
2022-08-24 10:17:17 -07:00
Fabian Parzefall
101344af1a [BOLT] Allocate FunctionFragment on heap
This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-specific information.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132050
2022-08-24 10:17:17 -07:00
Fabian Parzefall
587d265342 [BOLT] Towards FunctionLayout const-correctness
A const-qualified reference to function layout allows accessing
non-const qualified basic blocks on a const-qualified function. This
patch adds or removes const-qualifiers where necessary to indicate where
basic blocks are used in a non-const manner.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132049
2022-08-24 10:17:17 -07:00
Simon Tatham
79f99bf622 [bolt] Fix a test affected by D131589.
This test contained some data tables that llvm-objdump was
disassembling as code, so the test was recovering the 32-bit values in
the table from the instruction encoding column of the disassembly.

D131589 changed how llvm-objdump decides what to disassemble as code
or as data. As a result, these data tables are now being disassembled
as data, which I think is actually more sensible -- but the test
wasn't expecting it, and got confused.
2022-08-24 15:52:06 +01:00
John Ericson
ad8c34bc30 [CMake] Avoid LLVM_BINARY_DIR when other more specific variable are better-suited
A simple sed doing these substitutions:

- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?lib(${LLVM_LIBDIR_SUFFIX})?\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`

where `\>` means "word boundary".

The only manual modifications were reverting changes in

- `compiler-rt/cmake/Modules/CompilerRTUtils.cmake
- `runtimes/CMakeLists.txt`

because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.

This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.

Reviewed By: sebastian-ne

Differential Revision: https://reviews.llvm.org/D132316
2022-08-24 10:14:05 -04:00
Amir Ayupov
37cbbea674 [BOLT][NFC] Move out handleAArch64IndirectCall
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
255 to 233 LoC.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132104
2022-08-23 17:37:01 -07:00
Amir Ayupov
c844850bdf [BOLT][NFC] Move out handleIndirectBranch
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
295 to 255 LoC.

Differential Revision: https://reviews.llvm.org/D132101
2022-08-23 17:36:51 -07:00
Amir Ayupov
ec1fbf229e [BOLT][NFC] Move out handleExternalReference
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
338 to 295 LoC.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132100
2022-08-23 17:36:41 -07:00
Amir Ayupov
6cd475f8ca [BOLT][NFC] Move out handlePCRelOperand
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
377 to 338 LoC.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132099
2022-08-23 17:36:29 -07:00
Kazu Hirata
258531b7ac Remove redundant initialization of Optional (NFC) 2022-08-20 21:18:28 -07:00
John Ericson
90dcdc4b6e [bolt][llvm][cmake] Use CMAKE_INSTALL_LIBDIR too
Working back towards D130586.

Bolt didn't use `LLVM_LIBDIR_SUFFIX` before, and has no in-tree reverse
dependencies, it seems easier to add.

The change in LLVM itself is to prevent some unexpected `lib64` from
cropping up due to the `CMAKE_INSTALL_LIBDIR` defaulting logic.

Differential Revision: https://reviews.llvm.org/D132297
2022-08-20 13:08:06 -04:00
Alexander Yermolovich
928c2ba179 [DWARF][BOLT] Fix handling of converting range accesss from ofset to index.
Wasn't handling correctly creating DW_AT_rnglists_base in UnitDie when
converting access pattern for DW_AT_ranges from offset to index for DWARF5.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132087
2022-08-19 15:28:12 -07:00
Fabian Parzefall
e001a4e489 [BOLT] Insert EH trampolines for multiple fragments
This patch adds exception handling trampolines when a function is split
into more than two fragments. Trampolines are tracked per-fragment, such
that they can be removed if splitting is reversed.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132048
2022-08-18 21:55:08 -07:00
Fabian Parzefall
ac830664b2 [BOLT] Update buildCallGraph to check for split blocks
Use isSplit() instead of isCold() when building the call graph and
update parameter names to reflect this.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132047
2022-08-18 21:55:08 -07:00
Fabian Parzefall
48ff38ce5d [BOLT] Add randomN split strategy
This adds a strategy to split functions into a random number of
fragments at randomly chosen split points.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D130647
2022-08-18 21:55:07 -07:00
Fabian Parzefall
f428db7a00 [BOLT] Add split all blocks strategy
This adds a function splitting strategy that splits each outlineable
basic block into its own fragment. This is exposed through a new command
line option `--split-strategy`.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D129827
2022-08-18 21:55:07 -07:00
Fabian Parzefall
0f74d191d1 [BOLT] Generate sections for multiple fragments
This patch adds support to generate any number of sections that are
assigned to fragments of functions that are split more than two-way.
With this, a function's *nth* split fragment goes into section
`.text.cold.n`.

This also changes `FunctionLayout::erase` to make sure, that there are
no empty fragments at the end of the function. This sometimes happens
when blocks are erased from the function. To avoid creating symbols
pointing to these fragments, they need to be removed.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D130521
2022-08-18 21:55:06 -07:00
Fabian Parzefall
a191ea7d59 [BOLT] Make exception handling fragment aware
This adds basic fragment awareness in the exception handling passes and
generates the necessary symbols for fragments.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D130520
2022-08-18 21:55:06 -07:00
Fabian Parzefall
275e075cbe [BOLT] Support passing fragments to code emission
This changes code emission such that it can emit specific function
fragments instead of scanning all basic blocks of a function and just
emitting those that are hot or cold.

To implement this, `FunctionLayout` explicitly distinguishes the "main"
fragment (i.e. the one that contains the entry block and is associated
with the original symbol) from "split" fragments. Additionally,
`BinaryFunction` receives support for multiple cold symbols - one for
each split fragment.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D130052
2022-08-18 21:55:06 -07:00
John Ericson
e941b031d3 Revert "[cmake] Use CMAKE_INSTALL_LIBDIR too"
This reverts commit f7a33090a91015836497c75f173775392ab0304d.

Unfortunately this causes a number of failures that didn't show up in my
local build.
2022-08-18 22:46:32 -04:00
Amir Ayupov
129dfc8a9a Revert "[BOLT][NFC] Simplify addRelocation"
This reverts commit 29f23013226097a18ce90c48c9435f324956779b.

This change breaks one of the internal tests.
2022-08-18 17:26:26 -07:00
John Ericson
f7a33090a9 [cmake] Use CMAKE_INSTALL_LIBDIR too
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.

`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.

I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.

Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi

Differential Revision: https://reviews.llvm.org/D130586
2022-08-18 15:33:35 -04:00
Denis Revunov
d0e29e87cd [BOLT][AArch64] Ignore functions with islandsInfo during VeneerEliminarion and ICF
Differential Revision: https://reviews.llvm.org/D131881

Reviewed By: yota9
2022-08-18 11:08:47 -04:00
Amir Ayupov
e33599371e [BOLT][NFC] Reformat strings in handleRelocation
With reduced indentation, some strings can be reformatted to take less lines.
Also strategically apply `formatv` to shorten them.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132088
2022-08-17 20:45:18 -07:00
Amir Ayupov
70d0134f1d [BOLT][NFC] Split out handleRelocation
Split out the body of a for-loop in `RewriteInstance::readRelocations` into a
separate function (`handleRelocation`). It's still over 300 lines of code,
so it's worth splitting down further.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132078
2022-08-17 20:43:51 -07:00
Amir Ayupov
330eec139e [BOLT][UTILS] Add nfc-check-setup --switch-back option
Add an option to switch repo revision back, handling stashing automatically.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D128243
2022-08-17 17:37:27 -07:00
Amir Ayupov
055f9f6d08 [BOLT][NFC] Simplify debug logging in case of JT heuristic failure
Move logging into LLVM_DEBUG scope.
Remove redundant printing of jump table parents:

Old logging:
```
failed to analyze jump table in function _ZN12_GLOBAL__N_116InitHeaderSearch23Ad
dDefaultCIncludePathsERKN4llvm6TripleERKN5clang19HeaderSearchOptionsE/1(*2)
PIC Jump table JUMP_TABLE/_ZN12_GLOBAL__N_116InitHeaderSearch23AddDefaultCInclud
ePathsERKN4llvm6TripleERKN5clang19HeaderSearchOptionsE/1.1 for function _ZN12_GL
OBAL__N_116InitHeaderSearch23AddDefaultCIncludePathsERKN4llvm6TripleERKN5clang19
HeaderSearchOptionsE/1(*2) at 0x65996e0 with a total count of 0:
  0x9dc

next jump table at 0x659a810 belongs to function _ZN5clang5Lexer40LexDependencyD
irectiveTokenWhileSkippingERNS_5TokenE
PIC Jump table JUMP_TABLE/_ZN5clang5Lexer40LexDependencyDirectiveTokenWhileSkipp
ingERNS_5TokenE.0 for function _ZN5clang5Lexer40LexDependencyDirectiveTokenWhile
SkippingERNS_5TokenE at 0x659a810 with a total count of 0:

jump table heuristic failure
```

New logging:
```
failed to analyze PIC Jump table JUMP_TABLE/_ZN12_GLOBAL__N_116InitHeaderSearch2
3AddDefaultCIncludePathsERKN4llvm6TripleERKN5clang19HeaderSearchOptionsE/1.1 for
function _ZN12_GLOBAL__N_116InitHeaderSearch23AddDefaultCIncludePathsERKN4llvm6T
ripleERKN5clang19HeaderSearchOptionsE/1(*2) at 0x65996e0 with a total count of 0:
  absolute offset: 0x52ac58c

next PIC Jump table JUMP_TABLE/_ZN5clang5Lexer40LexDependencyDirectiveTokenWhile
SkippingERNS_5TokenE.0 for function _ZN5clang5Lexer40LexDependencyDirectiveToken
WhileSkippingERNS_5TokenE at 0x659a810 with a total count of 0:

jump table heuristic failure
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D131243
2022-08-17 17:35:16 -07:00
Amir Ayupov
cdef841fe7 [BOLT][NFC] Simplify scanExternalRefs
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132013
2022-08-17 17:33:59 -07:00
Alexander Yermolovich
ccbf28b09d [BOLT][DWARF] Handle zero size DW_TAG_inlined_subroutine
We were resetting DW_AT_low_pc to zero when DW_AT_high_pc was zero, or
DW_AT_low_pc == DW_AT_high_pc. This resulted in LLDB to print error "adding
range [0x0-0x0) which has a base that is less than the function's low PC".

Changed it so that when this case arises we set DW_AT_low_pc to the start
address.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132059
2022-08-17 17:29:53 -07:00
Fabian Parzefall
fd159c2316 [BOLT] Fix ignored LP at fragment start
If the first block of a fragment is also a landing pad, the landing pad
is not used if an exception is thrown. This is because the landing pad
is at the same start address that the corresponding LSDA describes. In
that case, the offset in the call site records to refer to that landing
pad is zero, and a zero offset is interpreted by the personality
function as "no handler" and ignored.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D132053
2022-08-17 16:34:44 -07:00
Amir Ayupov
4ddc9c8e12 [BOLT][NFC] Move printRelocationInfo into a method
Move this large lambda out of readRelocations into a standalone method.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D131812
2022-08-17 16:28:33 -07:00
Amir Ayupov
29f2301322 [BOLT][NFC] Simplify addRelocation
Move the implementation out of the header file.
Simplify the method.
Add debug logging.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D131811
2022-08-17 16:07:28 -07:00
Alexander Yermolovich
b786e01f93 [DWARF][BOLT] Handle getBinaryFunctionContainingAddress returning nullptr for DW_TAG_call_site
DW_TAG_call_site/DW_AT_call_return_pc can contain address that is not in any
function. In this case getBinaryFunctionContainingAddress returns nullptr. For
this case preserving original address.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D132057
2022-08-17 16:04:34 -07:00
Fabian Parzefall
aed75748de [BOLT] Remove old layout from function layout
To track whether a function's new layout is different from its old
layout when updating it, the old layout would be kept around in memory
indefinitely (if the new layout is different). This was used only for
debugging/logging purposes. This patch forces the caller of function
layout's update method to copy the old layout into a temporary if they
need it by removing the old layout fields.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D131413
2022-08-17 15:06:17 -07:00
Fabian Parzefall
0f8412c19c [BOLT] Add main fragment to function layout
Functions that do not contain any code still have to be emitted. This
occurs on AArch64 where functions can consist only of a constant island.
To support fragment semantics in code emission, this commits adds a
guaranteed main fragment to function layout. This fragment might be
empty, but allows us omit checks whether the function is empty in most
places.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D130051
2022-08-17 14:51:31 -07:00