38 Commits

Author SHA1 Message Date
Kristof Beyls
ceb7214be0
[BOLT] Introduce binary analysis tool based on BOLT (#115330)
This initial commit does not add any specific binary analyses yet, it
merely contains the boilerplate to introduce a new BOLT-based tool.

This basically combines the 4 first patches from the prototype pac-ret
and stack-clash binary analyzer discussed in RFC
https://discourse.llvm.org/t/rfc-bolt-based-binary-analysis-tool-to-verify-correctness-of-security-hardening/78148
and published at
https://github.com/llvm/llvm-project/compare/main...kbeyls:llvm-project:bolt-gadget-scanner-prototype

The introduction of such a BOLT-based binary analysis tool was proposed
and discussed in at least the following places:
- The RFC pointed to above
- EuroLLVM 2024 round table
https://discourse.llvm.org/t/summary-of-bolt-as-a-binary-analysis-tool-round-table-at-eurollvm/78441
The round table showed quite a few people interested in being able to
build a custom binary analysis quickly with a tool like this.
- Also at the US LLVM dev meeting a few weeks ago, I heard interest from
a few people, asking when the tool would be available upstream.
- The presentation "Adding Pointer Authentication ABI support for your
ELF platform"
(https://llvm.swoogo.com/2024devmtg/session/2512720/adding-pointer-authentication-abi-support-for-your-elf-platform)
explicitly mentioned interest to extend the prototype tool to verify
correct implementation of pauthabi.
2024-12-12 10:06:27 +00:00
Tristan Ross
abc2eae682
[BOLT] Enable standalone build (#97130)
Continue from #87196 as author did not have much time, I have taken over
working on this PR. We would like to have this so it'll be easier to
package for Nix.

Can be tested by copying cmake, bolt, third-party, and llvm directories
out into their own directory with this PR applied and then build bolt.

---------

Co-authored-by: pca006132 <john.lck40@gmail.com>
2024-07-25 08:18:14 -07:00
Denis Revunov
dfc7599296 [BOLT][Instrumentation] Add test for append-pid option
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D154121
2023-08-23 23:50:32 +03:00
Tobias Hieta
f98ee40f4b
[NFC][Py Reformat] Reformat python files in the rest of the dirs
This is an ongoing series of commits that are reformatting our
Python code. This catches the last of the python files to
reformat. Since they where so few I bunched them together.

Reformatting is done with `black`.

If you end up having problems merging this commit because you
have made changes to a python file, the best way to handle that
is to run git checkout --ours <yourfile> and then reformat it
with black.

If you run into any problems, post to discourse about it and
we will try to help.

RFC Thread below:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Reviewed By: jhenderson, #libc, Mordante, sivachandra

Differential Revision: https://reviews.llvm.org/D150784
2023-05-25 11:17:05 +02:00
Amir Ayupov
19941b0468 [BOLT] Use MCInstPrinter in createRetpolineFunctionTag
Make retpoline functions invariant of X86 register numbers.
retpoline-synthetic.test is known to fail NFC testing due to shifting
register numbers. Use canonical register names instead of tablegen
numbers.

Before:
```
__retpoline_r51_
__retpoline_mem_r58+DATAat0x200fe8
__retpoline_mem_r51+0
__retpoline_mem_r132+0+8*53
```

After:
```
__retpoline_%rax_
__retpoline_mem_%rip+DATAat0x200fe8
__retpoline_mem_%rax+0
__retpoline_mem_%r12+0+8*%rbx
```

Test Plan:
- Revert 67bd3c58c0c7389e39c5a2f4d3b1a30459ccf5b7 that touches X86RegisterInfo.td.
- retpoline-synthetic.test passes in NFC mode with this diff, fails without it.

Reviewed By: #bolt, rafauler

Differential Revision: https://reviews.llvm.org/D150138
2023-05-08 18:50:49 -07:00
Job Noorman
b3780af3b3 [BOLT] Fix many tests detected as unsupported
Since D148847, many tests are detected as being unsupported. This is
caused by BOLT_TARGETS_TO_BUILD being ;-separated whereas the previously
used TARGETS_TO_BUILD is space-separated.

This patch fixes this by creating config.targets lit.cfg.py by splitting
on ';'.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D149026
2023-04-24 11:39:02 -07:00
Amir Ayupov
c49941bd0d [BOLT] Process fragment siblings in lite mode, keep lite mode on
In lite mode, include split function fragments to the list of functions to
process even if a fragment has no samples. This is required to properly
detect and update split jump tables (jump tables that contain pointers to code
in the main and cold fragments).

Reviewed By: #bolt, maksfb

Differential Revision: https://reviews.llvm.org/D140457
2023-02-08 19:11:27 -08:00
Rafael Auler
fc0ced73dc Add BAT testing framework
This patch refactors BAT to be testable as a library, so we
can have open-source tests on it. This further fixes an issue with
basic blocks that lack a valid input offset, making BAT omit those
when writing translation tables.

Test Plan: new testcases added, new testing tool added (llvm-bat-dump)

Differential Revision: https://reviews.llvm.org/D129382
2022-07-29 14:55:04 -07:00
Alexander Yermolovich
ab9a175990 [BOLT][DWARF] Fix TU Index handling for DWARF4/5
When we generate split dwarf with -fdebug-types-section we will have
.debug_types.dwo sections. These go into TU Index when we run llvm-dwp. BOLT was
not handling DWP input correctly with this section.

Added support for handling DWP with TU Index as an input and output for DWARF4.
Added support for handling DWP with TU Index as an input for DWARF5

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D126087
2022-06-01 18:16:12 -07:00
Rafael Auler
b8a6345554 [BOLT] Fix LIT tests on Windows VS2019
Fix newline issue in link_fdata.py, as well as how to call the tool.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126437
2022-05-31 11:45:39 -07:00
Amir Ayupov
4a58eb9e4e [BOLT][TEST] Remove -gdwarf-4 override from %cflags
As BOLT support for monolithic and split DWARF5 is added, remove DWARF version
override for BOLT tests.

Reviewed By: ayermolo

Differential Revision: https://reviews.llvm.org/D125366
2022-05-11 03:38:26 -07:00
Alexey Moksyakov
48e894a536 [BOLT] Add R_AARCH64_PREL16/32/64 relocations support
Reviewed By: yota9, rafauler

Differential Revision: https://reviews.llvm.org/D122294
2022-04-21 13:52:47 +03:00
Amir Ayupov
487570fb86 [BOLT][TEST] Remove -no-pie from cflags/cxxflags
Align with an upstream change D120305 to make PIE the default on linux-gnu.

Add `-no-pie` to tests that require it.

Reviewed By: maksfb, yota9

Differential Revision: https://reviews.llvm.org/D123329
2022-04-13 11:38:40 -07:00
Vladislav Khmelevsky
4956e0e197 [BOLT] Fix plt relocations symbol match
The bfd linker adds the symbol versioning string to the symbol name in symtab.
Skip the versioning part in order to find the registered PLT function.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D122039
2022-04-05 15:57:26 +03:00
Maksim Panchenko
f927106e10 [BOLT][test] Enable cross-target testing
Check for supported target architecture instead of the host arch when
deciding to execute non-runtime tests.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D122498
2022-04-04 23:00:54 -07:00
Amir Ayupov
03f014c9ae [BOLT] Add ld.lld substitution
Register ld.lld substition (tests were failing)

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D118738
2022-02-07 20:59:18 -08:00
Amir Ayupov
1ab13cc18b [BOLT][TEST] Add heatmap.test
Add a basic test for heatmap mode

Reviewed By: maksfb, ayermolo

Differential Revision: https://reviews.llvm.org/D118868
2022-02-02 18:56:15 -08:00
Vladislav Khmelevsky
dcc595ea3c [BOLT] Fix DWARFv5 for aarch64
This patch reverts patch "DWARFv5 default: Switch bolt tests to use
DWARFv4 since Bolt doesn't support v5 yet" and places the -gdwarf-4 flag
to the global cflags config file.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D118283
2022-01-27 02:14:58 +03:00
Amir Ayupov
5a4bf4c2b3 [BOLT][CMAKE] Use BOLT_CLANG_EXE and BOLT_LLD_EXE as is
Add an ability to provide paths that don't match tool name exactly:
e.g. clang-13.
Remove use_lld call that sets up unused extra tools.

Test plan:
```
cmake -G Ninja ../llvm-project/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="bolt" -DBOLT_CLANG_EXE=/usr/bin/clang-13 -DBOLT_LLD_EXE=/usr/bin/lld-13
...
llvm-lit: /data/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using clang: /usr/bin/clang-13
llvm-lit: /data/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using lld: /usr/bin/lld-13

cmake -G Ninja ../llvm-project/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="bolt;lld" -DBOLT_CLANG_EXE=/usr/bin/clang-13
...
llvm-lit: /data/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using clang: /usr/bin/clang-13
llvm-lit: /data/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using lld: /data/llvm-build2/bin/lld

cmake -G Ninja ../llvm-project/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="bolt;clang;lld"
...
llvm-lit: /data/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using clang: /data/llvm-build3/bin/clang
llvm-lit: /data/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using lld: /data/llvm-build3/bin/lld
```

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D117446
2022-01-18 21:14:00 -08:00
Amir Ayupov
de3e3fcfa3 [BOLT][CMAKE] Accept BOLT_CLANG_EXE and BOLT_LLD_EXE
Add CMake options to supply clang and lld binaries for use in check-bolt
instead of requiring the build of clang and lld projects.

Suggested by Mehdi Amini in https://lists.llvm.org/pipermail/llvm-dev/2021-December/154426.html

Test Plan:
```
cmake -G Ninja ~/local/llvm-project/llvm \
-DLLVM_TARGETS_TO_BUILD="X86"  \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DLLVM_ENABLE_PROJECTS="bolt"  \
-DBOLT_CLANG_EXE=~/local/bin/clang \
-DBOLT_LLD_EXE=~/local/bin/lld

ninja check-bolt
...
llvm-lit: /home/aaupov/local/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using clang: /home/aaupov/local/bin/clang
llvm-lit: /home/aaupov/local/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using ld.lld: /home/aaupov/local/bin/ld.lld
llvm-lit: /home/aaupov/local/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using lld-link: /home/aaupov/local/bin/lld-link
llvm-lit: /home/aaupov/local/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using ld64.lld: /home/aaupov/local/bin/ld64.lld
llvm-lit: /home/aaupov/local/llvm-project/llvm/utils/lit/lit/llvm/config.py:436: note: using wasm-ld: /home/aaupov/local/bin/wasm-ld
...
```

Tested all configurations:
- LLVM_ENABLE_PROJECTS="bolt;clang;lld" + no BOLT_*_EXE
- LLVM_ENABLE_PROJECTS="bolt;clang" + BOLT_LLD_EXE
- LLVM_ENABLE_PROJECTS="bolt;lld" + BOLT_CLANG_EXE
- LLVM_ENABLE_PROJECTS="bolt" + BOLT_CLANG_EXE + BOLT_LLD_EXE
- LLVM_ENABLE_PROJECTS="bolt;clang;lld" + BOLT_CLANG_EXE + BOLT_LLD_EXE

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D117061
2022-01-15 04:37:29 -08:00
Rafael Auler
b392ec696b Re-enable Windows build and fix issues
Summary:
Fix missing string header file inclusion and link_fdata find
problem in lit tests. Change root-level tests to require
linux. Re-enable Windows in our root CMakeLists.txt.

(cherry picked from FBD33296290)
2021-12-23 05:59:35 -08:00
Rafael Auler
07d9e014ed [BOLT] Don't use ld.lld in tests
Summary: Addressing issue 270.

(cherry picked from FBD33255608)
2021-12-21 07:36:35 -08:00
Amir Ayupov
c7f8adb87f [BOLT][TEST] Add llvm-boltdiff to build/test requirements
Summary: llvm-boltdiff is required for `runtime/meta-merge-fdata.test`

(cherry picked from FBD32442220)
2021-11-15 13:53:18 -08:00
Amir Ayupov
3a16f2169d [BOLT][TEST] Import jump-table-icp.test, update link_fdata script
Summary:
Import the test. The assembly input has three functions with associated fdata.
The old link_fdata.sh script only replaces the symbol names with symbol values,
whereas fdata format expects to have symbol offsets against the anchor symbol.
Introduce the link_fdata.py script which is able to parse the input and produce
either an offset or an absolute symbol value.

(cherry picked from FBD32256351)
2021-11-08 10:56:21 -08:00
Amir Ayupov
d1df113e30 [BOLT][TEST] Add instrumentation test using merge-fdata
Summary:
BOLT meta test using merge-fdata tool.

This tests BOLT instrumentation for a non-trivial binary, running instrumented
binary, and using the instrumentation profile for BOLT optimizations.
The results are verified between original, instrumented, and optimized binaries.
Additional tested features: boltdiff mode and merge-fdata for two profiles.

merge-fdata tool is linked with relocs on Linux to support this test.

(cherry picked from FBD32141812)
2021-11-03 10:41:26 -07:00
Amir Ayupov
f808ea00bd [BOLT][TEST] Import small tests
Summary:
Imported small internal tests:
- asm_func_debug.test
- basic_instrumentation.test
- bolt_icf.test
- ctc_and_unreachable.test
- double_jump.test
- exceptions_args.test
- exceptions_instrumentation.test
- fptr.test

(cherry picked from FBD32032684)
2021-10-29 13:31:22 -07:00
Rafael Auler
0559dab546 [BOLT] Improve cmake configs for opensource
Summary:
Change cmake config in BOLT to only support Linux. In other
platforms, we print a warning that we won't build BOLT.  Change
configs to determine whether we will build BOLT runtime libs. This
only happens in x86 hosts. If true, we will build the runtime and
enable bolt-runtime tests. New tests that depend on the bolt_rt lib
needs to be marked REQUIRES:bolt-runtime. I updated the relevant
tests.  Fix cmake to do not crash when building llvm with a target
that BOLT does not support.

(cherry picked from FBD31935760)
2021-10-26 12:26:23 -07:00
Amir Ayupov
d4fdc98140 [BOLT][TEST] Remove dependence on host_cc and host_cxx
Summary: Add dependency on clang and clangxx instead.

(cherry picked from FBD31128140)
2021-09-22 15:53:38 -07:00
Vasily Leonenko
e2480fcc98 [PR] LIT: add checking if maxIndividualTestTime is availabe on the platform
Summary:
This commit adds checking if maxIndividualTestTime is availabe on
the platform. If available - it sets per test timeout to 60sec and
declares lit-max-individual-test-time feature for further checking
by particular test cases.
Based on https://reviews.llvm.org/D64251 implementation.

Vasily Leonenko,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD30821986)
2021-08-27 21:56:24 +03:00
Vladislav Khmelevsky
2a5790b670 [PR] Fdata: Escape whitespaces in symbol names
Summary:
This patch is part of preparation for golang support. The golang symbols
might have spaces in the name (for example "type..eq.[10]interface {}").
Since fdata uses spaces as a field separator such names brakes the fdata
format, so we need to escape whitespaces and backslashes in symbol names
using the backslash character.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD29999491)
2021-06-29 19:54:08 +03:00
Maksim Panchenko
7bccf8d25d [BOLT][NFC] Fix debug info printouts for inlined functions
Summary:
While printing debug info for instructions, we should use line tables
from the corresponding DWARF CU which could be different from the
containing function CU in case of inlined instructions.

(cherry picked from FBD28908324)
2021-06-04 12:31:31 -07:00
Vladislav Khmelevsky
de298c08fd [PR] Fix tests build with -no-pie option
Summary:
Since gcc/ld could produce and expect PIE files we need to pass -no-pie option to avoid linking errors for tests.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

(cherry picked from FBD28360045)
2021-05-11 03:25:49 +03:00
Amir Ayupov
94653797f3 Rebase: [BOLT][NFC] Avoid binutils in tests
Summary:
Replace binutils tools with llvm tools

(cherry picked from FBD29575630)
2021-05-04 16:45:28 -07:00
Alexander Yermolovich
0ec91a25df Update DW_AT_stmt_list for .debug_types
Summary:
There is no real link between CU and TU, so relying on fact
that address are the same, and we are updating all of them.

(cherry picked from FBD28112114)
2021-02-17 15:30:10 -08:00
Amir Ayupov
1c5d3a056c Rebase: Merge BOLT codebase in monorepo
Summary:
This commit is the first step in rebasing all of BOLT
history in the LLVM monorepo. It also solves trivial build issues
by updating BOLT codebase to use current LLVM. There is still work
left in rebasing some BOLT features and in making sure everything
is working as intended.

History has been rewritten to put BOLT in the /bolt folder, as
opposed to /tools/llvm-bolt.

(cherry picked from FBD33289252)
2020-12-01 16:29:39 -08:00
Amir Ayupov
dc48354f71 processInterproceduralReferences: record references to cold fragments as entry points
Summary:
For interprocedural references to fragments, record them as
fragment entry points. Not registering these entry points leads to
UCE removing the blocks and "Undefined temporary symbol"
assertion.

(cherry picked from FBD24511281)
2020-11-06 10:57:47 -08:00
Rafael Auler
340da8f294 [BOLT] Fix shrink wrapping to check pops
Summary:
Shrink wrapping has a mode where it will directly move push
pop pairs, instead of replacing them with stores/loads. This is an
ambitious mode that is triggered sometimes, but whenever matching with
a push, it would operate with the assumption that the restoring
instruction was a pop, not a load, otherwise it would assert. Fix this
assertion to bail nicely back to non-pushpop mode (use regular store and
load instructions).

(cherry picked from FBD20085905)
2020-02-18 16:00:40 -08:00
Rafael Auler
8f717dd25e [BOLT] Add initial bolt-only test infra
Summary:
Create folders and setup to make LIT run BOLT-only tests. Add
a test example. This will add a new make/ninja rule "check-bolt" that
the user can invoke to run LIT on this folder.

(cherry picked from FBD8595786)
2018-06-22 13:50:07 -07:00