17 Commits

Author SHA1 Message Date
Snehasish Kumar
a1bbf5ac3c [memprof] Record BuildIDs in the raw profile.
This patch adds support for recording BuildIds usng the sanitizer
ListOfModules API. We add another entry to the SegmentEntry struct and
change the memprof raw version.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145190
2023-03-14 18:16:38 +00:00
Snehasish Kumar
debe80cb8d Revert "[memprof] Record BuildIDs in the raw profile."
This reverts commit 287177a47a396ca6cc0bef7696108cdaa0c68e5f.
2023-03-13 20:09:46 +00:00
Snehasish Kumar
287177a47a [memprof] Record BuildIDs in the raw profile.
This patch adds support for recording BuildIds usng the sanitizer
ListOfModules API. We add another entry to the SegmentEntry struct and
change the memprof raw version.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145190
2023-03-13 19:28:38 +00:00
Snehasish Kumar
e99b5ad383 [memprof] Add scripts to automate testdata regeneration.
The memprof profiles and binaries need to be updated in case of version
updates. This change adds three scripts for llvm-profdata, clang and
llvm tests where memprof profiles are used as inputs. Also update the
tests, profiles and binaries in this change. Change based on the review
suggestions in D145023.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145644
2023-03-09 19:54:23 +00:00
Snehasish Kumar
e1b569b96a Revert "[memprof] Refactor tests to generate binaries and profiles on the fly."
This reverts commit 599b7690fa917ea4e9cd67275e34d0b5a0f51aa9. Since
adding a cross project dependency is a concern.
2023-03-06 23:48:52 +00:00
Snehasish Kumar
599b7690fa [memprof] Refactor tests to generate binaries and profiles on the fly.
This change replaces the binary profiles and executables used for
testing the memprof profile reader with tests where the profiles are
generated on the fly. This reduces toil when the profile version
changes. The tests are moved from tools/llvm-profdata to
compiler-rt/test/memprof due to the following reasons:
1. Adding dependency on memprof lit.cfg.py for llvm-profdata is
   preferable to adding a dependency on compiler-rt for llvm/test.
2. All the tests can now be run with `ninja check-memprof`.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145023
2023-03-06 21:24:40 +00:00
Teresa Johnson
ee73d240ab [MemProf] Collect access density statistics during profiling
Track min/max/avg access density (accesses per byte and accesses per
byte per lifetime second) metrics directly during profiling. This allows
more accurate use of these metrics in profile analysis and use, instead
of trying to compute them from already aggregated data in the profile.

This required regenerating some of the raw profile and executable inputs
for a few tests. While here, make the llvm-profdata memprof tests more
resilient to differences in things like memory mapping, timestamps and
cpu ids to make future test updates easier.

Differential Revision: https://reviews.llvm.org/D141558
2023-01-12 17:53:23 -08:00
Fangrui Song
103b28902f [llvm-profdata][test] Change -Wl,-no-pie to -no-pie after D127808
The driver option -no-pie is preferred: Clang selects different crt*.o files,
though the PIC one usually can replace the non-PIC one.
2022-06-15 10:46:37 -07:00
Snehasish Kumar
b0c51f00ae [memprof] Update the test comments to include -Wl,-no-pie
Until we have symbolization for position independent code lets update
this documentation since clang now defaults to position independent
code.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D127808
2022-06-15 16:49:19 +00:00
Snehasish Kumar
962db7de84 [memprof] Update summary output.
Update the YAML format print out of the profile to include a summary
instead of displaying the headers in the raw file buffer. This allows us
to release the raw buffer early saving memory.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D126834
2022-06-02 02:15:42 +00:00
Snehasish Kumar
11314f4059 [memprof] Filter out callstack frames which cannot be symbolized.
This patch filters out callstack frames which can't be symbolized or if
the frames belong to the runtime. Symbolization may not be possible if
debug information is unavailable or if the addresses are from a shared
library. For now we only support optimization of the main binary which
is statically linked to the compiler runtime.

Differential Revision: https://reviews.llvm.org/D120860
2022-03-04 11:10:08 -08:00
Snehasish Kumar
216575e581 Revert "Revert "[ProfileData] Read and symbolize raw memprof profiles.""
This reverts commit dbf47d227d080e4eb7239b589660f51d7b08afa9.

Reapply https://reviews.llvm.org/D116784 now that
https://reviews.llvm.org/D118413 has landed with a couple of fixes:
* fix raw profile reader unaligned access identified by ubsan
* fix windows build by using MOCK_CONST_METHOD3 instead of MOCK_METHOD.
2022-02-08 13:37:27 -08:00
Snehasish Kumar
dbf47d227d Revert "[ProfileData] Read and symbolize raw memprof profiles."
This reverts commit 26f978d4c5ad0d2217940ef7625b0c3c0d576988.

This patch added a transitive dependency on libcurl via symbolize.
See discussion
https://reviews.llvm.org/D116784#inline-1137928
https://reviews.llvm.org/D113717#3295350
2022-02-03 16:14:05 -08:00
Snehasish Kumar
26f978d4c5 [ProfileData] Read and symbolize raw memprof profiles.
This change extends the RawMemProfReader to read all the sections of the
raw profile and symbolize the virtual addresses recorded as part of the
callstack for each allocation. For now the symbolization is used to
display the contents of the profile with llvm-profdata.

Differential Revision: https://reviews.llvm.org/D116784
2022-02-03 14:33:50 -08:00
Snehasish Kumar
14f4f63af5 [memprof] Print out the summary in YAML format.
Print out the profile summary in YAML format to make it easier to for
tools and tests to read in the contents of the raw profile.

Differential Revision: https://reviews.llvm.org/D116783
2022-02-03 14:33:50 -08:00
Snehasish Kumar
86d5dc9afc [memprof] Disallow memprof profile reader tests on non-x86 archs.
The memprof profile reader tests rely on binary data which is generated
from and meant to be interpreted on little endian architectures. Add a
REQUIRES: x86_64-linux clause to both tests to ensure they don't fail on big
endian targets such as ppc.
2021-11-30 12:27:06 -08:00
Snehasish Kumar
7cca33b40f [memprof] Extend llvm-profdata to display MemProf profile summaries.
This commit adds initial support to llvm-profdata to read and print
summaries of raw memprof profiles.
Summary of changes:
* Refactor shared defs to MemProfData.inc
* Extend show_main to display memprof profile summaries.
* Add a simple raw memprof profile reader.
* Add a couple of tests to tools/llvm-profdata.

Differential Revision: https://reviews.llvm.org/D114286
2021-11-30 10:45:26 -08:00