17 Commits

Author SHA1 Message Date
Snehasish Kumar
a1bbf5ac3c [memprof] Record BuildIDs in the raw profile.
This patch adds support for recording BuildIds usng the sanitizer
ListOfModules API. We add another entry to the SegmentEntry struct and
change the memprof raw version.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145190
2023-03-14 18:16:38 +00:00
Snehasish Kumar
debe80cb8d Revert "[memprof] Record BuildIDs in the raw profile."
This reverts commit 287177a47a396ca6cc0bef7696108cdaa0c68e5f.
2023-03-13 20:09:46 +00:00
Snehasish Kumar
287177a47a [memprof] Record BuildIDs in the raw profile.
This patch adds support for recording BuildIds usng the sanitizer
ListOfModules API. We add another entry to the SegmentEntry struct and
change the memprof raw version.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145190
2023-03-13 19:28:38 +00:00
Snehasish Kumar
e99b5ad383 [memprof] Add scripts to automate testdata regeneration.
The memprof profiles and binaries need to be updated in case of version
updates. This change adds three scripts for llvm-profdata, clang and
llvm tests where memprof profiles are used as inputs. Also update the
tests, profiles and binaries in this change. Change based on the review
suggestions in D145023.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145644
2023-03-09 19:54:23 +00:00
Snehasish Kumar
e1b569b96a Revert "[memprof] Refactor tests to generate binaries and profiles on the fly."
This reverts commit 599b7690fa917ea4e9cd67275e34d0b5a0f51aa9. Since
adding a cross project dependency is a concern.
2023-03-06 23:48:52 +00:00
Snehasish Kumar
599b7690fa [memprof] Refactor tests to generate binaries and profiles on the fly.
This change replaces the binary profiles and executables used for
testing the memprof profile reader with tests where the profiles are
generated on the fly. This reduces toil when the profile version
changes. The tests are moved from tools/llvm-profdata to
compiler-rt/test/memprof due to the following reasons:
1. Adding dependency on memprof lit.cfg.py for llvm-profdata is
   preferable to adding a dependency on compiler-rt for llvm/test.
2. All the tests can now be run with `ninja check-memprof`.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D145023
2023-03-06 21:24:40 +00:00
Teresa Johnson
ee73d240ab [MemProf] Collect access density statistics during profiling
Track min/max/avg access density (accesses per byte and accesses per
byte per lifetime second) metrics directly during profiling. This allows
more accurate use of these metrics in profile analysis and use, instead
of trying to compute them from already aggregated data in the profile.

This required regenerating some of the raw profile and executable inputs
for a few tests. While here, make the llvm-profdata memprof tests more
resilient to differences in things like memory mapping, timestamps and
cpu ids to make future test updates easier.

Differential Revision: https://reviews.llvm.org/D141558
2023-01-12 17:53:23 -08:00
Eli Friedman
8f826fe723 Fix reverse-iteration buildbot.
A couple of instances of iterating over maps snuck in while the bot was
down; fix them to use maps with deterministic iteration.
2022-08-19 14:21:05 -07:00
Fangrui Song
103b28902f [llvm-profdata][test] Change -Wl,-no-pie to -no-pie after D127808
The driver option -no-pie is preferred: Clang selects different crt*.o files,
though the PIC one usually can replace the non-PIC one.
2022-06-15 10:46:37 -07:00
Snehasish Kumar
b0c51f00ae [memprof] Update the test comments to include -Wl,-no-pie
Until we have symbolization for position independent code lets update
this documentation since clang now defaults to position independent
code.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D127808
2022-06-15 16:49:19 +00:00
Snehasish Kumar
8a87f42fc6 [memprof] Print out the segment information in YAML format.
This change prints out the segment information in the raw profile in
YAML format for testing. Since we don't capture build ids yet, we print
out <None> for now.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D126840
2022-06-02 02:26:39 +00:00
Snehasish Kumar
962db7de84 [memprof] Update summary output.
Update the YAML format print out of the profile to include a summary
instead of displaying the headers in the raw file buffer. This allows us
to release the raw buffer early saving memory.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D126834
2022-06-02 02:15:42 +00:00
Snehasish Kumar
ec51971eae [memprof] Keep and display symbol names in the RawMemProfReader.
Extend the Frame struct to hold the symbol name if requested
when a RawMemProfReader object is constructed. This change updates the
tests and removes the need to pass --debug to obtain the mapping from
GUID to symbol names.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D126344
2022-05-25 21:17:44 +00:00
Snehasish Kumar
27a4f2545f Reland "[memprof] Store callsite metadata with memprof records."
This reverts commit f4b794427e8037a4e952cacdfe7201e961f31a6f.

Reland with underlying msan issue fixed in D122260.
2022-03-22 14:40:02 -07:00
Mitch Phillips
f4b794427e Revert "[memprof] Store callsite metadata with memprof records."
This reverts commit 0d362c90d335509c57c0fbd01ae1829e2b9c3765.

Reason: Causes the MSan buildbot to fail (see comments on
https://reviews.llvm.org/D121179 for more information
2022-03-21 15:59:13 -07:00
Snehasish Kumar
0d362c90d3 [memprof] Store callsite metadata with memprof records.
To ease profile annotation, each of the callsites in a function can be
annotated with profile data - "IR metadata format for MemProf" [1]. This
patch extends the on-disk serialized record format to store the debug
information for allocation callsites incl inline frames. This change is
incompatible with the existing format i.e. indexed profiles must be
regenerated, raw profiles are unaffected.

[1] https://groups.google.com/g/llvm-dev/c/aWHsdMxKAfE/m/WtEmRqyhAgAJ

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D121179
2022-03-21 13:58:29 -07:00
Snehasish Kumar
49c048add4 [memprof] Add a test to verify callstack order.
We add to ensure that we are observing the correct callstack order in memprof
during symbolization. There was some confusion whether the order of
DIFrame objects were reversed but in reality the leaf function is at
index 0 so no code changes are required.

Differential Revision: https://reviews.llvm.org/D121759
2022-03-16 10:10:57 -07:00