added new command "process trace save -d <directory>".
-it saves a JSON file as <directory>/trace.json, with the main properties of the trace session.
-it saves binary Intel-pt trace as <directory>/thread_id.trace; each file saves each thread.
-it saves modules to the directory <directory>/modules .
-it only works for live process and it only support Intel-pt right now.
Example:
```
b main
run
process trace start
n
process trace save -d /tmp/mytrace
```
A file named trace.json and xxx.trace should be generated in /tmp/mytrace. To load the trace that was just saved:
```
trace load /tmp/mytrace
thread trace dump instructions
```
You should see the instructions of the trace got printed.
To run a test:
```
cd ~/llvm-sand/build/Release/fbcode-x86_64/toolchain
ninja lldb-dotest
./bin/lldb-dotest -p TestTraceSave
```
Reviewed By: wallace
Differential Revision: https://reviews.llvm.org/D107669
Follow up on https://reviews.llvm.org/D105741
- Add new test that exhaustively checks the output file's content
- Fix typos in documentation and other minor fixes
Reviewed By: wallace
Original Author: jj10306
Differential Revision: https://reviews.llvm.org/D107674
This diff introduces Hierarchical Trace Representation (HTR) and creates the `thread trace export ctf -f <filename> -t <thread_id>` command to export an Intel PT trace's HTR to Chrome Trace Format (CTF) for visualization.
See `lldb/docs/htr.rst` for context/documentation on HTR.
**Overview of Changes**
- Add HTR documentation (see `lldb/docs/htr.rst`)
- Add HTR structures (layer, block, block metadata)
- Implement "Basic Super Block" HTR pass
- Add 'thread trace export ctf' command to export the HTR of an Intel PT
trace to Chrome Trace Format (CTF)
As this diff is the first iteration of HTR and trace visualization, future diffs will build on this work by generalizing the internal design of HTR and implementing new HTR passes that provide better trace summarization/visualization.
See attached video for an example of Intel PT trace visualization:
{F17851042}
Original Author: jj10306
Submitted by: wallace
Reviewed By: wallace, clayborg
Differential Revision: https://reviews.llvm.org/D105741
This diff introduces Hierarchical Trace Representation (HTR) and creates the `thread trace export ctf -f <filename> -t <thread_id>` command to export an Intel PT trace's HTR to Chrome Trace Format (CTF) for visualization.
See `lldb/docs/htr.rst` for context/documentation on HTR.
**Overview of Changes**
- Add HTR documentation (see `lldb/docs/htr.rst`)
- Add HTR structures (layer, block, block metadata)
- Implement "Basic Super Block" HTR pass
- Add 'thread trace export ctf' command to export the HTR of an Intel PT
trace to Chrome Trace Format (CTF)
As this diff is the first iteration of HTR and trace visualization, future diffs will build on this work by generalizing the internal design of HTR and implementing new HTR passes that provide better trace summarization/visualization.
See attached video for an example of Intel PT trace visualization:
{F17851042}
Original Author: jj10306
Submitted by: wallace
Reviewed By: wallace, clayborg
Differential Revision: https://reviews.llvm.org/D105741
When the user types that command 'thread trace dump info' and there's a running Trace session in LLDB, a raw trace in bytes should be printed; the command 'thread trace dump info all' should print the info for all the threads.
Original Author: hanbingwang
Reviewed By: clayborg, wallace
Differential Revision: https://reviews.llvm.org/D105717
D104422 added the interface for TraceCursor, which is the main way to traverse instructions in a trace. This diff implements the corresponding cursor class for Intel PT and deletes the now obsolete code.
Besides that, the logic for the "thread trace dump instructions" was adapted to use this cursor (pretty much I ended up moving code from Trace.cpp to TraceCursor.cpp). The command by default traverses the instructions backwards, and if the user passes --forwards, then it's not forwards. More information about that is in the Options.td file.
Regarding the Intel PT cursor. All Intel PT cursors for the same thread share the same DecodedThread instance. I'm not yet implementing lazy decoding because we don't need it. That'll be for later. For the time being, the entire thread trace is decoded when the first cursor for that thread is requested.
Differential Revision: https://reviews.llvm.org/D105531
This adds a basic SB API for creating and stopping traces.
Note: This doesn't add any APIs for inspecting individual instructions. That'd be a more complicated change and it might be better to enhande the dump functionality to output the data in binary format. I'll leave that for a later diff.
This also enhances the existing tests so that they test the same flow using both the command interface and the SB API.
I also did some cleanup of legacy code.
Differential Revision: https://reviews.llvm.org/D103500
At the moment nearly every test calls something similar to
`self.dbg.CreateTarget(self.getBuildArtifact("a.out"))` and them sometimes
checks if the created target is actually valid with something like
`self.assertTrue(target.IsValid(), "some useless text")`.
Beside being really verbose the error messages generated by this pattern are
always just indicating that the target failed to be created but now why.
This patch introduces a helper function `createTestTarget` to our Test class
that creates the target with the much more verbose `CreateTarget` overload that
gives us back an SBError (with a fancy error). If the target couldn't be created
the function prints out the SBError that LLDB returned and asserts for us. It
also defaults to the "a.out" build artifact path that nearly all tests are using
to avoid to hardcode "a.out" in every test.
I converted a bunch of tests to the new function but I'll do the rest of the
test suite as follow ups.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D102771
When dumping the traced instructions in a for loop, like this one
4: for (int a = 0; a < n; a++)
5: do something;
there might be multiple LineEntry objects for line 4, but with different address ranges. This was causing the dump command to dump something like this:
```
a.out`main + 11 at main.cpp:4
[1] 0x0000000000400518 movl $0x0, -0x8(%rbp)
[2] 0x000000000040051f jmp 0x400529 ; <+28> at main.cpp:4
a.out`main + 28 at main.cpp:4
[3] 0x0000000000400529 cmpl $0x3, -0x8(%rbp)
[4] 0x000000000040052d jle 0x400521 ; <+20> at main.cpp:5
```
which is confusing, as main.cpp:4 appears twice consecutively.
This diff fixes that issue by making the line entry comparison strictly about the line, column and file name. Before it was also comparing the address ranges, which we don't need because our output is strictly about what the user sees in the source.
Besides, I've noticed that the logic that traverses instructions and calculates symbols and disassemblies had too much coupling, and made my changes harder to implement, so I decided to decouple it. Now there are two methods for iterating over the instruction of a trace. The existing one does it on raw load addresses, but the one provides a SymbolContext and an InstructionSP, and does the calculations efficiently (not as efficient as possible for now though), so the caller doesn't need to care about these details. I think I'll be using that iterator to reconstruct the call stacks.
I was able to fix a test with this change.
Differential Revision: https://reviews.llvm.org/D100740
This implements the interactive trace start and stop methods.
This diff ended up being much larger than I anticipated because, by doing it, I found that I had implemented in the beginning many things in a non optimal way. In any case, the code is much better now.
There's a lot of boilerplate code due to the gdb-remote protocol, but the main changes are:
- New tracing packets: jLLDBTraceStop, jLLDBTraceStart, jLLDBTraceGetBinaryData. The gdb-remote packet definitions are quite comprehensive.
- Implementation of the "process trace start|stop" and "thread trace start|stop" commands.
- Implementaiton of an API in Trace.h to interact with live traces.
- Created an IntelPTDecoder for live threads, that use the debugger's stop id as checkpoint for its internal cache.
- Added a functionality to stop the process in case "process tracing" is enabled and a new thread can't traced.
- Added tests
I have some ideas to unify the code paths for post mortem and live threads, but I'll do that in another diff.
Differential Revision: https://reviews.llvm.org/D91679
Depends on D90490.
The stop command is simple and invokes the new method Trace::StopTracingThread(thread).
On the other hand, the start command works by delegating its implementation to a CommandObject provided by the Trace plugin. This is necessary because each trace plugin needs different options for this command. There's even the chance that a Trace plugin can't support live tracing, but instead supports offline decoding and analysis, which means that "thread trace dump instructions" works but "thread trace start" doest. Because of this and a few other reasons, it's better to have each plugin provide this implementation.
Besides, I'm using the GetSupportedTraceType method introduced in D90490 to quickly infer what's the trace plug-in that works for the current process.
As an implementation note, I moved CommandObjectIterateOverThreads to its header so that I can use it from the IntelPT plugin. Besides, the actual start and stop logic for intel-pt is not part of this diff.
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D90729
Depends on D89408.
This diff finally implements trace decoding!
The current interface is
$ trace load /path/to/trace/session/file.json
$ thread trace dump instructions
thread #1: tid = 3842849, total instructions = 22
[ 0] 0x40052d
[ 1] 0x40052d
...
[19] 0x400521
$ # simply enter, which is a repeat command
[20] 0x40052d
[21] 0x400529
...
This doesn't do any disassembly, which will be done in the next diff.
Changes:
- Added an IntelPTDecoder class, that is a wrapper for libipt, which is the actual library that performs the decoding.
- Added TraceThreadDecoder class that decodes traces and memoizes the result to avoid repeating the decoding step.
- Added a DecodedThread class, which represents the output from decoding and that for the time being only stores the list of reconstructed instructions. Later it'll contain the function call hierarchy, which will enable reconstructing backtraces.
- Added basic APIs for accessing the trace in Trace.h:
- GetInstructionCount, which counts the number of instructions traced for a given thread
- IsTraceFailed, which returns an Error if decoding a thread failed
- ForEachInstruction, which iterates on the instructions traced for a given thread, concealing the internal storage of threads, as plug-ins can decide to generate the instructions on the fly or to store them all in a vector, like I do.
- DumpTraceInstructions was updated to print the instructions or show an error message if decoding was impossible.
- Tests included
Differential Revision: https://reviews.llvm.org/D89283
Depends on D88841
As per the discussion in the RFC, we'll implement both
thread trace dump [instructions | functions]
This is the first step in implementing the "instructions" dumping command.
It includes:
- A minimal ProcessTrace plugin for representing processes from a trace file. I noticed that it was a required step to mimic how core-based processes are initialized, e.g. ProcessElfCore and ProcessMinidump. I haven't had the need to create ThreadTrace yet, though. So far HistoryThread seems good enough.
- The command handling itself in CommandObjectThread, which outputs a placeholder text instead of the actual instructions. I'll do that part in the next diff.
- Tests
{F13132325}
Differential Revision: https://reviews.llvm.org/D88769
With the feedback I was getting in different diffs, I realized that splitting the parsing logic into two classes was not easy to deal with. I do see value in doing that, but I'd rather leave that as a refactor after most of the intel-pt logic is in place. Thus, I'm merging the common parser into the intel pt one, having thus only one that is fully aware of Intel PT during parsing and object creation.
Besides, based on the feedback in https://reviews.llvm.org/D88769, I'm creating a ThreadIntelPT class that will be able to orchestrate decoding of its own trace and can handle the stop events correctly.
This leaves the TraceIntelPT class as an initialization class that glues together different components. Right now it can initialize a trace session from a json file, and in the future will be able to initialize a trace session from a live process.
Besides, I'm renaming SettingsParser to SessionParser, which I think is a better name, as the json object represents a trace session of possibly many processes.
With the current set of targets, we have the following
- Trace: main interface for dealing with trace sessions
- TraceIntelPT: plugin Trace for dealing with intel pt sessions
- TraceIntelPTSessionParser: a parser of a json trace session file that can create a corresponding TraceIntelPT instance along with Targets, ProcessTraces (to be created in https://reviews.llvm.org/D88769), and ThreadIntelPT threads.
- ProcessTrace: (to be created in https://reviews.llvm.org/D88769) can handle the correct state of the traces as the user traverses the trace. I don't think there'll be a need an intel-pt specific implementation of this class.
- ThreadIntelPT: a thread implementation that can handle the decoding of its own trace file, along with keeping track of the current position the user is looking at when doing reverse debugging.
Differential Revision: https://reviews.llvm.org/D88841
Recently https://reviews.llvm.org/D88103 introduced a nice API for
converting a JSON object into C++ types, which include nice error
messaging.
I'm using that new functioniality to perform the parsing in a much more
elegant way. As a result, the code looks simpler and more maintainable,
as we aren't parsing anymore individual fields manually.
I updated the test cases accordingly.
Differential Revision: https://reviews.llvm.org/D88264
This is the first in a series of patches that will adds a new processor trace plug-in to LLDB.
The idea for this first patch to to add the plug-in interface with simple commands for the trace files that can "load" and "dump" the trace information. We can test the functionality and ensure people are happy with the way things are done and how things are organized before moving on to adding more functionality.
Processor trace information can be view in a few different ways:
- post mortem where a trace is saved off that can be viewed later in the debugger
- gathered while a process is running and allow the user to step back in time (with no variables, memory or registers) to see how each thread arrived at where it is currently stopped.
This patch attempts to start with the first solution of loading a trace file after the fact. The idea is that we will use a JSON file to load the trace information. JSON allows us to specify information about the trace like:
- plug-in name in LLDB
- path to trace file
- shared library load information so we can re-create a target and symbolicate the information in the trace
- any other info that the trace plug-in will need to be able to successfully parse the trace information
- cpu type
- version info
- ???
A new "trace" command was added at the top level of the LLDB commmands:
- "trace load"
- "trace dump"
I did this because if we load trace information we don't need to have a process and we might end up creating a new target for the trace information that will become active. If anyone has any input on where this would be better suited, please let me know. Walter Erquinigo will end up filling in the Intel PT specific plug-in so that it works and is tested once we can agree that the direction of this patch is the correct one, so please feel free to chime in with ideas on comments!
Reviewed By: clayborg
Differential Revision: https://reviews.llvm.org/D85705
