Currently, source files are cached by their resolved path. This means
that before we can query the cache, we potentially have to resolve the
path, which can be slow. This patch avoids the call to FileSystem::Resolve
by caching both the resolved and unresolved path. We now only resolve
the path once when we create and cache a new file.
rdar://110787562
Differential revision: https://reviews.llvm.org/D153726
lldb-server for Android does not build with NDK r21 and above due to
RetryAfterSignal and Bionic ::open mismatch.
https://github.com/llvm/llvm-project/issues/54727
Apply the LLVM patch to LLDB.
0a0e411204
> In Bionic, open can be overloaded for _FORTIFY_SOURCE support, causing
> compile errors of RetryAfterSignal due to overload resolution. Wrapping
> the call in a lambda avoids this.
Differential Revision: https://reviews.llvm.org/D152712
Currently, all data buffers are assumed to be writable. This is a
problem on macOS where it's not allowed to load unsigned binaries in
memory as writable. To be more precise, MAP_RESILIENT_CODESIGN and
MAP_RESILIENT_MEDIA need to be set for mapped (unsigned) binaries on our
platform.
Binaries are mapped through FileSystem::CreateDataBuffer which returns a
DataBufferLLVM. The latter is backed by a llvm::WritableMemoryBuffer
because every DataBuffer in LLDB is considered to be writable. In order
to use a read-only llvm::MemoryBuffer I had to split our abstraction
around it.
This patch distinguishes between a DataBuffer (read-only) and
WritableDataBuffer (read-write) and updates LLDB to use the appropriate
one.
rdar://74890607
Differential revision: https://reviews.llvm.org/D122856
This patch removes the ability to instantiate the LLDB FileSystem class
with a FileCollector. It keeps the ability to collect files, but uses
the FileCollectorFileSystem to do that transparently.
Because the two are intertwined, this patch also removes the
finalization logic which copied the files over out of process.
This patch removes the ability to instantiate the LLDB FileSystem class
based on a VFS overlay. This also removes the "hack" where we cast the
VFS to a RedirectingFileSystem to obtain the external path. You can
still instantiate a FileSystem with a VFS, but with the caveat that
operations that rely on the external path won't work.
Differential revision: https://reviews.llvm.org/D120923
This is an updated version of the https://reviews.llvm.org/D113789 patch with the following changes:
- We no longer modify modification times of the cache files
- Use LLVM caching and cache pruning instead of making a new cache mechanism (See DataFileCache.h/.cpp)
- Add signature to start of each file since we are not using modification times so we can tell when caches are stale and remove and re-create the cache file as files are changed
- Add settings to control the cache size, disk percentage and expiration in days to keep cache size under control
This patch enables symbol tables to be cached in the LLDB index cache directory. All cache files are in a single directory and the files use unique names to ensure that files from the same path will re-use the same file as files get modified. This means as files change, their cache files will be deleted and updated. The modification time of each of the cache files is not modified so that access based pruning of the cache can be implemented.
The symbol table cache files start with a signature that uniquely identifies a file on disk and contains one or more of the following items:
- object file UUID if available
- object file mod time if available
- object name for BSD archive .o files that are in .a files if available
If none of these signature items are available, then the file will not be cached. This keeps temporary object files from expressions from being cached.
When the cache files are loaded on subsequent debug sessions, the signature is compare and if the file has been modified (uuid changes, mod time changes, or object file mod time changes) then the cache file is deleted and re-created.
Module caching must be enabled by the user before this can be used:
symbols.enable-lldb-index-cache (boolean) = false
(lldb) settings set symbols.enable-lldb-index-cache true
There is also a setting that allows the user to specify a module cache directory that defaults to a directory that defaults to being next to the symbols.clang-modules-cache-path directory in a temp directory:
(lldb) settings show symbols.lldb-index-cache-path
/var/folders/9p/472sr0c55l9b20x2zg36b91h0000gn/C/lldb/IndexCache
If this setting is enabled, the finalized symbol tables will be serialized and saved to disc so they can be quickly loaded next time you debug.
Each module can cache one or more files in the index cache directory. The cache file names must be unique to a file on disk and its architecture and object name for .o files in BSD archives. This allows universal mach-o files to support caching multuple architectures in the same module cache directory. Making the file based on the this info allows this cache file to be deleted and replaced when the file gets updated on disk. This keeps the cache from growing over time during the compile/edit/debug cycle and prevents out of space issues.
If the cache is enabled, the symbol table will be loaded from the cache the next time you debug if the module has not changed.
The cache also has settings to control the size of the cache on disk. Each time LLDB starts up with the index cache enable, the cache will be pruned to ensure it stays within the user defined settings:
(lldb) settings set symbols.lldb-index-cache-expiration-days <days>
A value of zero will disable cache files from expiring when the cache is pruned. The default value is 7 currently.
(lldb) settings set symbols.lldb-index-cache-max-byte-size <size>
A value of zero will disable pruning based on a total byte size. The default value is zero currently.
(lldb) settings set symbols.lldb-index-cache-max-percent <percentage-of-disk-space>
A value of 100 will allow the disc to be filled to the max, a value of zero will disable percentage pruning. The default value is zero.
Reviewed By: labath, wallace
Differential Revision: https://reviews.llvm.org/D115324
This converts a default constructor's member initializers into C++11
default member initializers. This patch was automatically generated with
clang-tidy and the modernize-use-default-member-init check.
$ run-clang-tidy.py -header-filter='lldb' -checks='-*,modernize-use-default-member-init' -fix
This is a mass-refactoring patch and this commit will be added to
.git-blame-ignore-revs.
Differential revision: https://reviews.llvm.org/D103483
The C headers are deprecated so as requested in D102845, this is replacing them
all with their (not deprecated) C++ equivalent.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D103084
For performance reasons the reproducers don't copy the files captured by
the file collector eagerly, but wait until the reproducer needs to be
generated.
This is a problematic when LLDB crashes and we have to do all this
signal-unsafe work in the signal handler. This patch uses a similar
trick to clang, which has the driver invoke a new cc1 instance to do all
this work out-of-process.
This patch moves the writing of the mapping file as well as copying over
the reproducers into a separate process spawned when lldb crashes.
Differential revision: https://reviews.llvm.org/D89600
When replaying the reproducer, lldb should source the .lldbinit file
that was captured by the reproducer and not the one in the current home
directory. This requires that we store the home directory as part of the
reproducer. By returning the virtual home directory during replay, we
ensure the correct virtual path gets constructed which the VFS can then
find and remap to the correct file in the reproducer root.
This patch adds a new HomeDirectoryProvider, similar to the existing
WorkingDirectoryProvider. As the home directory is not part of the VFS,
it is stored in LLDB's FileSystem instance.
Provider a wrapper around llvm::sys::path::home_directory in the
FileSystem class. This will make it possible for the reproducers to
intercept the call in a central place.
Files imported by the script interpreter aren't opened by LLDB so they
don't end up in the reproducer. The solution is to explicitly add them
to the FileCollector.
Differential revision: https://reviews.llvm.org/D76626
LLDB has a few different styles of header guards and they're not very
consistent because things get moved around or copy/pasted. This patch
unifies the header guards across LLDB and converts everything to match
LLVM's style.
Differential revision: https://reviews.llvm.org/D74743
The VFS mapping writer assumes that all the paths it gets are files.
When passed a directory, it ends up as a file in the VFS mapping twice,
once as a file and once as a directory.
{
'type': 'file',
'name': "Output",
'external-contents': "/root/path/to/Output"
},
{
'type': 'directory',
'name': "Output",
'contents': [ ... ]
}
Summary:
This patch re-types everywhere that passes a File::OpenOptions
as a uint32_t so it actually uses File::OpenOptions.
It also converts some OpenOptions related functions that fail
by returning 0 or NULL into llvm::Expected
split off from https://reviews.llvm.org/D68737
Reviewers: JDevlieghere, jasonmolenda, labath
Reviewed By: labath
Subscribers: lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D68853
llvm-svn: 374817
Summary:
This patch converts FileSystem::Open from this prototype:
Status
Open(File &File, const FileSpec &file_spec, ...);
to this one:
llvm::Expected<std::unique_ptr<File>>
Open(const FileSpec &file_spec, ...);
This is beneficial on its own, as llvm::Expected is a more modern
and recommended error type than Status. It is also a necessary step
towards https://reviews.llvm.org/D67891, and further developments
for lldb_private::File.
Reviewers: JDevlieghere, jasonmolenda, labath
Reviewed By: labath
Subscribers: mgorny, lldb-commits
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D67996
llvm-svn: 373003
Instead of passing the FileCollector around as a reference or raw
pointer, use a shared_ptr. This change's motivation is twofold. First it
adds compatibility for the newly added `FileCollectorFileSystem`.
Secondly, it addresses a lifetime issue we only see when LLDB is used
from Xcode, where a reference to the FileCollector outlives the
reproducer instance.
llvm-svn: 367258
The FileCollector got lifted into LLVM and a shim was introduced in LLDB
to keep the old API that takes FileSpecs. This patch removes that shim
and converts the arguments in place.
llvm-svn: 366975
In r353906 we hooked up clang and lldb's reproducer infrastructure to
capture files used by clang. This patch adds the necessary logic to have
clang reuse the files from lldb's reproducer during replay.
Differential revision: https://reviews.llvm.org/D58309
llvm-svn: 354283
Summary:
This commit contains the following changes:
- Rewrite vfile close/read/write packet handlers with portable routines from lldb.
This removes #if(s) and allows the handlers to work on Windows.
- Fix a bug in File::Write. This is intended to write data at an offset to a file
but actually writes at the current position of the file.
- Add a default boolean argument 'should_close_fd' to FileSystem::Open to
let the user decide whether to close the fd or not.
Reviewers: zturner, llvm-commits, labath
Reviewed By: zturner
Subscribers: Hui, labath, abidh, lldb-commits
Differential Revision: https://reviews.llvm.org/D56231
llvm-svn: 353446
This patch adds the file provider which is responsible for capturing
files used by LLDB.
When capturing a reproducer, we use a file collector that is very
similar to the one used in clang. For every file that we touch, we add
an entry with a mapping from its virtual to its real path. When we
decide to generate a reproducer we copy over the files and their
permission into to reproducer folder.
When replaying a reproducer, we load the VFS mapping and instantiate a
RedirectingFileSystem. The latter will transparently use the files
available in the reproducer.
I've tested this on two macOS machines with an artificial example.
Still, it is very likely that I missed some places where we (still) use
native file system calls. I'm hoping to flesh those out while testing
with more advanced examples. However, I will fix those things in
separate patches.
Differential revision: https://reviews.llvm.org/D54617
llvm-svn: 352538
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
When trying to fix the bots we expected that the cast would be needed in
different places. Ultimately it turned out only the
SymbolFileDWARFDebugMap was affected so, as Pavel correctly notes, it
makes more sense to do the cast just there instead of in teh FS.
llvm-svn: 347660
After a recent change in LLVM the TimePoint encoding become more
precise, exceeding the precision of the TimePoint obtained from the
DebugMap. This patch adds a flag to the GetModificationTime helper in
the FileSystem to return the modification time with less precision.
Thanks to Davide for bisecting this failure on the LLDB bots.
llvm-svn: 347615
This moves construction of data buffers into the FileSystem class. Like
some of the previous refactorings we don't translate the path yet
because the functionality hasn't been landed in LLVM yet.
Differential revision: https://reviews.llvm.org/D54272
llvm-svn: 346598
The whole point of this change was making it possible to resolve paths
without depending on the FileSystem, which is not what I did here. Not
sure what I was thinking...
llvm-svn: 346466
In order to call real_path from the TildeExpressionResolver we need
access to the FileSystem. Since the resolver lives under utility we have
to pass in the FS.
llvm-svn: 346457
Replace calls to LLVM's is_directory with calls to LLDB's FileSytem
class. For this I introduced a new convenience method that, like the
other methods, takes either a path or filespec. This still uses the LLVM
functions under the hood.
Differential revision: https://reviews.llvm.org/D54135
llvm-svn: 346375
This patch modifies how we open File instances in LLDB. Rather than
passing a path or FileSpec to the constructor, we now go through the
virtual file system. This is needed in order to make things work with
the VFS in the future.
Differential revision: https://reviews.llvm.org/D54020
llvm-svn: 346049
This patch removes the static accessor in File to get a file's
permissions. Permissions should be checked through the FileSystem class.
llvm-svn: 345901
This patch removes the ResolveExecutableLocation method from FileSpec
and updates its uses with calls to the FileSystem.
Differential revision: https://reviews.llvm.org/D53834
llvm-svn: 345853
The new implementation of EnumerateDirectory relies on `::no_push()`
being implemented for the VFS recursive directory iterators. However
this patch (D53465) hasn't been landed yet.
llvm-svn: 345787
This patch extends the FileSystem class with a bunch of functions that
are currently implemented as methods of the FileSpec class. These
methods will be removed in future commits and replaced by calls to the
file system.
The new functions are operated in terms of the virtual file system which
was recently moved from clang into LLVM so it could be reused in lldb.
Because the VFS is stateful, we turned the FileSystem class into a
singleton.
Differential revision: https://reviews.llvm.org/D53532
llvm-svn: 345783
This renames the LLDB error class to Status, as discussed
on the lldb-dev mailing list.
A change of this magnitude cannot easily be done without
find and replace, but that has potential to catch unwanted
occurrences of common strings such as "Error". Every effort
was made to find all the obvious things such as the word "Error"
appearing in a string, etc, but it's possible there are still
some lingering occurences left around. Hopefully nothing too
serious.
llvm-svn: 302872
