Depends on:
* https://github.com/llvm/llvm-project/pull/187229
(only second commit and onwards are relevant)
This patch implements the base infrastructure described in this [RFC re.
Moving libc++ data-formatters out of
LLDB](https://discourse.llvm.org/t/rfc-lldb-moving-libc-data-formatters-out-of-lldb/89591)
The intention is to provide vendors with a way to pre-configure a set of
paths that LLDB can automatically ingest formatter scripts from.
Three main changes:
1. Adds a CMake variable `LLDB_SAFE_AUTO_LOAD_PATHS` which is a
semi-colon separated list of paths. This is intended to be set by
vendors when building LLDB for distribution.
2. Adds a setting that only exists in asserts mode called
`testing.safe-auto-load-paths` which allows setting the paths without a
CMake configuration. Used for local development and, more crucially, the
shell and unit tests
3. Adds a `LocateScriptingResourcesFromSafePaths` which
`Platform::LocateExecutableScriptingResources` calls by default (and
hence used by all platforms). I add a
`LocateExecutableScriptingResourcesImpl` that platforms can override if
they have platform-specific resource script locations (e.g., dSYMs on
Darwin).
Whenever we load an image, we check the safe path (starting with the
last appended path) for a directory called
`/safe/path/to/<module>/<module>.py`. If such script exists, we import
it as a Python module. If not, we move on to the next safe path.
Currently the default for `LLDB_SAFE_AUTO_LOAD_PATHS` is empty.
Eventually the plan is to make those point to the libc++ installation
(where the formatters will live) depending on platform/vendor. For macOS
we'll add a special `$SDK_ROOT` that can be used in the path variable,
which `LocateScriptingResourcesFromSafePaths` will resolve to an actual
SDK path.
*AI Usage*:
* Claude assisted with the CMake machinery which I then reviewed and
cleaned up. I'm not sure this is the most idiomatic way of letting a
user provide lists of paths, but I couldn't find a better way.
* Wrote the basic auto-load Shell tests myself and asked claude to stamp
out a bunch more for different scenarios. Reviewed and cleaned those up
myself.
The `num_temps` was introduced under the mistaken understanding of the
index values given to the `pick` op. I thought the `pick` index was from
the top of the stack, but it's from the bottom. The model of indexing
from the bottom of the stack has the benefit of simplifying the
compiler.
This patch makes the warning message more concise (in my opinion). We
would duplicate the file path multiple times in the message. I'm
planning on factoring this logic into a standalone function, and having
it rely on fewer parameters helps with that.
Before:
```
warning: the symbol file '/path/to/.dSYM/Contents/Resources/DWARF/import' contains a debug script.
However, its name '/path/to/.dSYM/Contents/Resources/DWARF/../Python/import.py' conflicts with a keyword
and as such cannot be loaded. LLDB will load '/path/to/.dSYM/Contents/Resources/DWARF/../Python/_import.py' instead.
Consider removing the file with the malformed name to eliminate this warning.
```
After:
```
warning: debug script '/path/to/.dSYM/Contents/Resources/DWARF/../Python/import.py' cannot be loaded
because 'import.py' conflicts with the keyword 'import'. Ignoring 'import.py' and loading '_import.py' instead.
```
Before:
```
warning: the symbol file '/path/to/.dSYM/Contents/Resources/DWARF/import' contains a debug script.
However, its name conflicts with a keyword and as such cannot be loaded.
If you intend to have this script loaded, please rename '/path/to/.dSYM/Contents/Resources/DWARF/../Python/import.py'
to '/path/to/.dSYM/Contents/Resources/DWARF/../Python/_import.py' and retry.
```
After:
```
warning: debug script '/path/to/.dSYM/Contents/Resources/DWARF/../Python/import.py' cannot be loaded
because 'import.py' conflicts with the keyword 'import'. If you intend to have this script loaded, please rename
it to '_import.py' and retry.
```
Minimal infrastructure for a the SymbolLocator plugin that fetches debug
info from Microsoft SymStore repositories. This can work cross-platform
and for various debug info formats in principle, but the current plan is
focussed on PE/COFF on Windows with debug info in PDB files. Once we
have a stable first version, we'd like to add features like download,
environment variables, caching and progress feedback for users.
SymbolVendorPECOFF was tailored towards DWARF debug info so far. I added
code to load the PDB path from the executable (it only checked
gnu_debuglink so far) and not bail out if DWARF sections are missing, so
that in the PDB case we still call AddSymbolFileRepresentation() in the
very end of CreateInstance().
The API test in this patch mocks the directory layout from SymStore, so
it doesn't depend on SymStore.exe from the Windows SDK. It runs on all
platforms that link debug info in a PDB file, which is still just
Windows, but it could be cross-platform in principle.
-----
Relands with minor fixes: API tests create mocked SymStore in the test's
build directory. One log instruction was moved. One more object
access goes through module in SymbolFile.
I'm in the process of making `LoadScriptingResources` interactively ask
a user whether to load a script. I'd like to turn the existing warning
into the prompt. The simplest way to achieve this is to not print into a
`feedback_stream` parameter, and instead create a prompt right there.
This patch removes the `feedback_stream` parameter and emits a
`ReportWarning` instead. If we get around to adding the prompt instead
of the warning, those changes will be simpler to review. But even if we
don't end up replacing the warning with a prompt, moving away from
output parameters and towards more structured error reporting is a
nice-to-have (e.g., the `warning` prefix is now colored, IDEs have more
flexibility on how to present the warning, etc.).
For a command-line user nothing should change with this patch (apart
from `warning:` being highlighted).
#185735 added the `vbases.test`, which compiles with
`--target=x86_64-windows-msvc`. This will cause the final executable to
be linked to `libcmt.lib`. That doesn't work on ARM, so this PR changes
the command line to link without the default libraries. They're not
needed if we disable `/GS` (buffer security check) like in other tests.
We use `%clang_cl` over `%build` to be able to compile with DWARF as
well.
In #183172, I added support for reading `S_REGREL32_INDIR` records. This
adds support in LLDB. The record is emitted by MSVC for structured
bindings and in the compiler generated coroutine stubs.
It describes a location at `*(Register + Offset) + OffsetInUdt`.
Equivalent to
```
DW_OP_breg{reg} {Offset}
DW_OP_deref
DW_OP_plus_uconst {OffsetInUdt}
```
LLVM doesn't create this record - it only creates `S_LOCAL`s for local
variables. We'll probably need `S_DEFRANGE_REGISTER_REL_INDIR` for this
- should be simple to figure out the structure, but I haven't seen that
record emitted yet.
After #185537, `lldb-server` would crash with an assertion because it
calls`exit()` instead of returning from `main`. This means that we never
get an opportunity to call `Terminate` before the static
`SystemLifetimeManager` is destroyed.
I could've created a static RAII object to wrap this in (similar to what
Aiden did in #186133) but that's not necessary. I updated the helper
functions to return an `llvm::Error` while `main_gdbserver` and
`main_platform` now return an `int`.
Fixes#186207
Makes it obvious what the warning will look like (with the indenentation
etc.). Also adds a test since we had no coverage for the warning before
(as far as I'm aware).
This PR adds support for running the LLDB test suite under MTE. It's
enabled by default on capable hardware when asserts are enabled. It
relies on a launcher (#185921) which launches the process with the
appropriate posix_spawn attribute. One thing worth noting here is that
child processes inherit the MTE property, so binaries launched by the
test suite in this mode also run under MTE.
Besides the logic to detect the default and thread through the launcher,
I also had to make a small change to LLVM LIT's `ToolSubst` class to
support an optional launcher for the shell tests.
The `testing.XXX` settings (added in
https://github.com/llvm/llvm-project/pull/177279 and currently just
`testing.inject-variable-location-error`) are supposed to only exist in
asserts builds. However, we never added it as a global property to the
`Debugger`, so the setting wasn't actually usable, in any build.
The one test that did use it [skipped the test on
error](230e465617/lldb/test/API/commands/expression/diagnostics/TestExprDiagnostics.py (L286-L288))
as a way to mimick "only run test in asserts mode". However, this just
meant the test never ran.
This patch registers the property and adds a test that ensures an
asserts-LLDB does allow access to it from the CLI.
When a class indirectly inherits from a class with virtual bases, it
will get an `LF_IVBCLASS` record in its fieldlist, even though it
doesn't directly inherit that class.
In the following example, `UserUser` inherits from `User`, which
virtually inherits from `VBase`:
```cpp
struct User : public virtual VBase {};
struct UserUser : public User {};
```
For this we get
```
0x1015 | LF_FIELDLIST [size = 72]
- LF_BCLASS
type = 0x1002 (-> 0x102A), offset = 0, attrs = public
- LF_IVBCLASS
base = 0x1003, vbptr = 0x1005, vbptr offset = 0, vtable index = 1
attrs = public (...)
0x1016 | LF_STRUCTURE [size = 48] `UserUser`
unique name: `.?AUUserUser@@`
vtable: <no type>, base list: <no type>, field list: 0x1015
options: has ctor / dtor | has unique name | overloaded operator | overloaded operator=, sizeof 16
0x1029 | LF_FIELDLIST [size = 56]
- LF_VBCLASS
base = 0x1003, vbptr = 0x1005, vbptr offset = 0, vtable index = 1
attrs = public (...)
0x102A | LF_STRUCTURE [size = 40] `User`
unique name: `.?AUUser@@`
vtable: <no type>, base list: <no type>, field list: 0x1029
options: has ctor / dtor | has unique name | overloaded operator | overloaded operator=, sizeof 16
```
If I understand correctly, then `LF_IVBCLASS` indicates that if this
class (e.g. `UserUser`) is created as the most derived object, it will
host the class (e.g. `VBase`).
The VS debugger actually shows this as a separate field. LLDB on the
other hand doesn't, so I removed it.
---------
Co-authored-by: Zequan Wu <zequanwu@google.com>
Updates formatter_bytecode.py to support compilation and disassembly for
synthetic formatters, in other words support for multiple functions
(signatures).
This includes a number of other changes:
* String parsing and encoding have bugs fixed
* CLI args are updated, primarily to support an output file
* Added uleb encoding/decoding support
This work is a prelude the ongoing work of a Python to formatter
bytecode compiler. The python compiler to emit assembly, and this module
(formatter_bytecode) will compile it into binary bytecode.
This patch removes all our manual adjustments to the access control
specifiers of Clang decls we create from DWARF.
This has led to occasional subtle bugs in the past (the latest being
https://github.com/llvm/llvm-project/issues/171913) and it's ultimately
redundant because Clang already has provisions for LLDB to bypass access
control for C++ and Objective-C. Access control doesn't affect name
lookup so really we're doing a lot of bookkeeping for not much benefit.
The only "feature" that relied on this was that `type lookup <foo>`
would print the access specifier in the output structure layout. I'm not
convinced that's worth keeping the infrastructure in place for (but
happy to be convinced otherwise).
I'd rather lean fully into the Clang access control bypass instead.
Note, i still kept the `AccessType` parameters to the various
`TypeSystemClang` APIs to reduce the size of the diff. A follow-up NFC
change will remove those parameters and adjust all the call-sites.
Fails on Darwin after we made `-gsimple-template-names` the default (in https://github.com/llvm/llvm-project/pull/182297):
```
13:42:19 | # CHECK: DWARFASTParserClang::ParseTypeFromDWARF{{.*}}DW_TAG_structure_type (DW_TAG_structure_type) name = 't2<t1>'
13:42:19 | ^
13:42:19 | <stdin>:9:12: note: scanning from here
13:42:19 | (lldb) p v1
13:42:19 | ^
13:42:19 | <stdin>:10:278: note: possible intended match here
13:42:19 | (arm64) /Users/ec2-user/jenkins/workspace/llvm.org/as-lldb-cmake-os-verficiation/lldb-build/tools/lldb/test/Shell/SymbolFile/DWARF/Output/delayed-definition-die-searching.test.tmp.out: DWARFASTParserClang::ParseTypeFromDWARF (die = 0x0000000000000037, decl_ctx = 0x0000000B723D2030 (die 0x000000000000000c)) DW_TAG_structure_type (DW_TAG_structure_type) name = 't2')
13:42:19 |
```
This just checks the delayed definition search. It used to always run without `-gsimple-template-names`, so we're not losing coverage here. Also the failure is expected with `-gsimple-template-names` because the DIE name no longer include template parameters. I didn't want to make the `CHECK` less strict because it useful to check that the types being resolved are the correct instantiations.
Before:
```
(lldb) type synthetic add -l blah foo
warning: The provided class does not exist - please define it before attempting to use this synthetic provider
```
After:
```
(lldb) type synthetic add -l blah foo
warning: The provided class 'blah' does not exist - please define it before attempting to use this synthetic provider
```
Useful when many of these registration commands happen as part of
`~/.lldbinit`. Previously it wasn't immediately obvious which of those
commands failed.
This patch fixes where ELF cores will report all threads as `STOP REASON
0`.
This was/is a large personal annoyance of mine; added a test to verify a
default elf core process/thread has no valid stop reason.
Add handling for `STT_TLS` (thread-local storage) symbols in the ELF
symbol parsing code. Previously, TLS symbols like `errno` from glibc
were not recognized because `STT_TLS` was not handled in the symbol type
switch statement.
This treats TLS symbols as data symbols (`eSymbolTypeData`), similar to
`STT_OBJECT`. The actual TLS address resolution is already implemented
in `DynamicLoaderPOSIXDYLD::GetThreadLocalData()` which uses the DWARF
`DW_OP_form_tls_address` opcode to calculate thread-local addresses.
LLDB's DWARF parser didn't support parsing DW_FORM_GNU_ref_alt and
DW_FORM_GNU_strp_alt forms which would cause any file loaded by LLDB to
fail to parse any DWARF. Added support for parsing this information
only, not for actually finding the debug info reference to an alternate
file or a string in an alternate file. These extensions are used by DWZ
files which are present in some linux distros, so it will be good for
LLDB to just be able to parse these without emitting an error like:
(lldb) b bar
warning: (arm64) /tmp/a.out unsupported DW_FORM values: 0x1f20 0x1f21
Python's internal stderr may differ from sys.stderr.
When Python writes errors, it uses its internal stderr rather than the
overwritten sys.stderr.
This may not be the same file/handle
Fix the test to explicitly write to the specified stderr.
Due to some faulty logic, if you ran "memory region --all" twice, the
second time lldb would try to repeat the command. There's nothing to
repeat, so it failed with an error. It should treat each "--all" use as
starting from scratch.
The logic here was written in a confusing way, so I've refactored it to
first look at how many arguments there are (aka how many address
expressions there are) and then validate based on that.
For reasons unknown, I could not reproduce this issue using the API test
TestMemoryRegion.py. So I have added a shell test that I confirmed does
fail without this fix.
Currently we forcefully complete C++ types if we can't find their
definition for layout purposes. This ensures that we at least don't
crash in Clang when laying out the type. The definition is required for
types of members/array elements/base classes for the purposes of
calculating their layout. This is also true for Obj-C types, but we
haven't been forcefully completing those.
The test-case that's being un-XFAILed in this patch demonstrates a case
where not completing the super-class forcefully causes a clang crash.
rdar://168440264
LLDB currently crashes when the super-class of an Objective-C type can't
be completed (i.e., has no definition). For Foundation types such as
`NSObject`, the debug-info would usually only contain forward
declarations. The definitions live in the Clang module `.pcm` files. But
if the source of the definition fails to be loaded (e.g., if we just
delete the module cache), then we can no longer guarantee that the
super-class has a definition. This breaks a key Clang invariant, which
requires base-classes to have definitions by the time we try to lay them
out. This patch adds an XFAILed test for such scenario.
rdar://168440264
Corrected various spelling mistakes such as 'occurred', 'receiver',
'initialized', 'length', and others in comments, variable names,
function names, and documentation throughout the project. These
changes improve code readability and maintain consistency in naming
and documentation.
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
Implementation files using the Intel syntax typically explicitly specify it.
Do the same for the few files using AT&T syntax.
This enables building LLVM with `-mllvm -x86-asm-syntax=intel` in one's Clang config files
(i.e. a global preference for Intel syntax).
Unreverts: 14c69497b31038b37c273417f43bd2cfe169c86f
This is another instance where we weren't checking that the result of
FileSystem::CreateDataBuffer and unconditionally accessing it, similar
to the bug in SourceManager last week. In this particular case,
ObjectFile was assuming that we can read the contents non-zero, which
isn't true for directory nodes.
Jim figured this one out yesterday. I'm just putting up the patch and
adding a test.
rdar://167796036
There is no indication this ever worked on windows as this is the first
test that checks python interactive console from a file.
Looking at the error from the CI, It closed the interpreter before
running any python commands.
Will reconfirm this when I have access to a windows machine.
From https://github.com/llvm/llvm-project/pull/174216
