This changes `PdbAstBuilder` to a language-neutral abstract interface
and moves all of its functionality to the `PdbAstBuilderClang` derived
class.
All Clang-specific methods with external callers are now public methods
on `PdbAstBuilderClang`. `TypeSystemClang` and `UdtRecordCompleter` use
`PdbAstBuilderClang` directly.
Did my best to clean up includes and unused methods.
RFC for context:
https://discourse.llvm.org/t/rfc-lldb-make-pdbastbuilder-language-agnostic/89117
Currently, `SymbolFileNativePDB` calls several `PdbAstBuilder` methods
for side-effects to ensure the AST is populated.
This change adds new void-returning methods for `SymbolFileNativePDB` to
use as a hook instead, so that it doesn't depend on Clang-specific parts
of `PdbAstBuilder`'s interface.
This is part of the work to allow language-agnostic `PdbAstBuilder` (see
RFC:
https://discourse.llvm.org/t/rfc-lldb-make-pdbastbuilder-language-agnostic/89117)
This change implements part of
https://discourse.llvm.org/t/rfc-lldb-make-pdbastbuilder-language-agnostic/89117,
the goal of which is make `PdbAstBuilder` language-agnostic.
It adds several new interface methods to `PdbAstBuilder` which allow
`SymbolFileNativePDB` to interact with it without using Clang-specific
types. These new methods would make up much of the interface of a future
abstract `PdbAstBuilder`. `SymbolFileNativePDB` is updated to use these.
The old Clang-specific methods were renamed if used by
`UdtRecordCompleter` or internally, or removed where possible.
When a PDB is loaded through `target symbols add <pdb-path>`, its
`m_objectfile_sp` is an `ObjectFilePDB` instead of `ObjectFilePECOFF`
(the debugged module). In both the native and DIA plugin, some paths
assumed that `m_objectfile_sp` is the debugged module. With this PR,
they go through `m_objfile_sp->GetModule()->GetObjectFile()`.
For the DIA plugin, this lead to an assertion failure
(https://github.com/llvm/llvm-project/issues/169628#issuecomment-3582555277)
and for both plugins, it meant that the symbol table wasn't loaded.
In #165604, a test was skipped on Windows, because the native PDB plugin
didn't set sizes on symbols. While the test isn't compiled with debug
info, it's linked with `-gdwarf`, causing a PDB to be created on
Windows. This PDB will only contain the public symbols (written by the
linker) and section information. The symbols themselves don't have a
size, however the DIA SDK sets a size for them.
It seems like, for these data symbols, the size given from DIA is the
distance to the next symbol (or the section end).
This PR implements the naive approach for the native plugin. The main
difference is in function/code symbols. There, DIA searches for a
corresponding `S_GPROC32` which have a "code size" that is sometimes
slightly smaller than the difference to the next symbol.
When creating all types in a compilation unit, simple types (~>
primitive and pointer types) that were only used in function arguments
or return types weren't created as LLDB `Type`s.
With this PR, they're created when creating the function/method types.
This makes it possible to run the `SymbolFile/PDB/typedefs.test` with
both plugins.
This aligns the simple types created by the native plugin with the ones
from DIA as well as LLVM and the original cvdump.
- A few type names weren't handled when creating the LLDB `Type` name
(e.g. `short`)
- 64-bit integers were created as `(u)int64_t` and are now created as
`(unsigned) long long` (matches DIA)
- 128-bit integers (only supported by clang-cl) weren't created as types
(they have `SimpleTypeKind::(U)Int128Oct`)
- All complex types had the same name - now they have `_Complex
<float-type>`
Some types like `SimpleTypeKind::Float48` can't be tested because they
can't be created in C++.
Before this PR, the native PDB plugin would create the following LLDB
`Type` for `using SomeTypedef = long`:
```
Type{0x00002e03} , name = "SomeTypedef", size = 4, compiler_type = 0x000002becd8d8620 long
```
with this PR, the following is created:
```
Type{0x00002e03} , name = "SomeTypedef", size = 4, compiler_type = 0x0000024d6a7e3c90 typedef SomeTypedef
```
This matches the behavior of the DIA PDB plugin and works towards making
[`Shell/SymbolFile/PDB/typedefs.test`](https://github.com/llvm/llvm-project/blob/main/lldb/test/Shell/SymbolFile/PDB/typedefs.test)
pass with the native plugin.
I added a similar test to the `NativePDB` shell tests to capture the
current state, which doesn't quite match that of DIA yet. I'll add some
comments on what's missing on this PR, because I'm not fully sure what
the correct output would be.
Relands #149701 which was reverted in
185ae5cdc6
because it broke demangling of Itanium symbols on i386.
The last commit in this PR adds the fix for this (discussed in #160930).
On x86 environments, the prefix of `__cdecl` functions will now be
removed to match DWARF. I opened #161676 to discuss this for the other
calling conventions.
Before, functions created using the NativePDB plugin would not know
about their mangled name. This showed when printing a stacktrace. There,
only the function name was shown. For
https://github.com/llvm/llvm-project/issues/143149, the mangled function
name is required to separate different parts.
This PR adds that name if available.
The Clang AST nodes also take in a mangled name, which was previously
unset. I don't think this unblocks anything further, because Clang can
mangle the function anyway.
When creating LLDB types from `LF_MODIFIER` records, the type name of
the modified type was used. This didn't include the modifiers
(`const`/`volatile`/`__unaligned`). With this PR, they're included.
The DIA plugin had a test for this. That test also assumed that function
types had a name. I removed that check here, because function/procedure
types themselves in PDB don't have a name:
```
0x1015 | LF_ARGLIST [size = 20, hash = 0xBCB6]
0x0074 (int): `int`
0x1013: `int* __restrict`
0x1014: `int& __restrict`
0x1016 | LF_PROCEDURE [size = 16, hash = 0x3F611]
return type = 0x0003 (void), # args = 3, param list = 0x1015
calling conv = cdecl, options = None
```
I assume DIA gets the name from the function symbol itself. In the
native plugin, that name isn't included and multiple functions with the
same signature will reuse one type, whereas DIA would create a new type
for each function. The
[Shell/SymbolFile/PDB/func-symbols.test](b29c7ded31/lldb/test/Shell/SymbolFile/PDB/func-symbols.test)
also relies on this.
This patch causes the various AST dump commands (`target modules dump
ast`/`target dump typesystem`) to be color-highlighted. I added a `bool
show_color` parameter to `SymbolFile::DumpClangAST` and
`TypeSystem::Dump`. In `TypeSystemClang` I temporarily sets the
`getShowColors` flag on the owned Clang AST (using an RAII helper) for
the duration of the AST dump. We use `Debugger::GetUseColors` to decide
whether to color the AST dump.
After parsing blocks in a function, the blocks should be marked as
parsed for them to be dumped (see
[Function::Dump](e6aefbec78/lldb/source/Symbol/Function.cpp (L446-L447))).
As explained in
https://github.com/llvm/llvm-project/issues/114906#issuecomment-3255016266,
this happens (accidentally?) in the DIA plugin when parsing variables,
because it calls `function.GetBlock(can_create=true)` which marks blocks
as parsed. In the native plugin, this was never called, so blocks and
variables were never included in the `lldb-test symbols` output.
The `variables.test` for the DIA plugin tests this. One difference
between the plugins is how they specify the location of local variables.
This causes the output of the native plugin to be two lines per
variable, whereas the DIA plugin has one line:
```
(native):
000002C4B7593020: Variable{0x1c800001}, name = "var_arg1", type = {0000000000000744} 0x000002C4B6CA7900 (int), scope = parameter, location = 0x00000000:
[0x000000014000102c, 0x000000014000103e): DW_OP_breg7 RSP+8
```
```
(DIA):
000002778C827EE0: Variable{0x0000001b}, name = "var_arg1", type = {0000000000000005} 0x000002778C1FBAB0 (int), scope = parameter, decl = VariablesTest.cpp:32, location = DW_OP_breg7 RSP+8
```
In the test, I filtered lines starting with spaces followed by `[0x`, so
we can still use `CHECK-NEXT`.
---
Another difference between the plugins is that DIA marks the `this`
pointer as artificial (equivalent to DWARF). This is done if a
variable's object kind is `ObjectPtr`
([source](ab898f32c6/lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp (L1050))).
As far as I know, there isn't anything in the debug info that says "this
variable is the `this` pointer" other than the name/type of a variable
and the type of the function.
To find global variables, `SymbolFileNativePDB` used to search the
globals stream for the name passed to `FindGlobalVariables`. However,
the symbols in the globals stream contain the fully qualified name and
`FindGlobalVariables` only gets the basename. The approach here is
similar to the one for types and functions.
As we already search the globals stream for functions, we can cache the
basenames for global variables there as well.
This makes the `expressions.test` from the DIA PDB plugin pass with the
native one (#114906).
https://github.com/llvm/llvm-project/pull/153160 created those function
maps and uses default sort comparator which is not deterministic when
there are multiple entries with same name because llvm::sort is unstable
sort.
This fixes it by comparing the id value when tie happens and sort
`m_type_base_names` deterministically as well.
This patch fixes:
lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp:647:3:
error: 'llvm::Expected' may not intend to support class template
argument deduction [-Werror,-Wctad-maybe-unsupported]
lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp:677:3:
error: 'llvm::Expected' may not intend to support class template
argument deduction [-Werror,-Wctad-maybe-unsupported]
Tag types like stucts or enums didn't have a declaration attached to
them. The source locations are present in the IPI stream in
`LF_UDT_MOD_SRC_LINE` records:
```
0x101F | LF_UDT_MOD_SRC_LINE [size = 18, hash = 0x1C63]
udt = 0x1058, mod = 3, file = 1, line = 0
0x2789 | LF_UDT_MOD_SRC_LINE [size = 18, hash = 0x1E5A]
udt = 0x1253, mod = 35, file = 93, line = 17069
```
The file is an ID in the string table `/names`:
```
ID | String
1 | '\<unknown>'
12 | 'D:\a\_work\1\s\src\ExternalAPIs\WindowsSDKInc\c\Include\10.0.22621.0\um\wingdi.h'
93 | 'D:\a\_work\1\s\src\ExternalAPIs\WindowsSDKInc\c\Include\10.0.22621.0\um\winnt.h'
```
Here, we're not interested in `mod`. This would indicate which module
contributed the UDT.
I was looking at Rustc's PDB and found that it uses `<unknown>` for some
types, so I added a check for that.
This makes two DIA PDB shell tests to work with the native PDB plugin.
---------
Co-authored-by: Michael Buch <michaelbuch12@gmail.com>
Relands #152295.
Checking for the overloads did not account for them being out of order.
For example, [the failed
output](https://github.com/llvm/llvm-project/pull/152295#issuecomment-3177563247)
contained the overloads, but out of order. The last commit here fixes
that by using `-DAG`.
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
This adds the ability for functions to be matched by their basename.
Before, the globals were searched for the name. This works if the full
name is available but fails for basenames.
PDB only includes the full names of functions, so we need to cache all
basenames. This is (again) very similar to
[SymbolFilePDB](b242150b07/lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp (L1291-L1383)).
There are two main differences:
- We can't just access the parent of a function to determine that it's a
member function - we need to check the type of the function, and its
"this type".
- SymbolFilePDB saved the full method name in the map. However, when
searching for methods, only the basename is passed, so the function
never found any methods.
Fixes#51933.
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Languages other than C/C++ don't necessarily emit mangled names in the
`UniqueName` field of type records. Rust specifically emits a unique ID
that doesn't contain the name.
For example, `(i32, i32)` is emitted as
```llvm
!266 = !DICompositeType(
tag: DW_TAG_structure_type, name: "tuple$<i32,i32>", file: !9, size: 64, align: 32,
elements: !267, templateParams: !17, identifier: "19122721b0632fe96c0dd37477674472"
)
```
which results in
```
0x1091 | LF_STRUCTURE [size = 72, hash = 0x1AC67] `tuple$<i32,i32>`
unique name: `19122721b0632fe96c0dd37477674472`
vtable: <no type>, base list: <no type>, field list: 0x1090
options: has unique name, sizeof 8
```
In C++ with Clang and MSVC, a structure similar to this would result in
```
0x136F | LF_STRUCTURE [size = 44, hash = 0x30BE2] `MyTuple`
unique name: `.?AUMyTuple@@`
vtable: <no type>, base list: <no type>, field list: 0x136E
options: has unique name, sizeof 8
```
With this PR, if a `UniqueName` is encountered that couldn't be parsed,
it will fall back to using the undecorated (→ do the same as if the
unique name is empty/unavailable).
I'm not sure how to test this. Maybe compiling the LLVM IR that rustc
emits?
Fixes#152051.
Initially suggested in
https://github.com/llvm/llvm-project/pull/149305#issuecomment-3113413702
- this PR adds the setting `plugin.symbol-file.pdb.use-native-reader`.
It doesn't remove support for `LLDB_USE_NATIVE_PDB_READER` to allow some
backwards compatibility. This was the suggested way to use the native
reader - changing that would mean users who set this, now use the DIA
reader. The setting has priority over the environment variable, though.
If the default gets flipped on Windows, the environment variable could
probably be removed as well.
This would make it possible to test both native PDB and DIA PDB in the
API tests (see linked PR).
Previously, `type lookup` for types in namespaces didn't work with the
native PDB plugin, because `FindTypes` would only look for types whose
base name was equal to their full name. PDB/CodeView does not store the
base names in the TPI stream, but the types have their full name (e.g.
`std::thread` instead of `thread`). So `findRecordsByName` would only
return types in the top level namespace.
This PR changes the lookup to go through all types and check their base
name. As that could be a bit expensive, the names are first cached
(similar to the function lookup in the DIA PDB plugin). Potential types
are checked with `TypeQuery::ContextMatches`.
To be able to handle anonymous namespaces, I changed
`TypeQuery::ContextMatches`. The [`TypeQuery`
constructor](9ad7edef42/lldb/source/Symbol/Type.cpp (L76-L79))
inserts all name components as `CompilerContextKind::AnyDeclContext`. To
skip over anonymous namespaces, `ContextMatches` checked if a component
was empty and exactly of kind `Namespace`. For our query, the last check
was always false, so we never skipped anonymous namespaces. DWARF
doesn't have this problem, as it [constructs the context
outside](abe93d9d7e/lldb/source/Plugins/SymbolFile/DWARF/DWARFIndex.cpp (L154-L160))
and has proper information about namespaces. I'm not fully sure if my
change is correct and that it doesn't break other users of `TypeQuery`.
This enables `type lookup <type>` to work on types in namespaces.
However, expressions don't work with this yet, because `FindNamespace`
is unimplemented for native PDB.
Depends on https://github.com/llvm/llvm-project/pull/142163
This patch makes the `-ast-dump-filter` Clang option available to the
`target modules dump ast` command. This allows us to selectively dump
parts of the AST by name.
The AST can quickly grow way too large to skim on the console. This will
aid in debugging AST related issues.
Example:
```
(lldb) target modules dump ast --filter func
Dumping clang ast for 48 modules.
Dumping func:
FunctionDecl 0xc4b785008 <<invalid sloc>> <invalid sloc> func 'void (int)' extern
|-ParmVarDecl 0xc4b7853d8 <<invalid sloc>> <invalid sloc> x 'int'
`-AsmLabelAttr 0xc4b785358 <<invalid sloc>> Implicit "_Z4funcIiEvT_"
Dumping func<int>:
FunctionDecl 0xc4b7850b8 <<invalid sloc>> <invalid sloc> func<int> 'void (int)' implicit_instantiation extern
|-TemplateArgument type 'int'
| `-BuiltinType 0xc4b85b110 'int'
`-ParmVarDecl 0xc4b7853d8 <<invalid sloc>> <invalid sloc> x 'int'
```
The majority of this patch is adjust the `Dump` API. The main change in
behaviour is in `TypeSystemClang::Dump`, where we now use the
`ASTPrinter` for dumping the `TranslationUnitDecl`. This is where the
`-ast-dump-filter` functionality lives in Clang.
Add an overloaded `GetTypeSystem` to specify the expected type system subclass. Changes code from `GetTypeSystem().dyn_cast_or_null<TypeSystemClang>()` to `GetTypeSystem<TypeSystemClang>()`.
This patch pushes the error handling boundary for the GetBitSize()
methods from Runtime into the Type and CompilerType APIs. This makes it
easier to diagnose problems thanks to more meaningful error messages
being available. GetBitSize() is often the first thing LLDB asks about a
type, so this method is particularly important for a better user
experience.
rdar://145667239
This is the behavior expected by DWARF. It also requires some fixups to
algorithms which were storing the addresses of some objects (Blocks and
Variables) relative to the beginning of the function.
There are plenty of things that still don't work in this setups, but
this change is sufficient for the expression evaluator to correctly
recognize the entry point of a function in this case.
Many calls to Function::GetAddressRange() were not interested in the
range itself. Instead they wanted to find the address of the function
(its entry point) or the base address for relocation of function-scoped
entities (technically, the two don't need to be the same, but there's
isn't good reason for them not to be). This PR creates a separate
function for retrieving this, and changes the existing
(non-controversial) uses to call that instead.
This reverts commit 2526d5b1689389da9b194b5ec2878cfb2f4aca93, reapplying
ba14dac481564000339ba22ab867617590184f4c after fixing the conflict with
#117532. The change is that Function::GetAddressRanges now recomputes
the returned value instead of returning the member. This means it now
returns a value instead of a reference type.
This is a follow-up/reimplementation of #115730. While working on that
patch, I did not realize that the correct (discontinuous) set of ranges
is already stored in the block representing the whole function. The
catch -- ranges for this block are only set later, when parsing all of
the blocks of the function.
This patch changes that by populating the function block ranges eagerly
-- from within the Function constructor. This also necessitates a
corresponding change in all of the symbol files -- so that they stop
populating the ranges of that block. This allows us to avoid some
unnecessary work (not parsing the function DW_AT_ranges twice) and also
results in some simplification of the parsing code.
It's basically true already (except for a brief time during
construction). This patch makes sure the objects are constructed with a
valid parent and enforces this in the type system, which allows us to
get rid of some nullptr checks.
In case of an error GetBlock would return a reference to a Block without
adding it to a parent. This doesn't seem like a good idea, and none of
the other plugins do that.
This patch fixes that by propagating errors (well, null pointers...) up
the stack.
I don't know of any specific problem that this solves, but given that
this occurs only when something goes very wrong (e.g. a corrupted PDB
file), it's quite possible noone has run into this situation, so we
can't say the code is correct either. It also gets in the way of a
refactor I'm contemplating.
This is the beginning of a different, more fundamental approach to
handling. This PR tries to tries to minimize functional changes. It only
makes sure that we store the true set of ranges inside the function
object, so that subsequent patches can make use of it.
This doesn't parse S_CONSTANT case yet, because I found that the global
variable `std::strong_ordering::equal` is a S_CONSTANT and has type of
LF_STRUCTURE which is not currently handled when creating dwarf
expression for the variable. Left a TODO for it to finish later.
This makes `lldb/test/Shell/SymbolFile/PDB/ast-restore.test` and
`lldb/test/Shell/SymbolFile/PDB/calling-conventions-x86.test` pass on
windows with native pdb plugin only.
Updates:
- The previous patch changed the default behavior to not load dwos in
`DWARFUnit`
~~`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info =
false);`~~
`SymbolFileDWARFDwo *GetDwoSymbolFile(bool load_all_debug_info = true);`
- This broke some lldb-shell tests (see
https://green.lab.llvm.org/green/view/LLDB/job/as-lldb-cmake/16273/)
- TestDebugInfoSize.py
- with symbol on-demand, by default statistics dump only reports
skeleton debug info size
- `statistics dump -f` will load all dwos. debug info = skeleton debug
info + all dwo debug info
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.
This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
Currently running `statistics dump` will trigger lldb to load debug info
that's not yet loaded (eg. dwo files). Resulted in a delay in the
command return, which, can be interrupting.
This patch also added a new option `--load-all-debug-info` asking
statistics to dump all possible debug info, which will force loading all
debug info available if not yet loaded.
Store a SupportFile, rather than a FileSpec, in CompileUnit. This commit
works towards having the SourceManager operate on SupportFiles so that
it can (1) validate the Checksum and (2) materialize the content of
inline source information.
