In #165604, a test was skipped on Windows, because the native PDB plugin
didn't set sizes on symbols. While the test isn't compiled with debug
info, it's linked with `-gdwarf`, causing a PDB to be created on
Windows. This PDB will only contain the public symbols (written by the
linker) and section information. The symbols themselves don't have a
size, however the DIA SDK sets a size for them.
It seems like, for these data symbols, the size given from DIA is the
distance to the next symbol (or the section end).
This PR implements the naive approach for the native plugin. The main
difference is in function/code symbols. There, DIA searches for a
corresponding `S_GPROC32` which have a "code size" that is sometimes
slightly smaller than the difference to the next symbol.
Most of the cases were where a C++ file was being compiled with the C substitution.
There were a few cases of the opposite though.
LLDB seems to be the only real culprit in the LLVM codebase for these mismatches.
Rest of the LLVM presumably sticks at least language-specific options in the common substitutions
making the mistakes immediately apparent.
I found these by using Clang frontend configuration files containing language-specific options for
both C and C++ (e.g. `-std=c2y` and `-std=c++26`).
All PDB tests now pass when compiled without DIA on Windows, so they
pass with the native reader.
With this PR, the default reader changes to the native reader.
The plan is to eventually remove the DIA reader (see
https://discourse.llvm.org/t/rfc-removing-the-dia-pdb-plugin-from-lldb/87827
and #114906).
For now, DIA can be used by setting `plugin.symbol-file.pdb.reader` to
`dia` or by setting `LLDB_USE_NATIVE_PDB_READER=0` (mostly undocumented,
but used in tests).
When creating all types in a compilation unit, simple types (~>
primitive and pointer types) that were only used in function arguments
or return types weren't created as LLDB `Type`s.
With this PR, they're created when creating the function/method types.
This makes it possible to run the `SymbolFile/PDB/typedefs.test` with
both plugins.
The `pointers.test` was only run with the DIA plugin. I made the
following changes:
- Remove the check for the function type.
The types of the function are different in the plugins:
```
Native:
Type{0x00010084} , size = 0, compiler_type = 0x00000209aff60060 int
(int) __attribute__((thiscall))
DIA:
Type{0x0000000a} , name = "f", decl = PointerTypeTest.cpp:8,
compiler_type = 0x0000020bc22356c0 int (int) __attribute__((thiscall))
```
In DIA, each function gets its own type with a name and decl. In the
native plugin, only one unnamed type is created per signature. This
matches DWARF.
- The check for the `struct ST` fields was split, because the order of
members and methods is swapped between the plugins. In DIA, the member
is first and in the native plugin the method is first. We still check
that both are in the struct.
- The type names for the local variables are different. The native
plugin includes <code>\`extern "C" main'::\`2'::ST</code> which I added
as an allowed prefix. This comes from the mangled name of the struct
`ST` - `.?AUST@?1??main@@9@`.
- The location of local variables is different. DIA creates one static
location (e.g. `DW_OP_breg6 ESI-52`) whereas the native plugin limits
the location to the block (e.g. `[0x0040100d, 0x00401038): DW_OP_breg6
ESI-52`). This gets printed on a second line and the `location` starts
with `0x00000000:`
- DIA adds a decl for each parameter (and local variable). However, this
information is not contained in the PDB. I'm not sure how DIA calculates
this. It's often wrong and assumes variables are declared earlier. For
example, in this test
([PointerTypeTest.cpp](2b135b9313/lldb/test/Shell/SymbolFile/PDB/Inputs/PointerTypeTest.cpp)),
it assumes that all local variables of `main` are created on line 4. The
native plugin doesn't include this, so I made the check optional.
The test checks that functions have the correct type assigned. Because
of the differences between the two PDB plugins, I split the test.
DIA creates one named `Type` per function and uses identical UIDs for
`Type` and `Function`, whereas native creates one unnamed type per
signature and has different UIDs.
The native test has the same input and checks the same functions.
I also removed the `target-windows` requirement from the test, since it
only uses `lldb-test`.
This aligns the simple types created by the native plugin with the ones
from DIA as well as LLVM and the original cvdump.
- A few type names weren't handled when creating the LLDB `Type` name
(e.g. `short`)
- 64-bit integers were created as `(u)int64_t` and are now created as
`(unsigned) long long` (matches DIA)
- 128-bit integers (only supported by clang-cl) weren't created as types
(they have `SimpleTypeKind::(U)Int128Oct`)
- All complex types had the same name - now they have `_Complex
<float-type>`
Some types like `SimpleTypeKind::Float48` can't be tested because they
can't be created in C++.
Before this PR, the native PDB plugin would create the following LLDB
`Type` for `using SomeTypedef = long`:
```
Type{0x00002e03} , name = "SomeTypedef", size = 4, compiler_type = 0x000002becd8d8620 long
```
with this PR, the following is created:
```
Type{0x00002e03} , name = "SomeTypedef", size = 4, compiler_type = 0x0000024d6a7e3c90 typedef SomeTypedef
```
This matches the behavior of the DIA PDB plugin and works towards making
[`Shell/SymbolFile/PDB/typedefs.test`](https://github.com/llvm/llvm-project/blob/main/lldb/test/Shell/SymbolFile/PDB/typedefs.test)
pass with the native plugin.
I added a similar test to the `NativePDB` shell tests to capture the
current state, which doesn't quite match that of DIA yet. I'll add some
comments on what's missing on this PR, because I'm not fully sure what
the correct output would be.
Relands #149701 which was reverted in
185ae5cdc6
because it broke demangling of Itanium symbols on i386.
The last commit in this PR adds the fix for this (discussed in #160930).
On x86 environments, the prefix of `__cdecl` functions will now be
removed to match DWARF. I opened #161676 to discuss this for the other
calling conventions.
First time check was introduced in
`fa3ab4599d717feedbb83e08e7f654913942520b` to work around a debug-info
generation bug in Clang. This bug was fixed in Clang-4. The check has
since been adjusted (first in
`808ff186f6a6ba1fd38cc7e00697cd82f4afe540`, and then most recently in
`370db9c62910195e664e82dde6f0adb3e255a4fd`).
This check is getting quite convoluted, and all it does is turn an
`array[1]` into an `array[0]` type when it is deemed correct. At this
point the workaround probably never fires, apart from actually valid
codegen. This patch removes the special conditions and emits the error
specifically in those cases where we know the DWARF is malformed.
Added some shell tests for the error case.
Fixes#159401
On Windows there is no hex prefix, I suspect because somewhere we
print a pointer. I'd prefer to fix that itself but can't get to
a Windows machine at the moment. It's not important to the purpose
of the test anyway.
GCC doesn't add DW_AT_data_member_location attributes to the
DW_TAG_member children of DW_TAG_union_type types. An error was being
emitted incorrectly for these cases fr om the DWARFASTParserClang. This
fixes that issue and adds a test.
Before, functions created using the NativePDB plugin would not know
about their mangled name. This showed when printing a stacktrace. There,
only the function name was shown. For
https://github.com/llvm/llvm-project/issues/143149, the mangled function
name is required to separate different parts.
This PR adds that name if available.
The Clang AST nodes also take in a mangled name, which was previously
unset. I don't think this unblocks anything further, because Clang can
mangle the function anyway.
If LLDB was built without the DIA SDK and the DIA reader is explicitly
requested (through `LLDB_USE_NATIVE_PDB_READER=0` or `settings set
plugin.symbol-file.pdb.reader dia`), LLDB should print a warning,
because it will use the native reader in any case
(https://github.com/llvm/llvm-project/pull/159769#discussion_r2367316980).
This PR adds the warning and a test when LLDB is not built with the SDK
on Windows. I don't think any builder runs this configuration, as there
are still five failing tests. I tested this locally with and without the
SDK.
This test was failing with the native plugin due to two reasons:
1. The static `C::abc` was printed as `(int) ::C::abc = 123`
2. The order of the base classes of [`C`
(`List::Value`)](b7e4edca3d/lldb/test/Shell/SymbolFile/PDB/Inputs/UdtLayoutTest.cpp (L30))
is different between DIA and the native plugin. I don't know how the
order in the DIA plugin is determined - it prints `B<0>`, `B<1>`,
`B<2>`, `B<3>`, `A`. The native plugin follows the order of the bases in
memory and prints `B<2>`, `B<3>`, `A`, `B<0>`, `B<1>` (last three are
the virtual bases).
<details><summary>Class layout of C</summary>
```
class C size(88):
+---
0 | +--- (base class B<2>)
0 | | {vbptr}
8 | | _a
9. | | _b (bitstart=3,nbits=6)
11 | | _c
| +---
15 | +--- (base class B<3>)
15 | | {vbptr}
23 | | _a
24. | | _b (bitstart=3,nbits=6)
26 | | _c
| +---
| <alignment member> (size=2)
32 | _x
36 | _y
38 | _z
| <alignment member> (size=1)
| <alignment member> (size=2)
+---
+--- (virtual base A)
40 | {vfptr}
48 | U _u
| <alignment member> (size=4)
+---
+--- (virtual base B<0>)
56 | {vbptr}
64 | _a
65. | _b (bitstart=3,nbits=6)
67 | _c
+---
+--- (virtual base B<1>)
71 | {vbptr}
79 | _a
80. | _b (bitstart=3,nbits=6)
82 | _c
+---
```
</details>
I split the tests for the plugins for better readability.
When creating LLDB types from `LF_MODIFIER` records, the type name of
the modified type was used. This didn't include the modifiers
(`const`/`volatile`/`__unaligned`). With this PR, they're included.
The DIA plugin had a test for this. That test also assumed that function
types had a name. I removed that check here, because function/procedure
types themselves in PDB don't have a name:
```
0x1015 | LF_ARGLIST [size = 20, hash = 0xBCB6]
0x0074 (int): `int`
0x1013: `int* __restrict`
0x1014: `int& __restrict`
0x1016 | LF_PROCEDURE [size = 16, hash = 0x3F611]
return type = 0x0003 (void), # args = 3, param list = 0x1015
calling conv = cdecl, options = None
```
I assume DIA gets the name from the function symbol itself. In the
native plugin, that name isn't included and multiple functions with the
same signature will reuse one type, whereas DIA would create a new type
for each function. The
[Shell/SymbolFile/PDB/func-symbols.test](b29c7ded31/lldb/test/Shell/SymbolFile/PDB/func-symbols.test)
also relies on this.
If LLDB is built without the DIA SDK enabled, then the native plugin is
used regardless of `plugin.symbol-file.pdb.reader` or
`LLDB_USE_NATIVE_PDB_READER`. This made the test fail on Windows when
the DIA SDK was disabled
(https://github.com/llvm/llvm-project/issues/114906#issuecomment-3241796062).
This PR changes the requirement for the test from `target-windows` to
`diasdk` (only used in this test).
After parsing blocks in a function, the blocks should be marked as
parsed for them to be dumped (see
[Function::Dump](e6aefbec78/lldb/source/Symbol/Function.cpp (L446-L447))).
As explained in
https://github.com/llvm/llvm-project/issues/114906#issuecomment-3255016266,
this happens (accidentally?) in the DIA plugin when parsing variables,
because it calls `function.GetBlock(can_create=true)` which marks blocks
as parsed. In the native plugin, this was never called, so blocks and
variables were never included in the `lldb-test symbols` output.
The `variables.test` for the DIA plugin tests this. One difference
between the plugins is how they specify the location of local variables.
This causes the output of the native plugin to be two lines per
variable, whereas the DIA plugin has one line:
```
(native):
000002C4B7593020: Variable{0x1c800001}, name = "var_arg1", type = {0000000000000744} 0x000002C4B6CA7900 (int), scope = parameter, location = 0x00000000:
[0x000000014000102c, 0x000000014000103e): DW_OP_breg7 RSP+8
```
```
(DIA):
000002778C827EE0: Variable{0x0000001b}, name = "var_arg1", type = {0000000000000005} 0x000002778C1FBAB0 (int), scope = parameter, decl = VariablesTest.cpp:32, location = DW_OP_breg7 RSP+8
```
In the test, I filtered lines starting with spaces followed by `[0x`, so
we can still use `CHECK-NEXT`.
---
Another difference between the plugins is that DIA marks the `this`
pointer as artificial (equivalent to DWARF). This is done if a
variable's object kind is `ObjectPtr`
([source](ab898f32c6/lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp (L1050))).
As far as I know, there isn't anything in the debug info that says "this
variable is the `this` pointer" other than the name/type of a variable
and the type of the function.
This test was brokem by migrating to the lit internal shell due to a
lack of env prefix for setting environment variables. This was fixed in
prevented the breakpoint in the test from mapping to anything, causing
the test to file. This patch restores the original line numbering.
These tests were failing on darwin, because the internal shell needs
environment var definitions to start with 'env'. This PR (hopefully)
fixes that problem.
To find global variables, `SymbolFileNativePDB` used to search the
globals stream for the name passed to `FindGlobalVariables`. However,
the symbols in the globals stream contain the fully qualified name and
`FindGlobalVariables` only gets the basename. The approach here is
similar to the one for types and functions.
As we already search the globals stream for functions, we can cache the
basenames for global variables there as well.
This makes the `expressions.test` from the DIA PDB plugin pass with the
native one (#114906).
This patch updates the lld lit test config to use the internal shell by
default. This has some performance advantages (~10-15%) and also
produces nicer failure output. It also updates the two LLDB tests to not
require shell (so that they run under the internal shell), after first
verifying that they run and pass using the internal shell; and it fixes
one test that was not passing under the internal shell.
This upstreams https://github.com/swiftlang/llvm-project/pull/10313.
If we detect a situation where a forward declaration is C++ and the
definition DIE is Objective-C, then just don't try to complete the type
(it would crash otherwise). In the long term we might want to add
support for completing such types.
We've seen real world crashes when debugging WebKit and wxWidgets
because of this. Both projects forward declare ObjC++ decls in the way
shown in the test.
rdar://145959981
This uses split DWARF and from looking at other tests, it should not be
running on Darwin or Windows.
It does pass using the DIA PDB plugin but I think this is misleading
because it's not actually testing the intended feature.
When the native PDB plugin is used it fails because it cannot set a
breakpoint. I don't see a point to running this test on Windows at all.
Native PDB plugin test failures are being tracked in #114906.
Tag types like stucts or enums didn't have a declaration attached to
them. The source locations are present in the IPI stream in
`LF_UDT_MOD_SRC_LINE` records:
```
0x101F | LF_UDT_MOD_SRC_LINE [size = 18, hash = 0x1C63]
udt = 0x1058, mod = 3, file = 1, line = 0
0x2789 | LF_UDT_MOD_SRC_LINE [size = 18, hash = 0x1E5A]
udt = 0x1253, mod = 35, file = 93, line = 17069
```
The file is an ID in the string table `/names`:
```
ID | String
1 | '\<unknown>'
12 | 'D:\a\_work\1\s\src\ExternalAPIs\WindowsSDKInc\c\Include\10.0.22621.0\um\wingdi.h'
93 | 'D:\a\_work\1\s\src\ExternalAPIs\WindowsSDKInc\c\Include\10.0.22621.0\um\winnt.h'
```
Here, we're not interested in `mod`. This would indicate which module
contributed the UDT.
I was looking at Rustc's PDB and found that it uses `<unknown>` for some
types, so I added a check for that.
This makes two DIA PDB shell tests to work with the native PDB plugin.
---------
Co-authored-by: Michael Buch <michaelbuch12@gmail.com>
Relands #152295.
Checking for the overloads did not account for them being out of order.
For example, [the failed
output](https://github.com/llvm/llvm-project/pull/152295#issuecomment-3177563247)
contained the overloads, but out of order. The last commit here fixes
that by using `-DAG`.
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
This adds the ability for functions to be matched by their basename.
Before, the globals were searched for the name. This works if the full
name is available but fails for basenames.
PDB only includes the full names of functions, so we need to cache all
basenames. This is (again) very similar to
[SymbolFilePDB](b242150b07/lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp (L1291-L1383)).
There are two main differences:
- We can't just access the parent of a function to determine that it's a
member function - we need to check the type of the function, and its
"this type".
- SymbolFilePDB saved the full method name in the map. However, when
searching for methods, only the basename is passed, so the function
never found any methods.
Fixes#51933.
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
This is a major change on how we represent nested name qualifications in
the AST.
* The nested name specifier itself and how it's stored is changed. The
prefixes for types are handled within the type hierarchy, which makes
canonicalization for them super cheap, no memory allocation required.
Also translating a type into nested name specifier form becomes a no-op.
An identifier is stored as a DependentNameType. The nested name
specifier gains a lightweight handle class, to be used instead of
passing around pointers, which is similar to what is implemented for
TemplateName. There is still one free bit available, and this handle can
be used within a PointerUnion and PointerIntPair, which should keep
bit-packing aficionados happy.
* The ElaboratedType node is removed, all type nodes in which it could
previously apply to can now store the elaborated keyword and name
qualifier, tail allocating when present.
* TagTypes can now point to the exact declaration found when producing
these, as opposed to the previous situation of there only existing one
TagType per entity. This increases the amount of type sugar retained,
and can have several applications, for example in tracking module
ownership, and other tools which care about source file origins, such as
IWYU. These TagTypes are lazily allocated, in order to limit the
increase in AST size.
This patch offers a great performance benefit.
It greatly improves compilation time for
[stdexec](https://github.com/NVIDIA/stdexec). For one datapoint, for
`test_on2.cpp` in that project, which is the slowest compiling test,
this patch improves `-c` compilation time by about 7.2%, with the
`-fsyntax-only` improvement being at ~12%.
This has great results on compile-time-tracker as well:
This patch also further enables other optimziations in the future, and
will reduce the performance impact of template specialization resugaring
when that lands.
It has some other miscelaneous drive-by fixes.
About the review: Yes the patch is huge, sorry about that. Part of the
reason is that I started by the nested name specifier part, before the
ElaboratedType part, but that had a huge performance downside, as
ElaboratedType is a big performance hog. I didn't have the steam to go
back and change the patch after the fact.
There is also a lot of internal API changes, and it made sense to remove
ElaboratedType in one go, versus removing it from one type at a time, as
that would present much more churn to the users. Also, the nested name
specifier having a different API avoids missing changes related to how
prefixes work now, which could make existing code compile but not work.
How to review: The important changes are all in
`clang/include/clang/AST` and `clang/lib/AST`, with also important
changes in `clang/lib/Sema/TreeTransform.h`.
The rest and bulk of the changes are mostly consequences of the changes
in API.
PS: TagType::getDecl is renamed to `getOriginalDecl` in this patch, just
for easier to rebasing. I plan to rename it back after this lands.
Fixes#136624
Fixes https://github.com/llvm/llvm-project/issues/43179
Fixes https://github.com/llvm/llvm-project/issues/68670
Fixes https://github.com/llvm/llvm-project/issues/92757
Some DIA PDB tests pass with the native plugin already, but didn't test
this. This adds test runs with the native plugin - no functional
changes.
In addition to the x86 calling convention test, there's also
9f102a9004/lldb/test/Shell/SymbolFile/PDB/calling-conventions-arm.test,
but I can't test this.
Fixes a bug that surfaces in frame recognizers.
Details about the bug:
A new frame recognizer is configured to match a specific symbol
(`swift_willThrow`). This is an `extern "C"` symbol defined in a C++
source file. When Swift is built with debug info, the function
`ParseFunctionFromDWARF` will use the debug info to construct a function
name that looks like a C++ declaration (`::swift_willThrow(void *,
SwiftError**)`). The `Mangled` instance will have this string as its
`m_demangled` field, and have _no_ string for its `m_mangled` field.
The result is the frame recognizer would not match the symbol to the
name (`swift_willThrow` != `::swift_willThrow(void *, SwiftError**)`.
By changing `ParseFunctionFromDWARF` to assign both a demangled name and
a mangled, frame recognizers can successfully match symbols in this
configuration.
Languages other than C/C++ don't necessarily emit mangled names in the
`UniqueName` field of type records. Rust specifically emits a unique ID
that doesn't contain the name.
For example, `(i32, i32)` is emitted as
```llvm
!266 = !DICompositeType(
tag: DW_TAG_structure_type, name: "tuple$<i32,i32>", file: !9, size: 64, align: 32,
elements: !267, templateParams: !17, identifier: "19122721b0632fe96c0dd37477674472"
)
```
which results in
```
0x1091 | LF_STRUCTURE [size = 72, hash = 0x1AC67] `tuple$<i32,i32>`
unique name: `19122721b0632fe96c0dd37477674472`
vtable: <no type>, base list: <no type>, field list: 0x1090
options: has unique name, sizeof 8
```
In C++ with Clang and MSVC, a structure similar to this would result in
```
0x136F | LF_STRUCTURE [size = 44, hash = 0x30BE2] `MyTuple`
unique name: `.?AUMyTuple@@`
vtable: <no type>, base list: <no type>, field list: 0x136E
options: has unique name, sizeof 8
```
With this PR, if a `UniqueName` is encountered that couldn't be parsed,
it will fall back to using the undecorated (→ do the same as if the
unique name is empty/unavailable).
I'm not sure how to test this. Maybe compiling the LLVM IR that rustc
emits?
Fixes#152051.
Initially suggested in
https://github.com/llvm/llvm-project/pull/149305#issuecomment-3113413702
- this PR adds the setting `plugin.symbol-file.pdb.use-native-reader`.
It doesn't remove support for `LLDB_USE_NATIVE_PDB_READER` to allow some
backwards compatibility. This was the suggested way to use the native
reader - changing that would mean users who set this, now use the DIA
reader. The setting has priority over the environment variable, though.
If the default gets flipped on Windows, the environment variable could
probably be removed as well.
This would make it possible to test both native PDB and DIA PDB in the
API tests (see linked PR).
Previously, `type lookup` for types in namespaces didn't work with the
native PDB plugin, because `FindTypes` would only look for types whose
base name was equal to their full name. PDB/CodeView does not store the
base names in the TPI stream, but the types have their full name (e.g.
`std::thread` instead of `thread`). So `findRecordsByName` would only
return types in the top level namespace.
This PR changes the lookup to go through all types and check their base
name. As that could be a bit expensive, the names are first cached
(similar to the function lookup in the DIA PDB plugin). Potential types
are checked with `TypeQuery::ContextMatches`.
To be able to handle anonymous namespaces, I changed
`TypeQuery::ContextMatches`. The [`TypeQuery`
constructor](9ad7edef42/lldb/source/Symbol/Type.cpp (L76-L79))
inserts all name components as `CompilerContextKind::AnyDeclContext`. To
skip over anonymous namespaces, `ContextMatches` checked if a component
was empty and exactly of kind `Namespace`. For our query, the last check
was always false, so we never skipped anonymous namespaces. DWARF
doesn't have this problem, as it [constructs the context
outside](abe93d9d7e/lldb/source/Plugins/SymbolFile/DWARF/DWARFIndex.cpp (L154-L160))
and has proper information about namespaces. I'm not fully sure if my
change is correct and that it doesn't break other users of `TypeQuery`.
This enables `type lookup <type>` to work on types in namespaces.
However, expressions don't work with this yet, because `FindNamespace`
is unimplemented for native PDB.
%T has been deprecated for about seven years, mostly because it is not
unique to each test which can lead to races. This patch updates the few
remaining tests in lldb that use %T to not use it (either directly using
files or creating their own temp dir). The eventual goal is to remove
support for %T from llvm-lit given few tests use it and it still has
racey behavior.
This patch errors on the side of creating new temp dirs even when not
strictly necessary to avoid needing to update filenames inside filecheck
matchers.
https://github.com/llvm/llvm-project/pull/149282 changed
the max children depth and that caused one part of the
output to become `{...}`.
The original PR set a higher limit for a different test,
so I'm doing the same here.
Deeply nested structs can be noisy, so Apple's LLDB fork sets the
default to `4`:
9c93adbb28/lldb/source/Target/TargetProperties.td (L134-L136)
Thought it would be useful to upstream this. Though happy to pick a
different default or keep it as-is.
