This patch fixes:
lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp:415:7:
error: label at end of compound statement is a C++23 extension
[-Werror,-Wc++23-extensions]
lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp:536:7:
error: label at end of compound statement is a C++23 extension
[-Werror,-Wc++23-extensions]
lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp:672:7:
error: label at end of compound statement is a C++23 extension
[-Werror,-Wc++23-extensions]
While fixing bugs in the x86_64 LC_THREAD parser in ObjectFileMachO, I
noticed that the other LC_THREAD parsers are all less clear than they
should be.
To recap, a Mach-O LC_THREAD load command has a byte size for the entire
payload. Within the payload, there will be one or more register sets
provided. A register set starts with a UInt32 "flavor", the type of
register set defined in the system headers, and a UInt32 "count", the
number of UInt32 words of memory for this register set. After one
register set, there may be additional sets. A parser can skip an unknown
register set flavor by using the count field to get to the next register
set. When the total byte size of the LC_THREAD load command has been
parsed, it is completed.
This patch fixes the riscv/arm/arm64 LC_THREAD parsers to use the total
byte size as the exit condition, and to skip past unrecognized register
sets, instead of stopping parsing.
Instead of fixing the i386 corefile support, I removed it. The last
macOS that supported 32-bit Intel code was macOS 10.14 in 2018. I also
removed i386 KDP support, 32-bit intel kernel debugging hasn't been
supported for even longer than that.
It would be preferable to do these things separately, but I couldn't
bring myself to update the i386 LC_THREAD parser, and it required very
few changes to remove this support entirely.
reading, and one bug in the new RegisterContextUnifiedCore class.
The PR I landed a few days ago to allow Mach-O corefiles to augment
their registers with additional per-thread registers in metadata exposed
a few bugs in the x86_64 corefile reader when running under different CI
environments. It also showed a bug in my RegisterContextUnifiedCore
class where I wasn't properly handling lookups of unknown registers
(e.g. the LLDB_GENERIC_RA when debugging an intel target).
The Mach-O x86_64 corefile support would say that it had fpu & exc
registers available in every corefile, regardless of whether they were
actually present. It would only read the bytes for the first register
flavor in the LC_THREAD, the GPRs, but it read them incorrectly, so
sometimes you got more register context than you'd expect. The LC_THREAD
register context specifies a flavor and the number of uint32_t words;
the ObjectFileMachO method would read that number of uint64_t's,
exceeding the GPR register space, but it was followed by FPU and then
EXC register space so it didn't crash. If you had a corefile with GPR
and EXC register bytes, it would be written into the GPR and then FPU
register areas, with zeroes filling out the rest of the context.
The "process metadata" LC_NOTE allows for thread IDs to be specified in
a Mach-O corefile. This extends the JSON recognzied in that LC_NOTE to
allow for additional registers to be supplied on a per-thread basis.
The registers included in a Mach-O corefile LC_THREAD load command can
only be one of the register flavors that the kernel (xnu) defines in
<mach/arm/thread_status.h> for arm64 -- the general purpose registers,
floating point registers, exception registers.
JTAG style corefile producers may have access to many additional
registers beyond these that EL0 programs typically use, for instance
TCR_EL1 on AArch64, and people developing low level code need access to
these registers. This patch defines a format for including these
registers for any thread.
The JSON in "process metadata" is a dictionary that must have a
`threads` key. The value is an array of entries, one per LC_THREAD in
the Mach-O corefile. The number of entries must match the LC_THREADs so
they can be correctly associated.
Each thread's dictionary must have two keys, `sets`, and `registers`.
`sets` is an array of register set names. If a register set name matches
one from the LC_THREAD core registers, any registers that are defined
will be added to that register set. e.g. metadata can add a register to
the "General Purpose Registers" set that lldb shows users.
`registers` is an array of dictionaries, one per register. Each register
must have the keys `name`, `value`, `bitsize`, and `set`. It may provide
additional keys like `alt-name`, that
`DynamicRegisterInfo::SetRegisterInfo` recognizes.
This `sets` + `registers` formatting is the same that is used by the
`target.process.python-os-plugin-path` script interface uses, both are
parsed by `DynamicRegisterInfo`. The one addition is that in this
LC_NOTE metadata, each register must also have a `value` field, with the
value provided in big-endian base 10, as usual with JSON.
In RegisterContextUnifiedCore, I combine the register sets & registers
from the LC_THREAD for a specific thread, and the metadata sets &
registers for that thread from the LC_NOTE. Even if no LC_NOTE is
present, this class ingests the LC_THREAD register contexts and
reformats it to its internal stores before returning itself as the
RegisterContex, instead of shortcutting and returning the core's native
RegisterContext. I could have gone either way with that, but in the end
I decided if the code is correct, we should live on it always.
I added a test where we process save-core to create a userland corefile,
then use a utility "add-lcnote" to strip the existing "process metadata"
LC_NOTE that lldb put in it, and adds a new one from a JSON string.
rdar://74358787
---------
Co-authored-by: Jonas Devlieghere <jonas@devlieghere.com>
Some inter-plugin dependencies are okay, others are not. Yet others not,
but we're sort of stuck with them. The idea is to be able to prevent
backsliding while making sure that acceptable dependencies are..
accepted. For context, see
https://github.com/llvm/llvm-project/pull/139170 and the attached
changes to the documentation.
A unification of the DWARF section names,
https://github.com/llvm/llvm-project/pull/141344
broke dwarf5 debugging with Mach-O files. The str_offset and
str_offset.dwo names are different in Mach-O from other object
files.
Different object file formats support DWARF sections (COFF, ELF, MachO,
PE/COFF, WASM). COFF and PE/COFF only matched a subset. This caused some
GCC executables produced on MinGW to have issue later on when debugging.
One example is that `.debug_rnglists` was not matched, which caused
range-extraction to fail when printing a backtrace.
This unifies the parsing of section names in
`ObjectFile::GetDWARFSectionTypeFromName`, so all file formats can use
the same naming convention. Since the prefixes are different,
`GetDWARFSectionTypeFromName` only matches the suffixes (i.e. `.debug_`
needs to be stripped before).
I added two tests to ensure the sections are correctly identified on
Windows executables.
This reverts commit d4d2f069dec4fb8b13447f52752d4ecd08d976d6.
Temporarily reverting until we can find a way to get the correct
ObjectFile set in Module's Triples without adding "-macho" to the
triple string for each Module. This is breaking TestUniversal.py
on the x86_64 macOS CI bots.
# The Change
This patch sets the **default** object format of `ObjectFileMachO` to be
`MachO` (instead of what currently ends up to be `ELF`, see below). This
should be **the correct thing to do**, because the code before the line
of change has already verified the Mach-O header.
The existing logic:
* In `ObjectFileMachO`, the object format is unassigned by default. So
it's `UnknownObjectFormat` (see
[code](54d544b831/llvm/lib/TargetParser/Triple.cpp (L1024))).
* The code then looks at load commands like `LC_VERSION_MIN_*`
([code](54d544b831/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp (L5180-L5217)))
and `LC_BUILD_VERSION`
([code](54d544b831/lldb/source/Plugins/ObjectFile/Mach-O/ObjectFileMachO.cpp (L5231-L5252)))
and assign the Triple's OS and Environment if they exist.
* If the above sets the Triple's OS to macOS, then the object format
defaults to `MachO`; otherwise it is `ELF`
([code](54d544b831/llvm/lib/TargetParser/Triple.cpp (L936-L937)))
# Impact
For **production usage** where Mach-O files have the said load commands
(which is
[expected](https://www.google.com/search?q=Are+mach-o+files+expected+to+have+the+LC_BUILD_VERSION+load+command%3F)),
this patch won't change anything.
* **Important note**: It's not clear if there are legitimate production
use cases where the Mach-O files don't have said load commands. If there
is, the exiting code think they are `ELF`. This patch changes it to
`MachO`. This is considered a fix for such files.
For **unit tests**, this patch will simplify the yaml data by not
requiring the said load commands.
# Test
See PR.
If we're not touching them, we don't need to do anything special to pass
them along -- with one important caveat: due to how cmake arguments
work, the implicitly passed arguments need to be specified before
arguments that we handle.
This isn't particularly nice, but the alternative is enumerating all
arguments that can be used by llvm_add_library and the macros it calls
(it also relies on implicit passing of some arguments to
llvm_process_sources).
In #129307, we introduced read write in chunks, and during the final
revision of the PR I changed the behavior for 64b memory regions and did
not test an actual 64b memory range.
This caused LLDB to crash whenever we generated a 64b memory region.
64b regions has been a problem in testing for some time as it's a waste
of test resources to generation a 5gb+ Minidump. I will work with
@clayborg and @labath to come up with a way to specify creating a 64b
list instead of a 32b list (likely via the yamilizer).
Basically, disable everything except the eh_frame unwind plan, as that's
the only one which supports this right now. The other plans are working
with now trying the interpret everything in between the function parts
as a part of the function, which is more likely to produce wrong results
than correct ones.
I changed the interface for object file plans, to give the
implementations a chance to implement this correctly, but I haven't
actually converted PECallFrameInfo (its only implementation) to handle
that. (from the looks of things, it should be relatively easy to do, if
it becomes necessary)
I'm also deleting UnwindPlan::GetFirstNonPrologueInsn, as it's not used,
and it doesn't work for discontinuous functions.
Add a generous amount of buffer directories. I found out some LLDB forks
(internal and external) had custom ranges that could fail because we
didn't pre-account for those. To prevent this from being a problem, I've
added a large number of buffer directories at the cost of 240 bytes.
Add support for reading a macho corefile with CPU_TYPE_RISCV and the
riscv32 general purpose register file. I added code for the floating
point and exception registers too, but haven't exercised this. If we
start putting the full CSR register bank in a riscv corefile, it'll be
in separate 4k byte chunks, but I don't have a corefile to test against
that so I haven't written the code to read it.
The RegisterContextDarwin_riscv32 is copied & in the style of the other
RegisterContextDarwin classes; it's not the first choice I would make
for representing this, but it wasn't worth changing for this cputype.
rdar://145014653
I've recently been working on Minidump File Builder again, and some of
the comments are out of date, and many of the includes are no longer
used. This patch removes unneeded includes and rephrases some comments
to better fit with the current state after the read write chunks pr.
MinidumpFileBuilder calls std::min(MAX_WRITE_CHUNK_SIZE,
func_returning_uint64_t) and on Darwin this errors out with
unsigned long long & unsigned long not being the same type.
I recently received an internal error report that LLDB was OOM'ing when
creating a Minidump. In my 64b refactor we made a decision to acquire
buffers the size of the largest memory region so we could read all of
the contents in one call. This made error handling very simple (and
simpler coding for me!) but had the trade off of large allocations if
huge pages were enabled.
This patch is one I've had on the back burner for awhile, but we can
read and write the Minidump memory sections in discrete chunks which we
already do for writing to disk.
I had to refactor the error handling a bit, but it remains the same. We
make a best effort attempt to read as much of the memory region as
possible, but fail immediately if we receive an error writing to disk. I
did not add new tests for this because our existing test suite is quite
good, but I did manually verify a few Minidumps couldn't read beyond the
red_zone.
```
(lldb) reg read $sp
rsp = 0x00007fffffffc3b0
(lldb) p/x 0x00007fffffffc3b0 - 128
(long) 0x00007fffffffc330
(lldb) memory read 0x00007fffffffc330
0x7fffffffc330: 60 c3 ff ff ff 7f 00 00 60 cd ff ff ff 7f 00 00 `.......`.......
0x7fffffffc340: 60 c3 ff ff ff 7f 00 00 65 e6 26 00 00 00 00 00 `.......e.&.....
(lldb) memory read 0x00007fffffffc329
error: could not parse memory info (Success!)
```
I'm not sure how to quantify the memory improvement other than we would
allocate the largest size regardless of the size. So a 2gb unreadable
region would cause a 2gb allocation even if we were reading 4096 kb. Now
we will take the range size or the max chunk size of 128 mb.
To be able to describe discontinuous functions, this patch changes the
UnwindPlan to accept more than one address range.
I've also squeezed in a couple improvements/modernizations, for example
using the lower_bound function instead of a linear scan.
In most places, the rows are copied anyway (because they are generated
by cumulating modifications) immediately after adding them to the unwind
plans. In others, they can be moved into the unwind plan. This lets us
remove some backflip copies and make `const UnwindPlan` actually mean
something.
I've split this patch into two (and temporarily left both APIs) as this
patch was getting a bit big. This patch covers all the interesting
cases. Part two all about converting "architecture default" unwind plans
from ABI and InstructionEmulation plugins.
We have a binary image on Darwin that has no code, only metadata. It has
a large symbol table with many external symbol names that will not be
needed in the debugger. And it is possible to not have this binary on
the debugger system - so lldb must read all of the symbol names out of
memory, one at a time, which can be quite slow.
We're adding a section __TEXT,__lldb_no_nlist, to this binary to
indicate that lldb should not read the nlist symbols for it when we are
reading out of memory. If lldb is run with an on-disk version of the
binary, we will load the symbol table as we normally would, there's no
benefit to handling this binary differently.
I added a test where I create a dylib with this specially named section,
launch the process. The main binary deletes the dylib from the disk so
lldb is forced to read it out of memory. lldb attaches to the binary,
confirms that the dylib is present in the process and is a memory
Module. If the binary is not present, or lldb found the on-disk copy
because it hasn't been deleted yet, we delete the target, flush the
Debugger's module cache, sleep and retry, up to ten times. I create the
specially named section by compiling an assembly file that puts a byte
in the section which makes for a bit of a messy Makefile (the pre-canned
actions to build a dylib don't quite handle this case) but I don't think
it's much of a problem. This is a purely skipUnlessDarwin test case.
Relanding this change with a restructured Makefiles for the test case
that should pass on the CI bots.
rdar://146167816
This reverts commit 397696bb3d26c1298bf265e4907b0b6416ad59c9.
This breaks the macOS CI bots, I need to use $LDFLAGS in the $LD
invocation when building the dylib to get the dylibs to build on
the CI bots. But I've added "-lno-nlists -lhas-nlists" to the LDFLAGS
for the main binary in the same directory, so using LDFLAGS will
result in a compile error for the dylibs. I'll need to build the
dylibs in a subdir with a different Makefile, will reland with that
change in a bit.
We have a binary image on Darwin that has no code, only metadata. It has
a large symbol table with many external symbol names that will not be
needed in the debugger. And it is possible to not have this binary on
the debugger system - so lldb must read all of the symbol names out of
memory, one at a time, which can be quite slow.
We're adding a section __TEXT,__lldb_no_nlist, to this binary to
indicate that lldb should not read the nlist symbols for it when we are
reading out of memory. If lldb is run with an on-disk version of the
binary, we will load the symbol table as we normally would, there's no
benefit to handling this binary differently.
I added a test where I create a dylib with this specially named section,
launch the process. The main binary deletes the dylib from the disk so
lldb is forced to read it out of memory. lldb attaches to the binary,
confirms that the dylib is present in the process and is a memory
Module. If the binary is not present, or lldb found the on-disk copy
because it hasn't been deleted yet, we delete the target, flush the
Debugger's module cache, sleep and retry, up to ten times. I create the
specially named section by compiling an assembly file that puts a byte
in the section which makes for a bit of a messy Makefile (the pre-canned
actions to build a dylib don't quite handle this case) but I don't think
it's much of a problem. This is a purely skipUnlessDarwin test case.
rdar://146167816
Mach-O corefiles have LC_NOTE metadata, one LC_NOTE that lldb recognizes
is `main bin spec` which can specify that this is a kernel corefile,
userland corefile, or firmware/standalone corefile. With a userland
corefile, the LC_NOTE would specify the virtual address of the dyld
binary's Mach-O header. lldb would create a Module from that in-memory
binary, find the `dyld_all_image_infos` object in dyld's DATA segment,
and use that object to find all of the binaries present in the corefile.
ProcessMachCore takes the metadata from this LC_NOTE and passes the
address to the DynamicLoader plugin via its `GetImageInfoAddress()`
method, so the DynamicLoader can find all of the binaries and load them
in the Target at their correct virtual addresses.
We have a corefile creator who would prefer to specify the address of
`dyld_all_image_infos` directly, instead of specifying the address of
dyld and parsing that to find the object. DynamicLoaderMacOSX, the
DynamicLoader plugin being used here, will accept either a dyld virtual
address or a `dyld_all_image_infos` virtual address from
ProcessMachCore, and do the correct thing with either value.
lldb's process save-core mach-o corefile reader will continue to specify
the virtual address of the dyld binary.
rdar://144322688
A section of ObjectFileMachO is ifdef compiled only when
building to run on iOS etc natively, so this old method
call rename wasn't detected by normal on-mac building.
Recognize the visionOS Triple::OSType::XROS os type. Some of these have
already been landed on main, but I reviewed the downstream sources and
there were a few that still needed to be landed upstream.
Lots of code around LLDB was directly accessing the target's section
load list. This NFC patch makes the section load list private so the
Target class can access it, but everyone else now uses accessor
functions. This allows us to control the resolving of addresses and will
allow for functionality in LLDB which can lazily resolve addresses in
JIT plug-ins with a future patch.
Note that PointerUnion::{is,get} have been soft deprecated in
PointerUnion.h:
// FIXME: Replace the uses of is(), get() and dyn_cast() with
// isa<T>, cast<T> and the llvm::dyn_cast<T>
In #119598 my recent TLS feature seems to break crashpad symbols. I have
a few ideas on how this is happening, but for now as a mitigation I'm
checking if the Minidump was LLDB generated, and if so leveraging the
dynamic loader.
Compared to the python version, this also does type checking and error
handling, so it's slightly longer, however, it's still comfortably
under 500 lines.
Relanding with more explicit type conversions.
This reverts commit f6012a209dca6b1866d00e6b4f96279469884320.
Revert "[lldb] Add cast to fix compile error on 32-but platforms"
This reverts commit d300337e93da4ed96b044557e4b0a30001967cf0.
Revert "[lldb] Improve log message to include missing strings"
This reverts commit 0be33484853557bc0fd9dfb94e0b6c15dda136ce.
Revert "[lldb] Add comment"
This reverts commit e2bb47443d2e5c022c7851dd6029e3869fc8835c.
Revert "[lldb] Implement a formatter bytecode interpreter in C++"
This reverts commit 9a9c1d4a6155a96ce9be494cec7e25731d36b33e.
Compared to the python version, this also does type checking and error
handling, so it's slightly longer, however, it's still comfortably
under 500 lines.
Add support for type summaries embedded into the binary.
These embedded summaries will typically be generated by Swift macros,
but can also be generated by any other means.
rdar://115184658
The Mach-O load commands have an LC_SYMTAB / struct symtab_command which
represents the offset of the symbol table (nlist records) and string
table for this binary. In a mach-o binary on disk, these are file
offsets. If a mach-o binary is loaded in memory with all segments
consecutive, the `symoff` and `stroff` are the offsets from the TEXT
segment (aka the mach-o header) virtual address to the virtual address
of the start of these tables.
However, if a Mach-O binary is a part of the shared cache, then the
segments will be separated -- they will have different slide values. And
it is possible for the LINKEDIT segment to be greater than 4GB away from
the TEXT segment in the virtual address space, so these 32-bit offsets
cannot express the offset from TEXT segment to these tables.
Create separate uint64_t variables to track the offset to the symbol
table and string table, instead of reusing the 32-bit ones in the
symtab_command structure.
rdar://140432279
