**Note:** The register reading and writing depends on new register
flavor support in thread_get_state/thread_set_state in the kernel, which
will be first available in macOS 15.4.
The Apple M4 line of cores includes the Scalable Matrix Extension (SME)
feature. The M4s do not implement Scalable Vector Extension (SVE),
although the processor is in Streaming SVE Mode when the SME is being
used. The most obvious side effects of being in SSVE Mode are that (on
the M4 cores) NEON instructions cannot be used, and watchpoints may get
false positives, the address comparisons are done at a lowered
granularity.
When SSVE mode is enabled, the kernel will provide the Streaming Vector
Length register, which is a maximum of 64 bytes with the M4. Also
provided are SVCR (with bits indicating if SSVE mode and SME mode are
enabled), TPIDR2, SVL. Then the SVE registers Z0..31 (SVL bytes long),
P0..15 (SVL/8 bytes), the ZA matrix register (SVL*SVL bytes), and the M4
supports SME2, so the ZT0 register (64 bytes).
When SSVE/SME are disabled, none of these registers are provided by the
kernel - reads and writes of them will fail.
Unlike Linux, lldb cannot modify the SVL through a thread_set_state
call, or change the processor state's SSVE/SME status. There is also no
way for a process to request a lowered SVL size today, so the work that
David did to handle VL/SVL changing while stepping through a process is
not an issue on Darwin today. But debugserver should be providing
everything necessary so we can reuse all of David's work on resizing the
register contexts in lldb if it happens in the future. debugbserver
sends svl, svcr, and tpidr2 in the expedited registers when a thread
stops, if SSVE|SME mode are enabled (if the kernel allows it to read the
ARM_SME_STATE register set).
While the maximum SVL is 64 bytes on M4, the AArch64 maximum possible
SVL is 256; this would give us a 64k ZA register. If debugserver sized
all of its register contexts assuming the largest possible SVL, we could
easily use 2MB more memory for the register contexts of all threads in a
process -- and on iOS et al, processes must run within a small memory
allotment and this would push us over that.
Much of the work in debugserver was changing the arm64 register context
from being a static compile-time array of register sets, to being
initialized at runtime if debugserver is running on a machine with SME.
The ZA is only created to the machine's actual maximum SVL. The size of
the 32 SVE Z registers is less significant so I am statically allocating
those to the architecturally largest possible SVL value today.
Also, debugserver includes information about registers that share the
same part of the register file. e.g. S0 and D0 are the lower parts of
the NEON 128-bit V0 register. And when running on an SME machine, v0 is
the lower 128 bits of the SVE Z0 register. So the register maps used
when defining the VFP registers must differ depending on the
capabilities of the cpu at runtime.
I also changed register reading in debugserver, where formerly when
debugserver was asked to read a register, and the thread_get_state read
of that register failed, it would return all zero's. This is necessary
when constructing a `g` packet that gets all registers - because there
is no separation between register bytes, the offsets are fixed. But when
we are asking for a single register (e.g. Z0) when not in SSVE/SME mode,
this should return an error.
This does mean that when you're running on an SME capabable machine, but
not in SME mode, and do `register read -a`, lldb will report that 48 SVE
registers were unavailable and 5 SME registers were unavailable. But
that's only when `-a` is used.
The register reading and writing depends on new register flavor support
in thread_get_state/thread_set_state in the kernel, which is not yet in
a release. The test case I wrote is skipped on current OSes. I pilfered
the SME register setup from some of David's existing SME test files;
there were a few Linux specific details in those tests that they weren't
easy to reuse on Darwin.
rdar://121608074
TestFirmwareCorefiles.py has a helper utility,
create-empty-corefile.cpp, which creates corefiles with different
metadata to specify the binary that should be loaded. It normally uses
an actual binary's UUID for the metadata, and it uses the binary's
cputype/cpusubtype for the corefile's mach header.
There is one test where it creates a corefile with metadata for a UUID
that cannot be found -- it is given no binary -- and in that case, the
cputype/cpusubtype it sets in the core file mach header was
uninitialized data. Through luck, on Darwin systems, the uninitialized
data typically matched a CPU_TYPE from machine.h and the test would
work. But when the value doens't match one of thoes defines, lldb would
reject the corefile entirely, and the test would fail. This has been an
infrequent failure on the CI bots for a while and I couldn't ever repo
it. There's a recent configuration where it was happening every time and
I was able to track it down.
rdar://141727563
This makes tests more portable.
Make variables for LLVM utils are passed to `make` on Darwin as well.
Co-authored-by: Vladimir Vereschaka <vvereschaka@accesssoftek.com>
Some gdb remote serial protocol stubs will send the thread IDs and PCs
for all threads in a process in the stop-reply packet. lldb often needs
to know the pc values for all threads while at a private stop, and that
results in <n-1> read-register packets for <n> threads, and can be a big
performance problem when this is a hot code path.
GDBRemoteRegisterContext tracks the StopID of when its values were set,
and when the thread's StopID has incremented, it marks all values it has
as Invalid, and knows to refetch them.
We have a code path that resulted in setting the PCs for all the
threads, and then `ProcessGDBRemote::CalculateThreadStopInfo` *forcing*
an invalidation of all the register contexts, forcing us to re-read the
pc values for all threads except the one that stopped.
There are times when it is valid to force an invalidation of the
regsiter cache - for instance, if the layout of the registers has
changed because the processor state is different, or we've sent a
write-all-registers packet to the inferior and we want to make sure we
stay in sync with the inferior. But there was no reason for this method
to be forcing the register context to be invalid.
I added a test when running on Darwin systems, where debugserver always
sends the thread IDs and PCs, which turns on packet logging. The test
runs against an inferior which has 4 threads; it steps over a dlopen()
call, steps in to a user function with debug info, steps-over and
steps-in across source lines with multiple function calls, and then
examines the packet log and flags it as an error if lldb asked for the
pc value of any thread at any point in the debug session.
For this program and the operations we're doing, with debugserver that
provides thread IDs and PCs, we should never ask for the value of a pc
register.
rdar://136247381
This fix is based on a problem with cxx_compiler and cxx_linker macros
on Windows.
There was an issue with compiler detection in paths containing "icc". In
such case, Makefile.rules thought it was provided with icc compiler.
To solve that, utilities detection has been rewritten in Python.
The last element of compiler's path is separated, taking into account
the platform path delimiter, and compiler type is extracted, with regard
of possible cross-toolchain prefix.
---------
Co-authored-by: Pavel Labath <pavel@labath.sk>
This is used by various system routines (the capabilities checker and
dyld to name a few) to add extra color to an abort. This patch adds a
frame recognizer so people can easily see the details, and also adds the
information to the ExtendedCrashInformation dictionary.
I also had to rework how the dictionary is held; previously it was
created on demand, but that was inconvenient since it meant all the
entries had to be produced at that same time. That didn't work for the
recognizer.
This test sets a breakpoint on malloc, as a way to stop early in
dyld's setting up code, before the system libraries are initialized
so we can confirm that we don't fetch the Objective-C class table
before it's initialized.
In macOS 15 (macOS Sonoma), dyld doesn't call malloc any longer,
so this heuristic/trick isn't working. It does call other things
called *alloc though, so I'm changing this to use a regex breakpoint
on that, to keep the test working.
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
This relands https://github.com/llvm/llvm-project/pull/95963. It had to
be reverted because the `TestEarlyProcessLaunch.py` test was failing
on the incremental macOS bots. The test failed because it was relying on
expression log output from the ObjC introspection routines (but was
the expression was called from a C++ context). The relanded patch
simply ensures that the test runs the expressions as `ObjC` expressions.
When LLDB isn't able to find a `clang::Decl` in response
to a `FindExternalVisibleDeclsByName`, it will fall-back
to looking into the Objective-C runtime for that decl. This
ends up doing a lot of work which isn't necessary when we're
debugging a C++ program. This patch makes the ObjC lookup
conditional on the language that the ExpressionParser deduced
(which can be explicitly set using the `expr --language` option
or is set implicitly if we're stopped in an ObjC frame or a
C++ frame without debug-info).
rdar://96236519
The test has a check that the static linker supports the new option,
but it assumed the Xcode 16 linker also meant it was running on
macOS 15 and the dynamic linker would honor dependencies flagged
this way. But Xcode 16 can be run on macOS 14.5, so we need to
skip the test in that combination.
Add comments and a test for delay-init libraries on macOS. I originally
added the support in 954d00e87cdd77d0e9e367be52e62340467bd779 a month
ago, but without these additional clarifications.
rdar://126885033
using the macOS version as a proxy. I can't reproduce any of these
failures locally, but the tests all use pexpect and probably have bad
timeout behavior under high load.
https://github.com/llvm/llvm-project/pull/87409 removed the broadcast
bits from SBDebugger and placed them in `lldb-enumerations.h`. This is
API-breaking so this commits places the enum back into `SBDebugger.h`
and references the bits from `lldb-enumerations.h`.
rdar://127128536
Make SymbolFileCTF::ParseFunctions resilient against not being able to
resolve the argument or return type of a function. ResolveTypeUID can
fail for a variety of reasons so we should always check its result.
The type that caused the crash was `_Bool` which we didn't recognize
as a basic type. This commit also fixes the underlying issue and adds
a test.
rdar://126943722
…db-enumerations.h" (#88324)"
This reverts commit 9f6d08f2566a26144ea1753f80aebb1f2ecfdc63. This broke
the build because of a usage of one of the original SBDebugger broadcast
bits that wasn't updated in the original commit.
When the `eBroadcastBitProgressCategory` bit was originally added to
Debugger.h and SBDebugger.h, each corresponding bit was added in order
of the other bits that were previously there. Since `Debugger.h` has an
enum bit that `SBDebugger.h` does not, this meant that their offsets did
not match.
Instead of trying to keep the bit offsets in sync between the two, it's
preferable to just move SBDebugger's enum into the main enumerations
header and use the bits from there. This also requires that API tests using the bits from SBDebugger update their usage.
Darwin AArch64 application processors are run with Top Byte Ignore mode
enabled so metadata may be stored in the top byte, it needs to be
ignored when reading/writing memory. David Spickett handled this already
in the base class Process::ReadMemory but ProcessMachCore overrides that
method (to avoid the memory cache) and did not pick up the same change.
I add a test case that creates a pointer with metadata in the top byte
and dereferences it with a live process and with a corefile.
rdar://123784501
Looking ast the definition of both functions this is *almost* an NFC
change, except that Triple also looks at the SubArch (important) and
ObjectFormat (less so).
This fixes a bug that only manifests with how Xcode uses the SBAPI to
attach to a process by name: it guesses the architecture based on the
system. If the system is arm64 and the Process is arm64e Target fails
to update the triple because it deemed the two to be equivalent.
rdar://123338218
Looking ast the definition of both functions this is *almost* an NFC
change, except that Triple also looks at the SubArch (important) and
ObjectFormat (less so).
This fixes a bug that only manifests with how Xcode uses the SBAPI to
attach to a process by name: it guesses the architecture based on the
system. If the system is arm64 and the Process is arm64e Target fails to
update the triple because it deemed the two to be equivalent.
rdar://123338218
Any time we see the pattern `assertEqual(value, bool)`, we can replace
that with `assert<bool>(value)`. Likewise for `assertNotEqual`.
Technically this relaxes the test a bit, as we may want to make sure
`value` is either `True` or `False`, and not something that implicitly
converts to a bool. For example, `assertEqual("foo", True)` will fail,
but `assertTrue("foo")` will not. In most cases, this distinction is not
important.
There are two such places that this patch does **not** transform, since
it seems intentional that we want the result to be a bool:
*
5daf2001a1/lldb/test/API/python_api/sbstructureddata/TestStructuredDataAPI.py (L90)
*
5daf2001a1/lldb/test/API/commands/settings/TestSettings.py (L940)
Followup to 9c2468821ec51defd09c246fea4a47886fff8c01. I patched `teyit`
with a `visit_assertEqual` node handler to generate this.
This uses [teyit](https://pypi.org/project/teyit/) to modernize asserts,
as recommended by the [unittest release
notes](https://docs.python.org/3.12/whatsnew/3.12.html#id3).
For example, `assertTrue(a == b)` is replaced with `assertEqual(a, b)`.
This produces better error messages, e.g. `error: unexpectedly found 1
and 2 to be different` instead of `error: False`.
assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a
vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.
This removes the dependency LLDB API tests have on
lldb/third_party/Python/module/unittest2, and instead uses the standard
one provided by Python.
This does not actually remove the vendored dep yet, nor update the docs.
I'll do both those once this sticks.
Non-trivial changes to call out:
- expected failures (i.e. "bugnumber") don't have a reason anymore, so
those params were removed
- `assertItemsEqual` is now called `assertCountEqual`
- When a test is marked xfail, our copy of unittest2 considers failures
during teardown to be OK, but modern unittest does not. See
TestThreadLocal.py. (Very likely could be a real bug/leak).
- Our copy of unittest2 was patched to print all test results, even ones
that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped,
...)`, but standard unittest prints a terser message that omits test
result types that didn't happen, e.g. `OK (skipped=1)`. Our lit
integration parses this stderr and needs to be updated w/ that
expectation.
I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance
non-Linux tests have similar quirks, but I'm not able to uncover those.
This is a speculative fix for TestRosetta.py which is currently failing
on Green Dragon.
TestRosetta just makes sure we can debug an x86_64 process on Apple
Silicon. However, we're failing to build the x86_64 test binary. The
linker is failing with some warnings about libc++ and libunwind being
build for arm64 while the target binary is x86_64. I'm going to try
building with the system standard libraries instead of the just-built
ones to workaround it.
The "kern ver str" LC_NOTE gives lldb a kernel version string -- with a
UUID and/or a load address (stext) to load it at. The LC_NOTE specifies
a size of the identifier string in bytes. In
ObjectFileMachO::GetIdentifierString, I copy that number of bytes into a
std::string, and in case there were additional nul characters at the end
of the sting for padding reasons, I tried to shrink the std::string to
not include these extra nul's.
However, I did this resizing without handling the case of an empty
identifier string. I don't know why any corefile creator would do that,
but of course at least one does. This patch removes the resizing
altogether; I was solving something that hasn't ever shown to be a
problem. I also added a test case for this, to check that lldb doesn't
crash when given one of these corefiles.
rdar://120390199
Add a new LC_NOTE for Mach-O corefiles, "proces metadata", which is a
JSON string. Currently there may be a `threads` key in the JSON,
and if `threads` is present, it is an array with the same number of
elements as there are LC_THREADs in the corefile. This patch adds
support for a `thread_id` key-value for each `thread` entry, to
supply a thread ID for that LC_THREAD.
Differential Revision: https://reviews.llvm.org/D158785
rdar://113037252
DynamicLoader::LoadBinaryWithUUIDAndAddress can create a Module based
on the binary image in memory, which in some cases contains symbol
names and can be genuinely useful. If we don't have a filename, it
creates a name in the form `memory-image-0x...` with the header address.
In practice, this is most useful with Darwin userland corefiles
where the binary was stored in the corefile in whole, and we can't
find a binary with the matching UUID. Using the binary out of
the corefile memory in this case works well.
But in other cases, akin to firmware debugging, we merely end up
with an oddly named binary image and no symbols.
Add a flag to control whether we will create these memory images
and add them to the Target or not; only set it to true when working
with a userland Mach-O image with the "all image infos" LC_NOTE for
a userland corefile.
Differential Revision: https://reviews.llvm.org/D157167
Support recursive record types in CTF, for example a struct that
contains a pointer to itself:
struct S {
struct S *n;
};
We are now more lazy when creating LLDB types. When encountering a
record type (struct or union) we create a forward declaration and only
complete it when requested.
Differential revision: https://reviews.llvm.org/D156498
Fix parsing of large structs. If the size of a struct exceeds a certain
threshold, the offset is encoded using two 32-bit integers instead of
one.
Differential revision: https://reviews.llvm.org/D156490
Separate parsing CTF and creating LLDB types. This is a prerequisite to
parsing forward references and recursive types.
Differential revision: https://reviews.llvm.org/D156447
Add support for compressed CTF data. The flags in the header can
indicate whether the CTF body is compressed with zlib deflate. This
patch supports inflating the data before parsing.
Differential revision: https://reviews.llvm.org/D155221
Add support for the Compact C Type Format (CTF) in LLDB. The format
describes the layout and sizes of C types. It is most commonly consumed
by dtrace.
We generate CTF for the XNU kernel and want to be able to use this in
LLDB to debug kernels for which we don't have dSYMs (anymore). CTF is a
much more limited debug format than DWARF which allows is to be an order
of magnitude smaller: a 1GB dSYM can be converted to a handful of
megabytes of CTF. For XNU, the goal is not to replace DWARF, but rather
to have CTF serve as a "better than nothing" debug info format when
DWARF is not available.
It's worth noting that the LLVM toolchain does not support emitting CTF.
XNU uses ctfconvert to generate CTF from DWARF which is used for
testing.
Differential revision: https://reviews.llvm.org/D154862
ObjectFileMachO::SetLoadAddress() should allow for a DATA segment
that has no file content to be slid in the vmaddr, it is valid
to have such a section.
Differential Revision: https://reviews.llvm.org/D154037
rdar://99744343
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
