Fixes#161510
Addresses the Linux parts of #138085
The situation we have to handle here is systems where Yama ptrace_scope
set to 1.
> 1 - restricted ptrace: a process must have a predefined relationship
> with the inferior it wants to call PTRACE_ATTACH on. By default,
> this relationship is that of only its descendants when the above
> classic criteria is also met. To change the relationship, an
> inferior can call prctl(PR_SET_PTRACER, debugger, ...) to declare
> an allowed debugger PID to call PTRACE_ATTACH on the inferior.
> Using PTRACE_TRACEME is unchanged.
(https://www.kernel.org/doc/Documentation/security/Yama.txt)
The inferior was addressing this by calling this at the start of main():
prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY, 0, 0, 0);
Which is ok if lldb-server tries to attach after that call has happened,
but there was nothing to synchronise this. So if the system was heavily
loaded, the inferior may be stalled, delaying the call, causing
lldb-server to fail to attach with EPERM (permission denied).
We were not using any mechanism to retry the attach or wait for some
signal from the inferior.
Except we do do this in other tests, even other lldb-server tests.
So I have adopted that mechanism to these tests:
* The inferior is launched with `syncfile:<path>` as its first argument.
* It creates this file at `<path>`, at a point where we know attaching
has been allowed.
* The test framework launches the inferior then waits for the file to
appear.
* This check is retried a few times, increasing the delay each time and
eventually giving up.
* Only once it has seen the file does it start lldb-server and tell it
to attach to the inferior.
I have tested this by insterting a `sleep()` call before the attach
enable call and running the test on a machine with ptrace_scope set to
1. I was able to increase the sleep to 6 seconds before tests failed
(when running just these tests, single threaded).
With OS scheduling, you could be stalled indefinitely, so we may have to
increase this timeout but this is easy to do with
wait_for_file_on_target.
The alternative is to have the test runner check ptrace_scope and only
enable these on systems where it's 0. Would be good to keep them running
if we can though.
# Benefit
This patch fixes:
1. After `platform select ios-simulator`, `platform process list` will
now print processes which are running in the iOS simulator. Previously,
no process will be listed.
2. After `platform select ios-simulator`, `platform attach --name
<name>` will succeed. Previously, it will error out saying no process is
found.
# Several bugs that is being fixed
1. During the process listing, add `aarch64` to the list of CPU types
for which iOS simulators are checked for.
2. Given a candidate process, when checking for simulators, the original
code will find the desired environment variable (`SIMULATOR_UDID`) and
set the OS to iOS, but then the immediate next environment variable will
set it back to macOS.
3. For processes running on simulator, set the triple's `Environment` to
`Simulator`, so that such processes can pass the filtering [in this
line](https://fburl.com/8nivnrjx). The original code leave it as the
default `UnknownEnvironment`.
# Manual test
**With this patch:**
```
royshi-mac-home ~/public_llvm/build % bin/lldb
(lldb) platform select ios-simulator
(lldb) platform process list
240 matching processes were found on "ios-simulator"
PID PARENT USER TRIPLE NAME
====== ====== ========== ============================== ============================
40511 28844 royshi arm64-apple-ios-simulator FocusPlayground // my toy iOS app running on simulator
... // omit
28844 1 royshi arm64-apple-ios-simulator launchd_sim
(lldb) process attach --name FocusPlayground
Process 40511 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
frame #0: 0x0000000104e3cb70 libsystem_kernel.dylib`mach_msg2_trap + 8
libsystem_kernel.dylib`mach_msg2_trap:
-> 0x104e3cb70 <+8>: ret
... // omit
```
**Without this patch:**
```
$ bin/lldb
(lldb) platform select ios-simulator
(lldb) platform process list
error: no processes were found on the "ios-simulator" platform
(lldb) process attach --name FocusPlayground
error: attach failed: could not find a process named FocusPlayground
```
# Unittest
See PR.
The architectures provided to skipIf / expectedFail are regular
expressions (v. _match_decorator_property() in decorators.py
so on Darwin systems "arm64" would match the skips for "arm" (32-bit
Linux). Update these to "arm$" to prevent this, and also update
three tests (TestBuiltinFormats.py, TestCrossDSOTailCalls.py,
TestCrossObjectTailCalls.py) that were skipped for arm64 via this
behavior, and need to be skipped or they will fail.
This was moviated by the new TestDynamicValue.py test which has
an expected-fail for arm, but the test was passing on arm64 Darwin
resulting in failure for the CIs.
test_common is force-included into every compilation, which causes
problems when we're compiling assembly code, as we were in #138805.
This avoids that as we can include the header only when it's needed.
During lldb testing on remote targets TestGdbRemoteForkNonStop.py
freezes because in this test we try to create file on remote machine
using absolute file path from local machine. This patch fixes this error
In 2013 we added the QSaveRegisterState and QRestoreRegisterState
packets to checkpoint a thread's register state while executing an
inferior function call, instead of using the g packet to read all
registers into lldb, then the G packet to set them again after the func
call.
Since then, lldb has not sent g/G (except as a bug) - it either asks for
registers individually (p/P) or or asks debugserver to save and restore
the entire register set with these lldb extensions.
Felipe recently had a codepath that fell back to using g/G and found
that it does not work with the modern signed fp/sp/pc/lr registers that
we can get -- it sidesteps around the clearing of the non-addressable
bits that we do when reading/writing them, and results in a crash. (
https://github.com/llvm/llvm-project/pull/132079 )
Instead of fixing that issue, I am removing g/G from debugserver because
it's not needed by lldb, and it will would be easy for future bugs to
creep in to this packet that lldb should not use, but it can
accidentally fall back to and result in subtle bugs.
This does mean that a debugger using debugserver on darwin which doesn't
use QSaveRegisterState/QRestoreRegisterState will need to fall back to
reading & writing each register individually. I'm open to re-evaluating
this decision if this proves to be needed, and supporting these lldb
extensions is onerous.
Split test cases out of TestLldbGdbServer.py and TestGdbRemoteFork.py
into separate files to avoid hitting the 600s timeout limit. The
inferior used by these tests (main.cpp) takes approximately 20s to
compile with a Debug build of clang, causing timeouts when a single test
file contains many tests. By grouping similar tests into separate files,
we can prevent timeouts and improve overall test efficiency.
Some lldb tests from llgs category fail on RISC-V target due to lack of
necessary condition checks. This patch adapts these tests by taking into
account the peculiarities of the RISC-V architecture
During testing of LLDB on RISC-V target, tests from the llgs category
were built with an error: `Error when building test subject.`
```
llvm-project/lldb/test/API/tools/lldb-server/main.cpp:151:40: error: missing ')' after '__builtin_debugtrap'
151 | #elif __has_builtin(__builtin_debugtrap())
| ~~~~~~~~~~~~~~~~~~~^
llvm-project/lldb/test/API/tools/lldb-server/main.cpp:151:20: note: to match this '('
151 | #elif __has_builtin(__builtin_debugtrap())
| ^
```
This patch fixes this error.
When a test depends on a new debugserver feature/fix, the API test must
be marked @skipIfOutOfTreeDebugserver because the macOS CI bots test
using the latest Xcode release debugserver. But over time all of these
fixes & new features are picked up in the Xcode debugserver and these
skips can be removed.
We may see unexpected test failures from removing all of these 1+ year
old skips, but that's likely a separate reason the test is failing that
is being papered over by this skip.
This fix is based on a problem with cxx_compiler and cxx_linker macros
on Windows.
There was an issue with compiler detection in paths containing "icc". In
such case, Makefile.rules thought it was provided with icc compiler.
To solve that, utilities detection has been rewritten in Python.
The last element of compiler's path is separated, taking into account
the platform path delimiter, and compiler type is extracted, with regard
of possible cross-toolchain prefix.
---------
Co-authored-by: Pavel Labath <pavel@labath.sk>
Adding a "port" property to the VsCode "attach" command likely extends
the functionality of the debugger configuration to allow attaching to a
process using PID or PORT number.
Currently, the "Attach" configuration lets the user specify a pid. We
tell the user to use the attachCommands property to run "gdb-remote ".
Followed the below conditions for "attach" command with "port" and "pid"
We should add a "port" property. If port is specified and pid is not,
use that port to attach. If both port and pid are specified, return an
error saying that the user can't specify both pid and port.
Ex - launch.json
{
"version": "0.2.0",
"configurations": [
{
"name": "lldb-dap Debug",
"type": "lldb-dap",
"request": "attach",
"gdb-remote-port":1234,
"program": "${workspaceFolder}/a.out",
"args": [],
"stopOnEntry": false,
"cwd": "${workspaceFolder}",
"env": [],
}
]
}
---------
Co-authored-by: Santhosh Kumar Ellendula <sellendu@hu-sellendu-hyd.qualcomm.com>
Co-authored-by: Santhosh Kumar Ellendula <sellendu@hu-sellendu-lv.qualcomm.com>
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
Use the packaging [1] module for parsing version numbers, instead of
pkg_resources which is distributed with setuptools. I recently switched
over to using the latter, knowing it was deprecated (in favor of the
packaging module) because it comes with Python out of the box. Newer
versions of setuptools have removed `pkg_resources` so we have to use
packaging.
[1] https://pypi.org/project/packaging/
The local PTY is not available for the remotely executed lldb-server to
pass the test. Also, in general, we cannot execute the local lldb-server
instance because it could be compiled for the different system/cpu
target.
The TestGdbRemoteLibrariesSvr4Support test failed in case of Linux
x86_64 host and Linux Aarch64 target. Installing libraries to the remote
target is not enough. This test actively uses self.getBuildDir() and
os.path.realpath() which does not work in case of the remote target. So,
disable this test for remote target now.
The tests TestPty and TestPtyServer use the Unix specific python builtin
module termios. They are failed in case of Windows host and Linux
target. Disable them for Windows host too.
Windows path is case insensitive. Tests `test_QMemTags_packets` and
`test_qMemTags_packets` will use the same build dir and conflict. Added
a suffix to resolve conflicts.
It is necessary to transfer the test file to/from the really remote
target (for example Windows host and Linux target). Also ignore chmod
check in case of the Windows host.
On a case insensitive file sytem, the build dir for `test_multiple_c` and
`test_multiple_C` are the same and therefore the log files are in the same
place. This means one tries to clear the log file for the other.
To fix this, make the names unique by adding the meaning of each
protocol packet.
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Packets.html#Packets
This uses [teyit](https://pypi.org/project/teyit/) to modernize asserts,
as recommended by the [unittest release
notes](https://docs.python.org/3.12/whatsnew/3.12.html#id3).
For example, `assertTrue(a == b)` is replaced with `assertEqual(a, b)`.
This produces better error messages, e.g. `error: unexpectedly found 1
and 2 to be different` instead of `error: False`.
The distutils package has been deprecated and was removed from Python
3.12. The migration page [1] advises to use the packaging module
instead. Since Python 3.6 that's vendored into pkg_resources.
[1] https://peps.python.org/pep-0632/#migration-advice
The distutils package has been deprecated and was removed from Python
3.12. The migration page [1] advises to use the packaging module
instead. Since Python 3.6 that's vendored into pkg_resources.
[1] https://peps.python.org/pep-0632/#migration-advice
assertRegexpMatches is a deprecated alias for assertRegex and has been
removed in Python 3.12. This wasn't an issue previously because we used
a vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.
assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a
vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.
This removes the dependency LLDB API tests have on
lldb/third_party/Python/module/unittest2, and instead uses the standard
one provided by Python.
This does not actually remove the vendored dep yet, nor update the docs.
I'll do both those once this sticks.
Non-trivial changes to call out:
- expected failures (i.e. "bugnumber") don't have a reason anymore, so
those params were removed
- `assertItemsEqual` is now called `assertCountEqual`
- When a test is marked xfail, our copy of unittest2 considers failures
during teardown to be OK, but modern unittest does not. See
TestThreadLocal.py. (Very likely could be a real bug/leak).
- Our copy of unittest2 was patched to print all test results, even ones
that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped,
...)`, but standard unittest prints a terser message that omits test
result types that didn't happen, e.g. `OK (skipped=1)`. Our lit
integration parses this stderr and needs to be updated w/ that
expectation.
I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance
non-Linux tests have similar quirks, but I'm not able to uncover those.
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
This has been flaky on the Windows on Arm LLDB bot.
https://lab.llvm.org/buildbot/#/builders/219/builds/826
Given that test_stop_reply_reports_multiple_threads is already expected
to fail on Windows, this is not suprising.
So that the XML isn't one giant line. Which wasn't
a problem for lldb but was for me trying to troubleshoot
it using the logs.
It now looks like:
```
<?xml version="1.0"?>
<target version="1.0">
<architecture>aarch64</architecture>
<feature>
<...>
<reg name="fpcr" .../>
</feature>
</target>
```
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D134035
While auxv keys are usually small, e.g. less than 50, they can sometimes be larger, especially on a downstream kernel where a custom auxv entry is intentionally high to avoid conflicting with the standard lower numbers. This test fails on a system with an auxv value bigger than 1000, but instead of putting this test at that value plus one, it looks like 2023 (i.e. `AT_SUN_CAP_HW2`) is another large one out there. Use 2500 as a limit to still have this be a reasonable "small" check but still allow all known auxv keys.
Semi-related change: this test case prints the auxv dict at the trace level, but only _after_ the assertion fails, making it not print what the offending value is as the test case aborts. Move it earlier so we can see what the "unreasonable" auxv value is.
