## Issue
Attempting to run the lldb API tests against a remote-android target
fails with the error `NameError: name 'urlparse' is not defined`.
## Root Cause
It looks the Python import of `urlparse` was removed by mistake in
22ea97d7bfd65abf68a68b13bf96ad69be23df54. This import is only used when
running the lldb API tests against a remote-android target so it went
unnoticed.
## Fix
This change simply puts back the missing import. It is a one line
change.
fixes#99931
## Validation
Tested on Fedora 39 with an attached Android device:
`cd llvm-project`
`cmake -S llvm -B build -G Ninja -DLLVM_ENABLE_PROJECTS='clang;lldb'
-DCMAKE_BUILD_TYPE=Release -DLLDB_ENABLE_PYTHON=On`
`ninja -C build`
`./build/bin/lldb-dotest --arch aarch64 --out-of-tree-debugserver
--platform-name=remote-android
--platform-working-dir=/data/local/tmp/ds2
--platform-url=connect://localhost:5432 --compiler
~/Android/Sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/bin/clang`
In MinGW make, the `%windir%` variable has a lowercase name `$(windir)`
when launched from cmd.exe, and `$(WINDIR)` name when launched from
MSYS2, since MSYS2 represents standard Windows environment variables
names in upper case.
This commit makes Makefile.rules consider both run variants.
Remove commands for OS/HOST_OS detection from Makefile.rules to simplify
it, since logic for these variables has been implemented in
`lldb/packages/Python/lldbsuite/test/lldbplatformutil.py`
(7021e44b2f0e11717c0d82456bad0fed4a0b48f9).
This is motivated by the upcoming refactor of libc++'s
`__compressed_pair` in https://github.com/llvm/llvm-project/pull/76756
As this will require changes to numerous LLDB libc++ data-formatters
(see early draft https://github.com/llvm/llvm-project/pull/96538), it
would be nice to have a test-suite that will actually exercise both the
old and new layout. We have a matrix bot that tests old versions of
Clang (but currently those only date back to Clang-15). Having them in
the test-suite will give us quicker signal on what broke.
We have an existing test that exercises various layouts of `std::string`
over time in `TestDataFormatterLibcxxStringSimulator.py`, but that's the
only STL type we have it for. This patch proposes a new
`libcxx-simulators` directory which will take the same approach for all
the STL types that we can feasibly support in this way (as @labath
points out, for some types this might just not be possible due to their
implementation complexity). Nonetheless, it'd be great to have a record
of how the layout of libc++ types changed over time.
Some related discussion:
*
https://github.com/llvm/llvm-project/pull/97568#issuecomment-2213426804
This reverts commit 2fa1220a37a3f55b76a29803d8333b3a3937d53a.
This reverts commit b9496a74eb4029629ca2e440c5441614e766f773.
The patch #98344 causes a crash in LLDB when parsing some files like `numpy.libs/libgfortran-daac5196.so.5.0.0` on graviton (you can download it in https://drive.google.com/file/d/12ygLjJwWpzdYsrzBPp1JGiFHxcgM0-XY/view?usp=drive_link if you want to troubleshoot yourself).
The assert that is hit is the following:
```
llvm-project/lldb/source/Plugins/ObjectFile/ELF/ObjectFileELF.cpp:2452: std::pair<unsigned int, std::map<long unsigned int, lldb_private::AddressClass> > ObjectFileELF::ParseSymbolTable(lldb_private::Symtab*, lldb::user_id_t, lldb_private::Section*): Assertion `strtab->GetObjectFile() == this' failed.
[383588:383636:20240716,025305.572639:ERROR crashpad_client_linux.cc:780] Crashpad isn't enabled
```
This object file doesn't have apparently a strings table but LLDB still tries to process it due to the code that is being reverted.
This is all the tests and fixes I've had percolating since my first
attempt at this in January. After 6 months of trying, I've given up on
adding the ability to test DWP files in LLDB API tests. I've left both
the tests (disabled) and the changes to Makefile.rules in place, in the
hopes that someone who can configure the build bots will be able to
enable the tests once a non-borked dwp tool is widely available.
Other than disabling the DWP tests, this continues to be the same diff
that I've tried to land and
[not](https://github.com/llvm/llvm-project/pull/90622)
[revert](https://github.com/llvm/llvm-project/pull/87676)
[five](https://github.com/llvm/llvm-project/pull/86812)
[times](https://github.com/llvm/llvm-project/pull/85693)
[before](https://github.com/llvm/llvm-project/pull/96802). There are a
couple of fixes that the testing exposed, and I've abandoned the DWP
tests because I want to get those fixes finally upstreamed, as without
them DebugInfoD is less useful.
Makefile.rules uses HOST_OS and OS variables for determining host and target
OSes for API tests compilation.
This commit moves the platform detection logic from Makefile to Python lldb
test suite.
This is useful for the case of Windows-to-Linux cross-testing.
Reverts llvm/llvm-project#96802
Attempt #5 fails. It's been 6 months. I despise Makefile.rules and have
no ability to even *detect* these failures without _landing_ a diff. In
the mean time, we have no testing for DWP files at all (and a regression
that was introduced, that I fix with this diff) so I'm going to just
remove some of the tests and try to land it again, but with less testing
I guess.
This is the same diff I've put up at many times before. I've been trying
to add some brand new functionality to the LLDB test infrastucture
(create split-dwarf files!), and we all know that no good deed goes
unpunished. The last attempt was reverted because it didn't work on the
Fuchsia build.
There are no code differences between this and
[the](https://github.com/llvm/llvm-project/pull/90622)
[previous](https://github.com/llvm/llvm-project/pull/87676)
[four](https://github.com/llvm/llvm-project/pull/86812)
[diffs](https://github.com/llvm/llvm-project/pull/85693) landed &
reverted (due to testing infra failures). The only change in this one is
the way `dwp` is being identified in `Makefile.rules`.
Thanks to @petrhosek for helping me figure out how the fuchsia builders
are configured. I now prefer to use llvm-dwp and fall back to gnu's dwp
if the former isn't found. Hopefully this will work everywhere it needs
to.
This test only runs on Windows and fails because we're passing a literal
of the wrong type to random.randrange.
Co-authored-by: kendal <kendal@thebrowser.company>
Adding a "port" property to the VsCode "attach" command likely extends
the functionality of the debugger configuration to allow attaching to a
process using PID or PORT number.
Currently, the "Attach" configuration lets the user specify a pid. We
tell the user to use the attachCommands property to run "gdb-remote ".
Followed the below conditions for "attach" command with "port" and "pid"
We should add a "port" property. If port is specified and pid is not,
use that port to attach. If both port and pid are specified, return an
error saying that the user can't specify both pid and port.
Ex - launch.json
{
"version": "0.2.0",
"configurations": [
{
"name": "lldb-dap Debug",
"type": "lldb-dap",
"request": "attach",
"gdb-remote-port":1234,
"program": "${workspaceFolder}/a.out",
"args": [],
"stopOnEntry": false,
"cwd": "${workspaceFolder}",
"env": [],
}
]
}
---------
Co-authored-by: Santhosh Kumar Ellendula <sellendu@hu-sellendu-hyd.qualcomm.com>
Co-authored-by: Santhosh Kumar Ellendula <sellendu@hu-sellendu-lv.qualcomm.com>
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
from PEP8
(https://peps.python.org/pep-0008/#programming-recommendations):
> Comparisons to singletons like None should always be done with is or
is not, never the equality operators.
Co-authored-by: Eisuke Kawashima <e-kwsm@users.noreply.github.com>
Use the packaging [1] module for parsing version numbers, instead of
pkg_resources which is distributed with setuptools. I recently switched
over to using the latter, knowing it was deprecated (in favor of the
packaging module) because it comes with Python out of the box. Newer
versions of setuptools have removed `pkg_resources` so we have to use
packaging.
[1] https://pypi.org/project/packaging/
This argument allows to specify the path to make which is used by
LLDB API tests to compile test programs.
It might come in handy for setting up cross-platform remote runs of API tests on Windows host.
It can be used to override the make path of LLDB API tests using `LLDB_TEST_USER_ARGS` argument:
```
cmake ...
-DLLDB_TEST_USER_ARGS="...;--make;C:\\Path\\to\\make.exe;..."
...
```
This argument allows to set specific sysroot pass which will be used for
building LLDB API test programs.
It might come in handy for setting up cross-platform remote runs of API
tests on Windows host.
It can be useful for cross-compiling LLDB API tests. The argument can be
set using `LLDB_TEST_USER_ARGS` argument:
```
cmake ...
-DLLDB_TEST_USER_ARGS="...;--sysroot;C:\path\to\sysroot;..."
...
```
The don't currently work (and they're also not particularly useful,
since all of the remote stuff happens inside lldb).
This saves us from annotating tests one by one.
Here we go with attempt number five. Again, no changes to the LLDB code
diff, which has been reviewed several times.
For the tests, I added a `@skipIfCurlSupportMissing` annotation so that
the Debuginfod mocked server stuff won't run, and I also disabled
non-Linux/FreeBSD hosts altogether, as they fail for platform reasons on
macOS and Windows. In addition, I updated the process for extracting the
GNU BuildID to no create a target, per some feedback on the previous
diff.
For reference, previous PR's (landed, backed out after the fact for
various reasons) #90622, #87676, #86812, #85693
---------
Co-authored-by: Kevin Frei <freik@meta.com>
runCmd() is called from Base.getCPUInfo() but implemented only in
TestBase(Base). Usually it works if TestBase is used. But call
getCPUInfo() from a class based on Base will cause something like
```
File "E:\projects\llvm-nino\lldb\llvm-project\lldb\packages\Python\lldbsuite\test\lldbtest.py", line 1256, in getCPUInfo
self.runCmd('platform get-file "/proc/cpuinfo" ' + cpuinfo_path)
AttributeError: 'TestGdbRemoteExpeditedRegisters' object has no attribute 'runCmd'
```
BTW, TestBase.setUp() called runCmd() before applying
LLDB_MAX_LAUNCH_COUNT and LLDB_TIME_WAIT_NEXT_LAUNCH.
This patch fixes the test TestGdbRemoteExpeditedRegisters in case of
Windows host and Linux target.
A lot of `TestConcurrent*.py` expect one of the threads to crash, but we
weren't checking for it properly.
Possibly because signal reporting got better on FreeBSD at some point,
and it now shows the same info as Linux does.
```
lldb-api :: functionalities/inferior-changed/TestInferiorChanged.py
lldb-api :: functionalities/inferior-crashing/TestInferiorCrashing.py
lldb-api :: functionalities/inferior-crashing/TestInferiorCrashingStep.py
lldb-api :: functionalities/inferior-crashing/recursive-inferior/TestRecursiveInferior.py
lldb-api :: functionalities/inferior-crashing/recursive-inferior/TestRecursiveInferiorStep.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithBreak.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithSignal.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpoint.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpointBreakpointSignal.py
```
Fixes#48777
`TestConcurrentTwoBreakpointsOneSignal.py` no longer fails, at least on
an AWS instance, so I've removed the xfail there.
While adding a UI feature in VSCode to toggle hex/dec in variables view
window. I noticed that it does not work after second toggle. Then I
noticed that there is a bug that we only explicitly set hex format not
reset back to default during further toggle. The new test demonstrates
the bug.
This PR resets the format back to default if not using hex. One
complexity is that, we explicitly set registers value format to
AddressInfo, which shouldn't be overridden by default or hex settings.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
And "LLDB Debuginfod tests and a fix or two (#90622)".
f8fedfb6802173372ec923f99f31d4af810fbcb0 /
2d4acb086541577ac6ab3a140b9ceb9659ce7094
As it has caused a test failure on 32 bit Arm:
https://lab.llvm.org/buildbot/#/builders/17/builds/52580
Expr/TestStringLiteralExpr.test. The follow up did fix
lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
but not the other failure.
I'm taking yet another swing at getting these tests going, on the
hypothesis that the problems with buildbots & whatnot are because
they're not configured with CURL support, which I've confirmed would
cause the previous tests to fail. (I have no access to an ARM64 linux
system, but I did repro the failure on MacOS configured without CURL
support)
So, the only difference between this diff and
[previous](https://github.com/llvm/llvm-project/pull/85693)
[diffs](https://github.com/llvm/llvm-project/pull/87676) that have
already been approved is that I've added a condition to the tests to
only run if Debuginfod capabilities should be built into the binary. I
had done this for these tests when they were [Shell
tests](https://github.com/llvm/llvm-project/pull/79181) and not API
tests, but I couldn't find a direct analog in any API test, so I used
the "plugins" model used by the intel-pt tests as well.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
using the macOS version as a proxy. I can't reproduce any of these
failures locally, but the tests all use pexpect and probably have bad
timeout behavior under high load.
that separates out language and version. To avoid reinventing the wheel
and introducing subtle incompatibilities, this API uses the table of
languages and versiond defined by the upcoming DWARF 6 standard
(https://dwarfstd.org/languages-v6.html). While the DWARF 6 spec is not
finialized, the list of languages is broadly considered stable.
The primary motivation for this is to allow the Swift language plugin to
switch between language dialects between, e.g., Swift 5.9 and 6.0 with
out introducing a ton of new language codes. On the main branch this
change is considered NFC.
Depends on https://github.com/llvm/llvm-project/pull/89980
This patch provides the initial implementation for the "Step Into
Specific/Step In Targets" feature in VSCode DAP.
The implementation disassembles all the call instructions in step range
and try to resolve operand name (assuming one operand) using debug info.
Later, the call target function name is chosen by end user and specified
in the StepInto() API call.
It is v1 because of using the existing step in target function name API.
This implementation has several limitations:
* Won't for indirect/virtual function call -- in most cases, our
disassembler won't be able to solve the indirect call target
address/name.
* Won't work for target function without debug info -- if the target
function has symbol but not debug info, the existing
ThreadPlanStepInRange won't stop.
* Relying on function names can be fragile -- if there is some middle
glue/thunk code, our disassembler can only resolve the glue/thunk code's
name not the real target function name. It can be fragile to depend
compiler/linker emits the same names for both.
* Does not support step into raw address call sites -- it is a valid
scenario that in Visual Studio debugger, user can explicitly choose a
raw address to step into which land in the function without debug
info/symbol, then choose UI to load the debug info on-demand for that
module/frame to continue exploring.
A more reliable design could be extending the ThreadPlanStepInRange to
support step in based on call-site instruction offset/PC which I will
propose in next iteration.
---------
Co-authored-by: jeffreytan81 <jeffreytan@fb.com>
The `LLDB_TEST_USE_VENDOR_PACKAGES` has defaulted to `Off` for a while.
Either installing `pexpect` or skipping those tests with
`-DLLDB_TEST_USER_ARGS=--skip-category=pexpect` seems to be enough that
we can fully remove this option.
This patch removes the `LLDB_TEST_USE_VENDOR_PACKAGES` cmake
configuration as well as the associated code to add
`third_party/Python/module` to the python path. I'll do the actual
deletion of `third_party/Python/module` in a followup PR in the
(unlikely, I hope) event this commit needs to be reverted.
I was debugging space separation issues when passing user arguments
and noticed this error is really hard to read in that scenario.
Put "" around the invalid compiler name so you can tell whether
you have spaces around it that's causing the problem.
This enables testing of the LLDB libc++ specific data formatters.
This is enabled in the bootstrap build since building LLDB requires
Clang and this
is quite expensive. Adding this test changes the build time from 31 to
34 minutes.
This reverts commit d6713ad80d6907210c629f22babaf12177fa329c.
This changed was reverted because of greendragon failures such
as
Unresolved Tests (2):
lldb-api :: debuginfod/Normal/TestDebuginfod.py
lldb-api :: debuginfod/SplitDWARF/TestDebuginfodDWP.py
I believe I've got the tests properly configured to only run on Linux
x86(_64), as I don't have a Linux AArch64/Arm device to diagnose what's
going wrong with the tests (I suspect there's some issue with generating
`.note.gnu.build-id` sections...)
The actual code fixes have now been reviewed 3 times:
https://github.com/llvm/llvm-project/pull/79181 (moved shell tests to
API tests), https://github.com/llvm/llvm-project/pull/85693 (Changed
some of the testing infra), and
https://github.com/llvm/llvm-project/pull/86812 (didn't get the tests
configured quite right). The Debuginfod integration for symbol
acquisition in LLDB now works with the `executable` and `debuginfo`
Debuginfod network requests working properly for normal, `objcopy
--only-keep-debug` stripped, split-dwarf, and `objcopy
--only-keep-debug` stripped *plus* split-dwarf symbols/binaries.
The reasons for the multiple attempts have been tests on platforms I
don't have access to (Linux AArch64/Arm + MacOS x86_64). I believe I've
got the tests properly disabled for everything except for Linux x86(_64)
now. I've built & tested on MacOS AArch64 and Linux x86_64.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
The previous diff (and it's subsequent fix) were reverted as the tests
didn't work properly on the AArch64 & ARM LLDB buildbots. I made a
couple more minor changes to tests (from @clayborg's feedback) and
disabled them for non Linux-x86(_64) builds, as I don't have the ability
do anything about an ARM64 Linux failure. If I had to guess, I'd say the
toolchain on the buildbots isn't respecting the `-Wl,--build-id` flag.
Maybe, one day, when I have a Linux AArch64 system I'll dig in to it.
From the reverted PR:
I've migrated the tests in my
https://github.com/llvm/llvm-project/pull/79181 from shell to API (at
@JDevlieghere's suggestion) and addressed a couple issues that were
exposed during testing.
The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when DebugInfoD returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Finally getting back to Debuginfod tests:
I've migrated the tests in my [earlier
PR](https://github.com/llvm/llvm-project/pull/79181) from shell to API
(at @JDevlieghere's suggestion) and addressed a couple issues that came
about during testing.
The tests first test the "normal" situation (no DebugInfoD involvement,
just normal debug files sitting around), then the "no debug info"
situation (to make sure the test is seeing failure properly), then it
tests to validate that when Debuginfod returns the symbols, things work
properly. This is duplicated for DWP/split-dwarf scenarios.
---------
Co-authored-by: Kevin Frei <freik@meta.com>
Instead of directly annotating pexpect-based tests with
`@skipIfWindows`, we can tag them with a new `pexpect` category. We
still automatically skip windows behavior by adding `pexpect` to the
skip category list if the platform is windows, but also allow
non-Windows users to skip them by configuring cmake with
`-DLLDB_TEST_USER_ARGS=--skip-category=pexpect`
As a prerequisite, remove the restriction that `@add_test_categories`
can only apply to test cases, and we make the test runner look for
categories on both the class and the test method.
When Apple released its new linker, it had a subtle bug that caused
LLDB's TLS tests to fail. Unfortunately this means that TLS tests are
not going to work on machines that have affected versions of the linker,
so we should annotate the tests so that they only work when we are
confident the linker has the required fix.
I'm not completely satisfied with this implementation. That being said,
I believe that adding suport for linker versions in general is a
non-trivial change that would require far more thought. There are a few
challenges involved:
- LLDB's testing infra takes an argument to change the compiler, but
there's no way to switch out the linker.
- There's no standard way to ask a compiler what linker it will use.
- There's no standard way to ask a linker what its version is. Many
platforms have the same name for their linker (ld).
- Some platforms automatically switch out the linker underneath you. We
do this for Windows tests (where we use LLD no matter what).
Given that this is affecting the tests on our CI, I think this is an
acceptable solution in the interim.
The timeout for this test was set to 1.0s which is very low, it should
be a default of 10s and be increased by a factor of 10 if ASAN is
enabled. This will help reduce the falkiness of the test, especially in
ASAN builds.
For whatever reason on Windows, it is not at this point.
The copy of unit test we used to use would ignore failures during
teardown but Python's does not.
This implements functionality to handle DataBreakpointInfo request and
SetDataBreakpoints request.
Previous commit
8c56e78ec5
was reverted because setting 1 byte watchpoint failed in the new test on
ARM64. So, I changed the test to setting 4 byte watchpoint instead, and
hope this won't break it again. It also adds the fixes from
https://github.com/llvm/llvm-project/pull/81680.
