42 Commits

Author SHA1 Message Date
Jim Ingham
b9e08cbf30 I can't make this test fail locally, but it is failing on the macOS
bots as well.  I'll have to figure out a different way to test this.
2023-09-21 13:30:45 -07:00
Jim Ingham
de94c109b6 The test: test_run_then_attach_wait_interrupt was flakey on Linux & Windows.
I changed the test so I could tell whether the problem was sometimes the
interrupt was failing, or maybe the was just racy.  It failed again, but
in the new failures we waited 20 seconds for the attach-wait to get interrupted
and that never happened.

So there seems to be some real raciness in the feature of interrupting an
attach-wait, but only on Linux & Windows.  The bug fix that this test was
testing was for a bug that would cause us to never try to interrupt in this
case.  So it looks like this test is uncovering some flakiness in the underlying
interrupt support when in this state.  That's a separate bug that needs fixing.

For now, I disabled the test except on macOS where it seems to run reliably.
2023-09-20 17:38:27 -07:00
Daniel Thornburgh
7f3467412e Revert "Remove some raciness from the TestProcessAttach.test_run_then_attach_wait_interrupt"
This reverts commit df93c4ffdf220630ecceba5c9a7822c0aca7deaa.
This change is breaking the LLDB CI builders, e.g.
https://lab.llvm.org/buildbot/#/builders/68/builds/60350
2023-09-20 14:30:24 -07:00
Jim Ingham
df93c4ffdf Remove some raciness from the TestProcessAttach.test_run_then_attach_wait_interrupt
command.

We were reading the command output right after sending the interrupt, but
sometimes that wasn't long enough for the command result text to have been emitted.
I added a poll for the state change to eStateExited, and then added a bit more sleep
to give the command a chance to complete.
2023-09-20 13:08:40 -07:00
David Spickett
1446e3cf76 Revert "Fix a bug with cancelling "attach -w" after you have run a process previously (#65822)"
This reverts commit 7265f792dc8e1157a3874aee5f8aed6d4d8236e7.

The new test case is flaky on Linux AArch64 (https://lab.llvm.org/buildbot/#/builders/96)
and more flaky on Windows on Arm (https://lab.llvm.org/buildbot/#/builders/219/builds/5735).
2023-09-20 08:19:53 +00:00
Jim Ingham
74338bfe0c A test was changing directory and then incorrectly restoring the directory
to the "testdir" which is the build directory for that test, not the
original source directory.  That caused subsequent tests to fail.
2023-09-19 16:46:08 -07:00
jimingham
7265f792dc
Fix a bug with cancelling "attach -w" after you have run a process previously (#65822)
The problem is that the when the "attach" command is initiated, the
ExecutionContext for the command has a process - it's the exited one
from the previour run. But the `attach wait` creates a new process for
the attach, and then errors out instead of interrupting when it finds
that its process and the one in the command's ExecutionContext don't
match.

This change checks that if we're returning a target from
GetExecutionContext, we fill the context with it's current process, not
some historical one.
2023-09-19 11:25:53 -07:00
Jonas Devlieghere
2238dcc393
[NFC][Py Reformat] Reformat python files in lldb
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.

RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Differential revision: https://reviews.llvm.org/D151460
2023-05-25 12:54:09 -07:00
David Spickett
193259cbce [LLDB] Remove __future__ imports from tests
Not needed now that we require python 3.

Reviewed By: kastiglione, JDevlieghere

Differential Revision: https://reviews.llvm.org/D131761
2022-08-15 08:54:06 +00:00
Dave Lee
56f9cfe30c [lldb] Remove uses of six module (NFC)
With lldb (& llvm) requiring Python 3.6+, use of the `six` module can be removed.

Differential Revision: https://reviews.llvm.org/D131304
2022-08-11 19:06:15 -07:00
Jonas Devlieghere
0f821339da
[lldb] Add assertStopReason helper function
Add a function to make it easier to debug a test failure caused by an
unexpected stop reason. This is similar to the assertState helper that
was added in ce825e46743b.

Before:

  self.assertEqual(stop_reason, lldb.eStopReasonInstrumentation)
  AssertionError: 5 != 10

After:

  self.assertStopReason(stop_reason, lldb.eStopReasonInstrumentation)
  AssertionError: signal (5) != instrumentation (10)

Differential revision: https://reviews.llvm.org/D131083
2022-08-03 11:44:13 -07:00
Dave Lee
4655400b21 [lldb] Delete more mydir references (NFC) 2022-07-10 18:56:06 -07:00
Jonas Devlieghere
1b8c73522e
[lldb] Use assertState in even more tests (NFC)
Followup to D127355 and D127378, converting more instances of
assertEqual to assertState.
2022-06-29 17:05:58 -07:00
Jim Ingham
4298b1b8d1 Add a "-b" option to "process continue" to run to a set of breakpoints,
temporarily ignoring the others.

Differential Revision: https://reviews.llvm.org/D126513
2022-06-22 09:55:30 -07:00
Dave Lee
4cc8f2a017 [lldb][tests] Automatically call compute_mydir (NFC)
Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.

Differential Revision: https://reviews.llvm.org/D128077
2022-06-17 14:34:49 -07:00
Jonas Devlieghere
ce825e4674
[lldb] Add assertState function to the API test suite
Add a function to make it easier to debug a test failure caused by an
unexpected state.

Currently, tests are using assertEqual which results in a cryptic error
message: "AssertionError: 5 != 10". Even when a test provides a message
to make it clear why a particular state is expected, you still have to
figure out which of the two was the expected state, and what the other
value corresponds to.

We have a function in lldbutil that helps you convert the state number
into a user readable string. This patch adds a wrapper around
assertEqual specifically for comparing states and reporting better error
messages.

The aforementioned error message now looks like this: "AssertionError:
stopped (5) != exited (10)". If the user provided a message, that
continues to get printed as well.

Differential revision: https://reviews.llvm.org/D127355
2022-06-08 16:16:38 -07:00
Jim Ingham
134d7f9a4b Store a by name list of signals with their actions in the Target
so that they can be used to prime new Process runs.  "process handle"
was also changed to populate the dummy target if there's no selected
target, so that the settings will get copied into new targets.

Differential Revision: https://reviews.llvm.org/D126259
2022-05-26 14:50:33 -07:00
Stella Stamenova
58917054c2 [lldb] Skip several lldb tests that are flaky on Windows
These tests fail every 10 or so runs on Windows causing both local failures as well as buildbot failures.

Differential Revision: https://reviews.llvm.org/D111659
2021-10-13 09:46:41 -07:00
Jonas Devlieghere
b505ed9d31 [lldb] Remove support for replaying the test suite from a reproducer
This patch removes the infrastructure to replay the test suite from a
reproducer, as well as the modifications made to the individual tests.
2021-09-30 10:47:19 -07:00
Jim Ingham
2303391d1f Make "process attach -c" work correctly, and add a test for it.
The issue here was that we were not updating the interpreter's
execution context when calling HandleCommand to continue the process.
Since we had just created the process, it wasn't in the interpreter's
execution context so HandleCommand failed at CheckRequirements.  The
patch fixes that by passing the process execution context directly
to HandleCommand.

Differential Revision: https://reviews.llvm.org/D110787
2021-09-29 19:38:09 -07:00
Adrian Prantl
648844fd69 Make testcase more robust against codegen changes 2021-07-29 16:23:13 -07:00
Adrian Prantl
0fd813cf19 Fix typo 2021-07-29 16:23:13 -07:00
Adrian Prantl
26ba774f68 Simplify testcase to use v instead of p (NFC) 2021-07-29 15:15:00 -07:00
Raphael Isemann
54c2687292 [lldb] Introduce createTestTarget for creating a valid target in API tests
At the moment nearly every test calls something similar to
`self.dbg.CreateTarget(self.getBuildArtifact("a.out"))` and them sometimes
checks if the created target is actually valid with something like
`self.assertTrue(target.IsValid(), "some useless text")`.

Beside being really verbose the error messages generated by this pattern are
always just indicating that the target failed to be created but now why.

This patch introduces a helper function `createTestTarget` to our Test class
that creates the target with the much more verbose `CreateTarget` overload that
gives us back an SBError (with a fancy error). If the target couldn't be created
the function prints out the SBError that LLDB returned and asserts for us. It
also defaults to the "a.out" build artifact path that nearly all tests are using
to avoid to hardcode "a.out" in every test.

I converted a bunch of tests to the new function but I'll do the rest of the
test suite as follow ups.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D102771
2021-05-24 16:18:44 +02:00
Dave Lee
0ed758b260 [lldb] Convert more assertTrue to assertEqual (NFC)
Follow up to D95813, this converts multiline assertTrue to assertEqual.

Differential Revision: https://reviews.llvm.org/D95899
2021-02-03 21:15:08 -08:00
Stella Stamenova
3471455384 [lldb/test] Skip TestProcessAttach: test_attach_to_process_from_different_dir_by_id on Windows
This test is flakey on Windows and on failure it hangs causing the test suite to fail and future builds (on the buildbot, especially) to fail because they cannot re-write the files that are currently in use
2021-01-19 14:09:09 -08:00
Michał Górny
99562332e3 [lldb] [test] Update test status for NetBSD 2020-12-20 11:32:43 +01:00
Michał Górny
dbfdb139f7 [lldb] [POSIX-DYLD] Update the cached exe path after attach
Fix the POSIX-DYLD plugin to update the cached executable path after
attaching.  Previously, the path was cached in DYLDRendezvous
constructor and not updated afterwards.  This meant that if LLDB was
attaching to a process (e.g. via connecting to lldb-server), the code
stored the empty path before DidAttach() resolved it.  The fix updates
the cached path in DidAttach().

This fixes a new instance of https://llvm.org/pr17880

Differential Revision: https://reviews.llvm.org/D92264
2020-12-17 09:31:22 +01:00
Michał Górny
1a1cc0ba7d [lldb] [Platform/POSIX] Use gdb-remote plugin when attaching
Force gdb-remote plugin when attaching using the derivatives
of PlatformPOSIX class.  This is consistent with the behavior
for launching processes (via DebugProcess() method) and guarantees
consistent plugin choice on FreeBSD.

Differential Revision: https://reviews.llvm.org/D92667
2020-12-07 09:56:51 +01:00
Michał Górny
266c90fec8 [lldb] [test] Link FreeBSD test failures to bugs
Differential Revision: https://reviews.llvm.org/D92740
2020-12-07 09:56:50 +01:00
Raphael Isemann
d85cc03c9c [lldb] Add expect_var_path to test variable path results
This adds `expect_var_path` to test variable paths so we no longer have to
use `frame var` and find substrs in the command output. The behaviour
is identical with `expect_expr` (and it also uses the same checking backend),
but it instead calls `GetValueForVariablePath` to evaluate the string as a variable
path.

Also rewrites a few of the tests that previously used `frame variable` to use
`expect_var_path`.

Reviewed By: labath

Differential Revision: https://reviews.llvm.org/D90450
2020-11-12 16:14:48 +01:00
Raphael Isemann
66ae40ebfb [lldb][test] Remove not_remote_testsuite_ready in favor of skipIfRemote decorator
Those two decorators have identical behaviour. This removes
`not_remote_testsuite_ready` as `skipIfRemote` seems more consistent with the
other decorator names we have

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D89376
2020-11-11 09:14:54 +01:00
Michał Górny
f21e704d4a [lldb] [Process/NetBSD] Copy the recent improvements from FreeBSD
Copy the recent improvements from the FreeBSDRemote plugin, notably:

- moving event reporting setup into SetupTrace() helper

- adding more debug info into SIGTRAP handling

- handling user-generated (and unknown) SIGTRAP events

- adding missing error handling to the generic signal handler

- fixing attaching to processes

- switching watchpoint helpers to use llvm::Error

- minor style and formatting changes

This fixes a number of tests, mostly related to fixed attaching.

Differential Revision: https://reviews.llvm.org/D91167
2020-11-10 20:20:44 +01:00
Michał Górny
98257c3006 [lldb] [test] Update XFAILs/skips for FreeBSD
Update expected failures and test skips based on common results
for the old and new FreeBSD plugins.
2020-11-03 22:01:59 +01:00
Jonas Devlieghere
4aafc479f2 [lldb/Test] Always set the cleanupSubprocesses tear down hook
Always clean up subprocesses on tear down instead of relying on the
caller to do so. This is not only less error prone but also means the
tests can be more concise.

Differential revision: https://reviews.llvm.org/D83787
2020-07-14 14:05:56 -07:00
Raphael Isemann
9010cef2af [lldb] Replace StringConvert with llvm::to_integer when parsing integer values in CommandObjects
Summary:

This replaces the current use of LLDB's own `StringConvert` with LLVM's
`to_integer` which has a less error-prone API and doesn't use special 'error
values' to designate parsing problems.

Where needed I also added missing error handling code that prints a parsing
error instead of continuing with the error value returned from `StringConvert`
(which either gave a cryptic error message or just took the error value
performed an incorrect action with it. For example, `frame recognizer delete -1`
just deleted the frame recognizer at index 0).

Reviewers: #lldb, labath

Reviewed By: labath

Subscribers: labath, abidh, JDevlieghere

Differential Revision: https://reviews.llvm.org/D82297
2020-07-01 17:19:31 +02:00
Jonas Devlieghere
5f97a540ad [lldb/Reproducers] Differentiate active and passive replay unexpected packet. 2020-05-27 13:52:38 -07:00
Jonas Devlieghere
bf02bcffcf [lldb/Test] Modify more tests for API replay
Skip tests or parts thereof that aren't expected to work when run from a
reproducer. Also improve the doc comments in configuration.py to prevent
mistakes in the future.
2020-05-14 17:58:27 -07:00
Jonas Devlieghere
13062d0fb7 [lldb/Test] Skip more tests that are not expected to work with passive replay
This skips some tests that pass with active replay (which doesn't check
the output) but fail with passive replay. Valid reasons for this
include:

 - Checking the output of the process (which doesn't run during replay),
 - Checking files that cannot be captured in the VFS (non-existing or
   unreadable files or files that are removed during test),

Unfortunately there's no good way to mark a test as supported for active
replay but unsupported for passive replay because the number and order
of API calls needs to be identical during capture and replay. I don't
think this is a huge loss however.
2020-05-07 15:16:52 -07:00
Jonas Devlieghere
1e566f6b47 [lldb/Test] Add skipIfReproducer for tests that diverge during replay
Add the skipIfReproducer decorator to the remaining tests that fail to
replay because the GDB remote packets diverge during replay. This is
*not* expected and should be fixed, but figuring out exactly what caused
the divergence has proven pretty difficult to track down.

I've marked these tests as skipped for now so we can get clean results
and detect new regressions. I have no evidence to believe that these
failures have the same root cause, so I've not assigned them a PR.
2020-04-21 14:42:14 -07:00
Raphael Isemann
b3a0c4d7dc [lldb] Replace assertTrue(a == b, "msg") with assertEquals(a, b, "msg") in the test suite
Summary:
The error message from the construct `assertTrue(a == b, "msg") ` are nearly always completely useless for actually debugging the issue.
This patch is just replacing this construct (and similar ones like `assertTrue(a != b, ...)` with the proper call to assertEqual or assertNotEquals.

This patch was mostly written by a shell script with some manual verification afterwards:
```
lang=python
import sys

def sanitize_line(line):
  if line.strip().startswith("self.assertTrue(") and " == " in line:
    line = line.replace("self.assertTrue(", "self.assertEquals(")
    line = line.replace(" == ", ", ", 1)
  if line.strip().startswith("self.assertTrue(") and " != " in line:
    line = line.replace("self.assertTrue(", "self.assertNotEqual(")
    line = line.replace(" != ", ", ", 1)
  return line

for a in sys.argv[1:]:
  with open(a, "r") as f:
    lines = f.readlines()
  with open(a, "w") as f:
    for line in lines:
      f.write(sanitize_line(line))
```

Reviewers: labath, JDevlieghere

Reviewed By: labath

Subscribers: abidh, lldb-commits

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D74475
2020-02-13 15:00:55 +01:00
Jordan Rupprecht
99451b4453 [lldb][test] Remove symlink for API tests.
Summary: Moves lldbsuite tests to lldb/test/API.

This is a largely mechanical change, moved with the following steps:

```
rm lldb/test/API/testcases
mkdir -p lldb/test/API/{test_runner/test,tools/lldb-{server,vscode}}
mv lldb/packages/Python/lldbsuite/test/test_runner/test lldb/test/API/test_runner
for d in $(find lldb/packages/Python/lldbsuite/test/* -maxdepth 0 -type d | egrep -v "make|plugins|test_runner|tools"); do mv $d lldb/test/API; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-vscode -maxdepth 1 -mindepth 1 | grep -v ".py"); do mv $d lldb/test/API/tools/lldb-vscode; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-server -maxdepth 1 -mindepth 1 | egrep -v "gdbremote_testcase.py|lldbgdbserverutils.py|socket_packet_pump.py"); do mv $d lldb/test/API/tools/lldb-server; done
```

lldb/packages/Python/lldbsuite/__init__.py and lldb/test/API/lit.cfg.py were also updated with the new directory structure.

Reviewers: labath, JDevlieghere

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D71151
2020-02-11 10:03:53 -08:00