Failing on the macOS matrix bot for Clang-15 with the following error:
```
07:16:08 FAIL: LLDB (/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/clang_1501_build/bin/clang-arm64) :: test_jump_offset_dwarf (TestThreadJump.ThreadJumpTestCase)
07:16:08 UNSUPPORTED: LLDB (/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/clang_1501_build/bin/clang-arm64) :: test_jump_offset_dwo (TestThreadJump.ThreadJumpTestCase) (test case does not fall in any category of interest for this run)
07:16:08 Restore dir to: /Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/lldb-build/tools/lldb/test
07:16:08 ======================================================================
07:16:08 FAIL: test_jump_offset_dsym (TestThreadJump.ThreadJumpTestCase)
07:16:08 Test Thread Jump by negative or positive offset
07:16:08 ----------------------------------------------------------------------
07:16:08 Traceback (most recent call last):
07:16:08 File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1804, in test_method
07:16:08 return attrvalue(self)
07:16:08 File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/test/API/functionalities/thread/jump/TestThreadJump.py", line 112, in test_jump_offset
07:16:08 self.expect(f"print {var_2}", substrs=[var_2_value])
07:16:08 File "/Users/ec2-user/jenkins/workspace/llvm.org/lldb-cmake-matrix/llvm-project/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 2512, in expect
07:16:08 self.fail(log_msg)
07:16:08 AssertionError: Ran command:
07:16:08 "print var_2"
07:16:08
07:16:08 Got output:
07:16:08 (int) 20
07:16:08
07:16:08 Expecting sub string: "40" (was not found)
```
Fixes#45326
When you thread jump by calling
`j +2` or `thread jump --by +2` the offset is not recognised. This
commit fixes that.
---------
Signed-off-by: Ebuka Ezike <yerimyah1@gmail.com>
The architectures provided to skipIf / expectedFail are regular
expressions (v. _match_decorator_property() in decorators.py
so on Darwin systems "arm64" would match the skips for "arm" (32-bit
Linux). Update these to "arm$" to prevent this, and also update
three tests (TestBuiltinFormats.py, TestCrossDSOTailCalls.py,
TestCrossObjectTailCalls.py) that were skipped for arm64 via this
behavior, and need to be skipped or they will fail.
This was moviated by the new TestDynamicValue.py test which has
an expected-fail for arm, but the test was passing on arm64 Darwin
resulting in failure for the CIs.
test_common is force-included into every compilation, which causes
problems when we're compiling assembly code, as we were in #138805.
This avoids that as we can include the header only when it's needed.
The implementation has an optimization which detects the range of line
table entries covered by the function and then only searches for a
matching line between them.
This optimization was interfering with the logic for detecting whether a
line belongs to the function because the first call to FindLineEntry was
made with exact=false, which meant that if the function did not contain
any exact matches, we would just pick the closest line number in that
range, even if it was very far away.
This patch fixes that by first attempting an inexact search across the
entire line table, and then use the (potentially inexact) result of that
for searching within the function. This makes the optimization a less
effective, but I don't think we can differentiate between a line that
belongs to the function (but doesn't have any code) and a line outside
the function without that.
The patch also avoids the use of (deprecated) Function::GetAddressRange
by iterating over the GetAddressRanges result to find the full range of
line entries for the function.
I will be changing breakpoint hitting behavior soon, where currently
lldb reports a breakpoint as being hit when a thread is *at* a
BreakpointSite, but possibly has not executed the breakpoint instruction
and trapped yet, to having lldb only report a breakpoint hit when the
breakpoint instruction has actually been executed.
One corner case bug with this change is that when you are stopped at a
breakpoint (that has been hit) on the last instruction of a function,
and you do `finish`, a ThreadPlanStepOut is pushed to the thread's plan
stack to put a breakpoint on the return address and resume execution.
And when the thread is asked to resume, it sees that it is at a
BreakpointSite that has been hit, and pushes a
ThreadPlanStepOverBreakpoint on the thread. The StepOverBreakpoint
plan sees that the thread's state is eStateRunning (not eStateStepping),
so it marks itself as "auto continue" -- so once the breakpoint has
been stepped over, we will execution on the thread.
With current lldb stepping behavior ("a thread *at* a BreakpointSite is
said to have stopped with a breakpoint-hit stop reason, even if the
breakpoint hasn't been executed yet"),
`ThreadPlanStepOverBreakpoint::DoPlanExplainsStop` has a special bit of
code which detects when the thread stops with a eStopReasonBreakpoint.
It first checks if the pc is the same as when we started -- did our
"step instruction" not actually step? -- says the stop reason is
explained. Otherwise it sets auto-continue to false (because we've hit
an *unexpected* breakpoint, and we have advanced past our original pc,
and returns false - the stop reason is not explained.
So we do the "finish", lldb instruction steps, we stop *at* the
return-address breakpoint and lldb sets the thread's stop reason to
breakpoint-hit. ThreadPlanStepOverBreakpoint sees an
eStopReasonBreakpoint, sets its auto-continue to false, and says we
stopped for osme reason other than this plan. (and it will also report
`IsPlanStale()==true` so it will remove itself) Meanwhile the
ThreadPlanStepOut sees that it has stopped in the StackID it wanted to
run to, and return success.
This all changes when stopping at a breakpoint site doesn't report
breakpoint-hit until we actually execute the instruction. Now the
ThraedPlanStepOverBreakpoint looks at the thread's stop reason, it's
eStopReasonTrace (we've instruction stepped), and so it leaves its
auto-continue to `true`. ThreadPlanStepOut sees that it has reached its
goal StackID, removes its breakpoint, and says it is done.
Thread::ShouldStop thinks the auto-continue == yes vote from
ThreadPlanStepOverBreakpoint wins, and we lose control of the process.
This patch changes ThreadPlanStepOut to require that *both* (1) we are
at the StackID of the caller function, where we wanted to end up, and
(2) we have actually hit the breakpoint that we inserted.
This in effect means that now lldb instruction-steps over the breakpoint
in the callee function, stops at the return address of the caller
function. StepOverBreakpoint has completed. StepOut is still running,
and we continue the thread again. We immediatley hit the breakpoint
(that we're sitting at), and now ThreadPlanStepOut marks itself as
completed, and we return control to the user.
Jim suggests that ThreadPlanStepOverBreakpoint is a bit unusual because
it's not something pushed on the stack by a higher-order thread plan
that "owns" it, it is inserted by the Thread as it is about to resume,
if we're at a BreakpointSite. It has no connection to the thread plans
above it, but tries to set the auto-continue mode based on the state of
the thread when it is inserted (and tries to detect an unexpected
breakpoint and unset that auto-continue it previously decided on,
because it now realizes it should not influence execution control any
more). Instead maybe the
ThreadPlanStepOverBreakpoint should be inserted as a child plan of
whatever the lowest plan is on the stack at the point it is added.
I added an API test that will catch this bug in the new thread
breakpoint algorithm.
When a test depends on a new debugserver feature/fix, the API test must
be marked @skipIfOutOfTreeDebugserver because the macOS CI bots test
using the latest Xcode release debugserver. But over time all of these
fixes & new features are picked up in the Xcode debugserver and these
skips can be removed.
We may see unexpected test failures from removing all of these 1+ year
old skips, but that's likely a separate reason the test is failing that
is being papered over by this skip.
I think the only issue here was that we would erroneously consider
functions which are "in the middle" of the function were stepping to as
a part of the function, and would try to step into them (likely stepping
out of the function instead) instead of giving up early.
A lot of `TestConcurrent*.py` expect one of the threads to crash, but we
weren't checking for it properly.
Possibly because signal reporting got better on FreeBSD at some point,
and it now shows the same info as Linux does.
```
lldb-api :: functionalities/inferior-changed/TestInferiorChanged.py
lldb-api :: functionalities/inferior-crashing/TestInferiorCrashing.py
lldb-api :: functionalities/inferior-crashing/TestInferiorCrashingStep.py
lldb-api :: functionalities/inferior-crashing/recursive-inferior/TestRecursiveInferior.py
lldb-api :: functionalities/inferior-crashing/recursive-inferior/TestRecursiveInferiorStep.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithBreak.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithSignal.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpoint.py
lldb-api :: functionalities/thread/concurrent_events/TestConcurrentCrashWithWatchpointBreakpointSignal.py
```
Fixes#48777
`TestConcurrentTwoBreakpointsOneSignal.py` no longer fails, at least on
an AWS instance, so I've removed the xfail there.
self.wait_for_running_event(process) is always called after
self.runCmd("continue"). It is strange to expect eStateConnected here.
This test failed in case of a remote target. The correct state is
eStateRunning. Removed incorrect checking.
This actually passes on Windows but I don't know how to convey
that with an xfail without clashing with the xfail for all
platforms.
At least this avoids a UPASS.
Any time we see the pattern `assertEqual(value, bool)`, we can replace
that with `assert<bool>(value)`. Likewise for `assertNotEqual`.
Technically this relaxes the test a bit, as we may want to make sure
`value` is either `True` or `False`, and not something that implicitly
converts to a bool. For example, `assertEqual("foo", True)` will fail,
but `assertTrue("foo")` will not. In most cases, this distinction is not
important.
There are two such places that this patch does **not** transform, since
it seems intentional that we want the result to be a bool:
*
5daf2001a1/lldb/test/API/python_api/sbstructureddata/TestStructuredDataAPI.py (L90)
*
5daf2001a1/lldb/test/API/commands/settings/TestSettings.py (L940)
Followup to 9c2468821ec51defd09c246fea4a47886fff8c01. I patched `teyit`
with a `visit_assertEqual` node handler to generate this.
This uses [teyit](https://pypi.org/project/teyit/) to modernize asserts,
as recommended by the [unittest release
notes](https://docs.python.org/3.12/whatsnew/3.12.html#id3).
For example, `assertTrue(a == b)` is replaced with `assertEqual(a, b)`.
This produces better error messages, e.g. `error: unexpectedly found 1
and 2 to be different` instead of `error: False`.
assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a
vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.
This removes the dependency LLDB API tests have on
lldb/third_party/Python/module/unittest2, and instead uses the standard
one provided by Python.
This does not actually remove the vendored dep yet, nor update the docs.
I'll do both those once this sticks.
Non-trivial changes to call out:
- expected failures (i.e. "bugnumber") don't have a reason anymore, so
those params were removed
- `assertItemsEqual` is now called `assertCountEqual`
- When a test is marked xfail, our copy of unittest2 considers failures
during teardown to be OK, but modern unittest does not. See
TestThreadLocal.py. (Very likely could be a real bug/leak).
- Our copy of unittest2 was patched to print all test results, even ones
that don't happen, e.g. `(5 passes, 0 failures, 1 errors, 0 skipped,
...)`, but standard unittest prints a terser message that omits test
result types that didn't happen, e.g. `OK (skipped=1)`. Our lit
integration parses this stderr and needs to be updated w/ that
expectation.
I tested this w/ `ninja check-lldb-api` on Linux. There's a good chance
non-Linux tests have similar quirks, but I'm not able to uncover those.
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
After D133376, jumping to the return line in the otherfn function became
ambiguous because it has two line entries associated with it. Work
around that problem by changing the function. Filed PR59458 to track
possible improvements in jump target disambiguation.
This commit combines the initial commit (7c240de609af), a fix for x86_64 Linux
(3a0581501e76) and a fix for thinko in a last minute rewrite that I really
should have run the testsuite on.
Also, make sure that all the "I need to step over watchpoint" plans execute
before we call a public stop. Otherwise, e.g. if you have N watchpoints and
a Signal, the signal stop info will get us to stop with the watchpoints in a
half-done state.
Differential Revision: https://reviews.llvm.org/D130674
Add a function to make it easier to debug a test failure caused by an
unexpected stop reason. This is similar to the assertState helper that
was added in ce825e46743b.
Before:
self.assertEqual(stop_reason, lldb.eStopReasonInstrumentation)
AssertionError: 5 != 10
After:
self.assertStopReason(stop_reason, lldb.eStopReasonInstrumentation)
AssertionError: signal (5) != instrumentation (10)
Differential revision: https://reviews.llvm.org/D131083
This reverts commit 5778ada8e54edb2bc2869505b88a959d1915c02f.
The watchpoint tests all stall on aarch64-ubuntu bots. Reverting till I can
get my hands on an system to test this out.
Since we want to present the "new & old" values for watchpoint hits, on architectures,
including the ARM family, that stop before the triggering instruction is run, we need
to single step over the instruction before stopping for realz. This was incorrectly
done directly in the StopInfoWatchpoint::ShouldStop. That causes problems if more than
one thread stops "for a reason" at the same time as the watchpoint, since the other actions
didn't expect the process to make progress in this part of the execution control machinery.
The correct way to do this is to schedule the step over using ThreadPlans, and then to restore
the stop info after that plan stops, so that the rest of the stop info actions can happen when
all the other threads have handled their immediate actions as well.
Differential Revision: https://reviews.llvm.org/D129814
The requirements for "thread until <line number>" are:
a) If any code contributed by <line number> or the nearest subsequent of <line number> is executed before leaving the function, stop
b) If you end up leaving the function w/o triggering (a), then stop
In case of (a), since the <line number> may have multiple entries in the line table and the compiler might have scheduled/moved the relevant code across, and the lldb does not know the control flow, set breakpoints on all the line table entries of best match of <line number> i.e. exact or the nearest subsequent line.
Along with the above, currently, CommandObjectThreadUntil is also setting the breakpoints on all the subsequent line numbers after the best match and this latter part is wrong.
This issue is discussed at http://lists.llvm.org/pipermail/lldb-dev/2018-August/013979.html.
In fact, currently `TestStepUntil.py` is not actually testing step until scenarios and `test_missing_one` test fails without this patch if tests are made to run. Fixed the test as well.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D50304
The requirements for "thread until <line number>" are:
a) If any code contributed by <line number> or the nearest subsequent of <line number> is executed before leaving the function, stop
b) If you end up leaving the function w/o triggering (a), then stop
In case of (a), since the <line number> may have multiple entries in the line table and the compiler might have scheduled/moved the relevant code across, and the lldb does not know the control flow, set breakpoints on all the line table entries of best match of <line number> i.e. exact or the nearest subsequent line.
Along with the above, currently, CommandObjectThreadUntil is also setting the breakpoints on all the subsequent line numbers after the best match and this latter part is wrong.
This issue is discussed at http://lists.llvm.org/pipermail/lldb-dev/2018-August/013979.html.
In fact, currently `TestStepUntil.py` is not actually testing step until scenarios and `test_missing_one` test fails without this patch if tests are made to run. Fixed the test as well.
Reviewed By: jingham
Differential Revision: https://reviews.llvm.org/D50304
Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.
Differential Revision: https://reviews.llvm.org/D128077
This test depends on multiple threads with one of them
hitting a watchpoint at the same time as a breakpoint, and
can fail because of the way arm64 watchpoints are handled.
I added skips to most of these via
```
commit bef4da4a6aef8196f007f44e3e9c8e3419ffb623
Author: Jason Molenda <jason@molenda.com>
Date: Wed May 25 16:05:16 2022 -0700
Skip testing of watchpoint hit-count/ignore-count on multithreaded
```
but missed that this test is susceptable to the same issue.
Skip all watchpoint hit-count/ignore-count tests for multithreaded
API tests for now on arm64 Darwin.
On AArch64, insns that trigger a WP are rolled back and we are
notified. lldb needs to disable the WP, insn step, re-enable it,
then report it to the user. lldb only does this full step action
for the "selected thread", and so when a program stops with
multiple threads hitting a stop reason, some of them watchpoints,
any non-selected-thread will not be completed in this way. But
all threads with the initial watchpoint exception will have their
hit-count/ignore-counts updated. When we resume execution, the
other threads sitting at the instruction will again execute &
trigger the WP exceptoin again, repeating until we've gone through
all of the threads.
This bug is being tracked in llvm.org/pr49433 and inside apple
in rdar://93863107
This patch handles the situation where the main thread exits (through
the SYS_exit syscall). In this case, the process as a whole continues
running until all of the other threads exit, or one of them issues an
exit_group syscall.
The patch consists of two changes:
- a moderate redesign of the handling of thread exit (WIFEXITED) events.
Previously, we were removing (forgetting) a thread once we received
the WIFEXITED (or WIFSIGNALED) event. This was problematic for the
main thread, since the main thread WIFEXITED event (which is better thought
of as a process-wide event) gets reported only after the entire process
exits. This resulted in deadlocks, where we were waiting for the
process to stop (because we still considered the main thread "live").
This patch changes the logic such that the main thread is removed as
soon as its PTRACE_EVENT_EXIT (the pre-exit) event is received. At
this point we can consider the thread gone (for most purposes). As a
corrolary, I needed to add special logic to catch process-wide exit
events in the cases where we don't have the main thread around.
- The second part of the patch is the removal of the assumptions that
the main thread is always available. This generally meant replacing
the uses of GetThreadByID(process_id) with GetCurrentThread() in
various process-wide operations (such as memory reads).
Differential Revision: https://reviews.llvm.org/D122716
After applying the same for as in TestThreadBacktraceRepeat, the test
appears to pass reliably. The skip decorator was added many years ago,
so it's not clear whether this is what caused it to hang.
Replace forms of `assertTrue(err.Success())` with `assertSuccess(err)` (added in D82759).
* `assertSuccess` prints out the error's message
* `assertSuccess` expresses explicit higher level semantics, both to the reader and for test failure output
* `assertSuccess` seems not to be well known, using it where possible will help spread knowledge
* `assertSuccess` statements are more succinct
Differential Revision: https://reviews.llvm.org/D119616
This test is completely nondeterministic, environment-dependent and does
not test what it was supposed to test (reverting the associated patch
does not make it fail).
I tried to figure out what the patch was meant to fix to see if I can
create a better test with the current tools, but I was not able to
understand the problem (it sounds like it has something to do with local
classes, but I don't understand the details).
D116372, while fixing one kind of a race, ended up creating a new one.
The new issue could occur when one inferior thread exits while another
thread initiates termination of the entire process (exit_group(2)).
With some bad luck, we could start processing the exit notification
(PTRACE_EVENT_EXIT) only to have the become unresponsive (ESRCH) in the
middle of the MonitorCallback function. This function would then delete
the thread from our list even though it wasn't completely dead (it stays
zombified until we read the WIFEXITED event). The linux kernel will not
deliver the exited event for the entire process until we process
individual thread exits.
In a pre-D116372 world, this wouldn't be a problem because we would read
this event (even though we would not know what to do with it) with
waitpid(-1). Now, when we issue invididual waitpids, this event will
never be picked up, and we end up hanging.
The fix for this is actually quite simple -- don't delete the thread in
this situation. The thread will be deleted when the WIFEXITED event
comes.
This situation was kind of already tested by
TestCreateDuringInstructionStep (which is how I found this problem), but
it was mostly accidental, so I am also creating a dedicated test which
reproduces this situation.
It was being used only in some very old tests (which pass even without
it) and its implementation is highly questionable.
These days we have different mechanisms for requesting a build with a
particular kind of c++ library (USE_LIB(STD)CPP in the makefile).
These tests work fine with VS2017, but become more flaky with VS2019 and the buildbot is about to get upgraded.
Differential Revision: https://reviews.llvm.org/D114907
