PR #181720 introduced shared builds for LLDB API tests to improve test
efficiency. But several data formatter tests requiring PDB debug info
are failing on Windows x64 and AArch64 platforms.
This patch disables shared builds for these tests by setting
SHARED_BUILD_TESTCASE = False
The shared build optimization breaks these tests because they reuse
build artifacts between test methods
The test runs may could use multiple methods with different debug
formats or compiler flags. When a test runs first it builds with one set
of flags, but then it runs again but **make** sees the source unchanged
so it skips rebuilding and reuses the same old binary instead of
rebuilding with correct flags.
This changes Python API tests to use a single build shared across all
test functions, instead of the previous default behavior of a separate
build dir for each test function.
This build behavior opt-out, tests can use the previous behavior of one
individual (unshared) build directory per test function, by setting
`SHARED_BUILD_TESTCASE` to False (in the test class).
The motivation is to make the test suite more efficient, by not
repeatedly building the same test source. When running tests on my macOS
machine, this reduces the time of `ninja check-lldb-api` by almost 60%
(sample numbers: from ~492s down to ~207s = 58%). Almost 5min time
saved.
Each test function still calls `self.build()`, but only the first call
will do a build, in the subsequent tests `make` will be a no-op because
the sources won't have changed.
From
https://github.com/llvm/llvm-project/pull/148554#issuecomment-3083261858
- this adds an option for API tests to be run with the both PDB readers
on Windows. As there are a lot of failures with PDB, this is an opt-in
per test.
To get PDB, `-g -gcodeview` has to be used on Clang. `-gcodeview` alone
isn't enough, because it won't cause clang to pass `-debug` to the
linker.
#149498 tracks the (currently) failing tests.
In one of my recent PRs I mistakenly had two test-cases with the same
name, preventing one of them to run. Since it's an easy mistake to make
(e.g., copy pasting existing test-cases), I ran following sanity-check
script over `lldb/test/API`, which found couple of tests which were
losing coverage because of this (or in some cases simply had duplicate
tests):
```
import ast
import sys
filename = sys.argv[1]
print(f'Checking {filename}...')
tree = ast.parse(open(filename, 'r').read())
for node in ast.walk(tree):
if not isinstance(node, ast.ClassDef):
continue
func_names = []
for child in ast.iter_child_nodes(node):
if isinstance(child, ast.FunctionDef):
func_names.append(child.name)
seen_func_names = set()
duplicate_func_names = []
for name in func_names:
if name in seen_func_names:
duplicate_func_names.append(name)
else:
seen_func_names.add(name)
if len(duplicate_func_names) != 0:
print(f'Multiple func names found:\n\t{duplicate_func_names}\n\tclass {node.name}\n\tfile: {filename}')
```
This patch fixes these cases.
The @expectedFailureAll and @skipIf decorators will mark the test case
as xfail/skip if _all_ conditions passed in match, including debug_info.
* If debug_info is not one of the matching conditions, we can
immediately evaluate the check and decide if it should be decorated.
* If debug_info *is* present as a match condition, we need to defer
whether or not to decorate until when the `LLDBTestCaseFactory`
metaclass expands the test case into its potential variants. This is
still early enough that the standard `unittest` framework will recognize
the test as xfail/skip by the time the test actually runs.
TestDecorators exhibits the edge cases more thoroughly. With the
exception of `@expectedFailureIf` (added by this commit), all those test
cases pass prior to this commit.
This is a followup to 212a60ec37322f853e91e171b305479b1abff2f2.
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.
Differential Revision: https://reviews.llvm.org/D128077
This changes the decorator helper `_match_decorator_property` to
consider `None` as the actual value as not a match. Using `None` for the
pattern continues to be considered a match.
I discovered the issue because marking a test as NO_DEBUG_INFO_TESTCASE
will cause the call to `self.getDebugInfo()` to return `None` and
incorrectly skip or XFAIL the corresponding test.
I used the above scenario to create a test for the decorators.
Differential revision: https://reviews.llvm.org/D123401
Running tests with -t prints all lldb commands being run. It makes sense
to print all the build commands as well.
Differential Revision: https://reviews.llvm.org/D112212
This test supposed to check the test base we are using for pexpect tests, but instead it used the normal TestBase
class we use for all other tests. TestBase already had the substrs type check since D88792 so this
test was passing because of that.
This just changes the test base of the test to the pexpect one so that the `expect` calls find their intended
target function. Also moves the check to the very start so that we can check the argument without
actually having to start a terminal and all that jazz.
(I found this by accident as D88792 got somehow reverted in a downstream branch so this test started
failing).
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D96556
This is a follow up to D88792 which found an issue in a call to PExpectTest's
expect function that allows passing a string to the `substrs` parameter. However
this issue was found by just grepping and TestPExpect's expect function is still
accepting a single string as a value to `substrs`.
This patch adds the same sanity check that D88792 added to the PExpectTest's
implementation of `expect` and also adds a small test for it.
Reviewed By: kastiglione, JDevlieghere
Differential Revision: https://reviews.llvm.org/D89302
