The goal here is to remove the third_party/Python/module tree, which
LLDB tests only use to `import pexpect`. This package is available on
`pip`, and I believe should not be hard to obtain.
However, in case it isn't easily available, deleting the tree right now
could cause disruption. This introduces a
`LLDB_TEST_USE_VENDOR_PACKAGES` cmake param that can be enabled, and the
tests will continue loading that tree. By default, it is enabled,
meaning there's really no change here. A followup change will disable it
by default once all known build bots are updated to include this
package. When disabled, an eager cmake check runs that makes sure
`pexpect` is available before waiting for the test to fail in an obscure
way.
Later, this option will go away, and when it does, we can delete the
tree too. Ideally this is not disruptive, and we can remove it in a week
or two.
Fix the bug where merge-fdata unconditionally outputs boltedcollection
line, regardless of whether input files have it set.
Test Plan:
Added bolt/test/X86/merge-fdata-nobat-mode.test which fails without this
fix.
There are some uses of "vscode" in
`lldb/packages/Python/lldbsuite/test/tools/lldb-dap/dap_server.py` where
I'm not sure if it's referring to the adapter or VS Code itself, so
those remain.
Set the halt_on_error runtime flag to make TSan errors fatal when
running the test suite. For the API tests the environment variables are
set conditionally on whether the TSan is enabled. The Shell and Unit
tests don't have that logic but setting the environment variable is
harmless. For consistency, I've also mirrored the ASAN option
(detect_stack_use_after_return=1) for the Shell tests.
Differential revision: https://reviews.llvm.org/D157152
This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).
If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.
RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style
Differential revision: https://reviews.llvm.org/D151460
In certain configurations, libc++ headers all exist in the same directory, and libc++ binaries exist in the same directory as lldb libs. When `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR` is enabled (*and* the host is not Apple, which is why I assume this wasn't caught by others?), this is not the case: most headers will exist in the usual `include/c++/v1` directory, but `__config_site` exists in `include/$TRIPLE/c++/v1`. Likewise, the libc++.so binary exists in `lib/$TRIPLE/`, not `lib/` (where LLDB libraries reside).
This also adds the just-built-libcxx functionality to the lldb-dotest tool.
The `LIBCXX_` cmake config is borrowed from `libcxx/CMakeLists.txt`. I could not figure out a way to share the cmake config; ideally we would reuse the same config instead of copy/paste.
Reviewed By: JDevlieghere, fdeazeve
Differential Revision: https://reviews.llvm.org/D133973
The CMake variable LLDB_HAS_LIBCXX is passed to
`llvm_canonicalize_cmake_booleans`, which transforms TRUE/FALSE into
'1'/'0'. It also transforms undefined variables to '0'.
In particular, this means that the configuration script for LLDB API's
test always has _some_ value for the `has_libcxx` configuration:
```
config.has_libcxx = '@LLDB_HAS_LIBCXX@'
```
When deciding whether a libcxx exist, the testing scripts would only
check for the existence of `has_libcxx`, but not for its value. In other
words, because `if ('0')` is true in python we always think there is a
libcxx.
This was caught once D132940 was merged and most tests started to use
libcxx by default if `has_libcxx` is true. Prior to that, no failures
were seen because only tests are marked with
`@add_test_categories(["libc++"])` would require a libcxx, and these
would be filtered out on builds without the libcxx target. Furthermore,
the MacOS bots always build libcxx.
We fix this by making `has_libcxx` a boolean (instead of a string) and
by checking its value in the test configuration.
Differential Revision: https://reviews.llvm.org/D133639
When running LLDB API tests, a user can override test arguments with
LLDB_TEST_USER_ARGS. However, these flags used to be concatenated with a
CMake-derived variable LLDB_TEST_COMMON_ARGS, as below:
```
set(LLDB_DOTEST_ARGS ${LLDB_TEST_COMMON_ARGS};${LLDB_TEST_USER_ARGS}
CACHE INTERNAL STRING)
```
This is problematic, because LLDB_TEST_COMMON_ARGS must be processed
first, while LLDB_TEST_USER_ARGS args must be processed last, so that
user overrides are respected. Currently, if a user attempts to override
one of the "inferred" flags, the user's request is ignored. This is the
case, for example, with `--libcxx-include-dir` and
`--libcxx-library-dir`. Both flags are needed by the greendragon bots.
This commit removes the concatenation above, keeping the two original
variables throughout the entire flow, processing the user's flag last.
The variable LLDB_TEST_COMMON_ARGS needs to be a CACHE property, but it
is modified throughout the CMake file with `set` or `list` or `string`
commands, which don't work with properties. As such, a temporary
variable `LLDB_TEST_COMMON_ARGS_VAR` is created.
This was tested locally by invoking CMake with:
-DLLDB_TEST_USER_ARGS="--libcxx-include-dir=blah --libcxx-library-dir=blah2"
and checking that tests failed appropriately.
Differential Revision: https://reviews.llvm.org/D132642
This patch combines D129166 (to always pick the just-built libc++) and
D132257 (to allow customizing the libc++ for testing). The common goal
is to avoid picking up an unexpected libc++ for testing.
Differential revision: https://reviews.llvm.org/D132263
Make sure we use the libc++ from the build dir. Currently, by passing
-stdlib=libc++, we might pick up the system libc++. This change ensures
that if LLVM_LIBS_DIR is set, we try to use the libc++ from there.
Differential revision: https://reviews.llvm.org/D129166
Summary:
This is an essential piece of infrastructure for us to be
continuously testing debug info with BOLT. We can't only make changes
to a test repo because we need to change debuginfo tests to call BOLT,
hence, this diff needs to sit in our opensource repo. But when upstreaming
to LLVM, this should be kept BOLT-only outside of LLVM. When upstreaming,
we need to git diff and check all folders that are being modified by our
commits and discard this one (and leave as an internal diff).
To test BOLT in debuginfo tests, configure it with -DLLVM_TEST_BOLT=ON.
Then run check-lldb and check-debuginfo.
Manual rebase conflict history:
https://phabricator.intern.facebook.com/D29205224https://phabricator.intern.facebook.com/D29564078https://phabricator.intern.facebook.com/D33289118https://phabricator.intern.facebook.com/D34957174
Test Plan:
tested locally
Configured with:
-DLLVM_ENABLE_PROJECTS="clang;lld;lldb;compiler-rt;bolt;debuginfo-tests"
-DLLVM_TEST_BOLT=ON
Ran test suite with:
ninja check-debuginfo
ninja check-lldb
Reviewers: #llvm-bolt
Subscribers: ayermolo, phabricatorlinter
Differential Revision: https://phabricator.intern.facebook.com/D35317341
Tasks: T92898286
Avoid "TERM environment variable not set" by either propagating the TERM
environment variable or defaulting to vt100. All of our CI is already
doing this explicitly through the --env dotest arg, but it's easy to
forget when setting up a new job. I don't see any downside in making it
the default.
Remove the remaining references to LLDB_CAPTURE_REPRODUCER. I removed
the functionality in an earlier commit but forgot that there was a
corresponding test and logic to unset it in our test suite.
lit has grown a feature where it stores the runtimes of all tests.
Normally, these times should be stored in the build directory, but
because our API tests have set test_exec_root to point to the source
tree, it has ended up polluting our checkout and led to the
.lit_test_times.txt being committed to the repository.
Delete this file, and adjust the exec root of API tests. I've also
needed to adjust the root of Shell tests, in order to avoid the two
overlapping.
Make the API, Shell and Unit tests independent lit test suites. This
allows us to specify different dependencies and skip rebuilding all the
unit test (which is particularly expensive) when running check-lldb-api
or check-lldb-shell.
This does not change the autogenerated targets such as
check-lldb-shell-driver or the top level check-lldb target, which all
continue to work as before.
Differential revision: https://reviews.llvm.org/D98842
This variable is used to reducing the likelihood of hitting module cache
issues in CI where different branches can potentially run on the same
machine.
Replace the dotest command line options and various cmake variables,
which are used for passing the locations of llvm tools to the API tests
with a single variable, which points to the directory these tools are
placed in. Besides reducing repetition, this also makes things more
similar to how "normal" llvm tests are configured.
Differential Revision: https://reviews.llvm.org/D95261
Test runs log some of their output to files inside the LLDB session dir. This
session dir is shared between all tests, so all the tests have to make sure they
choose a unique file name inside that directory. We currently choose by default
`<test-class-name>-<test-method-name>` as the log file name. However, that means
that if not every test class in the test suite has a unique class name, then we
end up with a race condition as two tests will try to write to the same log
file.
I already tried in D83767 changing the format to use the test file basename
instead (which we already require to be unique for some other functionality),
but it seems the code for getting the basename didn't work on Windows.
This patch instead just changes that dotest stores the log files in the build
directory for the current test. We know that directory is unique for this test,
so no need to generate some unique file name now. Also removes all the
environment vars and parameters related to the now unused session dir.
The new log paths now look like this for a failure in 'TestCppOperators`:
```
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dwarf/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_dsym/Failure.log
./lldb-test-build.noindex/lang/cpp/operators/TestCppOperators.test_gmodules/Failure.log
```
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D92498
The new FreeBSDRemote plugin has reached feature parity on i386
and amd64 targets. Use it by default on these architectures, while
allowing the use of the legacy plugin via FREEBSD_LEGACY_PLUGIN envvar.
Revisit the method of switching plugins. Apparently, the return value
of PlatformFreeBSD::CanDebugProcess() is what really decides whether
the legacy or the new plugin is used.
Update the test status. Reenable the tests that were previously
disabled on FreeBSD and do not cause hangs or are irrelevant to FreeBSD.
Mark all tests that fail reliably as expectedFailure. For now, tests
that are flaky (i.e. produce unstable results) are left enabled
and cause unpredictable test failures.
Differential Revision: https://reviews.llvm.org/D90757
Replace the inline x86 watchpoint handling code with the reusable
NativeRegisterContextWatchpoint_x86. Implement watchpoint support
in NativeThreadFreeBSD and SIGTRAP handling for watchpoints.
Un-skip all concurrent_events tests as they pass with the new plugin.
Differential Revision: https://reviews.llvm.org/D90102
This patch adds several build system targets that run the normal test suite but
against the Watch/TV/iPhone simulators.
Reviewed By: JDevlieghere
Differential Revision: https://reviews.llvm.org/D89224
LIT uses a model where the test suite is configurable trough a
lit.site.cfg file. Most of the time we use the lit.site.cfg with values
that match the current build configuration, generated by CMake.
Nothing prevents you from running the test suite with a different
configuration, either by overriding some of these values from the
command line, or by passing a different lit.site.cfg.
The latter is currently tedious. Many configuration values are optional
but they still need to be set because lit.cfg.py is accessing them
directly. This patch changes the code to use getattr to return the
attribute if it exists. This makes it possible to specify a minimal
lit.site.cfg with only the mandatory/desired configuration values.
Differential revision: https://reviews.llvm.org/D86821
The Python 3 interpreter in Xcode has a relative RPATH and dyld fails to
load it when we copy it into the build directory.
This patch adds an additional check that the copied binary can be
executed. If it doesn't, we assume we're dealing with the Xcode python
interpreter and return the path to the real executable. That is
sufficient for the sanitizers because only system binaries need to be
copied to work around SIP.
This patch also moves all that logic out of LLDBTest and into the lit
configuration so that it's executed only once per test run, instead of
once for every test. Although I didn't benchmark the difference this
should result in a mild speedup.
Differential revision: https://reviews.llvm.org/D81696
These tests are flaky on the reproducer bot. I suspect it has something
to do with the module cache. Skipping the whole category while I
investigate the issue.
[intel-pt] Improve the way the test determines whether to run
- Now I'm creating a default value for the new test parameter
- I fixed a small mistake in the skipping logic of the test
... I forgot to clear the cmake cache when testing my diff
Summary:
@labath raised a concern on the way I was skipping this test. I think that was
fair and I found a better way.
Now I'm skipping if the CMAKE flag LLDB_BUILD_INTEL_PT is false.
I added an enabled_plugins entry in the dotest configuration, which gets
set by lit or lldb-dotest. The only available plugin right now is
'intel-pt', but I imagine it will be useful in the future for other
kinds of plugins that get determined at configuration time. I didn't
want to add a new argument option --enable-intel-pt or something or the
sort, as it wouldn't be useful for other cases.
Reviewers: labath, clayborg
Subscribers: lldb-commits, labath
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D77452
Make it possible to capture reproducers from the API test suite. Given
the symmetry between capture and replay, this patch also adds the
necessary code for replay. For now this is a NO-OP until the
corresponding reproducer instrumentation changes land.
For more info please refer to the RFC on lldb-dev:
http://lists.llvm.org/pipermail/lldb-dev/2020-April/016100.html
Differential revision: https://reviews.llvm.org/D77588
Summary: Moves lldbsuite tests to lldb/test/API.
This is a largely mechanical change, moved with the following steps:
```
rm lldb/test/API/testcases
mkdir -p lldb/test/API/{test_runner/test,tools/lldb-{server,vscode}}
mv lldb/packages/Python/lldbsuite/test/test_runner/test lldb/test/API/test_runner
for d in $(find lldb/packages/Python/lldbsuite/test/* -maxdepth 0 -type d | egrep -v "make|plugins|test_runner|tools"); do mv $d lldb/test/API; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-vscode -maxdepth 1 -mindepth 1 | grep -v ".py"); do mv $d lldb/test/API/tools/lldb-vscode; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-server -maxdepth 1 -mindepth 1 | egrep -v "gdbremote_testcase.py|lldbgdbserverutils.py|socket_packet_pump.py"); do mv $d lldb/test/API/tools/lldb-server; done
```
lldb/packages/Python/lldbsuite/__init__.py and lldb/test/API/lit.cfg.py were also updated with the new directory structure.
Reviewers: labath, JDevlieghere
Tags: #lldb
Differential Revision: https://reviews.llvm.org/D71151
Pass the correct library directory from CMake to dotest.py when linking
liblldb, instead of trying to reconstruct the path from executable path.
This fixes link failures on platforms having non-null
LLVM_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D73767
This allows an unsanitized test process which loads a sanitized DSO (the
motivating example is a Swift runtime dylib) to launch on Darwin.
rdar://57290132
Differential Revision: https://reviews.llvm.org/D71379
Rather than serializing every argument through LLDB_TEST_COMMON_ARGS, we
can pass some of them directly using their CMake variable. Although this
does introduce some code duplication between lit's site config and the
lldb-dotest utility, it also means that it becomes easier to override
these values (WIP).
