1. This commit adds LLDB_TEST_PLATFORM_URL, LLDB_TEST_SYSROOT,
LLDB_TEST_PLATFORM_WORKING_DIR, LLDB_SHELL_TESTS_DISABLE_REMOTE cmake
flags to pass arguments for cross-compilation and remote running of both Shell&API tests.
2. To run Shell tests remotely, it adds 'platform select' and 'platform connect' commands to %lldb
substitution.
3. 'remote-linux' feature added to lit to disable tests failing with
remote execution.
4. A separate working directory is assigned to each test to avoid
conflicts during parallel test execution.
5. Remote Shell testing is run only when LLDB_TEST_SYSROOT is set for
building test sources. The recommended compiler for that is Clang.
---------
Co-authored-by: Vladimir Vereschaka <vvereschaka@accesssoftek.com>
https://discourse.llvm.org/t/running-lldb-in-a-container/76801/4 found
that the obvious way to use this variable doesn't work, despite what our
docs and examples say.
Perhaps in the past it did but now you need to use ";" as a separator to
make sure the final command line works properly.
For example "-A foo" becomes "-A foo" when python goes to run the runner
script. The script sees this as one command line element, not two. What
you actually want is "-A;foo" which we convert to "-A" "foo" which the
script sees as one option and one value for that option.
The "Script:" printout from dotest is misleading here because it does `"
".join(cmd)` so it looks like it's ok but in fact it's not.
I'm not changing that format though because printing the command as a
Python list is not useful outside of this specific situation.
The instructions for running single tests in the LLDB test suite used an
older directory structure from before the LLVM project became a
monorepo. This commit updates the references to these directories.
Bridge network means that you can get to any port on the VM,
from the host, which is great. However it is quite involved to
setup in some cases, and I've certainly messed it up in the past.
An alternative is forwarding a block of ports and using some
hidden options to lldb-server to limit what it uses. This
commit documents that and the pitfall that the port list isn't shared.
The theory also works for Arm's FVP (which inspired me to write
this up) but since QEMU is the preferred option upstream, it goes
in that document.
Along the way I fixed a link to the QEMU page that used the URL
not a relative link to the document.
This patch brings a fresh coat of paint to the LLDB website.
This uses the `furo` theme from the suggested theme list, adds some
changes to the css to make it use the full browser window width and
removes the table of contents since the theme generates it automatically.
This patch also rewrites the tables for "GDB to LLDB command map" to
convert them from raw HTML to native inlined code blocks.
To generate the website, make sure you install the theme first with
`pip install furo`, enable sphinx in the cmake invocation` then run
`ninja docs-lldb-html`
Discourse: https://discourse.llvm.org/t/rfc-revamping-lldbs-website
Differential Revision: https://reviews.llvm.org/D158323
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch reverts the lldb website revamp (eea8874) since the `furo` theme
that we use is not installed on the publisher bot. The prevents other
website from getting there documentation updated.
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
This patch brings a fresh coat of paint to the LLDB website.
This uses the `furo` theme from the suggested theme list, adds some
changes to the css to make it use the full browser window width and
removes the table of contents since the theme generates it automatically.
This patch also rewrites the tables for "GDB to LLDB command map" to
convert them from raw HTML to native inlined code blocks.
To generate the website, make sure you install the theme first with
`pip install furo`, enable sphinx in the cmake invocation` then run
`ninja docs-lldb-html`
Discourse: https://discourse.llvm.org/t/rfc-revamping-lldbs-website
Differential Revision: https://reviews.llvm.org/D158323
Signed-off-by: Med Ismail Bennani <ismail@bennani.ma>
Currently, by default LLDB runs an API test with 3 variants,
one of which, depending on platform, is `gmodules`. However,
most tests don't actually make use of any specific `gmodules`
feature since they compile a single `main.cpp` file without
imporint types from other modules.
Instead of running all tests an extra time with `-gmodules`
enabled, we plan to test `gmodules` features with dedicated
API tests that explicitly opt-into compiling with `-gmodules`.
One of the main benefits of this will be a reduction in total
API test-suite run-time (by around 1/3).
This patch adds a flag to `debug_info_categories` that indicates
whether a category is eligible to be replicated by `lldbtest`.
Keeping `gmodules` a debug-info category is desirable because
`builder.py` knows how to inject the appropriate Makefile flags
into the build command for debug-info categories already. Whereas
for non-debug-info categories we'd have to teach it how to. The
category is inferred from the test-name debug-info suffix currently.
Differential Revision: https://reviews.llvm.org/D134524
Add a function to make it easier to debug a test failure caused by an
unexpected stop reason. This is similar to the assertState helper that
was added in ce825e46743b.
Before:
self.assertEqual(stop_reason, lldb.eStopReasonInstrumentation)
AssertionError: 5 != 10
After:
self.assertStopReason(stop_reason, lldb.eStopReasonInstrumentation)
AssertionError: signal (5) != instrumentation (10)
Differential revision: https://reviews.llvm.org/D131083
As raised here: https://lists.llvm.org/pipermail/llvm-dev/2021-November/153881.html
Now that VS2022 is on general release, LLVM is expected to build on VS2017, VS2019 and VS2022, which is proving hazardous to maintain due to changes in behaviour including preprocessor and constexpr changes. Plus of the few developers that work with VS, many have already moved to VS2019/22.
This patch proposes to raise the minimum supported version to VS2019 (16.x) - I've made the hard limit 16.0 or later, with the soft limit VS2019 16.7 - older versions of VS2019 are "allowed" (at your own risk) via the LLVM_FORCE_USE_OLD_TOOLCHAIN cmake flag.
Differential Revision: https://reviews.llvm.org/D114639
This file contain some old reference to files those are now either renamed or replaced.
Also this .txt file didn't generate to html during the sphnix documentation build so I send its contents to resources/test.rst file.
Signed-off-by: Shivam Gupta <shivam98.tkg@gmail.com>
Reviewed By: teemperor, mgorny, JDevlieghere
Differential Revision: https://reviews.llvm.org/D108812
This patch specifies a few guidelines that our API tests should follow.
The motivations for this are twofold:
1. API tests have unexpected pitfalls that especially new contributors run into
when writing tests. To prevent the frustration of letting people figure those
pitfalls out by trial-and-error, let's just document them briefly in one place.
2. It prevents some arguing about what is the right way to write tests. I really
like to have fast and reliable API test suite, but I also don't want to be the
bogeyman that has to insist in every review that the test should be rewritten to
not launch a process for no good reason. It's much easier to just point to a
policy document.
I omitted some guidelines that I think could be controversial (e.g., the whole
"should assert message describe failure or success").
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D101153
Debugging tests sometimes involves debugging the Python source. This adds a paragraph to
the "Debugging Test Failures" section about using `pdb`, and also describes how to run
lldb commands from pdb.
Differential Revision: https://reviews.llvm.org/D99744
This patch adds a HowTo document to lldb docs which gives instruction for
setting up a virtual environment based on QEMU emulator for LLDB testing.
Instruction in this document are tested on Arm and AArch64 targets but
can easily be duplicated for other targets supported by QEMU.
This helps test LLDB in absence for modern AArch64 features not released
in publicly available hardware till date.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D82064
Expand on the structure of the LLDB test suite. So far this information
has been mostly "tribal knowledge". By writing it down I hope to make it
easier to understand our test suite for anyone that's new to the
project.
Remove all beginning > from the sample commands as my accidental
copy-paste (multiple times...) will discard ./bin/llvm-lit which is
difficult to rebuild (I have to rm -rf and cmake it all again).
Differential Revision: https://reviews.llvm.org/D74296
CMake allows you to set a custom CXX compiler for the API test suite.
However, this variable is never used, because dotest uses the same
compiler to build C and CXX sources.
I'm not sure if this variable was added with the intention of supporting
a different compiler or if this is just a remnant of old functionality.
Given that this hasn't been working for a while, I assume it's safe to
remove.
Differential revision: https://reviews.llvm.org/D69401
The testing documentation appears to be from an era when the only kind
of tests were the lldbsuite python tests. This patch adds a short
description of the unittests and LIT tests and how to run them.
Patch by: Lawrence D'Anna
Differential revision: https://reviews.llvm.org/D67943
llvm-svn: 372797
This patch removes the -q (quiet) flag and changing the default
behavior. Currently the flag serves two purposes that are somewhat
contradictory, as illustrated by the difference between the argument
name (quiet) and the configuration flag (parsable). On the one hand it
reduces output, but on the other hand it prints more output, like the
result of individual tests. My proposal is to guard the extra output
behind the verbose flag and always print the individual test results.
Differential revision: https://reviews.llvm.org/D66837
llvm-svn: 370226
Given that LLDB_TEST_USE_CUSTOM_C_COMPILER and LLDB_TEST_C_COMPILER are
both set at configuration time, I don't really see the point of having
both. This patch simplifies things and uses the custom C/C++ compiler
when the variable is set, and uses the default one when it's not set.
The variable can be unset by passing -ULLDB_TEST_C_COMPILER to CMake.
Differential revision: https://reviews.llvm.org/D66429
llvm-svn: 369435
Now that the Xcode project is removed, I want to focus on dotest as a
test framework, and remove its driver capabilities for which we already
rely on llvm's lit. Removing multiprocessing is the first step in that
direction.
Differential revision: https://reviews.llvm.org/D65311
llvm-svn: 367331
The current LLDB website is written in HTML which is hard to maintain.
We have quite a bit of HTML code checked in which can make it hard to
differentiate between documentation written by us and documentation
generated by a tool.
In line with the other LLVM projects, I propose generating the
documentation with Sphix. I think text/rst files provide a lower barrier
for new or casual contributors to fix or update.
This patch adds a copy of the LLDB website and documentation in
reStructuredText. It also adds a new ninja target `docs-lldb-html` when
-DLLVM_ENABLE_SPHINX:BOOL is enabled.
This is the first step in having the website and documentation being
generated from the repository, rather than having the output checked-in
under the www folder. During the hopefully short transition period,
please also update the reStructuredText files when modifying the
website.
Differential revision: https://reviews.llvm.org/D55376
llvm-svn: 352644
