34ca553d30
Flang and Flang-RT have two flavours of unittests:
1. GTest unittests, using lit's `lit.formats.GoogleTest` format ending
with `Tests${CMAKE_EXECUTABLE_SUFFIX}`
2. "non-GTest" or "OldUnit" unittests, a plain executable ending with
`.test${CMAKE_EXECUTABLE_SUFFIX}`
Both executables are emitted into the same unittests/ subdirectory. When
running ...
1. `tests/Unit/lit.cfg.py`, only considers executable ending with
`Tests` (or `Tests.exe` on Windows), hence skips the non-GTest tests.
2. `tests/NonGtestUnit/lit.cfg.py` considers all tests ending with
`.test` or `.exe`. On Windows, The GTest unitests also end with `.exe`.
In Flang-RT, `.exe` is considered an extension for non-GTest unitests
which causes tests such as Flang's `RuntimeTests.exe` to be executed for
both on Windows. This particular test includes a file write test, using
a hard-coded filename `ucsfile`. If the two instances are executed
concurrently, they might interfere with each other reading/writing
`ucsfile` which results in a flaky test.
This patch avoids the redundant execution by requiring the suffix
`.test.exe` on Windows. lit has to be modified because it uses
`os.path.splitext` the extract the extension, which would only recognize
the last component. It was changed from the orginal `endswith` in
c865abe747aa72192f02ebfdcabe730f2553e42f
for unknown reasons.
In Flang, `.exe` is not considered a suffix for non-GTest unittests and
hence they are not run at all. Fixing by also added `.test.exe` as valid
suffix, like with Flang-RT.
Unfortunately, the ` Evaluate/real.test.exe` test was failing on
Windows:
```
FAIL: flang-OldUnit :: Evaluate/real.test.exe (3592 of 3592)
******************** TEST 'flang-OldUnit :: Evaluate/real.test.exe' FAILED ********************
..\_src\flang\unittests\Evaluate\real.cpp:511: FAIL: FlagsToBits(prod.flags) == 0x18, not 0x10
0 0x800001 * 0xbf7ffffe
..\_src\flang\unittests\Evaluate\real.cpp:511: FAIL: FlagsToBits(prod.flags) == 0x18, not 0x10
0 0x800001 * 0x3f7ffffe
..\_src\flang\unittests\Evaluate\real.cpp:511: FAIL: FlagsToBits(prod.flags) == 0x18, not 0x10
0 0x80800001 * 0xbf7ffffe
..\_src\flang\unittests\Evaluate\real.cpp:511: FAIL: FlagsToBits(prod.flags) == 0x18, not 0x10
0 0x80800001 * 0x3f7ffffe
...
```
This is due to the `__x86_64__` macro not being set by Microsoft's
cl.exe and hence floating point status flags not being read out. The
equivalent macro for Microsofts compiler is `_M_X64` (or `_M_X64`).
===============================
lit - A Software Testing Tool
===============================
About
=====
*lit* is a portable tool for executing LLVM and Clang style test suites,
summarizing their results, and providing indication of failures. *lit* is
designed to be a lightweight testing tool with as simple a user interface as
possible.
Features
========
* Portable!
* Flexible test discovery.
* Parallel test execution.
* Support for multiple test formats and test suite designs.
Documentation
=============
The official *lit* documentation is in the man page, available online at the LLVM
Command Guide: http://llvm.org/cmds/lit.html.
Source
======
The *lit* source is available as part of LLVM, in the LLVM source repository:
https://github.com/llvm/llvm-project/tree/main/llvm/utils/lit
Contributing to lit
===================
Please browse the issues labeled *tools:llvm-lit* in LLVM's issue tracker for
ideas on what to work on:
https://github.com/llvm/llvm-project/labels/tools%3Allvm-lit
Before submitting patches, run the test suite to ensure nothing has regressed::
# From within your LLVM source directory.
utils/lit/lit.py \
--path /path/to/your/llvm/build/bin \
utils/lit/tests
Note that lit's tests depend on ``not`` and ``FileCheck``, LLVM utilities.
You will need to have built LLVM tools in order to run lit's test suite
successfully.
You'll also want to confirm that lit continues to work when testing LLVM.
Follow the instructions in http://llvm.org/docs/TestingGuide.html to run the
regression test suite:
make check-llvm
And be sure to run the llvm-lit wrapper script as well:
/path/to/your/llvm/build/bin/llvm-lit utils/lit/tests
Finally, make sure lit works when installed via setuptools:
python utils/lit/setup.py install
lit --path /path/to/your/llvm/build/bin utils/lit/tests