- Add option (--report-failures-only) to generate a reduced report for
lit tests that only includes failing tests
- This is a continuation of proposed patches by @gregbedwell here:
- https://reviews.llvm.org/D143516
- https://reviews.llvm.org/D143519
---------
Co-authored-by: Greg Bedwell <greg.bedwell@sony.com>
Co-authored-by: James Henderson <James.Henderson@sony.com>
When running a build like:
```
ninja check-clang check-llvm
```
Prior to my changes you ended up with one results file, in this specific case Junit XML:
```
results.xml
```
This would only include the last set of tests lit ran, which were for
llvm. To get around this, many CI systems will run one check target,
move the file away, then run another, somehow propgating the return code
as well.
```
rectode=0
for target in targets:
ninja target
retcode=$?
mv results.xml results-${target}.xml
<report the overall return code>
```
I want to use something like this Buildkite reporting plugin in CI, which needs to have all the results available:
https://buildkite.com/docs/agent/v3/cli-annotate#using-annotations-to-report-test-results
Modifying CI's build scripts for Windows and Linux is a lot of work. So
my changes instead make lit detect an existing result file and modify
the file name to find a new file to write to. Now you will get:
```
results.xml results.<tempfile generated value>.xml
```
This will work for all result file types since I'm doing it in the base
Report class. Now you've got separate files, it's easy to collect them
with `<path>/*.xml`.
Note that the `<tempfile generated value>` is not ordered.
Encountered several testcase failures when running `ninja check-all`. It
was due to the full path name were shown in the error message instead of
the binary name, and therefore causing the check string mismatch.
The machine was running CentOS 9 with binfmt_misc setup that uses
qemu-aarch64 (8.1.2). Built and ran the unittest as aarch64 host
(through qemu user).
Co-authored-by: h2h <h2h@meta.com>
Line ending policies were changed in the parent, dccebddb3b80. To make
it easier to resolve downstream merge conflicts after line-ending
policies are adjusted this is a separate whitespace-only commit. If you
have merge conflicts as a result, you can simply `git add --renormalize
-u && git merge --continue` or `git add --renormalize -u && git rebase
--continue` - depending on your workflow.
Historically, we've not automatically enforced how git tracks line
endings, but there are many, many commits that "undo" unintended CRLFs
getting into history.
`git log --pretty=oneline --grep=CRLF` shows nearly 100 commits
involving reverts of CRLF making its way into the index and then
history. As far as I can tell, there are none the other way round except
for specific cases like `.bat` files or tests for parsers that need to
accept such sequences.
Of note, one of the earliest of those listed in that output is:
```
commit 9795860250734e5c2a879546c534e35d9edd5944
Author: NAKAMURA Takumi <geek4civic@gmail.com>
Date: Thu Feb 3 11:41:27 2011 +0000
cmake/*: Add svn:eol-style=native and fix CRLF.
llvm-svn: 124793
```
...which introduced such a defacto policy for subversion.
With old versions of git, it's been a bit of a crap-shoot whether
enforcing storing line endings in the history will upset checkouts on
machines where such line endings are the norm. Indeed many users have
enforced that git checks out the working copy according to a global or
per-user config via core crlf, or core autocrlf.
For ~8 years now[1], however, git has supported the ability to "do as
the Romans do" on checkout, but internally store subsets of text files
with line-endings specified via a system of patterns in the
`.gitattributes` file. Since we now have this ability, and we've been
specifying attributes for various binary files, I think it makes sense
to rid us of all that work converting things "back", and just let git
handle the local checkout. Thus the new toplevel policy here is
* text=auto
In simple terms this means "unless otherwise specified, convert all
files considered "text" files to LF in the project history, but check
them out as expected on the local machine. What is "expected on the
local machine" is dependent on configuration and default.
For those files in the repository that *do* need CRLF endings, I've
adopted a policy of `eol=crlf` which means that git will store them in
history with LF, but regardless of user config, they'll be checked out
in tree with CRLF.
Finally, existing files have been "corrected" in history via `git add
--renormalize .`
End users should *not* need to adjust their local git config or
workflow.
[1]: git 2.10 was released with fixed support for fine-grained
line-ending tracking that respects user-config *and* repo policy. This
can be considered the point at which git will respect both the user's
local working tree preference *and* the history as specified by the
maintainers. See
https://github.com/git/git/blob/master/Documentation/RelNotes/2.10.0.txt#L248
for the release note.
Currently we write out a time taken to run all test suites:
```
<testsuites time="8.28">
```
And one for each test:
```
<testcase classname="lldb-shell.Breakpoint" name="breakpoint-command.test" time="2.38"/>
```
However, the schema says there should be one for each suite and test,
but none for testsuites:
cfa434d4b8/JUnit.xsd (L216)
I'm leaving the `testsuites` time in though because no one has
complained so far, and someone out there probably has a script relying
on it by now. Most XML tools handle unknown attributes quite well
anyway.
I'm adding a per testsuite time to comply with the schema and maybe be
more compatible with other JUnit tools.
```
<testsuite name="lldb-shell" ... time="12.34">
```
The test suite time is the sum of the time taken for all tests in the
suite. This will ignore some overhead in setting up the suite, and means
that the sum of the times for all individual suites may not equal the
`testsuites` time.
As we're usually focusing on the execution time of particular tests, not
lit's book keeping, I think this is a reasonable choice.
…tting" (#108104)"
This recommits 0f56ba13bff7ab72bfafcf7c5cf9e5b8bd16d895 (reverted by
6007ad79afeffb1288781b4a7241290386293aff). In the original patch
llvm/utils/lit/tests/escape-color.py failed on Windows because it diffed
llvm-lit output with a file containing '\n' newlines rather than '\r\n'.
This issue is avoided by calling 'diff --strip-trailing-cr'.
Original description below:
Test output that carried color across newlines previously resulted in
the formatting around the output also being colored. Detect the current
ANSI color and reset it when printing formatting, and then reapply it.
As an added bonus an unterminated color code is also detected,
preventing it from leaking out into the rest of the terminal.
Fixes#106633
Test output that carried color across newlines previously resulted in
the formatting around the output also being colored. Detect the current
ANSI color and reset it when printing formatting, and then reapply it.
As an added bonus an unterminated color code is also detected,
preventing it from leaking out into the rest of the terminal.
Fixes#106633
This reverts commit 5af4ba2684b9b59de3bf8135f62e05ab68cfc489.
The previous patch was missing the closing parenthesis `)` in the
`CHECK` statement in the `llvm/utils/lit/tests/shtest-glob.py` file:
`# CHECK: FAIL: shtest-glob :: glob-mkdir.txt ({{[^)]*}}`
This issue broke some build bots. This patch corrects the `CHECK`
statement by adding the closing parenthesis:
`# CHECK: FAIL: shtest-glob :: glob-mkdir.txt ({{[^)]*}})`
This patch introduces a precommit test to verify the current behavior of
glob expansion in lit's internal shell. The motivation for this test
stems from an issue encountered during the BOLT test suite when running
with the lit internal shell using the command:
`LIT_USE_INTERNAL_SHELL=1 ninja check-bolt`
During execution, the following error was observed:
```
File "/usr/local/google/home/harinidonthula/llvm-project/llvm/utils/lit/lit/TestRunner.py", line 416, in executeBuiltinEcho
stdout.write(encode(maybeUnescape(args[-1])))
TypeError: string argument expected, got 'GlobItem'
```
The `executeBuiltinEcho` function in the lit testing framework expects a
string to be passed to `stdout.write`, but it received a `GlobItem`
object instead. This precommit test is designed to check the current
behavior where the glob pattern isn't correctly expanded, leading to
this `TypeError`.
While this patch doesn't fix the issue, it helps in understanding and
verifying the current behavior. The feedback I received from this
[PR](https://github.com/llvm/llvm-project/pull/105925) suggests using
`cmd.args = expand_glob_expressions(cmd.args, shenv.cwd)` to match the
behavior of `executeBuiltinMkdir` and `executeBuiltinRm`, but it is
recognized that the internal shell should ideally expand globs before
calling any built-in command.
**Request for Feedback:**
I'm looking for feedback on how to improve this precommit test,
specifically regarding the handling and expansion of glob patterns for
commands like mkdir and rm within the internal shell. Currently, the
args are expanded at the beginning of these functions, which should
ensure proper glob expansion. However, I'd appreciate guidance on
whether I should write additional tests to verify that mkdir and rm are
handling glob expansions correctly.
If such tests are recommended, I would also appreciate advice on the
best approach to implement them, considering the existing framework and
the way glob expansion is expected to function in the internal shell.
Should these tests confirm that the current implementation passes, or
are there specific edge cases I should be aware of?
**Next Steps:**
In my follow-up PR, I plan to address the UNRESOLVED error by expanding
the entire command, ensuring correct and consistent behavior across all
commands. The current test checks for an unresolved issue with the glob
expansion, specifically looking for a `TypeError` due to an unexpanded
`GlobItem`. This will be updated to reflect the correct behavior once
the issue is resolved.
This change is relevant for [[RFC] Enabling the Lit Internal Shell by
Default](https://discourse.llvm.org/t/rfc-enabling-the-lit-internal-shell-by-default/80179/3)
This patch addresses an issue with lit's internal shell when env is
without any arguments, it fails with exit code 127 because `env`
requires a subcommand. This patch addresses the issue by encoding the
command to properly return environment variables even when no arguments
are provided.
The error occurred when running the command
` LIT_USE_INTERNAL_SHELL=1 ninja check-llvm`.
fixes: #102383
This is part of the test cleanups proposed in the RFC: [[RFC] Enabling
the Lit Internal Shell by
Default](https://discourse.llvm.org/t/rfc-enabling-the-lit-internal-shell-by-default/80179)
This patch fixes the incorrect usage of lit's built-in `export` command.
There is a typo in raising the error itself where the error being raised
had the wrong number of parameters passed in.
Fixes https://github.com/llvm/llvm-project/issues/102386.
Substitutions can be added in a couple different ways; they can be added
via the calling python scripts by adding entries to the
config.substitutions dictionary, or via DEFINE lines in the scripts
themselves.
The substitution strings passed to Python's re classes are interpreted
so that backslashes expand to escape sequences, and literal backslashes
need to be escaped.
On Unix, the script defined substitutions don't (usually, so far)
contain backslashes - but on Windows, they often do, due to paths
containing backslashes. This lead to a Windows specific escaping of
backslashes before doing Python re substitutions - since
7c9eab8fef0ed79a5911d21eb97b6b0fa9d39f82. There's nothing inherently
Windows specific about this though - any intended literal backslashes in
the substitution strings need to be escaped; this is how the Python re
API works.
The DEFINE lines were added later, and in order to cope with
backslashes, escaping of backslashes was added in the SubstDirective
class in TestRunner, applying to DEFINE lines in the tests only.
The fact that the escaping right before passing to the Python re API was
done conditionally on Windows led to two inconsistencies:
- DEFINE lines in the tests that contain backslashes got double
backslashes on Windows. (This was visible as a FIXME in
llvm/utils/lit/tests/Inputs/shtest-define/value-escaped.txt.)
- Script provided substitutions containing backslashes did not work on
Unix, but they did work on Windows.
By removing the escaping from SubstDirective and escaping it
unconditionally in the processLine function, before feeding the
substitutions to Python's re classes, we should have consistent
behaviour across platforms, and get rid of the FIXME in the lit test.
This fixes issues with substitutions containing backslashes on Unix
platforms, as encountered in PR #86649.
This patch separates the lit tests that check for the functionality of
lit's built-in cat command into its own test file and folder. This is a
prerequisite for https://github.com/llvm/llvm-project/pull/101530.
FileCheck has special handline for the `-EMPTY` suffix, that should
match empty lines. Overloading the suffix can be a source of confusion
when reading tests. Additionally, the current implementation seems to
match the following expressions, which appears to be a bug in FileCheck.
LLVM lit assumes control of the test parallelism when running a test
suite. This style of testing doesn't play nicely with build systems like
Buck or Bazel since it prefers finer grained actions on a per-test
level. In order for external build systems to control the test
parallelism, add an option to disable `.lit_test_times.txt` under the
`--skip-test-time-recording` flag, thus allowing other build systems
to determine the parallelism and avoid race conditions when writing
to that file. I went for `--skip-test-time-recording` instead of `--time-tests` in
order to preserve the original functionality of writing to `.lit_test_times.txt`
as the default behavior and only opt-in for those who do _not_ want
`.lit_test_times.txt` file.
This reverts commit 9f6dd1f761a5121e9a69e5d1f67c2438c065b499.
Reverting to investigate buildbot failures e.g.:
"new failure on builder ppc64le-mlir-rhel-clang running on ppc64le-mlir-rhel-test"
My #73541 added lines to
`llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt` so what
was previously on line 7 is now on line 12.
Before:
28412d1800/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt (L7-L8)
After:
6fb7c2d713/llvm/utils/lit/tests/Inputs/testrunner-custom-parsers/test.txt (L12-L13)
This didn't show up in the PR checks, but caused a buildbot failure
after merging, https://lab.llvm.org/buildbot/#/builders/139/builds/54597
:
```
# | ======================================================================
# | FAIL: test_commands (__main__.TestIntegratedTestKeywordParser)
# | ----------------------------------------------------------------------
# | Traceback (most recent call last):
# | File "/home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/utils/lit/tests/unit/TestRunner.py", line 135, in test_commands
# | self.assertEqual(value[1].command.strip(), "%dbg(MY_RUN: at line 7) foo bar")
# | AssertionError: '%dbg(MY_RUN: at line 12) foo bar' != '%dbg(MY_RUN: at line 7) foo bar'
# | - %dbg(MY_RUN: at line 12) foo bar
# | ? ^^
# | + %dbg(MY_RUN: at line 7) foo bar
# | ? ^
```
Apologies for the build break 🙀
This patch adds "nice-to-have" feature in lit.
it prints the total number of discovered tests at the beginning. It is
covenient to see the total number of tests and avoid scrolling up to the
beginning of log.
Further, this patch also prints %ge of tests.
This patch fixes tests pointed by previous attempt of landing this
patch.
Reviewed By: RoboTux, jdenny-ornl
Co-authored-by: Madhur A <madhura@nvidia.com>
Without this patch, the functions `executeScriptInternal` and thus
`runOnce` in `llvm/utils/lit/lit/TestRunner.py` return either a tuple like
`(out, err, exitCode, timeoutInfo)` or a `lit.Test.Result` object. They
return the latter only when there's a lit internal shell parse error in
a RUN line. In my opinion, a more straight-forward way to handle
exceptional cases like that is to use python exceptions.
For that purpose, this patch introduces `ScriptFatal`. Thus, this patch
changes `executeScriptInternal` to always either return the tuple or
raise the `ScriptFatal` exception. It updates `runOnce` and
`libcxx/utils/libcxx/test/format.py` to catch the exception rather than
check for the special return type.
This patch also changes `runOnce` to convert the exception to a
`Test.UNRESOLVED` result instead of `TEST.FAIL`. The former is the
proper result for such a malformed test, for which a rerun (given an
`ALLOW_RETRIES:`) serves no purpose. There are at least two benefits
from this change. First, `_runShTest` no longer has to specially and
cryptically handle this case to avoid unnecessary reruns. Second, an
`XFAIL:` directive no longer hides such a failure [as we saw
previously](https://reviews.llvm.org/D154987#4501125).
To facilitate the `_runShTest` change, this patch inserts the internal
shell parse error diagnostic into the format of the test's normal debug
output rather than suppressing the latter entirely. That change is also
important for [D154987](https://reviews.llvm.org/D154987), which
proposes to reuse `ScriptFatal` for python compile errors in PYTHON
lines or in `config.prologue`. In that case, the diagnostic might follow
debugging output from the test's previous RUN or PYTHON lines, so
suppressing the normal debug output would lose information.
This patch adds "nice-to-have" feature in lit.
it prints the total number of discovered tests at the beginning. It is
covenient to see the total number of tests and avoid scrolling up to the
beginning of log.
Further, this patch also prints %ge of tests.
Reviewed By: RoboTux, jdenny-ornl
Co-authored-by: Madhur A <madhura@nvidia.com>
This patch addresses the missed review comment from PR #67063. It
renames LIT flag "--disable-gtest-sharding" to "--no-gtest-sharding"
and corrects the code style issue.
[unittest] Add lit option to allow disabling sharding in unittest
By default, googletest based unit tests uses sharding to speed up the
testing. However, when these unit tests are run through wrapper program
on a remote platform with large round trip time, the sharding will increase
the time cost dramatically. This patch adds a LLVM LIT option
"--disable-gtest-sharding" to allow sharding on gtest based unittest to
be disabled.
In PR #65242 (landed as 9e739fdb85ac672f3e25e971d96e71823e07ebda), I
claimed that RUN lines cannot contain newlines. Actually, they can after
substitution expansion. More generally, a lit config file can define
substitutions or preamble commands containing newlines. While both of
those cases seem unlikely in practice,
[D154987](https://reviews.llvm.org/D154987) proposes PYTHON directives
where it seems very likely.
Regardless of the use case, without this patch, such newlines break
expansion of `%dbg(RUN: at line N)`, and the fix is simple.
This patch and D154984 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
Motivation
----------
D154984 removes the "Script:" section that lit prints along with a
test's output, and it makes -v and -a imply -vv. For example, after
D154984, the "Script:" section below is never shown, but -v is enough
to produce the execution trace following it:
```
Script:
--
: 'RUN: at line 1'; echo hello | FileCheck bogus.txt && echo success
--
Exit Code: 2
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello"
# command output:
hello
$ "FileCheck" "bogus.txt"
# command stderr:
Could not open check file 'bogus.txt': No such file or directory
error: command failed with exit status: 2
--
```
In the D154984 review, some reviewers point out that they have been
using the "Script:" section for copying and pasting a test's shell
commands to a terminal window. The shell commands as printed in the
execution trace can be harder to copy and paste for the following
reasons:
- They drop redirections and break apart RUN lines at `&&`, `|`, etc.
- They add `$` at the start of every command, which makes it hard to
copy and paste multiple commands in bulk.
- Command stdout, stderr, etc. are interleaved with the commands and
are not clearly delineated.
- They don't always use proper shell quoting. Instead, they blindly
enclose all command-line arguments in double quotes.
Changes
-------
D154984 plus this patch converts the above example into:
```
Exit Code: 2
Command Output (stdout):
--
# RUN: at line 1
echo hello | FileCheck bogus-file.txt && echo success
# executed command: echo hello
# .---command stdout------------
# | hello
# `-----------------------------
# executed command: FileCheck bogus-file.txt
# .---command stderr------------
# | Could not open check file 'bogus-file.txt': No such file or directory
# `-----------------------------
# error: command failed with exit status: 2
--
```
Thus, this patch addresses the above issues as follows:
- The entire execution trace can be copied and pasted in bulk to a
terminal for correct execution of the RUN lines, which are printed
intact as they appeared in the original RUN lines except lit
substitutions are expanded. Everything else in the execution trace
appears in shell comments so it has no effect in a terminal.
- Each of the RUN line's commands is repeated (in shell comments) as
it executes to show (1) that the command actually executed (e.g.,
`echo success` above didn't) and (2) what stdout, stderr, non-zero
exit status, and output files are associated with the command, if
any. Shell quoting in the command is now correct and minimal but is
not necessarily the original shell quoting from the RUN line.
- The start and end of the contents of stdout, stderr, or an output
file is now delineated clearly in the trace.
To help produce some of the above output, this patch extends lit's
internal shell with a built-in `@echo` command. It's like `echo`
except lit suppresses the normal execution trace for `@echo` and just
prints its stdout directly. For now, `@echo` isn't documented for use
in lit tests.
Without this patch, libcxx's custom lit test format tries to parse the
stdout from `lit.TestRunner.executeScriptInternal` (which runs lit's
internal shell) to extract the stdout and stderr produced by shell
commands, and that parse no longer works after the above changes.
This patch makes a small adjustment to
`lit.TestRunner.executeScriptInternal` so libcxx can just request
stdout and stderr without an execution trace.
(As a minor drive-by fix that came up in testing: lit's internal `not`
command now always produces a numeric exit status and never `True`.)
Caveat
------
This patch only makes the above changes for lit's internal shell. In
most cases, we do not know how to force external shells (e.g., bash,
sh, window's `cmd`) to produce execution traces in the manner we want.
To configure a test suite to use lit's internal shell (which is
usually better for test portability than external shells anyway), add
this to the test suite's `lit.cfg` or other configuration file:
```
config.test_format = lit.formats.ShTest(execute_external=False)
```
Reviewed By: MaskRay, awarzynski
Differential Revision: https://reviews.llvm.org/D156954
This patch and D156954 were discussed in
<https://discourse.llvm.org/t/rfc-improving-lits-debug-output/72839>.
**Motivation**: -a shows output from all tests, and -v shows output
from just failed tests. Without this patch, that output from each
test includes a section called "Script:", which includes all shell
commands that lit has computed from RUN directives and will attempt to
run for that test. The effect of -vv (which also implies -v if
neither -a or -v is specified) is to extend that output with shell
commands as they are executing so you can easily see which one failed.
For example, when using lit's internal shell and -vv:
```
Script:
--
: 'RUN: at line 1'; echo hello world
: 'RUN: at line 2'; 3c40 hello world
: 'RUN: at line 3'; echo hello world
--
Exit Code: 127
Command Output (stdout):
--
$ ":" "RUN: at line 1"
$ "echo" "hello" "world"
hello world
$ ":" "RUN: at line 2"
$ "3c40" "hello" "world"
'3c40': command not found
error: command failed with exit status: 127
--
```
Notice that all shell commands that actually execute appear in the
output twice, once for "Script:" and once for -vv. Especially for
tests with many RUN directives, the result is noisy. When searching
through the output for a particular shell command, it is easy to get
lost and mistake shell commands under "Script:" for shell commands
that actually executed.
**Change**: With this patch, a test's output changes in two ways.
First, the "Script:" section is never shown. Second, omitting -vv no
longer disables printing of shell commands as they execute. That is,
-a and -v imply -vv, and so -vv is deprecated as it is just an alias
for -v.
**Secondary motivation**: We are also working to introduce a PYTHON
directive, which can appear between RUN directives. How should PYTHON
directives be represented in the "Script:" section, which has
previously been just a shell script? We could probably think of
something, but adding info about PYTHON directive execution in the -vv
trace seems more straight-forward and more useful.
(This patch also removes a confusing point in the -vv documentation:
at least when using bash as an external shell, -vv echoes commands to
the shell's stderr not stdout.)
Reviewed By: awarzynski, Endill, ldionne, MaskRay
Differential Revision: https://reviews.llvm.org/D154984
D154280 (landed in 64d19542e78a in July, 2023) implements
`--per-test-coverage` (which can also be specified via
`lit_config.per_test_coverage`). However, it has a few issues, which
the current patch addresses:
1. D154280 implements `--per-test-coverage` only for the case that lit
is configured to use an external shell. The current patch extends
the implementation to lit's internal shell.
2. In the case that lit is configured to use an external shell,
regardless of whether `--per-test-coverage` is actually specified,
D154280 causes `%dbg(RUN: at line N)` to be expanded in RUN lines
early and in a manner that is specific to sh-like shells. As a
result, later code in lit that expands it in a shell-specific
manner is useless as there's nothing left to expand. The current
patch cleans up the implementation to avoid useless code.
3. Because of issue 2, D154280 corrupts support for windows `cmd` as
an external shell (effectively comments out all RUN lines with
`:`). The current patch happens to fix that particular corruption
by addressing issue 2. However, D122569 (landed in 1041a9642ba0 in
April, 2022) had already broken support for windows `cmd` as an
external shell (discards RUN lines when expanding `%dbg(RUN: at
line N)`). The current patch does not attempt to fix that bug.
For further details, see the PR discussion of the current patch.
The current patch addresses the above issues by implementing
`--per-test-coverage` before selecting the shell (internal or
external) and by leaving `%dbg(RUN: at line N)` unexpanded there.
Thus, it is expanded later in a shell-specific manner, as before
D154280.
This patch introduces `buildPdbgCommand` into lit's implementation to
encapsulate the process of building (or rebuilding in the case of the
`--per-test-coverage` implementation) a full `%dbg(RUN: at line N)
cmd` line and asserting that the result matches `kPdbgRegex`. It also
cleans up that and all other uses of `kPdbgRegex` to operate on the
full line with `re.fullmatch` not `re.match`. This change better
reflects the intention in every case, but it is expected to be NFC
because `kPdbgRegex` ends in `.*` and thus avoids the difference
between `re.fullmatch` and `re.match`. The only caveat is that `.*`
does not match newlines, but RUN lines cannot contain newlines
currently, so this caveat currently shouldn't matter in practice.
The original `--per-test-coverage` implementation avoided accumulating
`export LLVM_PROFILE_FILE={profile}` insertions across retries (due to
`ALLOW_RETRIES`) by skipping the insertion if `%dbg(RUN: at line N)`
was not present and thus had already been expanded. However, the
current patch makes sure the insertions also happen for commands
without `%dbg(RUN: at line N)`, such as preamble commands or some
commands from other lit test formats. Thus, the current patch
implements a different mechanism to avoid accumulating those
insertions (see code comments).
