llvm-project/llvm/utils/lit/tests/googletest-format-respect-gtest-sharding-env-vars.py
Joe Loser 27731f0475 [llvm][lit] Respect GTEST_TOTAL_SHARDS and GTEST_SHARD_INDEX env vars
There are a variety of issues with using GTest sharding by default for users of
`lit` using the Google Test formatter as mentioned in
https://github.com/llvm/llvm-project/issues/56492 and
https://github.com/llvm/llvm-project/issues/56491.

Currently, there is no way for users to explicitly control the sharding
behavior, even with the environment variables that GTest provides. This patch
teaches the `googletest` formatter to actually respect `GTEST_TOTAL_SHARDS`
and `GTEST_SHARD_INDEX` environment variables if they are set.

In practice, we could go one step further and not do any of the post-processing
of the JSON files if `GTEST_TOTAL_SHARDS` is `1` for example, but that it left
as a follow-up if desired.  There may be preferred alternative approaches to
disabling sharding entirely through another mechanism, such as a lit config
variable.

Differential Revision: https://reviews.llvm.org/D133542
2022-09-09 17:47:21 -06:00

19 lines
800 B
Python

# Check GTEST_TOTAL_SHARDS and GTEST_SHARD_INDEX environment variabls are
# respected when using the googletest formatter.
# RUN: env GTEST_TOTAL_SHARDS=1 GTEST_SHARD_INDEX=0 \
# RUN: not %{lit} -v --order=random %{inputs}/googletest-format-respect-gtest-sharding-env-vars > %t.out
# FIXME: Temporarily dump test output so we can debug failing tests on
# buildbots.
# RUN: cat %t.out
# RUN: FileCheck < %t.out %s
#
# END.
# CHECK: -- Testing:
# CHECK: FAIL: googletest-format :: [[PATH:[Dd]ummy[Ss]ub[Dd]ir/]][[FILE:OneTest\.py]]/0
# CHECK: *** TEST 'googletest-format :: [[PATH]][[FILE]]/0{{.*}} FAILED ***
# CHECK-NEXT: Script(shard):
# CHECK-NEXT: --
# CHECK-NEXT: GTEST_OUTPUT=json:{{[^[:space:]]*}} GTEST_SHUFFLE=1 GTEST_TOTAL_SHARDS=1 GTEST_SHARD_INDEX=0 GTEST_RANDOM_SEED=123 {{.*}}[[FILE]]