This patch adds a script to run a subset of libc++'s benchmarks for
uploading to LNT.
As part of this patch the test-at-commit script is modified to no longer
build the library itself. Indeed, this provides the necessary
flexibility to run the test suite multiple times on the same built
library, and also addresses previous concerns where test-at-commit
couldn't customize how the library is being built.
Based on review feedback in #160026.
This makes the substitution a lot more clear now that there is no
documentation around %T.
---------
Co-authored-by: Louis Dionne <ldionne.2@gmail.com>
This is extremely useful for analysis purposes like finding regressions.
The ability to run such historical analysis locally is extremely useful
for doing quick investigations that may involve non-mainstream libc++
configurations.
This is useful to perform historical analyses, bisections or establish a
benchmarking baseline after making some changes on a branch. For
example, one can run benchmarks against `main` and easily compare them
to the results on the current feature branch with:
libcxx/utils/test-at-commit --commit $(git merge-base main HEAD) \
-B build/baseline -- <lit args>
libcxx/utils/libcxx-lit build/candidate <lit args>
libcxx/utils/compare-benchmarks \
<(libcxx/utils/consolidate-benchmarks build/baseline) \
<(libcxx/utils/consolidate-benchmarks build/candidate)
Doing this without these scripts would require checking out the desired
baseline, setting up the build directory and running the tests manually.
With these scripts, this can automatically be automated without dirtying
the current checkout.
