Allow the main llvm-project repository to contain the buildbot builder
instructions, instead of storing them in llvm-zorg. The corresponding
llvm-zorg PR is https://github.com/llvm/llvm-zorg/pull/648.
Using polly-x86_64-linux-test-suite as a proof-of-concept because that
builder is currently offline, I am its maintainer, and is easier to
build than an configuration supporting offload. Once the design has been
decided, more builders can follow.
Advantages are:
* It is easier to make changes in the llvm-project repository. There are
more reviewers than for the llvm-zorg repository.
* Buildbot changes can be made in the same PR with changes that require
updating the buildbot, e.g. changing the name of a CMake option.
* Configuration changes take effect immeditately when landing; no
buildbot master restart needed.
* Some builders store a CMake cache file in the llvm-project repository
for the reasons above. However, the number of changes that can be made
with a CMake cache file alone are limited.
Compared to AnnotatedBuilder, advantages are:
* Reproducing a buildbot configuration locally made easy: just execute
the script in-place. No llvm-zorg, local buildbot worker, or buildbot
master needed.
* Same for testing a change of a builder before landing it in llvm-zorg.
Doing so with an AnnotatedBuilder requires two llvm-zorg checkouts:
One for making the change of the builder script itself, which then is
pushed to a private llvm-zorg branch on GitHub, and a second that is
modified to fetch that branch instead of
https://github.com/llvm/llvm-zorg/tree/main.
* The AnnotatedBuilder scripts are located in the llvm-zorg repository
and the buildbot-workers always checkout is always the top-of-trunk.
This means that a buildbot configuration is split over three checkouts:
* The checkout of llvm-project to be tested
* The checkout of llvm-zorg by the buildbot-worker fetches; always the
top-of-trunk, i.e may not match the revision of llvm-project that is
executed (such as the CMake cache files located there), especially when
using the "Force build" feature.
* The checkout of llvm-zorg that the buildbot-master is running, which
is updated only when the master is manually restarted.
* The "Force Build" feature also allows for test-building any
llvm-project PR. This is correctly handled by zorg's
`addGetSourcecodeSteps`, but does not work with AnnotatedBuilders that
checkout the llvm-project source on their own.
The goal is to move as much as possible into the llvm-project repository
such that there cannot be a mismatch between checkouts of different
repositories. Ideally, the buildbot-master only needs to be
updated+restarted for adding/removing workers, not for build
configuration changes.
---------
Co-authored-by: Jan Patrick Lehr <jp.lehr@gmail.com>
c55c2ab806