shylie/llvm-project
1
0
Fork 0
Code Issues Pull Requests Actions 6 Packages Projects Releases Wiki Activity

[Buildbot][Polly] Move polly-x86_64-linux-test-suite build instructions into main repository (#166809)

Browse Source 
Allow the main llvm-project repository to contain the buildbot builder
instructions, instead of storing them in llvm-zorg. The corresponding
llvm-zorg PR is https://github.com/llvm/llvm-zorg/pull/648.

Using polly-x86_64-linux-test-suite as a proof-of-concept because that
builder is currently offline, I am its maintainer, and is easier to
build than an configuration supporting offload. Once the design has been
decided, more builders can follow.

Advantages are:
* It is easier to make changes in the llvm-project repository. There are
  more reviewers than for the llvm-zorg repository.
* Buildbot changes can be made in the same PR with changes that require
  updating the buildbot, e.g. changing the name of a CMake option.
* Configuration changes take effect immeditately when landing; no
  buildbot master restart needed.
* Some builders store a CMake cache file in the llvm-project repository
  for the reasons above. However, the number of changes that can be made
  with a CMake cache file alone are limited.

Compared to AnnotatedBuilder, advantages are:
* Reproducing a buildbot configuration locally made easy: just execute
  the script in-place. No llvm-zorg, local buildbot worker, or buildbot
  master needed.
* Same for testing a change of a builder before landing it in llvm-zorg.
  Doing so with an AnnotatedBuilder requires two llvm-zorg checkouts:
  One for making the change of the builder script itself, which then is
  pushed to a private llvm-zorg branch on GitHub, and a second that is
  modified to fetch that branch instead of
  https://github.com/llvm/llvm-zorg/tree/main.
* The AnnotatedBuilder scripts are located in the llvm-zorg repository
  and the buildbot-workers always checkout is always the top-of-trunk.
  This means that a buildbot configuration is split over three checkouts:
     * The checkout of llvm-project to be tested
     * The checkout of llvm-zorg by the buildbot-worker fetches; always the
        top-of-trunk, i.e may not match the revision of llvm-project that is
        executed (such as the CMake cache files located there), especially when
        using the "Force build" feature.
     * The checkout of llvm-zorg that the buildbot-master is running, which
       is updated only when the master is manually restarted.
* The "Force Build" feature also allows for test-building any
  llvm-project PR. This is correctly handled by zorg's
  `addGetSourcecodeSteps`, but does not work with AnnotatedBuilders that
  checkout the llvm-project source on their own.

The goal is to move as much as possible into the llvm-project repository
such that there cannot be a mismatch between checkouts of different
repositories. Ideally, the buildbot-master only needs to be
updated+restarted for adding/removing workers, not for build
configuration changes.

---------

Co-authored-by: Jan Patrick Lehr <jp.lehr@gmail.com>
This commit is contained in:
Michael Kruse 2026-01-07 10:02:04 +01:00 committed by GitHub
parent c85b8ff4d7
commit c55c2ab806
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 714 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
.ci/buildbot/README.md Normal file
View File

@ -0,0 +1,49 @@
# ScriptedBuilder Buildbot Workers
This directory contains code shared by LLVM Buildbot workers. The typical
pipeline of a ScriptedBuilder-based builder is as follows.
1. A commit is pushed to [main](https://github.com/llvm/llvm-project/tree/main)
2. The [Buildbot master](https://lab.llvm.org/) polls the repository and finds
new commits. It schedules build requests on every relevant worker.
Alternatively, a build request of a specific llvm-project commit can be
created using the "Force Build" or "Rebuild" buttons.
3. When a worker is ready, the master sends the build steps determined by
[ScriptedBuilder](https://github.com/llvm/llvm-zorg/blob/main/zorg/buildbot/builders/ScriptedBuilder.py)
to the worker.
4. The checkout step checks out llvm-project commit into a directory named
`llvm.src` on the worker.
5. The annotate step executes a predefined Python script from the llvm-project
source tree on the worker. Its working directory is an initially empty
sibling directory named `build`. The argument `--workdir=.` is passed to
override the default build directory (which is different to avoid
accidentally spilling clutter into the cwd)
6. The script is expected to use the utilities from
[`worker.py`](https://github.com/llvm/llvm-project/blob/main/.ci/buildbot/worker.py)
to build LLVM. The `with w.step("stepname"):` pattern is used to visually
separate additional steps in the [Buildbot GUI](https://lab.llvm.org/).
## Reproducing Builds
Users can execute the worker script directly to reproduce a build problem with
a worker using the llvm-project source tree in which it is located. By default
it will use a new directory with `.workdir` suffix (so it can be
`.gitignore`-ignored) next to the script as build directory.
The ScriptedBuilder system tries to keep all worker/build settings within the
script, but some parameters can be overridden using command line parameters.
For instance, `--jobs` overrides the Ninja and llvm-lit `-j` argument that the
worker would use. Script should be written to honor these overrides where they
apply, and may also add additional ones.
See
[`worker.py`](https://github.com/llvm/llvm-project/blob/main/.ci/buildbot/worker.py)
or a
[reference script](https://github.com/llvm/llvm-project/blob/main/polly/polly-x86_64-linux-test-suite)
for further details.

577
.ci/buildbot/worker.py Normal file
View File

@ -0,0 +1,577 @@
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
"""Utilities for ScriptedBuilder Buildbot worker scripts"""
import argparse
import filecmp
import os
import stat
import pathlib
import re
import shlex
import shutil
import subprocess
import sys
import traceback
import platform
import multiprocessing
from contextlib import contextmanager
_SHQUOTE_WINDOWS_ESCAPEDCHARS = re.compile(r'(["\\])')
_SHQUOTE_WINDOWS_QUOTEDCHARS = re.compile("[ \t\n]")
def _shquote_windows(txt):
"""shlex.quote for Windows cmd.exe"""
txt = txt.replace("%", "%%")
quoted = re.sub(_SHQUOTE_WINDOWS_ESCAPEDCHARS, r"\\\1", txt)
if len(quoted) == len(txt) and not _SHQUOTE_WINDOWS_QUOTEDCHARS.search(txt):
return txt
else:
return '"' + quoted + '"'
def shjoin(args):
"""Convert a list of shell arguments to an appropriately quoted string."""
if os.name in set(("nt", "os2", "ce")):
return " ".join(map(_shquote_windows, args))
else:
return shlex.join(args)
def report(msg):
"""
Emit a message to the build log. Appears in red font. Lines surrounded
by @@@ may be interpreted as meta-instructions.
"""
print(msg, file=sys.stderr, flush=True)
def report_prog_version(name, cmd):
try:
p = subprocess.run(cmd, check=True, capture_output=True, text=True)
outlines = p.stdout.strip().splitlines()
report_list(name, outlines[0])
except BaseException:
pass
def report_list(category, *items):
items = list(items)
filtered = []
while items:
item = items.pop()
match item:
case tuple() | list():
items += item
continue
case None:
continue
case _:
item = str(item).strip()
if not item:
continue
if item in filtered:
continue
filtered.append(item)
category += ":"
report(f"{category:<9}{', '.join(reversed( filtered))}")
def report_platform():
report_list(
"CPU",
platform.machine(),
platform.architecture()[0],
platform.processor(),
f"{multiprocessing.cpu_count()} native threads",
)
try:
releaseinfo = platform.freedesktop_os_release()
except BaseException:
releaseinfo = dict()
report_list(
"OS",
platform.system(),
platform.architecture()[1],
platform.platform(),
releaseinfo.get("PRETTY_NAME"),
)
report_list("Python", platform.python_implementation(), platform.python_version())
report_prog_version("CMake", ["cmake", "--version"])
report_prog_version("Ninja", ["ninja", "--version"])
report_prog_version("Sphinx", ["sphinx-build", "--version"])
report_prog_version("Doxygen", ["doxygen", "--version"])
report_prog_version("gcc", ["gcc", "--version"])
report_prog_version("ld", ["ld", "--version"])
report_prog_version("LLVM", ["llvm-config", "--version"])
report_prog_version("Clang", ["clang", "--version"])
report_prog_version("LLD", ["ld.lld", "--version"])
def run_command(cmd, shell=False, **kwargs):
"""
Report which command is being run, then execute it using
subprocess.check_call.
"""
report(f"Running: {cmd if shell else shjoin(cmd)}")
sys.stderr.flush()
subprocess.check_call(cmd, shell=shell, **kwargs)
def _remove_readonly(func, path, _):
"""Clear the readonly bit and reattempt the removal."""
try:
os.chmod(path, stat.S_IWRITE)
except Exception:
pass
func(path)
def rmtree(path):
"""Remove directory path and all its subdirectories. Includes a workaround
for Windows where shutil.rmtree errors on read-only files.
Taken from official Python docs
https://docs.python.org/3/library/shutil.html#rmtree-example
"""
shutil.rmtree(path, onexc=_remove_readonly)
def try_delete(path):
"""
Delete the file or directory;
if not successful, print a warning but continue
"""
try:
os.unlink(path)
except Exception:
try:
_remove_readonly(os.unlink, path, _)
except Exception:
try:
rmtree(path)
except Exception as e:
print(f"Warning: Could not delete {path}: {e}")
def checkout(giturl, sourcepath):
"""
Use git to checkout the remote repository giturl at local directory
sourcepath.
If the repository already exists, clear all local changes and check out the
latest main branch.
"""
if not os.path.exists(sourcepath):
run_command(["git", "clone", giturl, sourcepath])
# Reset repository state no matter what there was before
run_command(["git", "-C", sourcepath, "stash", "--all"])
run_command(["git", "-C", sourcepath, "stash", "clear"])
# Fetch and checkout the newest
run_command(["git", "-C", sourcepath, "fetch", "origin"])
run_command(["git", "-C", sourcepath, "checkout", "origin/main", "--detach"])
@contextmanager
def step(step_name, halt_on_fail=False):
"""Report a new build step being started.
Use like this::
with step("greet-step"):
report("Hello World!")
"""
# Barrier to separate stdio output for the the previous step
sys.stderr.flush()
sys.stdout.flush()
report(f"@@@BUILD_STEP {step_name}@@@")
if halt_on_fail:
report("@@@HALT_ON_FAILURE@@@")
try:
yield
except Exception as e:
if isinstance(e, subprocess.CalledProcessError):
report(f"{shjoin(e.cmd)} exited with return code {e.returncode}.")
report("@@@STEP_FAILURE@@@")
else:
traceback.print_exc()
report("@@@STEP_EXCEPTION@@@")
if halt_on_fail:
# Do not continue with the next steps, but allow except/finally
# blocks to execute
raise e
class Worker:
"""Helper class to keep context in a worker.run() environment"""
def __init__(self, args, clean, clobber, workdir, jobs, cachefile, llvmsrcroot):
self.args = args
self.clean = clean
self.clobber = clobber
self.workdir = workdir
self.jobs = jobs
self.cachefile = cachefile
self.llvmsrcroot = llvmsrcroot
def in_llvmsrc(self, path):
"""
Convert a path in the llvm-project source checkout to an absolute path
"""
return os.path.join(self.llvmsrcroot, path)
def in_workdir(self, path):
"""Convert a path in the workdir to an absolute path"""
return os.path.join(self.workdir, path)
def run_ninja(
self, targets: list = [], *, builddir, ccache_stats: bool = False, **kwargs
):
"""
Run ninja in builddir. If self.jobs is set, automatically adds a
-j option to set the number of parallel jobs.
Parameters
----------
targets : list
List of build targets; build the default target 'all' if list is
empty
builddir
Directory of the build.ninja file
ccache_stats : bool
If true, also emit ccache statistics when finishing the build
"""
cmd = ["ninja"]
if builddir is not None:
cmd += ["-C", builddir]
cmd += targets
if self.jobs:
cmd.append(f"-j{self.jobs}")
if ccache_stats:
run_command(["ccache", "-z"])
try:
run_command(cmd, **kwargs)
finally:
# TODO: Pipe to stderr to separate from build log itself
run_command(["ccache", "-sv"])
else:
run_command(cmd, **kwargs)
@contextmanager
def step(self, step_name, halt_on_fail=False):
"""Convenience wrapper for step()"""
with step(step_name, halt_on_fail=halt_on_fail) as s:
yield s
def report(self, msg):
"""Convenience wrapper for report()"""
report(msg)
def run_command(self, *args, **kwargs):
"""Convenience wrapper for run_command()"""
return run_command(*args, **kwargs)
def rmtree(self, *args, **kwargs):
"""Convenience wrapper for rmtree()"""
return rmtree(*args, *kwargs)
def checkout(self, giturl, sourcepath):
"""Convenience wrapper for checkout()"""
return checkout(giturl, sourcepath)
def convert_bool(v):
"""Convert input to bool type
Use to convert the value of bool environment variables. Specifically, the
buildbot master sets 'false' to build properties, which by default Python
would interpret as true-ish.
"""
match v:
case None:
return False
case bool(b):
return b
case str(s):
return not s.strip().upper() in ["", "0", "N", "NO", "FALSE", "OFF"]
case _:
return bool(v)
def relative_if_possible(path, relative_to):
"""
Like os.path.relpath, but does not fail if path is not a parent of
relative_to; keeps the original path in that case
"""
path = os.path.normpath(path)
if not os.path.isabs(path):
# Path is already relative (assumed to relative_to)
return path
try:
result = os.path.relpath(path, start=relative_to)
return result if result else path
except ValueError:
return path
@contextmanager
def run(
scriptpath,
llvmsrcroot,
parser=None,
cachefile=None,
clobberpaths=[],
workerjobs=None,
incremental=None,
):
"""
Runs the boilerplate for a ScriptedBuilder buildbot. It is not necessary to
use this function (one can also call run_command() etc. directly), but
allows for some more flexibility and safety checks. Arguments passed to this
function represent the worker configuration.
We use the term 'clean' for resetting the worker to an empty state. This
involves deleting ${prefix}/llvm.src as well as ${prefix}/build.
The term 'clobber' means deleting build artifacts, but not already
downloaded git repositories. Build artifacts include build- and
install-directories. Changes in the llvm.src directory will
either be force-reset by the buildbot's 'checkout' step anyway,
or -- in case of local invocation -- represents the source the user wants
to reproduce without being tied to a specific commit. In either case the
source directories should not be touched. We consider 'clean' to comprise
'clobber'. llvm-zorg also uses the term 'clean_obj' instead of 'clobber'.
By default, we will always clobber to get the same starting point at every
build. If incremental=True or the --incremental command line option is used,
the starting point is the previous build.
A buildbot worker will invoke this script using this directory structure,
where ${prefix} is a dedicated directory for this builder:
${prefix}/llvm.src # Checkout location for the llvm-source
${prefix}/build # cwd when launching the build script
The build script is called with --workdir=. parameter, i.e. the build
artifacts are written into ${prefix}/build. When cleaning, the worker (NOT
the build script) will delete ${prefix}/llvm.src; Deleting any contents of
${prefix}/build is to be done by the builder script, e.g. by this function.
The builder script can choose to not delete the complete workdir, e.g.
additional source checkouts such as the llvm-test-suite.
The buildbot master will set the 'clean' build property and the environment
variable BUILDBOT_CLEAN when in the GUI the option "Clean source code and
build directory" is checked by the user. The 'clean_obj' build property and
the BUILDBOT_CLEAN_OBJ environment variable will be set when either the
"Clean build directory" GUI option is set, or the master detects a change
to a CMakeLists.txt or *.cmake file.
Parameters
----------
scriptpath
Pass __file__ from the main builder script.
llvmsrcroot
Absolute path to the llvm-project source checkout. Since the builder
script is supposed to be a part of llvm-project itself, the builder
script can compute it from __file__.
parser
Use this argparse.ArgumentParser instead of creating a new one. Allows
adding additional command line switches in addition to the pre-defined
ones. Build scripts are encouraged to apply the pre-defined switches.
cachefile
Path (relative to llvmsrcroot) of the CMake cache file to
use. `None` indicates that the script does not use a cache file. Can be
overridden using --cachefile.
clobberpaths
Directories relative to workdir that need to be deleted if the build
configuration changes (due to changes of CMakeLists.txt or changes of
configuration parameters). Typically, only source checkouts are not
deleted.
workerjobs
Default number of build and test jobs; If set, expected to be the number
of jobs of the actual buildbot worker that executes this script. Can be
overridden using the --jobs parameter so in case someone needs to
reproduce this build, they can adjust the number of jobs for the
reproducer platform. Alternatively, the worker can set the
BUILDBOT_JOBS environment variable or keep ninja/llvm-lit defaults.
incremental
Only clobber the build artifacts when the build configuration changes.
Can be overridden using --incremental.
"""
scriptpath = os.path.abspath(scriptpath)
llvmsrcroot = os.path.abspath(llvmsrcroot)
stem = pathlib.Path(scriptpath).stem
workdir_default = f"{stem}.workdir"
jobs_default = None
if jobs_env := os.environ.get("BUILDBOT_JOBS"):
jobs_default = int(jobs_env)
if not jobs_default:
jobs_default = workerjobs
if not jobs_default:
jobs_default = None
incremental_default = None if incremental else False
parser = parser or argparse.ArgumentParser(
allow_abbrev=True,
description="When executed without arguments, builds the worker's "
f"LLVM build configuration in {os.path.abspath(workdir_default)}. "
"Some build configuration parameters can be altered using the "
"following switches:",
)
parser.add_argument(
"--workdir",
default=workdir_default,
help="Use this dir (relative to cwd) as workdir to write the build "
"artifacts into; --workdir=. uses the current directory.\nWarning: The "
"content of this directory may be deleted",
)
if cachefile is not None:
parser.add_argument(
"--cachefile",
default=relative_if_possible(cachefile, llvmsrcroot),
help="File containing the initial values for the CMakeCache.txt "
"for the llvm build.",
)
parser.add_argument(
"--clean",
action=argparse.BooleanOptionalAction,
default=convert_bool(os.environ.get("BUILDBOT_CLEAN")),
help="Delete the entire workdir before starting the build, including "
"source directories",
)
parser.add_argument(
"--incremental",
action=argparse.BooleanOptionalAction,
default=incremental_default,
help="Keep previous build artifacts when starting the build",
)
parser.add_argument(
"--jobs",
"-j",
type=int,
default=jobs_default,
help="Number of build- and test-jobs",
)
args = parser.parse_args()
workdir = os.path.abspath(args.workdir)
incremental = args.incremental
clean = args.clean
if cachefile is not None:
cachefile = os.path.join(llvmsrcroot, args.cachefile)
if not os.path.isfile(cachefile):
raise Exception(f"--cachefile={cachefile} does not exist")
prevcachepath = os.path.join(workdir, "prevcache.cmake")
prevscriptpath = os.path.join(workdir, "prevscript.py")
if clean:
# Clean implies clobber
clobber = False
elif incremental is None:
# Automatically determine whether to clobber
def has_config_change():
# Has the master scheduler determined a CMakeLists.txt has changed?
if convert_bool(os.environ.get("BUILDBOT_CLOBBER")):
return True
if convert_bool(os.environ.get("BUILDBOT_CLEAN_OBJ")):
return True
# Has the build script changed?
if not os.path.isfile(prevscriptpath):
return True
if not filecmp.cmp(scriptpath, prevscriptpath, shallow=False):
return True
# Has the cache file (if any) changed?
if cachefile:
if not os.path.isfile(prevcachepath):
return True
if not os.path.isfile(cachefile):
return True
if not filecmp.cmp(cachefile, prevcachepath, shallow=False):
return True
return False
clobber = has_config_change()
else:
# Adhere to explicitly set incremental option
clobber = not incremental
# Safety check
parentdir = os.path.dirname(scriptpath)
while True:
if os.path.exists(workdir) and os.path.samefile(parentdir, workdir):
raise Exception(
f"Cannot use {args.workdir} as workdir; it contains the source "
"itself in '{parentdir}'"
)
newparentdir = os.path.dirname(parentdir)
if newparentdir == parentdir:
break
parentdir = newparentdir
w = Worker(
args,
clean=clean,
clobber=clobber,
workdir=workdir,
jobs=args.jobs,
cachefile=cachefile,
llvmsrcroot=llvmsrcroot,
)
with step("platform-info"):
report_platform()
# Ensure that the cwd is not the directory we are going to delete. This
# would not work e.g. under Windows. We will chdir to workdir in the next
# step anyway.
os.chdir("/")
if clean:
if os.path.exists(workdir):
print("Deleting previous build state including sources", file=sys.stderr)
with w.step(f"clean"):
if os.path.exists(workdir):
# Do not delete the directory itself, just the contents; it might be
# a symlink to somewhere else
for d in os.listdir(workdir):
try_delete(os.path.join(workdir, d))
elif clobber:
# Warn user if deleting anything
for p in clobberpaths:
if os.path.exists(os.path.join(workdir, p)):
print(
"Deleting previous build artifacts; use --incremental to keep",
file=sys.stderr,
)
break
with w.step(f"clobber"):
for d in clobberpaths:
try_delete(os.path.join(workdir, d))
try_delete(prevscriptpath)
try_delete(prevcachepath)
os.makedirs(workdir, exist_ok=True)
os.chdir(workdir)
# Remember used script and cachefile to detect changes
shutil.copy(scriptpath, prevscriptpath)
if cachefile:
shutil.copy(cachefile, prevcachepath)
os.environ["NINJA_STATUS"] = "[%p/%es :: %u->%r->%f (of %t)] "
yield w

1
polly/ci/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
/*.workdir

11
polly/ci/polly-x86_64-linux-test-suite.cmake Normal file
View File

@ -0,0 +1,11 @@
# General settings
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "")
set(CMAKE_C_COMPILER_LAUNCHER "ccache" CACHE STRING "")
set(CMAKE_CXX_COMPILER_LAUNCHER "ccache" CACHE STRING "")
set(LLVM_ENABLE_LLD ON CACHE BOOL "")
set(LLVM_ENABLE_ASSERTIONS ON CACHE BOOL "")
set(LLVM_ENABLE_PROJECTS "clang;polly" CACHE STRING "")
set(LLVM_TARGETS_TO_BUILD "X86" CACHE STRING "")
set(LLVM_POLLY_LINK_INTO_TOOLS ON CACHE BOOL "")

76
polly/ci/polly-x86_64-linux-test-suite.py Normal file
View File

@ -0,0 +1,76 @@
#! /usr/bin/env python3
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
# See https://llvm.org/LICENSE.txt for license information.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
"""Check Polly optimizations on llvm-test-suite"""
import os
import sys
# Adapt to location in source tree
llvmsrcroot = os.path.normpath(f"{__file__}/../../..")
sys.path.insert(0, os.path.join(llvmsrcroot, ".ci/buildbot"))
import worker
llvmbuilddir = "llvm.build"
llvminstalldir = "llvm.install"
testsuitesrcdir = "testsuite.src"
testsuitebuilddir = "testsuite.build"
with worker.run(
__file__,
llvmsrcroot,
cachefile="polly/ci/polly-x86_64-linux-test-suite.cmake",
clobberpaths=[llvmbuilddir, testsuitebuilddir, llvminstalldir],
incremental=True,
) as w:
with w.step("configure-llvm", halt_on_fail=True):
cmakecmd = [
"cmake",
f"-S{w.in_llvmsrc('llvm')}",
f"-B{llvmbuilddir}",
"-GNinja",
f"-C{w.in_llvmsrc(w.cachefile)}",
f"-DCMAKE_INSTALL_PREFIX={llvminstalldir}",
]
if w.jobs:
cmakecmd.append(f"-DLLVM_LIT_ARGS=-sv;-j{w.jobs}")
w.run_command(cmakecmd)
with w.step("build-llvm", halt_on_fail=True):
w.run_ninja(builddir=llvmbuilddir, ccache_stats=True)
with w.step("check-polly"):
w.run_ninja(["check-polly"], builddir=llvmbuilddir)
with w.step("install-llvm", halt_on_fail=True):
w.run_ninja(["install"], builddir=llvmbuilddir)
with w.step("checkout-testsuite", halt_on_fail=True):
w.checkout("https://github.com/llvm/llvm-test-suite", testsuitesrcdir)
with w.step("configure-testsuite", halt_on_fail=True):
jobsarg = f";-j{w.jobs}" if w.jobs else ""
w.run_command(
[
"cmake",
f"-S{testsuitesrcdir}",
f"-B{testsuitebuilddir}",
"-GNinja",
"-DCMAKE_BUILD_TYPE=Release",
f"-DCMAKE_C_COMPILER={os.path.abspath(llvminstalldir)}/bin/clang",
f"-DCMAKE_CXX_COMPILER={os.path.abspath(llvminstalldir)}/bin/clang++",
f"-DTEST_SUITE_LIT={os.path.abspath(llvmbuilddir)}/bin/llvm-lit",
f"-DTEST_SUITE_LLVM_SIZE={os.path.abspath(llvmbuilddir)}/bin/llvm-size",
"-DTEST_SUITE_EXTRA_C_FLAGS=-Wno-unused-command-line-argument -mllvm -polly",
"-DTEST_SUITE_EXTRA_CXX_FLAGS=-Wno-unused-command-line-argument -mllvm -polly",
f"-DLLVM_LIT_ARGS=-sv{jobsarg};-o;report.json",
]
)
with w.step("build-testsuite", halt_on_fail=True):
w.run_ninja(builddir=testsuitebuilddir)
with w.step("check-testsuite"):
w.run_ninja(["check"], builddir=testsuitebuilddir)