[Tools][Docker] Update Dockerfiles and Docker guide (#123841)
Fixes #123669 - Update Dockerfiles to work with the LLVM trunk - Adapt Documentation accordingly - Fix duplicate `-c` flag
This commit is contained in:
Marc Auberer
GitHub
parent
fbd86d05fe
commit
5136c6d9d2
@ -9,7 +9,7 @@ You can find a number of sources to build docker images with LLVM components in
|
||||
images for their own use, or as a starting point for someone who wants to write
|
||||
their own Dockerfiles.
|
||||
|
||||
We currently provide Dockerfiles with ``debian10`` and ``nvidia-cuda`` base images.
|
||||
We currently provide Dockerfiles with ``debian12`` and ``nvidia-cuda`` base images.
|
||||
We also provide an ``example`` image, which contains placeholders that one would need
|
||||
to fill out in order to produce Dockerfiles for a new docker image.
|
||||
|
||||
@ -72,13 +72,13 @@ checkout from git and provide a list of CMake arguments to use during when
|
||||
building LLVM inside docker container.
|
||||
|
||||
Here's a very simple example of getting a docker image with clang binary,
|
||||
compiled by the system compiler in the debian10 image:
|
||||
compiled by the system compiler in the debian12 image:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./llvm/utils/docker/build_docker_image.sh \
|
||||
--source debian10 \
|
||||
--docker-repository clang-debian10 --docker-tag "staging" \
|
||||
--source debian12 \
|
||||
--docker-repository clang-debian12 --docker-tag "staging" \
|
||||
-p clang -i install-clang -i install-clang-resource-headers \
|
||||
-- \
|
||||
-DCMAKE_BUILD_TYPE=Release
|
||||
@ -93,51 +93,45 @@ this command will do that:
|
||||
# LLVM_TARGETS_TO_BUILD=Native is to reduce stage1 compile time.
|
||||
# Options, starting with BOOTSTRAP_* are passed to stage2 cmake invocation.
|
||||
./build_docker_image.sh \
|
||||
--source debian10 \
|
||||
--docker-repository clang-debian10 --docker-tag "staging" \
|
||||
--source debian12 \
|
||||
--docker-repository clang-debian12 --docker-tag "staging" \
|
||||
-p clang -i stage2-install-clang -i stage2-install-clang-resource-headers \
|
||||
-- \
|
||||
-DLLVM_TARGETS_TO_BUILD=Native -DCMAKE_BUILD_TYPE=Release \
|
||||
-DBOOTSTRAP_CMAKE_BUILD_TYPE=Release \
|
||||
-DCLANG_ENABLE_BOOTSTRAP=ON -DCLANG_BOOTSTRAP_TARGETS="install-clang;install-clang-resource-headers"
|
||||
|
||||
This will produce a new image ``clang-debian10:staging`` from the latest
|
||||
This will produce a new image ``clang-debian12:staging`` from the latest
|
||||
upstream revision.
|
||||
After the image is built you can run bash inside a container based on your image
|
||||
like this:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker run -ti clang-debian10:staging bash
|
||||
docker run -ti clang-debian12:staging bash
|
||||
|
||||
Now you can run bash commands as you normally would:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
root@80f351b51825:/# clang -v
|
||||
clang version 5.0.0 (trunk 305064)
|
||||
clang version 19.1.7 (trunk 524462)
|
||||
Target: x86_64-unknown-linux-gnu
|
||||
Target: x86_64-unknown-linux-gnu
|
||||
Thread model: posix
|
||||
InstalledDir: /bin
|
||||
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8
|
||||
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.4
|
||||
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
|
||||
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.2
|
||||
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9
|
||||
Candidate multilib: .;@m64
|
||||
Selected multilib: .;@m64
|
||||
|
||||
|
||||
Which image should I choose?
|
||||
============================
|
||||
We currently provide two images: Debian10-based and nvidia-cuda-based. They
|
||||
We currently provide two images: Debian12-based and nvidia-cuda-based. They
|
||||
differ in the base image that they use, i.e. they have a different set of
|
||||
preinstalled binaries. Debian8 is very minimal, nvidia-cuda is larger, but has
|
||||
preinstalled CUDA libraries and allows to access a GPU, installed on your
|
||||
machine.
|
||||
|
||||
If you need a minimal linux distribution with only clang and libstdc++ included,
|
||||
you should try Debian10-based image.
|
||||
you should try Debian12-based image.
|
||||
|
||||
If you want to use CUDA libraries and have access to a GPU on your machine,
|
||||
you should choose nvidia-cuda-based image and use `nvidia-docker
|
||||
@ -150,7 +144,7 @@ If you have a different use-case, you could create your own image based on
|
||||
``example/`` folder.
|
||||
|
||||
Any docker image can be built and run using only the docker binary, i.e. you can
|
||||
run debian10 build on Fedora or any other Linux distribution. You don't need to
|
||||
run debian12 build on Fedora or any other Linux distribution. You don't need to
|
||||
install CMake, compilers or any other clang dependencies. It is all handled
|
||||
during the build process inside Docker's isolated environment.
|
||||
|
||||
@ -158,12 +152,12 @@ Stable build
|
||||
============
|
||||
If you want a somewhat recent and somewhat stable build, use the
|
||||
``branches/google/stable`` branch, i.e. the following command will produce a
|
||||
Debian10-based image using the latest ``google/stable`` sources for you:
|
||||
Debian12-based image using the latest ``google/stable`` sources for you:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./llvm/utils/docker/build_docker_image.sh \
|
||||
-s debian10 --d clang-debian10 -t "staging" \
|
||||
-s debian12 --d clang-debian12 -t "staging" \
|
||||
--branch branches/google/stable \
|
||||
-p clang -i install-clang -i install-clang-resource-headers \
|
||||
-- \
|
||||
|
||||
@ -23,7 +23,7 @@ Available options:
|
||||
General:
|
||||
-h|--help show this help message
|
||||
Docker-specific:
|
||||
-s|--source image source dir (i.e. debian10, nvidia-cuda, etc)
|
||||
-s|--source image source dir (i.e. debian12, nvidia-cuda, etc)
|
||||
-d|--docker-repository docker repository for the image
|
||||
-t|--docker-tag docker tag for the image
|
||||
Checkout arguments:
|
||||
@ -38,7 +38,7 @@ Available options:
|
||||
-p|--llvm-project Add the project to a list LLVM_ENABLE_PROJECTS, passed to
|
||||
CMake.
|
||||
Can be specified multiple times.
|
||||
-c|--checksums name of a file, containing checksums of llvm checkout.
|
||||
--checksums name of a file, containing checksums of llvm checkout.
|
||||
Script will fail if checksums of the checkout do not
|
||||
match.
|
||||
Build-specific:
|
||||
@ -51,18 +51,18 @@ Required options: --source and --docker-repository, at least one
|
||||
All options after '--' are passed to CMake invocation.
|
||||
|
||||
For example, running:
|
||||
$ build_docker_image.sh -s debian10 -d mydocker/debian10-clang -t latest \
|
||||
$ build_docker_image.sh -s debian12 -d mydocker/debian12-clang -t latest \
|
||||
-p clang -i install-clang -i install-clang-resource-headers
|
||||
will produce two docker images:
|
||||
mydocker/debian10-clang-build:latest - an intermediate image used to compile
|
||||
mydocker/debian12-clang-build:latest - an intermediate image used to compile
|
||||
clang.
|
||||
mydocker/clang-debian10:latest - a small image with preinstalled clang.
|
||||
mydocker/clang-debian12:latest - a small image with preinstalled clang.
|
||||
Please note that this example produces a not very useful installation, since it
|
||||
doesn't override CMake defaults, which produces a Debug and non-boostrapped
|
||||
version of clang.
|
||||
|
||||
To get a 2-stage clang build, you could use this command:
|
||||
$ ./build_docker_image.sh -s debian10 -d mydocker/clang-debian10 -t "latest" \
|
||||
$ ./build_docker_image.sh -s debian12 -d mydocker/clang-debian12 -t "latest" \
|
||||
-p clang -i stage2-install-clang -i stage2-install-clang-resource-headers \
|
||||
-- \
|
||||
-DLLVM_TARGETS_TO_BUILD=Native -DCMAKE_BUILD_TYPE=Release \
|
||||
@ -110,7 +110,7 @@ while [[ $# -gt 0 ]]; do
|
||||
CMAKE_ENABLED_PROJECTS="$CMAKE_ENABLED_PROJECTS;$PROJ"
|
||||
shift 2
|
||||
;;
|
||||
-c|--checksums)
|
||||
--checksums)
|
||||
shift
|
||||
CHECKSUMS_FILE="$1"
|
||||
shift
|
||||
@ -160,7 +160,7 @@ if [ $SEEN_INSTALL_TARGET -eq 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SOURCE_DIR=$(dirname $0)
|
||||
SOURCE_DIR=$(dirname "$0")
|
||||
if [ ! -d "$SOURCE_DIR/$IMAGE_SOURCE" ]; then
|
||||
echo "No sources for '$IMAGE_SOURCE' were found in $SOURCE_DIR"
|
||||
exit 1
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
#===- llvm/utils/docker/debian10/build/Dockerfile -------------------------===//
|
||||
#===- llvm/utils/docker/debian12/build/Dockerfile -------------------------===//
|
||||
#
|
||||
# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
||||
# See https://llvm.org/LICENSE.txt for license information.
|
||||
@ -6,8 +6,8 @@
|
||||
#
|
||||
#===----------------------------------------------------------------------===//
|
||||
# Stage 1. Check out LLVM source code and run the build.
|
||||
FROM launcher.gcr.io/google/debian10:latest as builder
|
||||
LABEL maintainer "LLVM Developers"
|
||||
FROM launcher.gcr.io/google/debian12:latest AS builder
|
||||
LABEL maintainer="LLVM Developers"
|
||||
# Install build dependencies of llvm.
|
||||
# First, Update the apt's source list and include the sources of the packages.
|
||||
RUN grep deb /etc/apt/sources.list | \
|
||||
@ -19,8 +19,8 @@ RUN apt-get update && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
# Install a newer ninja release. It seems the older version in the debian repos
|
||||
# randomly crashes when compiling llvm.
|
||||
RUN wget "https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip" && \
|
||||
echo "d2fea9ff33b3ef353161ed906f260d565ca55b8ca0568fa07b1d2cab90a84a07 ninja-linux.zip" \
|
||||
RUN wget "https://github.com/ninja-build/ninja/releases/download/v1.12.1/ninja-linux.zip" && \
|
||||
echo "6f98805688d19672bd699fbbfa2c2cf0fc054ac3df1f0e6a47664d963d530255 ninja-linux.zip" \
|
||||
| sha256sum -c && \
|
||||
unzip ninja-linux.zip -d /usr/local/bin && \
|
||||
rm ninja-linux.zip
|
||||
@ -37,11 +37,11 @@ RUN /tmp/scripts/build_install_llvm.sh --to /tmp/clang-install ${buildscript_arg
|
||||
|
||||
|
||||
# Stage 2. Produce a minimal release image with build results.
|
||||
FROM launcher.gcr.io/google/debian10:latest
|
||||
LABEL maintainer "LLVM Developers"
|
||||
FROM launcher.gcr.io/google/debian12:latest
|
||||
LABEL maintainer="LLVM Developers"
|
||||
# Install packages for minimal useful image.
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends libstdc++-7-dev binutils && \
|
||||
apt-get install -y --no-install-recommends binutils && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
# Copy build results of stage 1 to /usr/local.
|
||||
COPY --from=builder /tmp/clang-install/ /usr/local/
|
||||
@ -10,9 +10,9 @@
|
||||
|
||||
# Stage 1. Check out LLVM source code and run the build.
|
||||
# FIXME: Replace 'ubuntu' with your base image
|
||||
FROM ubuntu as builder
|
||||
FROM ubuntu AS builder
|
||||
# FIXME: Change maintainer name
|
||||
LABEL maintainer "Maintainer <maintainer@email>"
|
||||
LABEL maintainer="Maintainer <maintainer@email>"
|
||||
# FIXME: Install llvm/clang build dependencies here. Including compiler to
|
||||
# build stage1, cmake, subversion, ninja, etc.
|
||||
|
||||
@ -31,7 +31,7 @@ RUN /tmp/scripts/build_install_llvm.sh --to /tmp/clang-install ${buildscript_arg
|
||||
# FIXME: Replace 'ubuntu' with your base image.
|
||||
FROM ubuntu
|
||||
# FIXME: Change maintainer name.
|
||||
LABEL maintainer "Maintainer <maintainer@email>"
|
||||
LABEL maintainer="Maintainer <maintainer@email>"
|
||||
# FIXME: Install all packages you want to have in your release container.
|
||||
# A minimal useful installation should include at least libstdc++ and binutils.
|
||||
|
||||
|
||||
@ -6,11 +6,11 @@
|
||||
#
|
||||
#===----------------------------------------------------------------------===//
|
||||
# Stage 1. Check out LLVM source code and run the build.
|
||||
FROM nvidia/cuda:8.0-devel as builder
|
||||
LABEL maintainer "LLVM Developers"
|
||||
FROM nvidia/cuda:12.6.3-devel-ubuntu24.04 AS builder
|
||||
LABEL maintainer="LLVM Developers"
|
||||
# Install llvm build dependencies.
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends ca-certificates cmake python \
|
||||
apt-get install -y --no-install-recommends ca-certificates cmake 2to3 python-is-python3 \
|
||||
subversion ninja-build git && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
@ -26,8 +26,8 @@ RUN /tmp/scripts/build_install_llvm.sh --to /tmp/clang-install ${buildscript_arg
|
||||
|
||||
|
||||
# Stage 2. Produce a minimal release image with build results.
|
||||
FROM nvidia/cuda:8.0-devel
|
||||
LABEL maintainer "LLVM Developers"
|
||||
FROM nvidia/cuda:12.6.3-devel-ubuntu24.04
|
||||
LABEL maintainer="LLVM Developers"
|
||||
# Copy clang installation into this container.
|
||||
COPY --from=builder /tmp/clang-install/ /usr/local/
|
||||
# C++ standard library and binutils are already included in the base package.
|
||||
|
||||
@ -18,7 +18,7 @@ Checkout git sources into /tmp/clang-build/src. Used inside a docker container.
|
||||
Available options:
|
||||
-h|--help show this help message
|
||||
-b|--branch git branch to checkout, i.e. 'main',
|
||||
'release/10.x'
|
||||
'release/19.x'
|
||||
(default: 'main')
|
||||
-r|--revision git revision to checkout
|
||||
-c|--cherrypick revision to cherry-pick. Can be specified multiple times.
|
||||
@ -83,7 +83,7 @@ function apply_cherrypicks() {
|
||||
# This function is always called on a sorted list of cherrypicks.
|
||||
for CHERRY_REV in $CHERRYPICKS; do
|
||||
echo "Cherry-picking $CHERRY_REV into $CHECKOUT_DIR"
|
||||
EMAIL="someone@somewhere.net" git cherry-pick $CHERRY_REV
|
||||
EMAIL="someone@somewhere.net" git cherry-pick "$CHERRY_REV"
|
||||
done
|
||||
|
||||
popd
|
||||
@ -97,7 +97,7 @@ mkdir -p "$CLANG_BUILD_DIR/src"
|
||||
CHECKOUT_DIR="$CLANG_BUILD_DIR/src"
|
||||
|
||||
echo "Checking out https://github.com/llvm/llvm-project.git to $CHECKOUT_DIR"
|
||||
git clone -b $LLVM_BRANCH --single-branch \
|
||||
git clone -b "$LLVM_BRANCH" --single-branch \
|
||||
"https://github.com/llvm/llvm-project.git" \
|
||||
"$CHECKOUT_DIR"
|
||||
|
||||
@ -114,7 +114,7 @@ CHECKSUMS_FILE="/tmp/checksums/checksums.txt"
|
||||
|
||||
if [ -f "$CHECKSUMS_FILE" ]; then
|
||||
echo "Validating checksums for LLVM checkout..."
|
||||
python "$(dirname $0)/llvm_checksum/llvm_checksum.py" -c "$CHECKSUMS_FILE" \
|
||||
python "$(dirname "$0")/llvm_checksum/llvm_checksum.py" -c "$CHECKSUMS_FILE" \
|
||||
--partial --multi_dir "$CLANG_BUILD_DIR/src"
|
||||
else
|
||||
echo "Skipping checksumming checks..."
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user