270 Commits

Author SHA1 Message Date
Tom Stellard
9cea288bf7
[release] Fix version extraction in export.sh (#85328)
The LLVM_VERSION_* variables were moved to a new file in
81e20472a0c5a4a8edc5ec38dc345d580681af81.
2024-03-18 15:32:10 +01:00
Tom Stellard
0b9ce71a25
github-upload-release.py: Fix bug preventing release creation (#84571)
After aa02002491333c42060373bc84f1ff5d2c76b4ce we started passing the
user name to the create_release function and this was being interpreted
as the git tag.
2024-03-08 21:17:27 -08:00
Benoît Amiaux
52ada07ef5
build_llvm_release.bat: add tarball export to x64 release (#79840)
Like linux releases, export a tar.xz files containing most llvm tools,
including non toolchain utilities, llvm-config, llvm-link and others.

We do this by reconfiguring cmake one last time at the last step,
running the install target so we do not need to recompile anything.

Fix #51192
Fix #53052
2024-02-23 14:49:57 +01:00
Tom Stellard
2836d8edbf
[workflows] Fix permissions check for creating new releases (#81163)
The default GitHub token does not have read permissions on the org, so
we need to use a custom token in order to read the members of the
llvm-release-managers team.
2024-02-20 17:52:38 -08:00
Rainer Orth
f6ac598c10
[Release] Don't build during test-release.sh Phase 3 install (#82001)
As described in [test-release.sh ninja install does builds in Phase
3](https://github.com/llvm/llvm-project/issues/80999), considerable
parts of Phase 3 of a `test-release.sh` build are run by `ninja
install`, ignoring both `$Verbose` and the parallelism set via `-j NUM`.

This patches fixes this by not specifying any explicit build target for
Phase 3, thus running the full build as usual.

Tested on `sparc64-unknown-linux-gnu`.
2024-02-20 07:26:48 +01:00
azhan92
3af5c98200
[Release] Install compiler-rt builtins during Phase 1 on AIX (#81485)
The current test-release.sh script does not install the necessary
compiler-rt builtin's during Phase 1 on AIX, resulting on a
non-functional Phase 1 clang. Futhermore, the installation is also
necessary for Phase 2 on AIX.

Co-authored-by: Alison Zhang <alisonzhang@ibm.com>
2024-02-15 22:27:45 -04:00
Tom Stellard
f6ced3579a
[CMake][Release] Add option for enabling PGO to release cache file. (#78823)
The option is LLVM_RELEASE_ENABLE_PGO and it's turned on by default.

---------

Co-authored-by: Petr Hosek <phosek@google.com>
2024-01-23 11:32:37 -08:00
Tom Stellard
aa02002491
workflows: Refactor release-tasks.yml (#69523)
* Split out the lit release job and the documentation build job into
their own workflow files. This makes it possible to manually run these
jobs via workflow_dispatch.
    
* Improve tag/user validation and ensure it gets run for each release
task.
2024-01-17 17:17:00 -08:00
Alexandre Ganea
b7c81c1f91
On Windows, make the release script work with the local git checkout (#78273)
Add two new flags to the release script:
  `--skip-checkout` builds from the local source folder, instead of the downloaded source package
  `--local-python` uses whichever local Python version is installed, instead of a specific version (3.10)

If the LLVM source is already in `C:\git\llvm-project` then one can do in a admin prompt:
```
C:\git>llvm-project\llvm\utils\release\build_llvm_release.bat --version 18.0.0 --x64 --skip-checkout
...
```
This allows for iterating more easily on build issues, just before a new LLVM package is made.

Also fix some warnings on the second stage build (`-Wbackend-plugin`).
2024-01-17 11:14:49 -05:00
Tom Stellard
3096353477
test-release.sh: Add a CMake cache file for 3-stage release builds (#75903)
You can now pass the -use-cmake-cache option to test-release.sh and it
will use a predefined cache file for building the release. This will
make it easier to reproduce the builds and add other enhancements like
PGO or bolt optimizations.

---------

Co-authored-by: Konrad Kleine <konrad.kleine@posteo.de>
2024-01-04 16:33:06 -08:00
Tom Stellard
79a2cf5da3
test-release.sh: Default to letting ninja select the number of build jobs (#73187)
ninja defaults to using ncpus + 2 build jobs, and this extra parallelism
helps the script complete faster in my testing on the GitHub runners.
2023-11-26 07:48:51 -08:00
Tom Stellard
907ed77ad1
test-release.sh: Only build the clang target in stage 1 and 2 (#72703)
This skips the build of all the unittests and llvm/clang tools, reducing
the number of ninja targets from 4,826 to 3,816 in phase 1 and phase 2.
2023-11-21 13:15:47 -08:00
Hans
060af2697b
Use PGO for x86_64 windows release packaging (#71067)
Applying this to 17.0.4 makes the toolchain 22% faster (as measured by building clang).
2023-11-14 18:45:03 +01:00
azhan92
ddfee5d2d6
[Release] Build compiler-rt during Phase 1 on AIX (#70672)
Compiler-rt built-ins are needed to have a functional Phase 1 clang
compiler on AIX. This PR adds compiler-rt to the runtime_list during Phase 1 to avoid
this problem.

---------

Co-authored-by: Alison Zhang <alisonzhang@ibm.com>
2023-11-02 09:41:00 -04:00
David Spickett
600e38b11a
[llvm][Release] Add note about binaries to Github release description (#69698)
This appears on the announcements e.g.
https://discourse.llvm.org/t/llvm-17-0-3-released/74172 and it is
important context.

However a lot of folks see release pages e.g.
https://github.com/llvm/llvm-project/releases/tag/llvmorg-17.0.3 first
so it's good to include it there too.
2023-10-23 15:50:04 +01:00
René Rebe
0eed8ae7d2
Fix release/export.sh to export runtimes tarball, too (#67404)
This addresses missing cmake files needed to build some sub-projects
like libstdcxx.

Co-authored-by: René Rebe <rene@exactcode.de>
2023-09-28 16:37:24 +02:00
cor3ntin
b7ff03206d
[Documentation] Replace recommonmark by myst-parser (#65664)
Recommonmark has been deprecated, then archived last year. This was
tracked by: https://github.com/llvm/llvm-iwg/issues/30

See https://github.com/readthedocs/recommonmark

This patch migrates all our doc to use myst

Additional details for bot maintainers: https://discourse.llvm.org/t/maintenance-required-on-sphinx-build-bots/73612
2023-09-25 14:02:39 +02:00
Tobias Hieta
fe7fe6d343
build-docs: Add option to disable doxygen/sphinx docs (#66928)
Doxygen documentation takes very long to build, when making releases we
want to get the normal documentation up earlier so that we don't have to
wait for doxygen documentation.

This PR just adds the flag to disable doxygen builds, I will then later
make a PR that changes the actions to first build the normal docs and
another job to build the doxygen docs.
2023-09-22 10:41:33 +02:00
Hans Wennborg
beb339c0ef build_llvm_release.bat: Set -DCLANG_ENABLE_LIBXML2=OFF
Closes #64263
2023-08-04 16:54:51 +02:00
Hans Wennborg
277a7e8e33 build_llvm_release.bat: Update desired SWIG version for LLDB
Closes #64279
2023-08-01 14:43:49 +02:00
Tobias Hieta
b71edfaa4e
[NFC][Py Reformat] Reformat python files in llvm
This is the first commit in a series that will reformat
all the python files in the LLVM repository.

Reformatting is done with `black`.

See more information here:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Reviewed By: jhenderson, JDevlieghere, MatzeB

Differential Revision: https://reviews.llvm.org/D150545
2023-05-17 10:48:52 +02:00
Dimitry Andric
9b17f5ee0e test-release.sh: build projects and runtimes lists with semicolons
While doing a test-release.sh run on FreeBSD, I ran into a sed error due
to the introduction of the GNU extension '\s' in commit 500587e23dfd.

Scanning for blanks (spaces and tabs) could be done in a more portable
fashion using the [[:blank:]] character class. But it is easier to avoid
the original problem, which is that the projects and runtime lists have
to be separated by semicolons, and cannot start with a semicolon.

Instead, use the shell's alternate value parameter expansion mechanism,
which makes it easy to append items to lists with separators in between,
and without any leading separator. This also avoids having to run sed on
the end result.

In addition, build any selected runtimes in the second phase, otherwise
the third phase can fail to find several symbols in compiler-rt, if that
has been built. This is because the host's compiler-rt is not guaranteed
to have those symbols.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D145884
2023-04-14 20:57:33 +02:00
Peter Jung
b673135bb8 [Release] Produce bolt tarball
Source tarball's are used from some distribution to build the project

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D143809
2023-02-13 14:01:58 -08:00
Pierrick Bouvier
c5e1000b29 Add build for Windows on Arm in packaging script
Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D142983
2023-02-13 14:36:37 +05:00
Rainer Orth
8d2d8e022e [Release] Increase test-release.sh verbosity
`test-release.sh` is too silent in some cases:

- Only the build proper is run verbosely, but `check-all` is not.
- `lit` is run without `-v`, so in case of failures one cannot see what's
actually wrong.

This patch fixes both issues, running all `${MAKE}` invocations with
`$Verbose` (except for `${MAKE} install` where it would only add noise),
and running `lit` with `-v`.

Tested on `x86_64-pc-linux-gnu` and `arm64-apple-darwin21.6`.

Differential Revision: https://reviews.llvm.org/D143249
2023-02-06 09:30:36 +01:00
Tom Stellard
009048810a test-release.sh: Use parallel xz for packaging the binaries
Reviewed By: amyk, dim, hans

Differential Revision: https://reviews.llvm.org/D142417
2023-01-26 15:19:06 -08:00
Tom Stellard
19f100e89a test-release.sh: Only build clang for stage1 and stage2
The stage1 and stage2 builds aren't packaged, so we only need to build
enough of the toolchain to build the next phase.

Reviewed By: thieta, amyk

Differential Revision: https://reviews.llvm.org/D141552
2023-01-24 18:09:20 -08:00
Nikita Popov
fff6964374 [Release] Produce mlir tarball
MLIR supports standalone builds, so I think it makes sense to also
produce a release tarball for the MLIR subproject.

Differential Revision: https://reviews.llvm.org/D141919
2023-01-18 09:59:06 +01:00
NAKAMURA Takumi
f72601a89f [Bazel] Use LLVM_VERSION from llvm/CMakeLists.txt
* Generate `//:vars.bzl` from `llvm/CMakeLists.txt`

`_extract_cmake_settings()` generates `//:vars.bzl` in `llvm_configure()`.
It would be easier to use external commands like sed(1) and python.
For portability, I think the parser should run on Starlark.

`@llvm-project//:vars.bzl` may be loaded from both WORKSPACE and BUILD.
At the moment, `vars.bzl` provides some values as string.

- CMAKE_CXX_STANDARD = "17"
- LLVM_VERSION_MAJOR = "16"
- LLVM_VERSION_MINOR = "0"
- LLVM_VERSION_PATCH = "0"
- LLVM_VERSION = "16.0.0"
- llvm_vars = (dict of these values)

`CMAKE_CXX_STANDARD` may be used to configure toolchain.

* Use `//vars.bzl` for each BUILD files

It would be smarter if the BUILD phase could generate `llvm-config.h`.
Since I am afraid of the discussion in D126581, I just remove
LLVM_VERSION stuff out of the static `llvm-config.h`.

* Eliminate Bazel stuff in 'bump-version.py'

Current version of `bump-version.py` tries to substitute CLANG_VERSION.
It is the reason why I modify bump-version in this change rather than
incoming patch.

Differential Revision: https://reviews.llvm.org/D136392
2023-01-14 10:26:01 +09:00
David Spickett
500587e23d [LLVM][Release] Prevent empty runtime name in release script
Unlike projects, runtimes doesn't have a default set of names.
This means you get a leading space at the start, which gets converted
to a ';' giving ";<runtime name>;<runtime name>".

CMake then errors because the "" before the first ';' is treated
as a runtime name and of course it's not a valid name.

Fix this by removing the leading spaces from runtimes before we
insert the ';'.

Reviewed By: ldionne

Differential Revision: https://reviews.llvm.org/D139306
2022-12-06 09:36:50 +00:00
Konrad Kleine
fd8ba4f537 [release] Add third-party tarball to release for standalone builds
With the advent of https://reviews.llvm.org/D131919 and
a11cd0d94e
 the third-party directory is required to build LLVM and other packages and in standalone
builds the third-party directory is not available from the llvm tarball anymore.

Differential Revision: https://reviews.llvm.org/D137777
2022-11-10 20:55:19 +01:00
Rafael Auler
db698c535f [test-release] Build BOLT by default for x86/arm
Make BOLT build by default in X86 and AArch64 Linux boxes.

Reviewed By: thieta, xbolva00

Differential Revision: https://reviews.llvm.org/D137305
2022-11-03 13:25:31 -07:00
Fangrui Song
3a3603ff99 [clang] Replace BACKEND_PACKAGE_STRING with LLVM_VERSION_STRING
420d7ccbac0f499a6ff9595bdbfa99cd3376df22 introduced BACKEND_PACKAGE_STRING to
replace `PACKAGE_VERSION` (llvm/Config/config.h) to support standalone builds.
This is used in the output of `clang -cc1 -v`.

Since llvm-config.h is available for both standalone and non-standalone builds,
we can just use `LLVM_VERSION_STRING` from llvm-config.h.

clang/cmake/modules/AddClang.cmake uses `VERSION_STRING "${CLANG_VERSION} (${BACKEND_PACKAGE_STRING})"`.
Just simplify it to `"${CLANG_VERSION}"` so that we can remove the CMake
variable BACKEND_PACKAGE_STRING.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D136660
2022-10-25 00:24:25 -07:00
Pierrick Bouvier
b1e5e81efd
Detect Visual Studio automatically in Windows packaging script
Instead of hardcoding several VS paths, use vswhere.exe (available from
VS 2017) to get latest version available.

Reviewed By: hans, thieta

Differential Revision: https://reviews.llvm.org/D135873
2022-10-20 16:29:36 +02:00
Pierrick Bouvier
d7d05ffa74
Introduce options for Windows packaging script
Options:
--version: [required] version to build
--help: display this help
--x86: build and test x86 variant
--x64: build and test x64 variant

Note: At least one variant to build is required.

Example: build_llvm_release.bat --version 15.0.0 --x64

Reviewed By: hans, thieta

Differential Revision: https://reviews.llvm.org/D135255
2022-10-20 14:13:28 +02:00
Pierrick Bouvier
86e23c4e1f Detect Visual Studio in Windows packaging script
Instead of hardcoding a specific VS install, try sequentially:

- %VSINSTALLDIR% (already set from a vs prompt)
- 2019/Enterprise
- 2019/Professional
- 2019/Community
- 2019/BuildTools

It stops when one is found and set vsdevcmd env var.

Differential revision: https://reviews.llvm.org/D135173
2022-10-06 10:18:58 +02:00
Pierrick Bouvier
44073d319d Detect errors in Windows packaging script
- Detect VS devcmd error (missing VS)
- Detect missing python install
- Show commands executed
- Removed pause (blocking CI usage)

Differential revision: https://reviews.llvm.org/D135138
2022-10-04 15:22:36 +02:00
Tobias Hieta
12ba7f27b8
[Release] Add bump-version script.
There are many files that needs to be updated when you
bump the version of LLVM. This script tries to automate
that in order to make the release managers job easier.

Reviewed By: kwk, hans, ldionne

Differential Revision: https://reviews.llvm.org/D133923
2022-09-28 08:41:29 +02:00
Tobias Hieta
b27f6de35d
[docs] Fix build-docs.sh
If libcxxabi is not included CMake will error out:

Cannot find target libcxxabi-SHARED

I ran into this doing the 15.0.0 release

Differential Revision: https://reviews.llvm.org/D133475
2022-09-19 21:45:12 +02:00
Carlos Alberto Enciso
f671eb17be Add command line argument parsing to the Windows packaging script.
As discussed here:
https://discourse.llvm.org/t/build-llvm-release-bat-script-options

Add a function to parse command line arguments: `parse_args`.

The format for the arguments is:
  Boolean: --option
  Value:   --option<separator>value
    with `<separator>` being: space, colon, semicolon or equal sign

Command line usage example:
  my-batch-file.bat --build --type=release --version 123

It will create 3 variables:
  `build` with the value `true`
  `type` with the value `release`
  `version` with the value `123`

Usage:
  set "build="
  set "type="
  set "version="

  REM Parse arguments.
  call :parse_args %*

  if defined build (
    ...
  )
  if %type%=='release' (
    ...
  )
  if %version%=='123' (
    ...
  )
2022-09-09 14:36:40 +01:00
Rafael Auler
e99490512e [test-release] Add option to build BOLT
Add a flag to enable BOLT. Should be used in x86-64 and
AArch64 linux builds only, since BOLT doesn't really support other
targets and is mostly tested on these two systems as hosts.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D131703
2022-08-15 14:03:30 -07:00
Tobias Hieta
70165c55dc
[release] Use threaded compression with xz
Use xz -T0 to use your threads

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D131470
2022-08-10 10:01:47 +02:00
Tom Stellard
66d755bbf8 workflows: Add GitHub action for automating some release tasks
For each release tag, this action will create a new release on GitHub,
and for each -final tag, this action will build the documentation and
upload it to GitHub.

Reviewed By: hans, kwk

Differential Revision: https://reviews.llvm.org/D99780
2022-07-26 15:36:51 -07:00
Carlos Alberto Enciso
f8c13754af Update the Windows packaging script.
As discussed on:
  https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

Giving:
  call :function if errorlevel 1 exit /b 1

Due to a missing new line, the error code returned by the function
is taking as another argument.

Changed to use standard '||' to exit if the errorlevel greater than zero.

  call :function || exit /b 1

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D130154
2022-07-20 13:22:10 +01:00
Carlos Alberto Enciso
c96ddd7019 Update the Windows packaging script.
As discussed on:
https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

- In stage1 use the following binaries from stage0:
  lld-link, llvm-lib and llvm-windres

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D130067
2022-07-20 07:15:13 +01:00
Carlos Alberto Enciso
83e922562f Update the Windows packaging script.
As discussed on:
  https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

- Refactor the build/test steps into functions.
- Exit the script if the build directory already exists.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D129559
2022-07-19 05:55:14 +01:00
Hans Wennborg
77350198d3 Pass -DLIBXML2_INCLUDE_DIRS in the Windows release package script
As pointed out on https://reviews.llvm.org/D129571 this seems to
be the preferred variable to set.
2022-07-14 19:21:51 +02:00
Hans Wennborg
145835caae Build Windows releases with libxml enabled, to unbreak llvm-mt
Recent CMake versions have started to prefer llvm-mt when using
clang-cl, which doesn't work at all when llvm-mt is built without
libxml which has been the case so far.

See https://github.com/llvm/llvm-project/issues/55817

Differential revision: https://reviews.llvm.org/D129571
2022-07-12 20:14:00 +02:00
Carlos Alberto Enciso
df7c5772a2 Update the Windows packaging script.
As discussed on:
  https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

Latest 7-zip versions (21.x) require administrator permissions
to create symbolic links.

Check if any of the following conditions is true:
- Version of 7-zip is 20.x or older
- Script is running with administrator permissions

Reviewed By: hans, thieta

Differential Revision: https://reviews.llvm.org/D129263
2022-07-12 06:37:29 +01:00
Hans Wennborg
b2719349d2 Change build_llvm_package.bat to build_llvm_release.bat
We don't build snapshot packages anymore, so repurpose this
for doing release builds instead.
2022-05-30 17:08:01 +02:00