260 Commits

Author SHA1 Message Date
Tom Stellard
79a2cf5da3
test-release.sh: Default to letting ninja select the number of build jobs (#73187)
ninja defaults to using ncpus + 2 build jobs, and this extra parallelism
helps the script complete faster in my testing on the GitHub runners.
2023-11-26 07:48:51 -08:00
Tom Stellard
907ed77ad1
test-release.sh: Only build the clang target in stage 1 and 2 (#72703)
This skips the build of all the unittests and llvm/clang tools, reducing
the number of ninja targets from 4,826 to 3,816 in phase 1 and phase 2.
2023-11-21 13:15:47 -08:00
Hans
060af2697b
Use PGO for x86_64 windows release packaging (#71067)
Applying this to 17.0.4 makes the toolchain 22% faster (as measured by building clang).
2023-11-14 18:45:03 +01:00
azhan92
ddfee5d2d6
[Release] Build compiler-rt during Phase 1 on AIX (#70672)
Compiler-rt built-ins are needed to have a functional Phase 1 clang
compiler on AIX. This PR adds compiler-rt to the runtime_list during Phase 1 to avoid
this problem.

---------

Co-authored-by: Alison Zhang <alisonzhang@ibm.com>
2023-11-02 09:41:00 -04:00
David Spickett
600e38b11a
[llvm][Release] Add note about binaries to Github release description (#69698)
This appears on the announcements e.g.
https://discourse.llvm.org/t/llvm-17-0-3-released/74172 and it is
important context.

However a lot of folks see release pages e.g.
https://github.com/llvm/llvm-project/releases/tag/llvmorg-17.0.3 first
so it's good to include it there too.
2023-10-23 15:50:04 +01:00
René Rebe
0eed8ae7d2
Fix release/export.sh to export runtimes tarball, too (#67404)
This addresses missing cmake files needed to build some sub-projects
like libstdcxx.

Co-authored-by: René Rebe <rene@exactcode.de>
2023-09-28 16:37:24 +02:00
cor3ntin
b7ff03206d
[Documentation] Replace recommonmark by myst-parser (#65664)
Recommonmark has been deprecated, then archived last year. This was
tracked by: https://github.com/llvm/llvm-iwg/issues/30

See https://github.com/readthedocs/recommonmark

This patch migrates all our doc to use myst

Additional details for bot maintainers: https://discourse.llvm.org/t/maintenance-required-on-sphinx-build-bots/73612
2023-09-25 14:02:39 +02:00
Tobias Hieta
fe7fe6d343
build-docs: Add option to disable doxygen/sphinx docs (#66928)
Doxygen documentation takes very long to build, when making releases we
want to get the normal documentation up earlier so that we don't have to
wait for doxygen documentation.

This PR just adds the flag to disable doxygen builds, I will then later
make a PR that changes the actions to first build the normal docs and
another job to build the doxygen docs.
2023-09-22 10:41:33 +02:00
Hans Wennborg
beb339c0ef build_llvm_release.bat: Set -DCLANG_ENABLE_LIBXML2=OFF
Closes #64263
2023-08-04 16:54:51 +02:00
Hans Wennborg
277a7e8e33 build_llvm_release.bat: Update desired SWIG version for LLDB
Closes #64279
2023-08-01 14:43:49 +02:00
Tobias Hieta
b71edfaa4e
[NFC][Py Reformat] Reformat python files in llvm
This is the first commit in a series that will reformat
all the python files in the LLVM repository.

Reformatting is done with `black`.

See more information here:

https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Reviewed By: jhenderson, JDevlieghere, MatzeB

Differential Revision: https://reviews.llvm.org/D150545
2023-05-17 10:48:52 +02:00
Dimitry Andric
9b17f5ee0e test-release.sh: build projects and runtimes lists with semicolons
While doing a test-release.sh run on FreeBSD, I ran into a sed error due
to the introduction of the GNU extension '\s' in commit 500587e23dfd.

Scanning for blanks (spaces and tabs) could be done in a more portable
fashion using the [[:blank:]] character class. But it is easier to avoid
the original problem, which is that the projects and runtime lists have
to be separated by semicolons, and cannot start with a semicolon.

Instead, use the shell's alternate value parameter expansion mechanism,
which makes it easy to append items to lists with separators in between,
and without any leading separator. This also avoids having to run sed on
the end result.

In addition, build any selected runtimes in the second phase, otherwise
the third phase can fail to find several symbols in compiler-rt, if that
has been built. This is because the host's compiler-rt is not guaranteed
to have those symbols.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D145884
2023-04-14 20:57:33 +02:00
Peter Jung
b673135bb8 [Release] Produce bolt tarball
Source tarball's are used from some distribution to build the project

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D143809
2023-02-13 14:01:58 -08:00
Pierrick Bouvier
c5e1000b29 Add build for Windows on Arm in packaging script
Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D142983
2023-02-13 14:36:37 +05:00
Rainer Orth
8d2d8e022e [Release] Increase test-release.sh verbosity
`test-release.sh` is too silent in some cases:

- Only the build proper is run verbosely, but `check-all` is not.
- `lit` is run without `-v`, so in case of failures one cannot see what's
actually wrong.

This patch fixes both issues, running all `${MAKE}` invocations with
`$Verbose` (except for `${MAKE} install` where it would only add noise),
and running `lit` with `-v`.

Tested on `x86_64-pc-linux-gnu` and `arm64-apple-darwin21.6`.

Differential Revision: https://reviews.llvm.org/D143249
2023-02-06 09:30:36 +01:00
Tom Stellard
009048810a test-release.sh: Use parallel xz for packaging the binaries
Reviewed By: amyk, dim, hans

Differential Revision: https://reviews.llvm.org/D142417
2023-01-26 15:19:06 -08:00
Tom Stellard
19f100e89a test-release.sh: Only build clang for stage1 and stage2
The stage1 and stage2 builds aren't packaged, so we only need to build
enough of the toolchain to build the next phase.

Reviewed By: thieta, amyk

Differential Revision: https://reviews.llvm.org/D141552
2023-01-24 18:09:20 -08:00
Nikita Popov
fff6964374 [Release] Produce mlir tarball
MLIR supports standalone builds, so I think it makes sense to also
produce a release tarball for the MLIR subproject.

Differential Revision: https://reviews.llvm.org/D141919
2023-01-18 09:59:06 +01:00
NAKAMURA Takumi
f72601a89f [Bazel] Use LLVM_VERSION from llvm/CMakeLists.txt
* Generate `//:vars.bzl` from `llvm/CMakeLists.txt`

`_extract_cmake_settings()` generates `//:vars.bzl` in `llvm_configure()`.
It would be easier to use external commands like sed(1) and python.
For portability, I think the parser should run on Starlark.

`@llvm-project//:vars.bzl` may be loaded from both WORKSPACE and BUILD.
At the moment, `vars.bzl` provides some values as string.

- CMAKE_CXX_STANDARD = "17"
- LLVM_VERSION_MAJOR = "16"
- LLVM_VERSION_MINOR = "0"
- LLVM_VERSION_PATCH = "0"
- LLVM_VERSION = "16.0.0"
- llvm_vars = (dict of these values)

`CMAKE_CXX_STANDARD` may be used to configure toolchain.

* Use `//vars.bzl` for each BUILD files

It would be smarter if the BUILD phase could generate `llvm-config.h`.
Since I am afraid of the discussion in D126581, I just remove
LLVM_VERSION stuff out of the static `llvm-config.h`.

* Eliminate Bazel stuff in 'bump-version.py'

Current version of `bump-version.py` tries to substitute CLANG_VERSION.
It is the reason why I modify bump-version in this change rather than
incoming patch.

Differential Revision: https://reviews.llvm.org/D136392
2023-01-14 10:26:01 +09:00
David Spickett
500587e23d [LLVM][Release] Prevent empty runtime name in release script
Unlike projects, runtimes doesn't have a default set of names.
This means you get a leading space at the start, which gets converted
to a ';' giving ";<runtime name>;<runtime name>".

CMake then errors because the "" before the first ';' is treated
as a runtime name and of course it's not a valid name.

Fix this by removing the leading spaces from runtimes before we
insert the ';'.

Reviewed By: ldionne

Differential Revision: https://reviews.llvm.org/D139306
2022-12-06 09:36:50 +00:00
Konrad Kleine
fd8ba4f537 [release] Add third-party tarball to release for standalone builds
With the advent of https://reviews.llvm.org/D131919 and
a11cd0d94e
 the third-party directory is required to build LLVM and other packages and in standalone
builds the third-party directory is not available from the llvm tarball anymore.

Differential Revision: https://reviews.llvm.org/D137777
2022-11-10 20:55:19 +01:00
Rafael Auler
db698c535f [test-release] Build BOLT by default for x86/arm
Make BOLT build by default in X86 and AArch64 Linux boxes.

Reviewed By: thieta, xbolva00

Differential Revision: https://reviews.llvm.org/D137305
2022-11-03 13:25:31 -07:00
Fangrui Song
3a3603ff99 [clang] Replace BACKEND_PACKAGE_STRING with LLVM_VERSION_STRING
420d7ccbac0f499a6ff9595bdbfa99cd3376df22 introduced BACKEND_PACKAGE_STRING to
replace `PACKAGE_VERSION` (llvm/Config/config.h) to support standalone builds.
This is used in the output of `clang -cc1 -v`.

Since llvm-config.h is available for both standalone and non-standalone builds,
we can just use `LLVM_VERSION_STRING` from llvm-config.h.

clang/cmake/modules/AddClang.cmake uses `VERSION_STRING "${CLANG_VERSION} (${BACKEND_PACKAGE_STRING})"`.
Just simplify it to `"${CLANG_VERSION}"` so that we can remove the CMake
variable BACKEND_PACKAGE_STRING.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D136660
2022-10-25 00:24:25 -07:00
Pierrick Bouvier
b1e5e81efd
Detect Visual Studio automatically in Windows packaging script
Instead of hardcoding several VS paths, use vswhere.exe (available from
VS 2017) to get latest version available.

Reviewed By: hans, thieta

Differential Revision: https://reviews.llvm.org/D135873
2022-10-20 16:29:36 +02:00
Pierrick Bouvier
d7d05ffa74
Introduce options for Windows packaging script
Options:
--version: [required] version to build
--help: display this help
--x86: build and test x86 variant
--x64: build and test x64 variant

Note: At least one variant to build is required.

Example: build_llvm_release.bat --version 15.0.0 --x64

Reviewed By: hans, thieta

Differential Revision: https://reviews.llvm.org/D135255
2022-10-20 14:13:28 +02:00
Pierrick Bouvier
86e23c4e1f Detect Visual Studio in Windows packaging script
Instead of hardcoding a specific VS install, try sequentially:

- %VSINSTALLDIR% (already set from a vs prompt)
- 2019/Enterprise
- 2019/Professional
- 2019/Community
- 2019/BuildTools

It stops when one is found and set vsdevcmd env var.

Differential revision: https://reviews.llvm.org/D135173
2022-10-06 10:18:58 +02:00
Pierrick Bouvier
44073d319d Detect errors in Windows packaging script
- Detect VS devcmd error (missing VS)
- Detect missing python install
- Show commands executed
- Removed pause (blocking CI usage)

Differential revision: https://reviews.llvm.org/D135138
2022-10-04 15:22:36 +02:00
Tobias Hieta
12ba7f27b8
[Release] Add bump-version script.
There are many files that needs to be updated when you
bump the version of LLVM. This script tries to automate
that in order to make the release managers job easier.

Reviewed By: kwk, hans, ldionne

Differential Revision: https://reviews.llvm.org/D133923
2022-09-28 08:41:29 +02:00
Tobias Hieta
b27f6de35d
[docs] Fix build-docs.sh
If libcxxabi is not included CMake will error out:

Cannot find target libcxxabi-SHARED

I ran into this doing the 15.0.0 release

Differential Revision: https://reviews.llvm.org/D133475
2022-09-19 21:45:12 +02:00
Carlos Alberto Enciso
f671eb17be Add command line argument parsing to the Windows packaging script.
As discussed here:
https://discourse.llvm.org/t/build-llvm-release-bat-script-options

Add a function to parse command line arguments: `parse_args`.

The format for the arguments is:
  Boolean: --option
  Value:   --option<separator>value
    with `<separator>` being: space, colon, semicolon or equal sign

Command line usage example:
  my-batch-file.bat --build --type=release --version 123

It will create 3 variables:
  `build` with the value `true`
  `type` with the value `release`
  `version` with the value `123`

Usage:
  set "build="
  set "type="
  set "version="

  REM Parse arguments.
  call :parse_args %*

  if defined build (
    ...
  )
  if %type%=='release' (
    ...
  )
  if %version%=='123' (
    ...
  )
2022-09-09 14:36:40 +01:00
Rafael Auler
e99490512e [test-release] Add option to build BOLT
Add a flag to enable BOLT. Should be used in x86-64 and
AArch64 linux builds only, since BOLT doesn't really support other
targets and is mostly tested on these two systems as hosts.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D131703
2022-08-15 14:03:30 -07:00
Tobias Hieta
70165c55dc
[release] Use threaded compression with xz
Use xz -T0 to use your threads

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D131470
2022-08-10 10:01:47 +02:00
Tom Stellard
66d755bbf8 workflows: Add GitHub action for automating some release tasks
For each release tag, this action will create a new release on GitHub,
and for each -final tag, this action will build the documentation and
upload it to GitHub.

Reviewed By: hans, kwk

Differential Revision: https://reviews.llvm.org/D99780
2022-07-26 15:36:51 -07:00
Carlos Alberto Enciso
f8c13754af Update the Windows packaging script.
As discussed on:
  https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

Giving:
  call :function if errorlevel 1 exit /b 1

Due to a missing new line, the error code returned by the function
is taking as another argument.

Changed to use standard '||' to exit if the errorlevel greater than zero.

  call :function || exit /b 1

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D130154
2022-07-20 13:22:10 +01:00
Carlos Alberto Enciso
c96ddd7019 Update the Windows packaging script.
As discussed on:
https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

- In stage1 use the following binaries from stage0:
  lld-link, llvm-lib and llvm-windres

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D130067
2022-07-20 07:15:13 +01:00
Carlos Alberto Enciso
83e922562f Update the Windows packaging script.
As discussed on:
  https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

- Refactor the build/test steps into functions.
- Exit the script if the build directory already exists.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D129559
2022-07-19 05:55:14 +01:00
Hans Wennborg
77350198d3 Pass -DLIBXML2_INCLUDE_DIRS in the Windows release package script
As pointed out on https://reviews.llvm.org/D129571 this seems to
be the preferred variable to set.
2022-07-14 19:21:51 +02:00
Hans Wennborg
145835caae Build Windows releases with libxml enabled, to unbreak llvm-mt
Recent CMake versions have started to prefer llvm-mt when using
clang-cl, which doesn't work at all when llvm-mt is built without
libxml which has been the case so far.

See https://github.com/llvm/llvm-project/issues/55817

Differential revision: https://reviews.llvm.org/D129571
2022-07-12 20:14:00 +02:00
Carlos Alberto Enciso
df7c5772a2 Update the Windows packaging script.
As discussed on:
  https://discourse.llvm.org/t/build-llvm-release-bat-script-options/63146/6

Latest 7-zip versions (21.x) require administrator permissions
to create symbolic links.

Check if any of the following conditions is true:
- Version of 7-zip is 20.x or older
- Script is running with administrator permissions

Reviewed By: hans, thieta

Differential Revision: https://reviews.llvm.org/D129263
2022-07-12 06:37:29 +01:00
Hans Wennborg
b2719349d2 Change build_llvm_package.bat to build_llvm_release.bat
We don't build snapshot packages anymore, so repurpose this
for doing release builds instead.
2022-05-30 17:08:01 +02:00
Hans Wennborg
bac4934c84 Revert "build_llvm_package.bat: Produce zip files in addition to the installers"
The zip files were too large to be practical, so they were never
shipped. Reverting to reduce build time and complexity of the script.

This reverts commit 4486aa03c5f431ba33a1d1ac9991da912e3decd9.
2022-05-30 13:55:54 +02:00
Hans Wennborg
10d2195305 Update the Windows packaging script
Check in updates based on how the latest release was built [0] and add
the bug fix from [1] which allows LLDB to start.

Other changes which had accumulated in the local release script:
- Don't build the clang format plugin (VS has the functionality built
  in now)
- Disable tests that have been failing (I'll try to follow up and
  re-enable them)
- Switch to Python 3.10
- Jump through more hoops to make LLDB pick the right Python.

0. https://discourse.llvm.org/t/14-0-4-final-has-been-tagged/62750/3
1. https://github.com/llvm/llvm-project/issues/54589
2022-05-30 11:58:13 +02:00
Konrad Kleine
c0f5beef2f [release] Add cmake as an extra tarball and not bundle it
Revert "Add cmake/ to release tarballs via concatenation"

This reverts commit 3a33664e8838e8b77acd1bbb13b1cf5e580a1077.

Revert "Add cmake to source release tarballs"

This reverts commit 32a0482a65b86cf0d84ea7e784cca9852df5c67c.

Reviewed By: tstellar, aaronpuchert

Differential Revision: https://reviews.llvm.org/D125798
2022-05-19 11:12:54 +02:00
Tobias Hieta
a30972fbd7 [test-release] Added -silent-log flag to test-release.sh
This flag silents the build output of test-release.sh so that
it can be used in CI systems a bit better. It will still log
the build output to the log files but not echo it to stdout.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D122146
2022-03-31 08:34:53 +02:00
Aaron Puchert
3a33664e88 Add cmake/ to release tarballs via concatenation
The solution using append was reported not to work, but additionally it
would use the contents of the checked-out source tree instead of the git
tag or commit. This uses `git archive`, so it will use the right commit,
and at least for me (with GNU tar) it seems to work as intended.

Should fix #53281.

Reviewed By: kwk

Differential Revision: https://reviews.llvm.org/D121972
2022-03-21 15:28:29 +01:00
Louis Dionne
f34f7dfe3a [release] Use a supported way of building libc++ when building the documentation
Instead of using the deprecated LLVM_ENABLE_PROJECTS build, use the
default runtimes build. This is just as fast, but it's supported.

Differential Revision: https://reviews.llvm.org/D119275
2022-02-14 16:01:11 -05:00
Amy Kwan
db691903d2 [test-release.sh] Add option to disable building clang-tools-extra during release testing.
This patch adds an option (no-clang-tools) to disable building clang-tools-extra when
performing release testing. Prior to this patch, clang-tools-extra was built by default,
but on some platforms (such as AIX), clang-tools-extra is not supported, and so we do
not normally build it. Furthermore, this change should not change the invocation for
targets that build clang-tools-extra normally.

Differential Revision: https://reviews.llvm.org/D119520
2022-02-11 07:50:03 -06:00
Konrad Kleine
32a0482a65 Add cmake to source release tarballs
I've split the git archive generation into three steps:

1. generate pure tarball
2. append top-level cmake directory to all tarballs
3. compress the archive

This was inspired by D118252 and can be considered an alternative
approach for all projects to have access to the shared cmake
directory when building in standalone mode.

When generating source tarballs on my local laptop it takes 9 minutes and 45 seconds WITH this patch applied. When this patch is not applied, it takes 9minutes and 38 seconds. That means, this patch introduces a slowdown of 7 seconds, which seems fair.

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D118481
2022-02-11 11:54:44 +01:00
Diana Picus
e931f92550 test-release.sh: Remove test-suite from LLVM_ENABLE_PROJECTS
I think this was just being ignored before, but now it crashes because
we're checking if the projects that we're trying to enable are valid.
There is no test-suite project (it's a separate repo with separate
handling), so we should never try to enable it.

Differential Revision: https://reviews.llvm.org/D119322
2022-02-10 09:29:15 +01:00
Amy Kwan
413b35cd74 [test-release.sh] Set TEST_SUITE_HOST_CC to the release testing build compiler when compiling test-suite tools.
The tools used by test-suite are originally configured to compile with cc by
default, and this is dictated by TEST_SUITE_HOST_CC.
However, it is possible that on some systems that the version of cc may either
not be present or it may not be able to compile the tools as it may be too old,
which could be an issue seen during release testing.

This patch updates the compiler to be the default build compiler that is used
for release testing. If no such compiler it specified, then cc will be set as
the test-suite tools build compiler by default (as it already is set under
TEST_SUITE_HOST_CC).

Differential Revision: https://reviews.llvm.org/D118357
2022-02-03 21:31:44 -06:00