Update HIPSPV toolchain to support `--offload-new-driver`. Additionally,
tailor llvm-spirv invocation for
[chipStar](github.com/CHIP-SPV/chipStar) via `spirv64-*-chipstar`
offload triple.
AFAICT, all the relevant test failures in the previous PR (#178664) came
from tests involving `-Xoffload-compiler ‘-###’` in their RUN
directives. I have reworked those tests in this PR.
Make a Triple::OSType to support a generic "firmware" OS that isn't bare
metal, but isn't tied to a specific hardware platform like macOS or iOS.
Hook up support for the new OSType in the Darwin toolchain.
Update HIPSPV toolchain to support --offload-new-driver. Additionally,
tailor llvm-spirv invocation for
[chipStar](https://github.com/CHIP-SPV/chipStar) via
`spirv64-*-chipstar` offload triple.
The previous PR (#168043) had CI failures that were not caught early.
This one attempts to address them.
Update HIPSPV toolchain to support `--offload-new-driver`. Additionally,
tailor `llvm-spirv` invocation for
[chipStar](https://github.com/CHIP-SPV/chipStar) via
`spirv64-*-chipstar` offload triple.
Depends on one commit from #170467 and one from #170655.
---------
Co-authored-by: Henry Linjamäki <henry.mikael.linjamaki@intel.com>
Co-authored-by: Joseph Huber <huberjn@outlook.com>
We proceeded with frontend/clang changes, until we figure out how ABI
for BE should look like. Once it is final, we will proceed with codegen
changes.
In this patch several things addressed:
- Define riscv32be/riscv64be target triples
- Set correct data layout for BE targets
- Handle BE-specific ABI details
- Emit warning for BE case since it is still experimental
SPIR-V does not have a production-grade linker so it is often necessary
to stay in LLVM-IR as long as possible and only convert to SPIR-V at the
very end.
As such, it is common that we want to create a BC library that is linked
into a user program (motivating example here is the OpenMP device RTL).
We only convert to SPIR-V at the very end when we have a fully linked
program.
Other targets can achieve a similar goal by using LTO to get a linked-BC
file, but the SPIR-V linker does not support LTO, so we have no way to
do it with `clang`. We can do it with `llvm-link` directly, but my
understanding is that it is not intended to be directly called in
production workflows and does not fit well into code that supports
multiple targets.
Extend the existing behavior for `HIP` that allows `-emit-llvm` without
`-c`.
---------
Signed-off-by: Nick Sarnie <nick.sarnie@intel.com>
The `ExpansionContext` constructor calls `vfs::getRealFileSystem()` when
not given VFS explicitly, leading to sandbox violations. This PR passes
the VFS into the constructor instead of calling `setVFS()` later.
Commit 4c6f398 introduced a non-working compilation path for regular
C/C++ to AMD SPIRV; this commit fixes it. For example, 4c6f398 was
expecting an assembler phase but it was never supported since there is
no assembler available for it. Thus, the compilation starting from
source code failed.
The compilation path is fixed by taking into account that we cannot rely
on external dependencies such as `spirv-link` or `spirv-as`. Thus, the
backend emits bitcode and the SPIRVAMDToolChain's linker takes care of
generating the final SPIRV as we already do for HIP (same intent as in
4c6f398).
This reverts commit 9403c2d64d63c16a09739d943eaa22b8e3499b7a.
During the review of #152770, it was decided to remove the logic that
would have been used to implicitly enable the -fmodules-driver feature
in the future. (Currently, this logic is only used for diagnostics.)
If we find that implicitly enabling the -fmodules-driver feature is
really useful in practice later on, it may be re-added.
Link to review comment:
https://github.com/llvm/llvm-project/pull/152770#discussion_r2389642300
This PR is the first step towards introducing LFI into LLVM as a new
sub-architecture backend of AArch64. For details, please see the
[RFC](https://discourse.llvm.org/t/rfc-lightweight-fault-isolation-lfi-efficient-native-code-sandboxing-upstream-lfi-target-and-compiler-changes/88380),
which has been approved for AArch64.
This patch creates the `aarch64_lfi` architecture, and marks the
appropriate registers as reserved when it is targeted (`x25`, `x26`,
`x27`, `x28`). It also adds a Clang driver toolchain for targeting LFI,
and updates the compiler-rt CMake to allow builds for the `aarch64_lfi`
target. The patch also includes documentation for LFI and the rewrites
that will be implemented in future patches.
I am planning to split the relevant modifications for LFI into a series
of patches, organized as described below (after this one). Please let me
know if you'd like me to split the changes in a different way, or
provide one big patch.
1. The next patch will introduce the `MCLFIExpander` mechanism for
applying the MC-level rewrites needed by LFI, along with the
`.lfi_expand` and `.lfi_no_expand` assembly directives when targeting
LFI. A preview can be seen on the `lfi-project`
[fork](https://github.com/llvm/llvm-project/compare/main...lfi-project:llvm-project:lfi-patchset/aarch64-pr-2).
2. The following patch will create an `MCLFIExpander` for the AArch64
backend that performs LFI expansions. This patch will contain the
majority of the LFI-specific logic.
3. The final patch will add an optimization to the rewriter that can
eliminate redundant guard instructions that occur within the same basic
block.
We plan to introduce x86-64 support after further discussion and once
the `MCLFIExpander` infrastructure is in place.
Please let me know your feedback, and thank you very much for your help
and guidance in the review process.
This PR introduces a new mechanism for enforcing a sandbox around
filesystem reads coming from the compiler. A fatal error is raised
whenever the `llvm::sys::fs`, `llvm::MemoryBuffer::getFile*()` APIs get
used directly instead of going through the "blessed" virtual interface
of `llvm::vfs::FileSystem`.
Reapplication of #137828, changes:
* Workaround CMAKE_Fortran_PREPROCESS_SOURCE issue for CMake < 2.24: The
issue is that `try_compile` does not forward manually-defined compiler
flang variables to the test build environment; instead of just a
negative test result, it aborts the configuration step itself. To be
fair, manually defining these variables is deprecated since at least
CMake 3.6.
* Missing flang cmd line flags for CMake < 3.28 `-target=`, `-O2`, `-O3`
* It is now possible to set FLANG_RT_ENABLED_STATIC=OFF and
FLANG_RT_ENABLE_SHARED=OFF at the same and is the default for amdgpu and
nvptx targets. In this mode, only the .mod files are compiled --
necessary for module files in
lib/clang/22/finclude/flang/(nvptx64-nvidia-cuda|amdgpu-amd-amdhsa)/*.mod
to be available.
* For compiling omp_lib.mod for nvptx and amdgpu, the module build
functionality must be hoisted out if openmp's runtime/ directory which
is only included for host targets. This PR now requires #169909.
Move building the .mod files from openmp/flang to openmp/flang-rt using
a shared mechanism. Motivations to do so are:
1. Most modules are target-dependent and need to be re-compiled for each
target separately, which is something the LLVM_ENABLE_RUNTIMES system
already does. Prime example is `iso_c_binding.mod` which encodes the
target's ABI. Constants such as [`c_long_double` also have different
values](d748c81218/flang-rt/lib/runtime/iso_c_binding.f90 (L77-L81)).
Most other modules have `#ifdef`-enclosed code as well. For instance
this caused offload targets nvptx64-nvidia-cuda/amdgpu-amd-amdhsa to use
the modules files compiled for the host which may contrain uses of the
types REAL(10) or REAL(16) not available for nvptx/amdgpu.
#146876#128015#129742#158790
3. CMake has support for Fortran that we should use. Among other things,
it automatically determines module dependencies so there is no need to
hardcode them in the CMakeLists.txt.
4. It allows using Fortran itself to implement Flang-RT. Currently, only
`iso_fortran_env_impl.f90` emits object files that are needed by Fortran
applications (#89403). The workaround of #95388 could be reverted (PR
#169525).
If using Flang for cross-compilation or target-offloading, flang-rt must
now be compiled for each target not only for the library, but also to
get the target-specific module files. For instance in a bootstrapping
runtime build, this can be done by adding:
`-DLLVM_RUNTIME_TARGETS=default;nvptx64-nvidia-cuda;amdgpu-amd-amdhsa`.
Some new dependencies come into play:
* openmp depends on flang-rt for building `lib_omp.mod` and
`lib_omp_kinds.mod`. Currently, if flang-rt is not found then the
modules are not built.
* check-flang depends on flang-rt: If not found, the majority of tests
are disabled. If not building in a bootstrpping build, the location of
the module files can be pointed to using
`-DFLANG_INTRINSIC_MODULES_DIR=<path>`, e.g. in a flang-standalone
build. Alternatively, the test needing any of the intrinsic modules
could be marked with `REQUIRES: flangrt-modules`.
* check-flang depends on openmp: Not a change; tests requiring
`lib_omp.mod` and `lib_omp_kinds.mod` those are already marked with
`openmp_runtime`.
As intrinsic are now specific to the target, their location is moved
from `include/flang` to `<resource-dir>/finclude/flang/<triple>`. The
mechnism to compute the location have been moved from flang-rt
(previously used to compute the location of `libflang_rt.*.a`) to common
locations in `cmake/GetToolchainDirs.cmake` and
`runtimes/CMakeLists.txt` so they can be used by both, openmp and
flang-rt. Potentially the mechnism could also be shared by other
libraries such as compiler-rt.
`finclude` was chosen because `gfortran` uses it as well and avoids
misuse such as `#include <flang/iso_c_binding.mod>`. The search location
is now determined by `ToolChain` in the driver, instead of by the
frontend. Another subdirectory `flang` avoids accidental inclusion of
gfortran-modules which due to compression would result in
user-unfriendly errors. Now the driver adds `-fintrinsic-module-path`
for that location to the frontend call (Just like gfortran does).
`-fintrinsic-module-path` had to be fixed for this because ironically it
was only added to `searchDirectories`, but not
`intrinsicModuleDirectories_`. Since the driver determines the location,
tests invoking `flang -fc1` and `bbc` must also be passed the location
by llvm-lit. This works like llvm-lit does for finding the include dirs
for Clang using `-print-file-name=...`.
This relands "[clang][Driver] Support for the SPIR-V backend when compiling HIP" #167543. The only new change is a small fix for the multicall driver.
For HIP, the SPIR-V backend can be optionally activated with the -use-spirv-backend flag. This option uses the SPIR-V BE instead of the SPIR-V translator. These changes also ensure that -use-spirv-backend does not require external dependencies, such as spirv-as and spirv-link
This relands #165277 by reverting #169397.
This also relands the corresponding Bazel port by reverting #169410.
The original revert was due to a report of a broken build, which was
later resolved by fully clearing the build directory.
For HIP, the SPIR-V backend can be optionally activated with the -use-spirv-backend flag. This option uses the SPIR-V BE instead of the SPIR-V translator. These changes also ensure that -use-spirv-backend does not require external dependencies, such as spirv-as and spirv-link
Move building the .mod files from openmp/flang to openmp/flang-rt using
a shared mechanism. Motivations to do so are:
1. Most modules are target-dependent and need to be re-compiled for each
target separately, which is something the LLVM_ENABLE_RUNTIMES system
already does. Prime example is `iso_c_binding.mod` which encodes the
target's ABI. Most other modules have `#ifdef`-enclosed code as well.
2. CMake has support for Fortran that we should use. Among other things,
it automatically determines module dependencies so there is no need to
hardcode them in the CMakeLists.txt.
3. It allows using Fortran itself to implement Flang-RT. Currently, only
`iso_fortran_env_impl.f90` emits object files that are needed by Fortran
applications (#89403). The workaround of #95388 could be reverted.
Some new dependencies come into play:
* openmp depends on flang-rt for building `lib_omp.mod` and
`lib_omp_kinds.mod`. Currently, if flang-rt is not found then the
modules are not built.
* check-flang depends on flang-rt: If not found, the majority of tests
are disabled. If not building in a bootstrpping build, the location of
the module files can be pointed to using
`-DFLANG_INTRINSIC_MODULES_DIR=<path>`, e.g. in a flang-standalone
build. Alternatively, the test needing any of the intrinsic modules
could be marked with `REQUIRES: flangrt-modules`.
* check-flang depends on openmp: Not a change; tests requiring
`lib_omp.mod` and `lib_omp_kinds.mod` those are already marked with
`openmp_runtime`.
As intrinsic are now specific to the target, their location is moved
from `include/flang` to `<resource-dir>/finclude/flang/<triple>`. The
mechnism to compute the location have been moved from flang-rt
(previously used to compute the location of `libflang_rt.*.a`) to common
locations in `cmake/GetToolchainDirs.cmake` and
`runtimes/CMakeLists.txt` so they can be used by both, openmp and
flang-rt. Potentially the mechnism could also be shared by other
libraries such as compiler-rt.
`finclude` was chosen because `gfortran` uses it as well and avoids
misuse such as `#include <flang/iso_c_binding.mod>`. The search location
is now determined by `ToolChain` in the driver, instead of by the
frontend. Now the driver adds `-fintrinsic-module-path` for that
location to the frontend call (Just like gfortran does).
`-fintrinsic-module-path` had to be fixed for this because ironically it
was only added to `searchDirectories`, but not
`intrinsicModuleDirectories_`. Since the driver determines the location,
tests invoking `flang -fc1` and `bbc` must also be passed the location
by llvm-lit. This works like llvm-lit does for finding the include dirs
for Clang using `-print-file-name=...`.
This removes the dependency on clangDriver from clangFrontend and
flangFrontend.
This refactoring is part of a broader effort to support driver-managed
builds for compilations using C++ named modules and/or Clang modules.
It is required for linking the dependency scanning tooling against the
driver without introducing cyclic dependencies, which would otherwise
cause build failures when dynamic linking is enabled.
In particular, clangFrontend must no longer depend on clangDriver
for this to be possible.
This change was discussed in the following RFC:
https://discourse.llvm.org/t/rfc-new-clangoptions-library-remove-dependency-on-clangdriver-from-clangfrontend-and-flangfrontend/88773
… names
Fix non-RDC mode HIP compilation for the new driver on Windows due to
invalid temporary file names when offload arch is a target ID containing
':', which is invalid in file names on Windows.
Refactor the existing handling of ':' in file names on Windows from
clang driver into a shared function sanitizeTargetIDInFileName in
clang/Basic/TargetID.h. This function replaces ':' with '@' on Windows
only, preserving the original behavior.
Update both clang/lib/Driver/Driver.cpp and
clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp to use this
shared function, ensuring consistent handling across both tools.
Use clang linker wrapper to device-link and embed HIP fat binary
directly. Match CUDA non-RDC flow in new driver by producing .hipfb like
.fatbin.
Previously, llvm offload binary is used to package the device IR's and
embed them in the host object file, then clang linker wrapper is used
with each host object file to extract device IR's, perform device
linking, bundle code objects into a fat binary, wrap it in a host object
file, then merge it with the original host object by the host linker
with '-r' option. However, the host linker in MSVC toolchain does not
support '-r' option.
The new approach still package the device IR's with llvm offload binary,
but instead of embed it in a host object, it is passed to clang linker
wrapper directly, where device IR's are extracted and linked, fat binary
is generated, then embeded in the host object directly. Compared with
the old offload driver, this approach can parallelize the device linking
for different GPU's by using the parallelization feature of clang linker
wrapper.
Fixes: SWDEV-565994
This relands #167348.
The original PR was reverted due to a reported build failure, which was
later diagnosed as a local issue in the developer’s checkout or build
state. See discussion here:
https://github.com/llvm/llvm-project/pull/163659#discussion_r2511546964
No additional changes have been made in this reland.
This change moves option-related code from clangDriver into a new
clangOptions library.
This refactoring is part of a broader effort to support driver-managed
builds for compilations using C++ named modules and/or Clang modules.
It is required for linking the dependency scanning tooling against the
driver without introducing cyclic dependencies, which would otherwise
cause build failures when dynamic linking is enabled.
In particular, clangFrontend must no longer depend on clangDriver
for this to be possible.
This PR is motivated by the following review comment:
https://github.com/llvm/llvm-project/pull/152770#discussion_r2430756918
Currently, the command `clang -c -emit-llvm --no-gpu-bundle-output
--offload-arch=gfx900,gfx1030 -O3 -x hip square.hip` will lead to a
bundled output:
```
❯ ../bin/clang -c -emit-llvm --no-gpu-bundle-output --offload-arch=gfx900,gfx1030 -O3 -x hip square.hip
❯ ls
square.hip
square.bc
```
This doesn't match my expectation of the behavior of
`--no-gpu-bundle-output`, so this adds a check into
OffloadingActionBuilder for the flag when replacing the host compile
action for a bundling action.
`clang -x hip foo.c --offload-arch=amdgcnspirv --offload-new-driver
-save-temps` was crashing with the following error:
```
/usr/bin/ld: input file 'foo-x86_64-unknown-linux-gnu.o' is the same as output file
build/bin/clang-linker-wrapper: error: 'ld' failed
```
The `LinkerWrapperJobAction` [is
created](957598f71b/clang/lib/Driver/Driver.cpp (L4888))
with `types::TY_Object` which makes `Driver::GetNamedOutputPath` assign
the same name as the assembler's output and thus causing the crash.
Before this PR, `clang --print-runtime-dir` on Windows used to report a
non-existent directory if `LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF`.
We now check if any of the known runtime directories exist before
printing any of them on stdout. If none exists, we print `(runtime dir
is not present)`.
When an option that is only available in `flang -fc1` is provided to
`flang`, emit a diagnostic with a suggestion containing "did you mean
-Xflang '-foo'".
Partially addresses #163550.
When -nostdlib is specified, Clang should not report any
library‑provided module manifest, even if a manifest for the default
standard library is present.
This pr adds the `Qstrip-rootsignature` as a `DXC` driver option.
To do so, this pr introduces the `BinaryModifyJobClass` as an `Action`
to modify a produced object file before its final output.
Further, it registers `llvm-objcopy` as the tool to modify a produced
`DXContainer` on the `HLSL` toolchain.
This allows us to specify the `Qstrip-rootsignature` option to
`clang-dxc` which will invoke `llvm-objcopy` with a
`--remove-section=RTS0` argument to implement its functionality.
Resolves: https://github.com/llvm/llvm-project/issues/150275.
This patch is part of a series to support driver managed module builds
for C++ named modules and Clang modules.
This introduces a scanner that detects C++ named module usage early in
the driver with only negligible overhead.
For now, it is enabled only with the `-fmodules-driver` flag and serves
solely diagnostic purposes. In the future, the scanner will be enabled
for any (modules-driver compatible) compilation with two or more inputs,
and will help the driver determine whether to implicitly enable the
modules driver.
Since the scanner adds very little overhead, we are also exploring
enabling it for compilations with only a single input. This approach
could allow us to detect `import std` usage in a single-file
compilation, which would then activate the modules driver. For
performance measurements on this, see
https://github.com/naveen-seth/llvm-dev-cxx-modules-check-benchmark.
RFC for driver managed module builds:
https://discourse.llvm.org/t/rfc-modules-support-simple-c-20-modules-use-from-the-clang-driver-without-a-build-system
This patch relands the reland (2d31fc8) for commit ded1426. The earlier
reland failed due to a missing link dependency on `clangLex`. This
reland fixes the issue by adding the link dependency after discussing it
in the following RFC:
https://discourse.llvm.org/t/rfc-driver-link-the-driver-against-clangdependencyscanning-clangast-clangfrontend-clangserialization-and-clanglex
Summary:
This is a bit of an awkward transition point for the new and old
drivers. Previously AMDGPU uses this to generate offloading bundles, but
the new driver much prefers to output the file itself. This patch
changes the behavior to always respect `--gpu-bundle-output` instead of
having it be the default behavior. This means that we effectively get to
override the default new driver behavior with this flag now. This should
hoepfully fix some errors in the downstream comgr tests.
Summary:
Previously, querying for the offload architecture tool would invoke the
user's PATH, which is bad when potentially using the driver from a
direct path. This patch change this to *only* consider the
`offload-arch` that's supposed to live next to the driver executable.
Now we will no longer pick up a potentially conflicting version of this
tool and it should always be found (Since it's a clang tool that's
installazed alongside the driver)
