Summary:
This patch reworks how we create offloading toolchains. Previously we
would handle this separately for all the different kinds. This patch
instead changes this to use the target triple and the offloading kind to
determine the proper toolchain. In the old case where the user only
passes `--offload-arch` we instead infer the triple from the passed
arguments. This is a pretty major overhaul but currently passes all the
clang tests with only minor changes to error messages.
Summary:
Offloading tends to have a bound architecture that directly correponds
to the `-mcpu` argument for that embedded job. This is currently handled
by the GPU offloading toolchains, but is ignored by the CPU ones. This
is problematic for languages like SYCL or OpenMP which permit
'offloading' to non-GPU targets. This patch handles this by putting
generic handling in the GCC toolchain for those languages.
I would've made this fully generic but it regressed some HIP sanitizer
tests because their use-case is really weird. This also only goes for
the languages that inherit from 'generic_gcc`. I could've made it in the
base class, but I felt like it wasn't necessary as we only support
offloading based off of this toolchain. In the future if we need it we
can move it around.
---------
Co-authored-by: Nick Sarnie <nick.sarnie@intel.com>
Summary:
The test failed because it no longer passed Rpass by default without
LTO. I think that's desirable as it matches the standard behavior.
This reverts commit 6fd99de31864a5ef84ae8613b3a9034e05293461.
Summary:
Recent changes made a lot of this stuff redundant or unused, clean it up
a bit.
Also snuck in a change to pass the CUDA path since we still use it for
`fatbinary` internally.
Summary:
Currently we have a subset of arguments that are handled specially to
keep them consistent between host and device compiles, however, this is
extremely hacky as it only works on a few predetermined options. This is
a holdover from the days before the linker wrapper shuttled all of its
arguments through `clang`. Now that we just use clang, all we need to do
is just use the `--device-compiler=` option to forward it there and let
the normal toolchain handle it.
For example,
```console
clang -fopenmp --offload-arch=gfx1030,sm_89 -Xarch_nvptx64 -O3 -foffload-lto
```
will forward the `-O3` to the LTO compilation only for the NVPTX
compilation.
We see some tests are failing internally after 12949c9.
In some CAS systems, we might not see the exact binary name (clang),
this patch adds the `-no-canonical-prefixes` option to the command line to not
realpath-ify the binary name.
Removed foo-registered-target constraints from a bunch of tests, because
mostly the driver doesn't need to have a target availabile. I ran
check-clang-driver using a build with only the XCore target, and these
all passed.
There are ~50 tests that still have foo-registered-target, and it looks
like most of them are either doing codegen when they don't need to, or
don't really belong in the Driver tests. But that's a task for another
day.
The exit code for -### is inconsistent. Unrecognized options lead to
exit code 1, as expected. However, most others errors (including invalid
option value) lead to exit code 0, differing from GCC and most utilities.
This is a longstanding quirk of -###, and we didn't fix it because many
driver tests need adjustment.
Change -### to be similar to -fdriver-only -v and exit with code 1.
This requires fixing many driver tests, but the end result gives us
stronger tests.
* Existing `RUN: %clang -### ...` tests usually don't use `CHECK-NOT: error:` or `--implicit-check-not=error:`.
If a change introduces an error, such a change usually cannot be detected.
* Many folks contributing new tests don't know `-fdriver-only -v`.
To test no driver error/warning for new tests, they can use the familiar `-### -Werror`
instead of `-fdriver-only -v -Werror`.
An incomplete list of prerequisite test improvement:
* 2f79bb10461d114783a1548201928549ace09755: add -nogpulib to some AMDGPU tests
* 9155e517e6e1cda474d0d0fa82f71696c325bc10: add --cuda-path= (test w/ and w/o /usr/local/cuda)
* 80765ede5bbcca1364c2d4ae06127011eaba6389: -mcpu=native may return either 0 or 1, depending on whether `--target=` specifies a native target
* abae53f43f0d1da8d8e421f4a628d7ec64d6e365: fix -fuse-ld=lld misuses (test w/o and w/o /usr/local/bin/ld.lld)
* ab68df505e5bb8808ee44f53044b50ca7575098e: add -resource-dir= and -fvisibility=hidden
to some -fsanitize=cfi tests
* d5ca1602f64114f612ad5630f04e4aa90591c78d: --rtlib=platform without --unwindlib= may fail if CLANG_DEFAULT_UNWINDLIB=unwindlib
Reviewed By: jhuber6, yaxunl, dblaikie
Differential Revision: https://reviews.llvm.org/D156363
The exit code for -### is inconsistent. Unrecognized options lead to
exit code 1, as expected. However, most others errors (including invalid
option value) lead to exit code 0, differing from GCC and most utilities.
This is a longstanding quirk of -###, and we didn't fix it because many
driver tests need adjustment.
Change -### to be similar to -fdriver-only -v and exit with code 1.
This requires fixing many driver tests, but the end result gives us
stronger tests.
* Existing `RUN: %clang -### ...` tests usually don't use `CHECK-NOT: error:` or `--implicit-check-not=error:`.
If a change introduces an error, such a change usually cannot be detected.
* Many folks contributing new tests don't know `-fdriver-only -v`.
To test no driver error/warning for new tests, they can use the familiar `-### -Werror`
instead of `-fdriver-only -v -Werror`.
An incomplete list of prerequisite test improvement:
* 2f79bb10461d114783a1548201928549ace09755: add -nogpulib to some AMDGPU tests
* 9155e517e6e1cda474d0d0fa82f71696c325bc10: add --cuda-path= (test w/ and w/o /usr/local/cuda)
* 80765ede5bbcca1364c2d4ae06127011eaba6389: -mcpu=native may return either 0 or 1, depending on whether `--target=` specifies a native target
* abae53f43f0d1da8d8e421f4a628d7ec64d6e365: fix -fuse-ld=lld misuses (test w/o and w/o /usr/local/bin/ld.lld)
Reviewed By: jhuber6, yaxunl, dblaikie
Differential Revision: https://reviews.llvm.org/D156363
This reverts commit e39bf32b3bc2f0cc21d783ba789bd82553493875.
Some tests have different behaviors depent on whether certain directories/files are present on the host.
An incomplete list from https://lab.llvm.org/buildbot/#/builders/109/builds/70149
csky-toolchain.c
riscv*-toolchain.c
fuchsia.*
hip-*
ohos.c
The exit code for -### is inconsistent. Unrecognized options lead to
exit code 1, as expected. However, most others errors (including invalid
option value) lead to exit code 0, differing from GCC and most utilities.
This is a longstanding quirk of -###, and we didn't fix it because many
driver tests need adjustment.
Change -### to be similar to -fdriver-only -v and exit with code 1.
This requires fixing many driver tests, but the end result gives us
stronger tests.
* Existing `RUN: %clang -### ...` tests usually don't use `CHECK-NOT: error:` or `--implicit-check-not=error:`.
If a change introduces an error, such a change usually cannot be detected.
* Many folks contributing new tests don't know `-fdriver-only -v`.
To test no driver error/warning for new tests, they can use the familiar `-### -Werror`
instead of `-fdriver-only -v -Werror`.
Reviewed By: jhuber6, yaxunl, dblaikie
Differential Revision: https://reviews.llvm.org/D156363
This patch renames the `OpenMPIRBuilderConfig` flags to reduce confusion over
their meaning. `IsTargetCodegen` becomes `IsGPU`, whereas `IsEmbedded` becomes
`IsTargetDevice`. The `-fopenmp-is-device` compiler option is also renamed to
`-fopenmp-is-target-device` and the `omp.is_device` MLIR attribute is renamed
to `omp.is_target_device`. Getters and setters of all these renamed properties
are also updated accordingly. Many unit tests have been updated to use the new
names, but an alias for the `-fopenmp-is-device` option is created so that
external programs do not stop working after the name change.
`IsGPU` is set when the target triple is AMDGCN or NVIDIA PTX, and it is only
valid if `IsTargetDevice` is specified as well. `IsTargetDevice` is set by the
`-fopenmp-is-target-device` compiler frontend option, which is only added to
the OpenMP device invocation for offloading-enabled programs.
Differential Revision: https://reviews.llvm.org/D154591
Previously when using the `clang-offload-packager` we did not pass the
active offloading kinds. Then in Clang when we attempted to detect when
there was host-offloading action that needed to be embedded in the host
we did not find it. This patch adds the active offloading kinds so we
know when there is input to be embedded.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D134189
The new driver currently crashses when attempting to use the
'-fsyntax-only' option. This is because the option causes all output to
be given the `TY_Nothing' type which should signal the end of the
pipeline. The new driver was not treating this correctly and attempting
to use empty input. This patch fixes the handling so we do not attempt
to continue when the input is nothing.
One concession is that we must now check when generating the arguments
for Clang if the input is of 'TY_Nothing'. This is because the new
driver will only create code if the device code is a dependency on the
host, creating the output without the dependency would require a
complete rewrite of the logic as we do not maintain any state between
calls to 'BuildOffloadingActions' so I believe this is the most
straightforward method.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D133161
Recently OpenMP has transitioned to using the "new" driver which
primarily merges the device and host linking phases into a single
wrapper that handles both at the same time. This replaced a few tools
that were only used for OpenMP offloading, such as the
`clang-offload-wrapper` and `clang-nvlink-wrapper`. The new driver
carries some marked benefits compared to the old driver that is now
being deprecated. Things like device-side LTO, static library
support, and more compatible tooling. As such, we should be able to
completely deprecate the old driver, at least for OpenMP. The old driver
support will still exist for CUDA and HIP, although both of these can
currently be compiled on Linux with `--offload-new-driver` to use the new
method.
Note that this does not deprecate the `clang-offload-bundler`, although
it is unused by OpenMP now, it is still used by the HIP toolchain both
as their device binary format and object format.
When I proposed deprecating this code I heard some vendors voice
concernes about needing to update their code in their fork. They should
be able to just revert this commit if it lands.
Reviewed By: jdoerfert, MaskRay, ye-luo
Differential Revision: https://reviews.llvm.org/D130020
This patch adds the ability to use `-mllvm` options in the linker
wrapper when performing bitcode linking or the module compilation.
This is done by passing in the LLVM argument to the clang-linker-wrapper
tool. Inside the linker-wrapper tool we invoke the `CommandLine` parser
solely for forwarding command line options to the `clang-linker-wrapper`
to the LLVM tools that also use the `CommandLine` parser. The actual
arguments to the linker wrapper are parsed using the `Opt` library
instead.
For example, in the following command the `CommandLine` parser will attempt to
parse `abc`, while the `opt` parser takes `-mllvm <arg>` and ignores it so it is
not passed to the linker arguments.
```
clang-linker-wrapper -mllvm -abc -- <linker-args>
```
As far as I can tell this is the easiest way to forward arguments to
LLVM tool invocations. If there is a better way to pass these arguments
(such as through the LTO config) let me know.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D129424
Summary:
This patch reworks the command line argument handling in the linker
wrapper from using the LLVM `cl` interface to using the `Option`
interface with TableGen. This has several benefits compared to the old
method.
We use arguments from the linker arguments in the linker
wrapper, such as the libraries and input files, this allows us to
properly parse these. Additionally we can now easily set up aliases to
the linker wrapper arguments and pass them in the linker input directly.
That is, pass an option like `cuda-path=` as `--offload-arg=cuda-path=`
in the linker's inputs. This will allow us to handle offloading
compilation in the linker itself some day. Finally, this is also a much
cleaner interface for passing arguments to the individual device linking
jobs.
Previously an opt-in flag `-fopenmp-new-driver` was used to enable the
new offloading driver. After passing tests for a few months it should be
sufficiently mature to flip the switch and make it the default. The new
offloading driver is now enabled if there is OpenMP and OpenMP
offloading present and the new `-fno-openmp-new-driver` is not present.
The new offloading driver has three main benefits over the old method:
- Static library support
- Device-side LTO
- Unified clang driver stages
Depends on D122683
Differential Revision: https://reviews.llvm.org/D122831
It (introduced by 556d713c70bfaf58ac18d089883f9c34c581633a) appears to be
related to the removed dragonegg project. In addition, the feature was a bit
misnamed and may lur users to unnecessarily use it.
Currently, clang-offload-bundler has -inputs and -outputs options that accept
values with comma as the delimiter. This causes issues with file paths
containing commas, which are valid file paths on Linux.
This add two new options -input and -output, which accept one single file,
and allow multiple instances. This allows arbitrary file paths. The old
-inputs and -outputs options will be kept for backward compatibility, but
are not allowed to be used with -input and -output options for simplicity.
In the future, -inputs and -outputs options will be phasing out.
RFC: https://discourse.llvm.org/t/rfc-adding-input-and-output-options-to-clang-offload-bundler/60049
Patch by: Siu Chi Chan
Reviewed by: Yaxun Liu
Differential Revision: https://reviews.llvm.org/D120662
This patch adds support for linking CPU offloading applications in the
linker wrapper. We generate the necessary linking job using the host
linker's path and library arguments. This may not be true for more
complex offloading schemes, but this is sufficient for now.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D119613
Clang diagnostics should not start with a capital letter or use
trailing punctuation (https://clang.llvm.org/docs/InternalsManual.html#the-format-string),
but quite a few driver diagnostics were not following this advice. This
corrects the grammar and punctuation to improve consistency, but does
not change the circumstances under which the diagnostics are produced.
This ensures that the Darwin driver uses a consistent target triple
representation when the triple is printed out to the user.
This reverts the revert commit ab0df6c0346e515291a381467527621ab0ccf953.
Differential Revision: https://reviews.llvm.org/D100807
There are out-of-tree tools using clang-offload-bundler to extract
bundles from bundled files. When a bundle is not in the bundled
file, clang-offload-bundler is expected to emit an error message
and return non-zero value. However currently clang-offload-bundler
silently generates empty file for the missing bundles.
Since OpenMP/HIP toolchains expect the current behavior, an option
-allow-missing-bundles is added to let clang-offload-bundler
create empty file when a bundle is missing when unbundling.
The unbundling job action is updated to use this option by
default.
clang-offload-bundler itself will emit error when a bundle
is missing when unbundling by default.
Changes are also made to check duplicate targets in -targets
option and emit error.
Differential Revision: https://reviews.llvm.org/D93068
Summary: With OpenMP offloading host compilation is done in two phases to capture host IR that is passed to all device compilations as input. But it turns out that we currently run entire LLVM optimization pipeline on host IR on both compilations which may have unpredictable effects on the resulting code. This patch fixes this problem by disabling LLVM passes on the first compilation, so the host IR that is passed to device compilations will be captured right after front end.
Reviewers: ABataev, jdoerfert, hfinkel
Reviewed By: ABataev
Subscribers: guansong, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73721
This patch removes the remaining part of the OpenMP offload linker scripts which was used for inserting device binaries into the output linked binary. Device binaries are now inserted into the host binary with a help of the wrapper bit-code file which contains device binaries as data. Wrapper bit-code file is dynamically created by the clang driver with a help of new tool clang-offload-wrapper which takes device binaries as input and produces bit-code file with required contents. Wrapper bit-code is then compiled to an object and resulting object is appended to the host linking by the clang driver.
This is the second part of the patch for eliminating OpenMP linker script (please see https://reviews.llvm.org/D64943).
Differential Revision: https://reviews.llvm.org/D68166
llvm-svn: 374219
Linker automatically provides __start_<section name> and __stop_<section name> symbols to satisfy unresolved references if <section name> is representable as a C identifier (see https://sourceware.org/binutils/docs/ld/Input-Section-Example.html for details). These symbols indicate the start address and end address of the output section respectively. Therefore, renaming OpenMP offload entries section name from ".omp.offloading_entries" to "omp_offloading_entries" to use this feature.
This is the first part of the patch for eliminating OpenMP linker script (please see https://reviews.llvm.org/D64943).
Differential Revision: https://reviews.llvm.org/D68070
llvm-svn: 373118
New -cc1 arguments, such as -faddrsig, have started appearing after the
input name. I personally find it convenient for the input to be the last
argument to the compile command line, since I often need to edit it when
running crash reproduction scripts.
Differential Revision: https://reviews.llvm.org/D62270
llvm-svn: 361530
The CHECK lines as structured were requiring them to appear only in a certain
position while all that is really needed is to check that they are present.
llvm-svn: 354001
clang-offload-bundler should not be invoked with the unbundling action
when the input file type does not match the action type. For example,
.so files should be unbundled during linking phase and should be linked
only with the host code.
llvm-svn: 343335
types."
It reverts commit r342991 + several other commits intended to fix the
tests. Still have some failed tests, need to investigate it.
llvm-svn: 343002
clang-offload-bundler should not be invoked with the unbundling action
when the input file type does not match the action type. For example,
.so files should be unbundled during linking phase and should be linked
only with the host code.
llvm-svn: 342991
LLVM triple normalization is handling "unknown" and empty components
differently; for example given "x86_64-unknown-linux-gnu" and
"x86_64-linux-gnu" which should be equivalent, triple normalization
returns "x86_64-unknown-linux-gnu" and "x86_64--linux-gnu". autoconf's
config.sub returns "x86_64-unknown-linux-gnu" for both
"x86_64-linux-gnu" and "x86_64-unknown-linux-gnu". This changes the
triple normalization to behave the same way, replacing empty triple
components with "unknown".
This addresses PR37129.
Differential Revision: https://reviews.llvm.org/D50219
llvm-svn: 339294
The recent fix in D38258 was wrong: getAuxTriple() only returns
non-null values for the CUDA toolchain. That is why the now added
test for PPC and X86 failed.
Differential Revision: https://reviews.llvm.org/D38372
llvm-svn: 314902
AuxTriple is not set if host and device share a toolchain. Also,
removing an argument modifies the DAL which needs to be returned
for future use.
(Move tests back to offload-openmp.c as they are not related to GPUs.)
Differential Revision: https://reviews.llvm.org/D38258
llvm-svn: 314329
