Fix a regression of df7473673214.
cmake MSVC generator is multiple configurations. Build type is not known
at configure time and CMAKE_CFG_INTDIR is evaluated to $(Configuration)
at configure time. libclc install fails since $(Configuration) in
bitcode file path is unresolved in libclc/cmake_install.cmake at install time.
We need a solution that resolves libclc bitcode file path at install
time. This PR fixes the issue using CMAKE_INSTALL_CONFIG_NAME which can
be evaluated at install time. This is the same solution as in
https://reviews.llvm.org/D76827
The target's output bitcode `libclc_builtins_lib` is located in a
sub-directory in clang resource directory since df7473673214. Setting
TARGET_FILE property can allow targets in non-libclc project to obtain
the path to `libclc_builtins_lib`.
This removes the dependency on an external tool to build the SPIR-V
files. It may be of interest to projects such as Mesa.
Note that the option is off by default as using the SPIR-V backend, at
least on my machine, uses a *lot* of memory and the process is often
killed in a parallelized build. It does complete, however.
Fixes#135327.
With libclc being a 'runtime', the top-level build assumes that there is
a corresopnding 'libclc' target. We previously weren't providing this,
leading to a build failure if the user tried to build it.
This commit remedies this by adding support for building the 'libclc'
target. It does so by adding dependencies from the OpenCL builtins to
this target. It uses a configurable in-between target -
libclc-opencl-builtins - to ease the possibility of adding non-OpenCL
builtin libraries in the future.
Fix the symlink creation logic to use relative paths instead of
absolute, in order to ensure that the installed symlinks actually refer
to the installed .bc files rather than the ones from the build
directory. This was broken in #146833. The change is a bit roundabout
but it attempts to preserve the spirit of #146833, that is the ability
to use multiple output directories (provided they all resides in
`${LIBCLC_OUTPUT_LIBRARY_DIR}` and preserve the same structure in the
installed tree).
Signed-off-by: Michał Górny <mgorny@gentoo.org>
The prepare target was depending on the output of a custom command, but
wasn't the full path to that file. This tripped up CMake if the file was
removed as it didn't know how to rebuild that file.
These changes were split off from #146503.
This commit makes the output directories of libclc artefacts explicit.
It creates a variable for the final output directory -
LIBCLC_OUTPUT_LIBRARY_DIR - which has not changed. This allows future
changes to alter the output directory more simply, such as by pointing
it to somewhere inside clang's resource directory.
This commit also changes the output directory of each target's
intermediate builtins.*.bc files. They are now placed into each
respective libclc target's object directory, rather than the top-level
libclc binary directory. This should help keep the binary directory a
bit tidier.
This target provides a unified build target for all devices under the
single triple. This way a user doesn't have to know device names to
build a specific target's bytecode libraries.
Device names may be considered as internal implementation details as
they are not exposed to users of CMake; users only specify triples to
build. Now, instead of `prepare-{barts,cayman,cedar,cypress}-r600--.bc`,
for example, a user may now build simply `prepare-r600--` and have all
four of those libraries built.
This commit also refactors the CMake somewhat. We were previously
diverging between the SPIR-V and other targets, and duplicating a bit of
logic like the creation of the 'prepare' targets, the targets'
properties, and the installation directory. It's cleaner and hopefully
more robust to share this code between all targets. This commit also
takes this opportunity to improve some comments around this code.
This commits moves all OpenCL builtins under a top-level 'opencl'
directory, akin to how the CLC builtins are organized. This new
structure aims to better convey the separation of the two layers and
that 'CLC' is not a subset of OpenCL or a libclc target.
In doing so this commit moves the location of the 'lib' directory to
match CLC: libclc/generic/lib/ becomes libclc/opencl/lib/generic/. This
allows us to remove some special casing in CMake and ensure a common
directory structure.
It also tries to better communicate that the OpenCL headers are
libclc-specific OpenCL headers and should not be confused with or used
as standard OpenCL headers. It does so by ensuring includes are of the
form <clc/opencl/*>. It might be that we don't specifically need the
libclc OpenCL headers and we simply could use clang's built-in
declarations, but we can revisit that later.
Aside from the code move, there is some code formatting and updating a
couple of OpenCL builtin includes to use the readily available gentype
helpers. This allows us to remove some '.inc' files.
This enables file_specific_compile_options to take precedence over
ARG_COMPILE_FLAGS. For example, if we add -fno-slp-vectorize to
COMPILE_OPTIONS of a file, the behavior changes as follows:
* Before this PR: -fno-slp-vectorize is overwritten by -O3, resulting in
SLP vectorizer remaining enabled.
* After this PR: -fno-slp-vectorize overwrites -O3, effectively
disabling SLP vectorizer.
llvm-diff shows this PR has no changes to amdgcn--amdhsa.bc.
Motivation is that in our downstream the same category of target
built-ins, e.g. math, are organized in several different folders. For
example, in target SOURCES we have math-common/cos.cl, while in generic
SOURCES it is math/cos.cl. Based on current check rule that compares
both folder name and base filename, target math-common/cos.cl won't
override math/cos.cl when collecting source files from SOURCES files in
cmake function libclc_configure_lib_source.
With this PR, we allow folder name to be different in the process.
A notable change of this PR is that two entries in SOURCES with the same
base filename must not implements the same built-in.
In libclc, we observe that compiling OpenCL source files to bitcode is
executed sequentially on Windows, which increases debug build time by
about an hour.
add_custom_command may introduce additional implicit dependencies, see
https://gitlab.kitware.com/cmake/cmake/-/issues/17097
This PR adds a target for each command, enabling parallel builds of
OpenCL source files.
CMake 3.27 has fixed above issue with DEPENDS_EXPLICIT_ONLY. When LLVM
upgrades cmake vertion to 3.7, we can switch to DEPENDS_EXPLICIT_ONLY.
The libclc build system isn't well set up to pass arbitrary options to
arbitrary source files in a non-intrusive way. There isn't currently any
other motivating example to warrant rewriting the build system just to
satisfy this requirement. So this commit uses a filename-based approach
to inserting this option into the list of compile flags.
Currently link_bc command depends on the bitcode file that is associated
with custom target builtins.link.clc-arch_suffix.
On windows we randomly see following error:
`
Generating builtins.link.clc-${ARCH}--.bc
Generating builtins.link.libspirv-${ARCH}.bc
error : The requested operation cannot be performed on a file with a
user-mapped section open.
`
I suspect that builtins.link.clc-${ARCH}--.bc file is being generated
while it is being used in link_bc.
This PR adds target-level dependency to ensure
builtins.link.clc-${ARCH}--.bc is generated first.
When -internalize flag is passed to llvm-link, we only need to link in
needed symbols. This PR reduces size of linked bitcode, e.g. by removing
following symbols:
_Z12__clc_sw_fmaDv16_fS_S_
_Z12__clc_sw_fmaDv2_fS_S_
_Z12__clc_sw_fmaDv3_fS_S_
_Z12__clc_sw_fmaDv4_fS_S_
_Z12__clc_sw_fmaDv8_fS_S_
_Z12__clc_sw_fmafff
During a recent change, the build system accidentally dropped the
(theoretical) support for the CLC builtins library to build
target-specific builtins from the 'amdgpu' directory, due to a change in
variable names. This functionality wasn't being used but was spotted
during another code review.
This commit takes the opportunity to clean up and better document the
code that manages the list of directories to search for builtin
implementations.
While fixing this, some references to now-removed SOURCES files were
discovered which have been cleaned up.
libclc uses llvm-link to link together all of the individually built
libclc builtins files into one module. Some of these builtins files are
compiled from source by clang whilst others are converted from LLVM IR
directly to bytecode.
When llvm-link links a 'source' module into a 'destination' module, it
warns if the two modules have differing data layouts.
The LLVM IR files libclc links either have no data layout (shared
submodule files) or an explicit data layout in the case of certain
amdgcn/r600 files.
The warnings are very noisy and largely inconsequential. We can suppress
them exploiting a specific behaviours exhibited by llvm-link. When the
destination module has no data layout, it is given the source module's
data layout. Thus, if we link together all IR files first, followed by
the clang-compiled modules, 99% of the warnings are suppressed as they
arose from linking an empty data layout into a non-empty one.
The remaining warnings came from the amdgcn and r600 targets. Some of
these were because the data layouts were out of date compared with what
clang currently produced, so those could have been updated.
However, even with those changes and by grouping the IR files together,
the linker may still link explicit data layouts with empty ones
depending on the order the IR files are processed.
As it happens, the data layouts aren't essential. With the changes to
the link line we can rely on those IR files receiving the correct data
layout from the clang-compiled modules later in the link line. This also
makes the previously AMDGPU-specific IR files available to be used by
all targets in a generic capacity in the future.
In #127378 it was reported that builds without clspv targets enabled
were failing after #124727, as all targets had a dependency on a file
that only clspv targets generated.
A quick fix was merged in #127315 which wasn't correct. It moved the
dependency on those generated files to the spirv targets, instead of
onto the clspv targets. This means a build with spirv targets and
without clspv targets would see the same problems as #127378 reported.
I tried simply removing the requirement to explicitly add dependencies
to the custom command, relying instead on the file-level dependencies.
This didn't seem reliable enough; in some cases on a Makefiles build,
the clang command compiling (e.g.,) convert.cl would begin before the
file was fully written.
Instead, we keep the target-level dependency but automatically infer it
based on the generated file name, to avoid manual book-keeping of pairs
of files and targets.
This commit also fixes what looks like an unintended bug where, when
ENABLE_RUNTIME_SUBNORMAL was enabled, the OpenCL conversions weren't
being compiled.
Fix `add_libclc_builtin_set` to add an appropriate dependency to either
`clspv-generate_convert.cl` or `generate_convert.cl` based on the `ARCH`
argument, rather than to both unconditionally. This fixes build failures
due to missing dependencies when `clspv*` targets are not enabled.
The added check mirrors the one from `libclc/CMakeLists.txt`.
Fixes: #127378
Some libclc builtins currently use internal builtins prefixed with
'__clc_' for various reasons, e.g., to avoid naming clashes.
This commit formalizes this concept by starting to isolate the
definitions of these internal clc builtins into a separate
self-contained bytecode library, which is linked into each target's
libclc OpenCL builtins before optimization takes place.
The goal of this step is to allow additional libraries of builtins
that provide entry points (or bindings) that are not written in OpenCL C
but still wish to expose OpenCL-compatible builtins. By moving the
implementations into a separate self-contained library, entry points can
share as much code as possible without going through OpenCL C.
The overall structure of the internal clc library is similar to the
current OpenCL structure, with SOURCES files and targets being able to
override the definitions of builtins as needed. The idea is that the
OpenCL builtins will begin to need fewer target-specific overrides, as
those will slowly move over to the clc builtins instead.
Another advantage of having a separate bytecode library with the CLC
implementations is that we can internalize the symbols when linking it
(separately), whereas currently the CLC symbols make it into the final
builtins library (and perhaps even the final compiled binary).
This patch starts of with 'dot' as it's relatively self-contained, as
opposed to most of the maths builtins which tend to pull in other
builtins.
We can also start to clang-format the builtins as we go, which should
help to modernize the codebase.
This splits off several key parts of the build system into utility
methods. This will be used in upcoming patches to help provide
additional sets of target-specific builtin libraries.
Running llvm-diff on the resulting LLVM bytecode binaries, and regular
diff on SPIR-V binaries, shows no differences before and after this
patch.
The `ARCHIVE` artifact kind is not valid for `install(FILES ...)`.
Additionally, install wasn't resolving the target's `TARGET_FILE`
properly and was trying to find it in the top-level build directory, rather than
in the libclc binary directory. This is because our `TARGET_FILE`
properties were being set to relative paths. The cmake behaviour they
are trying to mimic - `$<TARGET_FILE:$tgt>` - provides an absolute path.
As such this patch updates instances where we set the `TARGET_FILE`
property to return an absolute path.
Reviewers of #89153 suggested to break up the patch into per-subproject
patches. This is the libclc part. See #89153 for the entire series and
motivation.
Update the folder titles for targets in the monorepository that have not
seen taken care of for some time. These are the folders that targets are
organized in Visual Studio and XCode
(`set_property(TARGET <target> PROPERTY FOLDER "<title>")`)
when using the respective CMake's IDE generator.
* Ensure that every target is in a folder
* Use a folder hierarchy with each LLVM subproject as a top-level folder
* Use consistent folder names between subprojects
* When using target-creating functions from AddLLVM.cmake, automatically
deduce the folder. This reduces the number of
`set_property`/`set_target_property`, but are still necessary when
`add_custom_target`, `add_executable`, `add_library`, etc. are used. A
LLVM_SUBPROJECT_TITLE definition is used for that in each subproject's
root CMakeLists.txt.
When performing cross in-tree builds, we need native versions of various
tools, we cannot assume the cross builds that are part of the current
build are executable. LLVM provides the setup_host_tool function to
handle this, either picking up versions of tools from
LLVM_NATIVE_TOOL_DIR, or implicitly building native versions as needed.
Use it for libclc too.
LLVM's setup_host_tool function assumes the project is LLVM, so this
also needs libclc's project() to be conditional on it being built
standalone. Luckily, the only change this needs is using
CMAKE_CURRENT_SOURCE_DIR instead of PROJECT_SOURCE_DIR.
With the Makefile generator and particularly high build parallelism some
intermediate dependencies may be generated redundantly and concurrently,
leading to build failures.
To fix this, arrange for libclc's add_custom_commands to depend on
targets in addition to files.
This follows CMake documentation's[^1] guidance on add_custom_command:
> Do not list the output in more than one independent target that may
> build in parallel or the instances of the rule may conflict. Instead,
> use the add_custom_target() command to drive the command and make the
> other targets depend on that one.
Eliminating the redundant commands also improves build times.
[^1]: https://cmake.org/cmake/help/v3.29/command/add_custom_command.html
We've recently seen the libclc llvm-link invocations become so long that
they exceed the character limits on certain platforms.
Using a 'response file' should solve this by offloading the list of
inputs into a separate file, and using special syntax to pass it to
llvm-link. Note that neither the response file nor syntax aren't
specific to Windows but we restrict it to that platform regardless. We
have the option of expanding it to other platforms in the future.
Commit #87622 broke the build. Ninja was happy with creating the output
directories as necessary, but Unix Makefiles isn't. Ensure they are
always created.
Fixes#88626.
The previous build system was adding custom "OpenCL" and "LLVM IR"
languages in CMake to build the builtin libraries. This was making it
harder to build in-tree because the tool binaries needed to be present
at configure time.
This commit refactors the build system to use custom commands to build
the bytecode files one by one, and link them all together into the final
bytecode library. It also enables in-tree builds by aliasing the
clang/llvm-link/etc. tool targets to internal targets, which are
imported from the LLVM installation directory when building out of tree.
Diffing (with llvm-diff) all of the final bytecode libraries in an
out-of-tree configuration against those built using the current tip
system shows no changes. Note that there are textual changes to metadata
IDs which confuse regular diff, and that llvm-diff 14 and below may show
false-positives.
This commit also removes a file listed in one of the SOURCEs which
didn't exist and which was preventing the use of
ENABLE_RUNTIME_SUBNORMAL when configuring CMake.
