The simplest way is:
1. Save `vtype` to a scalar register.
2. Insert a `vsetvli`.
3. Use segment load/store.
4. Restore `vtype` via `vsetvl`.
But `vsetvl` is usually slow, so this PR is not in this way.
Instead, we use wider whole load/store instructions if the register
encoding is aligned. We have done the same optimization for COPY in
https://github.com/llvm/llvm-project/pull/84455.
We found this suboptimal implementation when porting some video codec
kernels via RVV intrinsics.
The order entries in the tablegen API files are iterated is not the
order
they appear in the file. To avoid any issues with the order changing
in future, we now generate all definitions of a certain class before
class that can use them.
This is a NFC; the definitions don't actually change, just the order
they exist in in the OffloadAPI.h header.
This keeps it closer to the other legality checks like the FP exceptions
check.
It also means that isSupportedInstr only needs to check the opcode,
which allows it to be replaced with a TSFlags based check in a later
patch.
This patch works towards consolidating all Clang debug-info into the
`clang/test/DebugInfo` directory
(https://discourse.llvm.org/t/clang-test-location-of-clang-debug-info-tests/87958).
Here we move only the `clang/test/CodeGenCXX` tests. I created a `CXX`
subdirectory for now because many of the tests I checked actually did
seem C++-specific. There is probably overlap between the `Generic` and
`CXX` subdirectory, but I haven't gone through and audited them all.
The list of files i came up with is:
1. searched for anything with `*debug-info*` in the filename
2. searched for occurrences of `debug-info-kind` in the tests
There's a couple of tests in `clang/test/CodeGenCXX` that still set
`-debug-info-kind`. They probably don't need to do that, but I'm not
changing that as part of this PR.
This patch adds feature modules registry, as discussed with @kadircet in
[discourse](https://discourse.llvm.org/t/rfc-registry-for-feature-modules/87733).
Feature modules, which added into the feature module set from registry
entries, can't expose public API, but still can be used via
`FeatureModule` interface.
MSVC's STL marks `std::make_shared`, `std::allocate_shared`,
`std::bitset::to_ulong`, and `std::bitset::to_ullong` as
`[[nodiscard]]`, which causes these libcxx tests to emit righteous
warnings. They should use the traditional `(void)` cast technique to
ignore the return values.
Currently, Flang can generate no-loop kernels for all OpenMP target
kernels in the program if the flags
-fopenmp-assume-teams-oversubscription or
-fopenmp-assume-threads-oversubscription are set.
If we add an additional parameter, we can choose
in the future which OpenMP kernels should be generated as no-loop
kernels.
This PR doesn't modify current behavior of oversubscription flags.
RFC for no-loop kernels:
https://discourse.llvm.org/t/rfc-no-loop-mode-for-openmp-gpu-kernels/87517
The change in PR #154268 introduced a dependency on the `__GLIBC_PREREQ`
macro, which is not defined in musl libc. This caused the build to fail
in environments using musl.
This patch fixes the build by including
`sanitizer_common/sanitizer_glibc_version.h`. This header provides a
fallback definition for `__GLIBC_PREREQ` when LLVM is built against
non-glibc C libraries, resolving the compilation error.
It is going to grow, so it makes sense to move its definition
out of class. Instead, inline `populateInstruction()` into it.
Also, rename a couple of methods to better convey their meaning.
That is, on all targets except ARM and AArch64.
This field used to be required due to a bug, it was fixed long ago
by 23423c0ea8d414e56081cb6a13bd8b2cc91513a9.
This patch contains a list of tests that are currently failing in the
LLVM_ENABLE_PROFCHECK=ON build. This enables passing them to lit through
the LIT_XFAIL env variable. This is necessary for getting a buildbot
spun up to catch regressions while work is being done to fix the
existing issues.
We need to keep this in the LLVM tree so that tests can be removed from
the list at the same time the passes causing issues are fixed.
Issue #147390
LSan was recently refactored to call GetMaxUserVirtualAddress for
diagnostic purposes. This leads to failures for some of our downstream
tests which only run with lsan. This occurs because
GetMaxUserVirtualAddress depends on setting up shadow via a call to
__sanitizer_shadow_bounds, but shadow bounds aren't set for standalone
lsan because it doesn't use shadow. This updates the function to invoke
the same syscall used by __sanitizer_shadow_bounds calls for getting the
memory limit. Ideally this function would only be called once since we
only need to get the bounds once.
More context in https://fxbug.dev/437346226.
Operate directly on the existing Ops vector instead of copying to
a new vector. This is similar to what the autogenerated codegen
does for other intrinsics.
Currently only cases rooted at a full copy of an MFMA result are
handled.
Prepare to relax that by testing more intricate subregister usage.
Currently only full copies are handled, add some tests to help work
towards handling subregisters.
Previously it would just assert if the extract needed elements from
both halves. Extract the individual elements from both halves and
create a new vector, as the simplest implementation. This could
try to do better and create a partial extract or shuffle (or
maybe that's best left for the combiner to figure out later).
Fixes secondary issue noticed as part of #153808
`VPEVLBasedIVPHIRecipe` will lower to VPInstruction scalar phi and
generate scalar phi. This recipe will only occupy a scalar register just
like other phi recipes.
This patch fix the register usage for `VPEVLBasedIVPHIRecipe` from
vector
to scalar which is close to generated vector IR.
https://godbolt.org/z/6Mzd6W6ha shows that no register spills when
choosing `<vscale x 16>`.
Note that this test is basically copied from AArch64.
SimplifyBranchConditionForVFAndUF only recognized canonical IVs and a
few PHI
recipes in the loop header. With more IV-step optimizations,
the canonical widen-canonical-iv can be replaced by a canonical
VPWidenIntOrFpInduction,
which the pass did not handle, causing regressions (missed
simplifications).
This patch replaces canonical VPWidenIntOrFpInduction with a StepVector
in the vector preheader
since the vector loop region only executes once.
I'm trying to remove the redirection in SmallSet.h:
template <typename PointeeType, unsigned N>
class SmallSet<PointeeType*, N> : public SmallPtrSet<PointeeType*, N>
{};
to make it clear that we are using SmallPtrSet. There are only
handful places that rely on this redirection.
This patch replaces SmallSet to SmallPtrSet where the element type is
a pointer.
I'm trying to remove the redirection in SmallSet.h:
template <typename PointeeType, unsigned N>
class SmallSet<PointeeType*, N> : public SmallPtrSet<PointeeType*, N>
{};
to make it clear that we are using SmallPtrSet. There are only
handful places that rely on this redirection.
This patch replaces SmallSet to SmallPtrSet where the element type is
a pointer.
I'm trying to remove the redirection in SmallSet.h:
template <typename PointeeType, unsigned N>
class SmallSet<PointeeType*, N> : public SmallPtrSet<PointeeType*, N>
{};
to make it clear that we are using SmallPtrSet. There are only
handful places that rely on this redirection.
This patch replaces SmallSet to SmallPtrSet where the element type is
a pointer.
#153975 added a new test,
`test/CodeGen/AMDGPU/disable-preload-kernargs.ll`, that triggers an
assertion under `LLVM_ENABLE_EXPENSIVE_CHECKS` complaining about not
invalidating analyses even when the Pass made changes. It was caused by
the fact that the Pass only invalidates the analyses when number of
explicit arguments is greater than zero, while it is possible that some
functions will be removed even when there isn't any explicit argument,
hence the missed invalidation.
If src2 and dst aren't the same register, to fold a copy
to AGPR into the instruction we also need to reassign src2
to an available AGPR. All the other uses of src2 also need
to be compatible with the AGPR replacement in order to avoid
inserting other copies somewhere else.
Perform this transform, after verifying all other uses are
compatible with AGPR, and have an available AGPR available at
all points (which effectively means rewriting a full chain of
mfmas and load/store at once).
Currently the privatization recipe of a scalar allocatable is as follow:
```
acc.private.recipe @privatization_ref_box_heap_i32 : !fir.ref<!fir.box<!fir.heap<i32>>> init {
^bb0(%arg0: !fir.ref<!fir.box<!fir.heap<i32>>>):
%0 = fir.alloca !fir.box<!fir.heap<i32>>
%1:2 = hlfir.declare %0 {uniq_name = "acc.private.init"} : (!fir.ref<!fir.box<!fir.heap<i32>>>) -> (!fir.ref<!fir.box<!fir.heap<i32>>>, !fir.ref<!fir.box<!fir.heap<i32>>>)
acc.yield %1#0 : !fir.ref<!fir.box<!fir.heap<i32>>>
}
```
This change adds the allocation for the scalar.
Summary:
This patch changes the linux build to use the wide reads on the memory
operations by default. These memory functions will now potentially read
outside of the bounds explicitly allowed by the current function. While
technically undefined behavior in the standard, plenty of C library
implementations do this. it will not cause a segmentation fault on linux
as long as you do not cross a page boundary, and because we are only
*reading* memory it should not have atomic effects.
Operate directly on the existing Ops vector instead of copying to
a new vector. This is similar to what the autogenerated codegen
does for other intrinsics.
This reduced the clang binary size by ~96kb on my local Release+Asserts
build.
We used to abuse Operands list to store instruction encoding's
DecoderMethod there. Let's store it in the InstructionEncoding class
instead, where it belongs.
