Summary:
This warning exists because if you do `-mcpu=native` in some contexts it
may not be obvious which GPU you get. But if they are all the same then
it really don't make a difference since we just pass the first one to
`-mcpu` anyway. Relax this so it doesn't annoyingly warn on machines
with more than one of the same GPU.
---------
Co-authored-by: Jacob Lambert <jacob.lambert@amd.com>
Add CUFRegisterManagedVariable runtime wrapper in flang-rt that calls
__cudaRegisterManagedVar.
This is preparation for supporting non-allocatable managed variables.
No functional change -- nothing calls this yet.
SCEV isKnownPredicate may crash if the expressions are involved with
different loops. To verify if two loops have the same iteration space,
we do not need to use the SCEV apis, and it can be done by the equality
check.
Moreover, no pass (not even loop fusion) requires to check SameSD levels
for more than one level. In this patch, we limit the analysis of SameSD
levels to only one level after the common levels.
If the function dependencesAllowFusion returns false, in fuseCandidates
the reportLoopFusion function is used to increment InvalidDependencies
and to emit a OptimizationRemarkMissed. If both dependencesAllowFusion
and reportLoopFusion increment InvalidDependencies, statistics will
appear duplicated
Supported before: `fir.store`, `fir.box_addr`, and `fir.call` only for
PointerAllocate/PointerDeallocate.
Added now: fir.call support for PointerAllocateSource,
PointerDeallocatePolymorphic, AllocatableAllocate,
AllocatableAllocateSource, AllocatableDeallocate,
AllocatableDeallocatePolymorphic (found in
flang/include/flang/Runtime/allocatable.h).
Explicitly check whether we are building debugserver for arm64e. To
debug an arm64e binary, debugserver itself needs to be an arm64e
process.
This PR eliminates the possibility of configuring LLDB with Right now,
it's possible to configure CMake with
`LLDB_ENABLE_ARM64E_DEBUGSERVER=Off` and the decorator wouldn't account
for that.
The flang compiler intentionally issues a warning on duplicate
prefix-specs for procedures. This is not consistent with the standard
which says "shall contain at most one of each". Other tested compilers
correctly issue an error.
It is safe to leave this as a warning. It can be turned into an error
condition by using the "-Werror" flag.
However, it should be noted in the Extensions document, similar to the
mention of the SAVE attribute.
The non-constant size handling in `emitCXXNewAllocSize` was making the
incorrect assumption that the default behavior of the size value being
explicitly cast to size_t would be the only behavior we'd see. This is
actually only true with C++14 and later. To properly handle earlier
standards, we need the more robust checking that classic codegen does in
the equivalent function. This change adds that handling.
Assisted-by: Cursor / claude-4.6-opus-high
Implements a structural stall heuristic that considers both resource
hazards and latency constraints when selecting instructions. In coexec,
this changes the pending queue from a binary “not ready to issue”
distinction into part of a unified candidate comparison. Pending
instructions still identify structural stalls in the current cycle, but
they are now evaluated directly against available instructions by stall
cost, making the heuristics both more intuitive and more expressive.
- Add getStructuralStallCycles() to GCNSchedStrategy that computes the
number of cycles an instruction must wait due to:
- Resource conflicts on unbuffered resources (from the SchedModel)
- Sequence-dependent hazards (from GCNHazardRecognizer)
- Add getHazardWaitStates() to GCNHazardRecognizer that returns the
number
of wait states until all hazards for an instruction are resolved,
providing cycle-accurate hazard information for scheduling heuristics.
bitreverse(bitreverse(x)) == x and byte_swap(byte_swap(x)) == x are
mathematical involutions.
This adds MLIR Involution trait to CIR opetation, it encodes this
property and automatically folds away the outer application when an op's
input is produced by the same op type.
- Fix buildNDRange according to OpenCL and SPIRV specs.
- Fix tablegen SPIRV builtins for ndrange_* functions: despite of OpenCL
spec, the real call has additional first argument - structure return,
changed min and max num arguments accordingly.
- Update test, add checks, combined with BuildNDRange_2
Add support for inlining hlfir.assign operations where the LHS is an
allocatable and the RHS is an `hlfir.expr` (e.g., from
`hlfir.elemental`). Since `hlfir.expr` values cannot alias with memory
locations, these assignments can be safely inlined without alias
analysis.
The optimization is controlled by the
-inline-hlfir-allocatable-expr-assign flag:
fir-opt -inline-hlfir-allocatable-expr-assign --inline-hlfir-assign
file.fir
flang -mmlir -inline-hlfir-allocatable-expr-assign file.f90
The generated code uses `fir:🏭:genReallocIfNeeded` to properly
handle Fortran allocatable assignment semantics (F2018 10.2.1.3):
- If not allocated: allocate with RHS shape
- If allocated with same shape: reuse existing allocation
- If allocated with different shape: reallocate
This is motivated by the increased link time that I am seeing when
`__FortranAAssign` is called from the user code. The details of the
problem are documented in
https://github.com/llvm/llvm-project/issues/187720
Symbols that may be referenced indirectly by lastprivate or linear
DSAs in nested constructs must not have their privatization skipped
in the enclosing context. This avoids updates to the wrong symbols.
This is needed to unblock PRs that fix#170784.
Fixes#179345
When fir.array_coor carries an explicit shape_shift (non-default lower
bounds) and an explicit slice, the indices are Fortran indices rather
than 1-based section indices. The FIRToMemRef pass was unconditionally
subtracting 1 from sliced indices, which is only correct for 1-based
section indices (the embox-with-embedded-slice case).
For shape_shift + explicit slice, the correct adjustment is to subtract
the slice lower bound instead of 1. This produces proper 0-based memref
indices.
This pattern arises after the FIR inliner canonicalizes
fir.embox(shape_shift, slice) + fir.array_coor(box) into a single
fir.array_coor with explicit shape_shift and slice operands, where the
indices become Fortran indices.
Without this fix, arrays with non-default lower bounds (e.g., A(0:N) or
A(-1:N)) produce negative memref indices, writing before the array
allocation and causing a segfault.
The test was added in
4e9894498e.
Alternative fixes would be:
* Remove unused GEP, although not clear why we'd want to overwrite
stored `i64` with `ptr` store.
* Keep this patch, but perform both GEPs with `i64` element type to
reduce the diff. It's not clear if the scalarization caused by that type
mismatch is intentional/relevant for the original change.
PSADBW takes vXi8 inputs and gives a vXi64 result so we need to tweak
the bitcasts (shuffle types checks will already ensure that the result
type isn't affected).
Minor improvement to #187447
`LoopVectorizationCostModel::getPredBlockCostDivisor(...)` may return
large `uint64_t` values that get coerced to an `unsigned` by
`VPCostContext::getPredBlockCostDivisor(...)`, which can cause division
by zero.
Fixes#187584
This PR adds support for region control-flow. Region control-flow and
CFG can be mixed together in the same program. See the [accompanying
RFC](https://discourse.llvm.org/t/rfc-support-region-control-flow-in-mem2reg/90082)
for some design considerations.
Beyond the considerations in the RFC, a few minor changes were
introduced:
- Calling the visitor hook for defined values is now deferred to the end
of promotion.
- The lazy creation of default values has been moved to the places where
it happens to prepare for a future change where it is actually lazy.
Documentation about it not working as intended for now was also added.
All SCF operations are supported, including `forall` and `parallel`,
which is pretty cool I think.
I am sorry in advance for git diff displaying a really bad diff for
Mem2Reg.cpp around where the liveness analysis used to be. Do consider
simply reading this part of the code off the file.
As a disclaimer, I designed all the test cases myself, but I used a
large amount of matrix multiplications to produce the corresponding IR
and FileCheck tests. I have reviewed them carefully and they correspond
to my intent.
---------
Co-authored-by: Slava Zakharin <szakharin@nvidia.com>
Add RegBankLegalize rules for the amdgcn_permlane16
and amdgcn_permlanex16 intrinsics. Both intrinsics
are sources of divergence, so only the divergent
case is needed: result, old, and src0 map to VGPR,
while src1 and src2 are SGPR with ReadFirstLane if
divergent.
Update the GISEL RUN lines in llvm.amdgcn.permlane.ll
and permlane16_opsel.ll to use -new-reg-bank-select,
and regenerate check lines. The v8i16 test cases now
produce identical SDAG/GISEL output so their checks
are unified.
This PR removes the first argument from the `Args` list (which is `S()`)
before doing lifetime safety checks to ensure correct indexing.
It also adds a test to prevent regressions in the future
Fixes#187426
<details>
<summary>Bug details</summary>
When calling a `static operator()` directly (with `S()(...)`), we also
store `S()` in `Args` as the first argument, so all indexing is off by
one.
The most interesting part is that `S::operator()(...)` works correctly
and does not add `S()` at the beginning of the argument list, so it does
not crash during lifetime checks.
This solution is probably not the cleanest, but I would love to hear
feedback on where to put it!
</details>
Initially, the reduction root was detected using the last member of the UserIgnoreList set, which is unordered. Better to use the reduction root explicitly to avoid non-determinism in the reduction parent block, which may cause incorrect scale factor estimation for the reduction cost.
Updated LibcTest to handle Windows test executables:
* Added support for .exe extensions when identifying test executables.
* Skipped the executable bit check on Windows as it is not applicable.
* Updated .params file discovery to look for both <test>.exe.params and
<test>.params.
This allows running libc tests on Windows hosts.
Previously the optional parser would consume the token even when it
failed to match a value of the enum and prevented parsers later in the
op syntax from having an attempt. This PR changes that so that the token
is consumed only when the parsing succeeds. This change is made to the
emitted `FieldParser<std::optional<T>>` for enums.
This, for example, allows having a simple list of default valued props
in the assembly format without needing decorations around them. This
mimics the behaviour that is emitted for `DefaultValuedAttribute` when
it is used with `EnumAttr`.
This PR also adds `parseOptionalString` variant with an allow-list
argument as `parseOptionalKeyword` has and adds
`parseOptionalKeywordOrString` allow-list variant which combines these
two into a single utility wrapper. These methods do not consume the
token unless it is from the allow-list.
This patch adds the initial coexec scheduler scaffold for machine
learning workloads on gfx1250.
It introduces function and module-level controls for selecting the
AMDGPU preRA and postRA schedulers, including an `amdgpu-workload-type`
module flag that maps ML workloads to coexec preRA scheduling and a nop
postRA scheduler by default.
It also updates the coexec scheduler to use a simplified top-down
candidate selection path that considers both available and pending
queues through a single flow, setting up follow-on heuristic work.
In the builtins library, most functions have a portable C implementation
(e.g. `mulsf3.c`), and platforms might provide an optimized assembler
implementation (e.g. `arm/mulsf3.S`). The cmake script automatically
excludes the C source file corresponding to each assembly source file it
includes. Additionally, each source file name is automatically
translated into a flag that lit tests can query, with a name like
`librt_has_mulsf3`, to indicate that a function is available to be
tested.
In future commits I plan to introduce cases where a single .S file
provides more than one function (so that they can share code easily),
and therefore, must supersede more than one existing source file.
I've introduced the `crt_supersedes` cmake property, which you can set
on a .S file to name a list of .c files that it should supersede. Also,
the `crt_provides` property can be set on any source file to indicate a
list of functions it makes available for testing, in addition to the one
implied by its name.
fixes https://github.com/llvm/llvm-project/issues/129441
cc @lukel97 @badumbatish
https://github.com/llvm/llvm-project/pull/145108
I've been learning a bit about LLVM, trying to make progress on some of
these issues. The code below is based on
https://github.com/llvm/llvm-project/pull/145108#issuecomment-3004561085,
by implementing `shouldExpandReduction`.
The implementation works for the test cases I added, but (obviously)
fails for any existing cases. `ISD::VECREDUCE_AND` and
`ISD::VECREDUCE_OR` are now marked as legal, which is required for the
`Pat`s to fire, but when they don't that causes a selection failure.
So, I'm wondering, what is the right approach here. Should I mark these
intrinsics as `Custom` instead and manually perform the transformation
in C++? Or is there some trick to still get the default lowering (a
series of loads and scalar bitwise operations) when the patterns don't
fire?
Minor code cleanup: define variables at their first assignment as
opposed to at the start of functions, and use `[[maybe_unused]]` for
variables used in assert only.
