Flang currently adds loop metadata to a conditional branch in the loop
preheader, while clang adds it to the loop latch's branch instruction.
Langref says:
> Currently, loop metadata is implemented as metadata attached to the
branch instruction in the loop latch block.
>
> https://llvm.org/docs/LangRef.html#llvm-loop
I misread langref a couple times, but I think this is the appropriate
branch op for the LoopAnnotationAttr. In a couple examples I found that
the metadata was lost entirely during canonicalization. This patch makes
the codegen look more like clang's and the annotations persist through
codegen.
* current clang: https://godbolt.org/z/8WhbcrnG3
* current flang: https://godbolt.org/z/TrPboqqcn
Previously, a bug in the MemCptOpt LLVM IR pass caused issues with
adding alias tags for locally allocated objects for Fortran code.
However, the bug has now been fixed (
https://github.com/llvm/llvm-project/pull/129537 ), and we can safely
enable alias tags for these objects. This change should improve the
accuracy of the alias analysis.
This patch updates several passes to include the DLTI dialect, since
their use of the `fir::support::getOrSetMLIRDataLayout()` utility
function could, in some cases, require this dialect to be loaded in
advance.
Also, the `CUFComputeSharedMemoryOffsetsAndSize` pass has been updated
with a dependency to the GPU dialect, as its invocation to
`cuf::getOrCreateGPUModule()` would result in the same kind of error if
no other operations or attributes from that dialect were present in the
input MLIR module.
Ensure volatility is reflected not just on the reference to an
allocatable, but on the box, too. When we declare a volatile
allocatable, we now get a volatile reference to a volatile box.
Some related cleanups:
* SELECT TYPE constructs check the selector's type for volatility when
creating and designating the type used in the selecting block.
* Refine the verifier for fir.convert. In general, I think it is ok to
implicitly drop volatility in any ptr-to-int conversion because it means
we are in codegen (and representing volatility on the LLVM ops and
intrinsics) or we are calling an external function (are there any cases
I'm not thinking of?)
* An allocatable test that was XFAILed is now passing. Making
allocatables' boxes volatile resulted in accesses of those boxes being
volatile, which resolved some errors coming from the strict verifier.
* I noticed a runtime function was missing the fir.runtime attribute.
This change allows marking more designators producing an opaque
box with 'contiguous' attribute, e.g. like in test1 case
in flang/test/HLFIR/propagate-contiguous-attribute.fir.
This would make isSimplyContiguous() return true for such
designators allowing merging hlfir.eval_in_mem with hlfir.assign
where the LHS is a contiguous array section.
Depends on #139003
The `AddDebugInfo` pass currently has a dependency on the `DLTI` MLIR
dialect caused by a call to the `fir::support::getOrSetMLIRDataLayout()`
utility function.
This dependency is not captured in the pass definition. This patch adds
the dependency and simplifies several unit tests that had to explicitly
use the `DLTI` dialect to prevent the missing dependency from causing
compiler failures.
`FunctionOpInterface` assumed the fact that the function type (attribute
of the operation) can be cloned with arbirary lists of function
arguments and results to support argument and result list mutation. This
is not always correct, in particular, LLVM dialect functions require
exactly one result making it impossible to erase the result.
Allow function type cloning to fail and propagate this failure through
various APIs that use it. The common assumption is that existing IR has
not been modified.
Fixes#131142.
Reland a8c7ecdcbc3e89b493b495c6831cc93671c3b844 / #136300.
This patch produces the following TBAA tree for a function:
```
Function root
|
"any access"
|
|- "descriptor member"
|- "any data access"
|
|- "dummy arg data"
|- "target data"
|
|- "allocated data"
|- "direct data"
|- "global data"
```
The TBAA tags are assigned using the following logic:
* All POINTER variables point to the root of "target data".
* Dummy arguments without POINTER/TARGET point to their
leafs under "dummy arg data".
* Dummy arguments with TARGET point to the root of "target data".
* Global variables without descriptors point to their leafs under
"global data" (including the ones with TARGET).
* Global variables with descriptors point to their leafs under
"direct data" (including the ones with TARGET).
* Locally allocated variables point to their leafs under
"allocated data" (including the ones with TARGET).
This change makes it possible to disambiguate globals like:
```
module data
real, allocatable :: a(:)
real, allocatable, target :: b(:)
end
```
Indeed, two direct references to global vars cannot alias
even if any/both of them have TARGET attribute.
In addition, the dummy arguments without POINTER/TARGET cannot alias
any other variable even with POINTER/TARGET. This was not expressed
in TBAA before this change.
As before, any "unknown" memory references (such as with Indirect
source, as classified by FIR alias analysis) may alias with
anything, as long as they point to the root of "any access".
Please think of the counterexamples for which this structure
may not work.
This is similar to other configuration objects used across MLIR.
Rename some fields to better reflect that they are no longer booleans.
Reland 04d261101b4f229189463136a794e3e362a793af / #132253.
In CUDA Fortran the stream is encoded in an INTEGER(cuda_stream_kind)
variable.
This information is carried over the GPU dialect through the
`cuf.stream_cast` and the token in the GPU ops.
When converting the `gpu.launch_func` to runtime call, the
`cuf.stream_cast` becomes a no-op and the reference to the stream is
passed to the runtime.
The runtime is adapted to take integer references instead of value for
stream.
Update `cuf.kernel_launch` to take the stream as a reference. Update the
conversion to insert the `cuf.stream_cast` op so the stream can be set
as dependency.
When the mask is scalar, it is incorrect to cast it to
!fir.box<!fir.array<1xlogical<>>>, because the coordinate
operation will try to read the dim-1 stride from the box
to get the address of the first element. Even though
the stride value will be multiplied by 0, and does not matter,
it is still a read past the allocated box object.
Instead, we should just use box_addr to get the address
of the scalar mask.
I am making a CG pass to depend on `FIROpenACCSupport` in #134346.
This introduces a cyclic dependency between `FIROpenACCSupport`
and `FIRCodeGen`. This patch splits `FIRCodeGen` into
`FIRCodeGenDialect` (for FIR CG dialect definition) and `FIRCodeGen`
(for the CG passes).
Now, `FIROpenACCSupport` depends on `FIRCodeGenDialect`,
and `FIRCodeGen` depends on `FIROpenACCSupport`.
Allocatable or pointer module variables with the CUDA managed attribute
are defined with a double descriptor. One on the host and one on the
device. Only the data pointed to by the descriptor will be allocated in
managed memory.
Allow the registration of any allocatable or pointer module variables
like device or constant.
This change enables LoopVersioning when `fir.pack_array` is met
in the def-use chain. It fixes a couple of huge performance regressions
caused by enabling `-frepack-arrays`.
Currently only ctor/dtor list and their priorities are supported. This
PR adds support for the missing data field.
Few implementation notes:
- The assembly printer has a fixed form because previous `attr_dict`
will sort the dict by key name, making global_dtor and global_ctor
differ in the order of printed arguments.
- LLVM's `ptr null` is being converted to `#llvm.zero` otherwise we'd
have to create a region to use the default operation conversion from
`ptr null`, which is silly given that the field only support null or a
symbol.
flang/include/flang/Runtime/io-api.h was changed into io-api-consts.h,
then wrapped into a new io-api.h that includes io-api-consts.h, does
some redundant includes and declarations, and then declares the
prototype of one function, InquiryKeywordHashDecode.
Make that function static in io-stmt.cpp prior to its sole call site,
then undo the renaming, to reduce confusion and redundancy.
I am working on `-frepack-array` feature (#127147), which produces
non-trivial manipulations with arguments of `fir.declare`.
In this case, we end up with CFG computation of the `fir.declare`
argument, and AddDebugInfo pass incorrectly mapped two dummy arguments
to the same arg index in the debug attributes.
This patch makes sure that we assign the arg index only if we can prove
that we've traced the block argument to the function's entry block.
I believe this problem is not specific to `-frepack-arrays`, e.g.
it may appear due to MLIR inlining as well.
Add a pass to compute the size of the shared memory (static shared
memory) and the offsets of each variables to be placed in shared memory.
The global representing the shared memory is also created during this
pass.
In case of dynamic shared memory, the global as a type of
`!fir.array<0xi8>` and the size of the memory is set at kernel launch.
If mask is a scalar, it always converts to !fir.box<!fir.array<1xi1>>.
The wrong value may be picked up when passing to the function
on the big endian platform. This patch is to do the conversion
based on the original type of the mask and convert the value to
i1 after the load.
This change is inspired by a case in facerec benchmark, where
performance
of scalar code may improve by about 6%@aarch64 due to getting rid of
redundant
loads from Fortran descriptors. These descriptors are corresponding
to subroutine local ALLOCATABLE, SAVE variables. The scalar loop nest
in LocalMove subroutine contains call to Fortran runtime IO functions,
and LLVM globals-aa analysis cannot prove that these calls do not modify
the globalized descriptors with internal linkage.
This patch sets and propagates llvm.memory_effects attribute for
fir.call
operations calling Fortran runtime functions. In particular, it tries
to set the Other memory effect to NoModRef. The Other memory effect
includes accesses to globals and captured pointers, so we cannot set
it for functions taking Fortran descriptors with one exception
for calls where the Fortran descriptor arguments are all null.
As long as different calls to the same Fortran runtime function may have
different attributes, I decided to attach the attributes to the calls
rather than functions. Moreover, attaching the attributes to func.func
will require propagating these attributes to llvm.func, which is not
happening right now.
In addition to llvm.memory_effects, the new pass sets llvm.nosync
and llvm.nocallback attributes that may also help LLVM alias analysis
(e.g. see #127707). These attributes are ignored currently.
I will support them in LLVM IR dialect in a separate patch.
I also added another pass for developers to be able to print
declarations/calls of all Fortran runtime functions that are recognized
by the attributes setting pass. It should help with maintenance
of the LIT tests.
This change is inspired by a case in facerec benchmark, where
performance
of scalar code may improve by about 6%@aarch64 due to getting rid of
redundant
loads from Fortran descriptors. These descriptors are corresponding
to subroutine local ALLOCATABLE, SAVE variables. The scalar loop nest
in LocalMove subroutine contains call to Fortran runtime IO functions,
and LLVM globals-aa analysis cannot prove that these calls do not modify
the globalized descriptors with internal linkage.
This patch sets and propagates llvm.memory_effects attribute for
fir.call
operations calling Fortran runtime functions. In particular, it tries
to set the Other memory effect to NoModRef. The Other memory effect
includes accesses to globals and captured pointers, so we cannot set
it for functions taking Fortran descriptors with one exception
for calls where the Fortran descriptor arguments are all null.
As long as different calls to the same Fortran runtime function may have
different attributes, I decided to attach the attributes to the calls
rather than functions. Moreover, attaching the attributes to func.func
will require propagating these attributes to llvm.func, which is not
happening right now.
In addition to llvm.memory_effects, the new pass sets llvm.nosync
and llvm.nocallback attributes that may also help LLVM alias analysis
(e.g. see #127707). These attributes are ignored currently.
I will support them in LLVM IR dialect in a separate patch.
I also added another pass for developers to be able to print
declarations/calls of all Fortran runtime functions that are recognized
by the attributes setting pass. It should help with maintenance
of the LIT tests.
Currently when we do a CUDA data transfer from a constant, we embox it
and delegate the assignment to the runtime. When the type of the
constant is not exactly the same as the destination descriptor, the
runtime will emit an assignment mismatch error.
Convert the constant when necessary so the assignment is fine.
