Change the separator in the `uniqueCGIdent` method to `X`. This change
is required to enable OpenMP offloading for the NVPTX target, as dots
are not valid identifiers in PTX and `uniqueCGIdent` is used to mangle
some literals. Follow up patches will change the remainder of `.`
appearances in names to `X` and add support for the NVPTX target.
This patch removes the OMPEarlyOutliningPass as it is no longer required. The implicit map operand capture has now been moved to the PFT lowering stage.
Depends on #67318.
The stack arrays pass uses data flow analysis to determine whether heap
allocations are freed on all paths out of the function.
`interp_domain_em_part2` in spec2017 wrf generates over 120k operations,
including almost 5k fir.if operations and over 200 fir.do_loop
operations, all in the same function. The MLIR data flow analysis
framework cannot provide reasonable performance for such cases because
there is a combinatorial explosion in the number of control flow paths
through the function, all of which must be checked to determine if the
heap allocations will be freed.
This patch skips the stack arrays pass for ridiculously large functions
(defined as having more than 1000 fir.allocmem operations). This
threshold is configurable at runtime with a command line argument.
With this patch, compiling this file is more than 80% faster.
Propagate fast math flags through complex number lowering (when lowering
fir.*c directly to llvm floating point operations).
The lowering path through the MLIR complex dialect is unchanged.
This leads to a small improvement in spec2017 fotonik3d_r.
The front-end is making implicit conversions explicit in assignment and
structure constructors.
While this generally helps and is needed by semantics to fold structure
constructors correctly, this is incorrect when the LHS or component is
an allocatable. The RHS may have non default lower bounds that should be
propagated to the LHS, and making the conversion explicit changes the
semantics. In the structure constructor, the situation is even worse
since Fortran 2018 7.5.10 point 7 allows the value to be a reference to
an unallocated allocatable, and adding an explicit conversion in
semantics will cause a segfault.
This patch removes the explicit convert in semantics when the
LHS/component is a whole allocatable, and update lowering to deal with
the conversion insertion, dealing with preserving the lower bounds and
the tricky structure constructor case.
These turn out to be useful for spec2017/fotonik3d and safe so long as
they are not used along side TBAA tags for local allocations. LLVM may
be able to figure out local allocations by itself anyway.
PR #68727
!llvm.ptr<T> typed pointers are depreciated in MLIR LLVM dialects. Flang
codegen still generated them and relied on mlir.llvm codegen to LLVM to
turn them into opaque pointers.
This patch update FIR codegen to directly emit and work with LLVM opaque
pointers.
Addresses https://github.com/llvm/llvm-project/issues/69303
- All places generating GEPs need to add an extra type argument with the
base type (the T that was previously in the llvm.ptr<T> of the base).
- llvm.alloca must also be provided the object type. In the process, I
doscovered that we were shamelessly copying all the attribute from
fir.alloca to the llvm.alloca, which makes no sense for the operand
segments. The updated code that cannot take an attribute dictionnary in
the llvm.alloca builder with opaque pointers only propagate the "pinned"
and "bindc_name" attributes to help debugging the generated IR.
- Updating all the places that rely on getting the llvm object type from
lowered llvm.ptr<T> arguments to get it from a type conversion of the
original fir types.
- Updating all the places that were generating llvm.ptr<T> types to
generate the opaque llvm.ptr type.
- Updating all the codegen tests checking generated MLIR llvm dialect.
Many tests are testing directly LLVM IR, and this change is a no-op for
those (which is expected).
Expose a `MutableArrayRef<OpOperand>` instead of
`ValueRange`/`OperandRange`. This allows users of this interface to
change the yielded values and the init values. The names of the
interface methods are the same as the auto-generated op accessor names
(`get...()` returns `OperandRange`, `get...Mutable()` returns
`MutableOperandRange`).
Note: The interface methods return a `MutableArrayRef` instead of a
`MutableOperandRange` because a loop op may not implement
`getYieldedValuesMutable` etc. and there is no safe way to return an
"empty" range with a `MutableOperandRange`.
This patch makes changes to the early outlining pass to avoid compiler
crashes due to not handling `hlfir.declare` operations correctly. That
pass is intended to eventually be removed (#67319), but in the meantime
this fixes some issues arising in different parts of the OpenMP
offloading compilation process.
The main changes included in this patch are the following:
- Added support for mapped values defined by an `hlfir.declare`
operation. These operations are now kept in outlined target functions,
so that both of their outputs (base and original base) are available to
the corresponding `omp.target`'s map arguments and region.
- Added a fix by @agozillon to prevent unused map clauses from producing
a compiler crash. All these unused mapped variables are added to the
outlined function's inputs.
- Added a fix to the OpenMP translation to MLIR to support integer
arguments to these outlined functions. This enables successfully
compiling and running the tests in
opemp/libomptarget/test/offloading/fortran using HLFIR.
Co-authored-by: agozillon <Andrew.Gozillon@amd.com>
Add a new interface method that returns the yielded values.
Also add a verifier that checks the number of inits/iter_args/yielded
values. Most of the checked invariants (but not all of them) are already
covered by the `RegionBranchOpInterface`, but the `LoopLikeOpInterface`
now provides (additional) error messages that are easier to read.
This avoids trying to version loops that can't be versioned, and thus
avoids hitting an assert.
Co-authored with Slava Zakharin (who provided the test-code).
The ultimate intention is to have this pass enabled by default whenever
we are optimizing for speed. But for now, just add the arguments so this
can be more easily tested.
PR: https://github.com/llvm/llvm-project/pull/68595
This is important to ensure that tags end up in the same trees that were
created in the FIR TBAA pass. If they are in different trees then
everything in one tree will be assumed to MayAlias with everything in the
other tree. This leads to poor performance.
@vzakhari requested that the old (not-per-function) trees are
maintained so I left the old test intact.
PR: https://github.com/llvm/llvm-project/pull/68437
See RFC at
https://discourse.llvm.org/t/rfc-propagate-fir-alias-analysis-information-using-tbaa/73755
This pass adds TBAA tags to all accesses to non-pointer/target dummy
arguments. These TBAA tags tell LLVM that these accesses cannot alias:
allowing better dead code elimination, hoisting out of loops, and
vectorization.
Each function has its own TBAA tree so that accesses between funtions
MayAlias after inlining.
I also included code for adding tags for local allocations and for
global variables. Enabling all three kinds of tag is known to produce a
miscompile and so these are disabled by default. But it isn't much
code and I thought it could be interesting to play with these later if
one is looking at a benchmark which looks like it would benefit from
more alias information. I'm open to removing this code too.
TBAA tags are also added separately by TBAABuilder during CodeGen.
TBAABuilder has to run during CodeGen because it adds tags to box
accesses, many of which are implicit in FIR. This pass cannot (easily)
run in CodeGen because fir::AliasAnalysis has difficulty tracing values
between blocks, and by the time CodeGen runs, structured control flow
has already been lowered.
Coming in follow up patches
- Change CodeGen/TBAABuilder to use TBAAForest to add tags within the
same per-function trees as are used here (delayed to a later patch
to make it easier to revert)
- Command line argument processing to actually enable the pass
This interface allows (HL)FIR passes to add TBAA information to fir.load
and fir.store. If present, these TBAA tags take precedence over those
added during CodeGen.
We can't reuse mlir::LLVMIR::AliasAnalysisOpInterface because that uses
the mlir::LLVMIR namespace so it tries to define methods for fir
operations in the wrong namespace. But I did re-use the tbaa tag type to
minimise boilerplate code.
The new builders are to preserve the old interface without the tbaa tag.
Conversion of `hlfir.assign` operations inside OpenACC recipe operations
may result in `fir.alloca` insertion. FIRBuilder can only handle
alloca insertion inside FuncOp's and outlineable OpenMP operations.
I added a simple interface for OpenACC recipe operations that have
executable code inside all their regions, and alloca may be inserted
into the entry blocks of those regions always.
With our current approach the OptimizedBufferization pass is supposed
to lower these `hlfir.assign` operations into loops, because there
should not be conflicts between lhs/rhs. The pass is currently
only working on FuncOp, and this is why it does not optimize
`hlfir.assign` inside the recipes. I will fix it in a separate commit.
Since we run OptimizedBufferization only at >O0, these changes
should still be useful.
Note that the OpenACC codegen that applies the recipes should be aware
of potential alloca operations and produce appropriate stack clean-ups.
HLFIR was always calling Destroy runtime when doing derived type scalar
assignments because the IR did not contain the info of whether
finalization was needed or not.
This info is now available in fir.type_info which allow skipping the
runtime call when not needed.
Also, when finalization is needed, simply use Assign runtime. This makes
no difference from a semantic point of view with the current code that
generated a call to Destroy and did the assignment inline, but if some
piece of runtime must be called anyway, it is simpler to just call
Assign that deals with everything.
The goal is to progressively propagate all the derived type info that is
currently in the runtime type info globals into a FIR operation that can
be easily queried and used by FIR/HLFIR passes.
When this will be complete, the last step will be to stop generating the
runtime info global in lowering, but to do that later in or just before
codegen to keep the FIR files readable (on the added type-info.f90
tests, the lowered runtime info globals takes a whooping 2.6 millions
characters on 1600 lines of the FIR textual output. The fir.type_info that
contains all the info required to generate those globals for such
"trivial" types takes 1721 characters on 9 lines).
So far this patch simply starts by replacing the fir.dispatch_table
operation by the fir.type_info operation and to add the noinit/
nofinal/nodestroy flags to it. These flags will soon be used in HLFIR to
better rewrite hlfir.assign with derived types.
Add vscale range attirbute for the Scalable Vector Extension (SVE) if
provided on the command-line (options in a previous commit)
If no command-line option is provided, if the target-feature of SVE is
specified and the architecture is AArch64, it defualts to 128-2048. in
other words a vscale-min of 1, vscale-max of 16.
A pass is used to add the atribute to all functions. The vectorizer will
use this attribute to generate the SVE instruction to match the range
specified. The attribute is harmless if there is no vectorizable
operations in the function.
The representation produced by `getTypeAsString` is used by reduction,
privatization and firstprivatization recipes. The name of the recipe is
used as a symbol attribute. If the name has special symbol the symbol
becomes quoted. This patch remove the special character from the
representation so the representation is homogenous.
Since HLFIR bufferization can introduce shallow copies of derived
type values we have to be careful not to treat these load/store
operations as data-only-accesses. If a derived type has descriptor
members, we attach any-access tag now.
To avoid the overhead of deallocating allocatable components of the
elemental temporary result on every iteration of the elemental operation,
we can use a shallow copy instead of deep-copy assign.
Fixes https://github.com/llvm/llvm-project/issues/67658
The bug was that when instantiating a character array result variable,
the code inserted a cast from the result buffer to the proper array type
if it could see an fir.unboxchar op. But this is wrong for results and
on caller side because the fir.emboxchar is visible so,
charHelper.genUnboxChar() just takes the operand from that instead of
generating an unboxchar.
The fix is simply to move the cast at the place where fir.boxchar<>
argument are dealt with. The cast when creating fir.emboxchar is also
removed: it adds noise and causes constant length result type to be
lowered to fir.char<?>.
The main change from this patch is to deal with the lit test fallout of
this cast move and removal.
This is not standard but is vastly expected by existing code.
This was implemented by https://reviews.llvm.org/D149877 for simple
scalars, but MLIR lacked a generic way to deal with aggregate types
(arrays and derived type).
Support was recently added in
https://github.com/llvm/llvm-project/pull/65508. Leverage it to zero
initialize all types.
Assumed shape array are using descriptor and must be handled differently
than known shape arrays. This patch adds support to generate the `init`
and `combiner` region for the reduction recipe operation with assumed
shape array by using the descriptor and the HLFIR lowering path.
`createTempFromMold` function is moved from
`flang/lib/Optimizer/HLFIR/Transforms/BufferizeHLFIR.cpp` to
`flang/include/flang/Optimizer/Builder/HLFIRTools.h` to be reused to
create the private copy.
Semantics was replacing storage_size(func()) by the length specification
expression of func result (if any), which brought meaningless symbols.
Update FunctionRef::GetType to not copy its length parameter from the
procedure designator symbol if it is not a constant expression. Note
that the deferred aspect can and must be preserved because it matters
for POINTER function results (semantics test added to ensure this).
Update lowering code to deal with characters in storage_size: simply
always call createBox to ensure the BoxEleSizeOp is legal. This will
take care of dereferencing pointers/allocatables if needed (what the
load was intended for in the previous code).
PowerPC Vector type intrinsics currently crashes with
`-flang-experimental-hlfir` is specified.
This patch is to fix the HLFIR path for PowerPC Vector type intrinsics.
The patch:
1. Added the `flang-experimental-hlfir` option to all PowerPC vector intrinsic testing.
2. Removed the FIR/MLIR testing to reduce the maintenance cost.
3. Fixed a few verification IR for some non-functional changes in LLVM IR in HLFIR path.
The changes are needed to get leslie3d same performance with HLFIR
as with FIR lowering. The two module allocatable variables cannot
alias, so the optimized bufferization should be able to elide
the temporary and inline the assignment loop.
This revision replaces the LLVM dialect NullOp by the recently
introduced ZeroOp. The ZeroOp is more generic in the sense that it
represents zero values of any LLVM type rather than null pointers only.
This is a follow to https://github.com/llvm/llvm-project/pull/65508
This set of commits resolves some of the issues with elemental calls producing
results that may require finalization, and also some memory leak issues due to
the missing deallocation of allocatable components of the temporary buffers
created by the bufferization pass.
- [flang][runtime] Expose Finalize API for derived types.
- [flang][hlfir] Add 'finalize' attribute for DestroyOp.
- [flang][hlfir] Postpone result finalization for elemental calls.
The results of elemental calls generated inside hlfir.elemental must not
be finalized/destructed before they are copied into the resulting
array. The finalization must be done on the array as a whole
(e.g. there might be different scalar and array finalization routines).
The finalization work is left to the hlfir.destroy corresponding
to this hlfir.elemental.
- [flang][hlfir] Tighten requirements on hlfir.end_associate operand.
If component deallocation might be required for the operand of
hlfir.end_associate, we have to be able to get the variable
shape/params to create a descriptor for calling the runtime.
This commit adds verification that we can do so.
- [flang][hlfir] Lower argument clean-ups using valid hlfir.end_associate.
The operand must be a Fortran entity, when allocatable component
deallocation may be required.
- [flang][hlfir] Properly clean-up temporary buffers in bufferization pass.
This commit combines changes for proper finalization and component
deallocation of the temporary buffers. The finalization part
relates to hlfir.destroy operations with 'finalize' attribute.
The component deallocation might be invoked for both hlfir.destroy
and hlfir.end_associate, if the operand is of a derived type
with allocatable component(s).
The changes are mostly in one function, so I decided not to split them.
- [flang][hlfir] Disable optimizations for hlfir.elemental requiring finalization.
If hlfir.elemental is coupled with hlfir.destroy with 'finalize' attribute,
the temporary array result of hlfir.elemental needs to be created
for the purpose of finalization. We cannot do certain optimizations
on such hlfir.elemental operations.
I was not able to come up with a test for the OptimizedBufferization pass,
but I put the check there as well.
There are currently several places that automatically deallocate
allocatble if they are allocated:
- INTENT(OUT) allocatable are deallocated on entry in the callee
- INTENT(OUT) allocatable are also deallocated on the caller side of
BIND(C) function in case the implementation is in C.
- Results of function returning allocatable are deallocated after usage.
- OPENMP privatized allocatable are deallocated at the end of OPENMP
region.
Introduce genDeallocateIfAllocated that centralize all this code, except
for the function return that use genFreememIfAllocated since
finalization is done separately currently.
`fir:🏭:genFinalization` and
`fir:🏭:genInlinedDeallocation` are removed and replaced by
genFreemem since their name were misleading: finalization was not
called.
There is a fallout in the tests because previous generated code did not
check the allocated status when doing inline deallocation. This was OK
since free(null) is guaranteed to be a no-op, but this makes compiler
code more complex, is a bit surprising in the generated IR IMHO, and it
relied on knowing when genDeallocateBox inserts runtime calls or uses
inlined code.
This patch fixes two issues introduced by the D149368 patch, one is
a memory leak from using the removeFromParent rather
than eraseFromParent (the erase also had to be moved to not create
use after deletes).
And the other is a possible iterator invalidation bug, better to be safe
than sorry.
This patch adds initial lowering for DeclareTargetAttr on
GlobalOp's utilising registerTargetGlobalVariable
and getAddrOfDeclareTargetVar from the
OMPIRBuilder.
It also adds initial processing of declare target map
operands, populating the combinedInfo that the
OMPIRBuilder requires to generate kernels and
it's kernel argument structure.
The combination of these additions allows simple mapping
of declare target globals to Target regions, as such a simple
runtime test showcasing this and testing it has been added.
The patch currently does not factor in filtering
based on device_type clauses (e.g. no emission of
globals for device if host specified), this will come in
a future iteration. And for the moment it's only been
tested with 1-D arrays and basic fortran data types,
more complex types (such as user defined derived
types from Fortran, allocatables or Fortran pointers)
may need further work.
reviewers: kiranchandramohan, skatrak
Differential Revision: https://reviews.llvm.org/D149368
Defining a procedure with a BIND(C, NAME="...") where the binding label
matches the assembly name of a non BIND(C) external procedure in the
same file causes a failure when generating the LLVM IR because of the
assembly symbol name clash.
Prevent this crash with a clearer semantic error.
This reverts commit 775754e32840c6b6ca64c8bc0b7ae2c778b97d1e.
Relanding with removing part of the LIT test. There seems to be operations
ordering indeterminism that is unrelated to my change. I will address this
issue separately.
Add support for fir.box_addr, fir.array_corr, fir.coordinate, fir.embox,
fir.rebox and fir.load.
1) Through the use of boolean `followBoxAddr` determine whether the
analysis should apply to the address of the box or the address wrapped
by the box.
2) Some asserts have been removed to allow for more SourceKinds though
the flow, in a particular SourceKind::Direct
3) getSource was a public method but the returned type (SourceKind) was
not public making it impossible to be called publicly
4) About 12 tests have been added to check for real Fortran scenarios
5) More tests will be added with HLFIR
6) A few TODOs have been identified and will need to be addressed in
follow-up patches. I felt that more changes would increase the
complexity of the patch.
This patch places the finalization code for the RHS of a user-defined
assignment after the assignment code. The change only affects
standalone RegionAssignOp operations.
This commit implements `LoopLikeOpInterface` on `scf.while`. This
enables LICM (and potentially other transforms) on `scf.while`.
`LoopLikeOpInterface::getLoopBody()` is renamed to `getLoopRegions` and
can now return multiple regions.
Also fix a bug in the default implementation of
`LoopLikeOpInterface::isDefinedOutsideOfLoop()`, which returned "false"
for some values that are defined outside of the loop (in a nested op, in
such a way that the value does not dominate the loop). This interface is
currently only used for LICM and there is no way to trigger this bug, so
no test is added.
This fixes a bug where functions generated by the MLIR Math dialect, for
example ipowi, would fail to link with link.exe on Windows due to having
linkonce linkage but no associated comdat. Adding the comdat on ELF also
allows linkers to perform better garbage collection in the binary.
Simply adding comdats to all functions with this linkage type should
also cover future cases where linkonce or linkonce_odr functions might
be necessary.
This patch is a required change for the device side IR to
maintain apporpiate links for declare target variables to
their global variables for later lowering.
It is also a requirement to clone over map bounds and
entry operations to maintain the correct information for
later lowering of the IR.
It simply tries to clone over the relevant information
maintaining the appropriate links they would have
maintained prior to the pass, rather than redirecting
them to new function arguments which causes a
loss of information in the case of Declare Target
and map information.
Depends on D158734
reviewers: TIFitis, razvanlupusoru
Differential Revision: https://reviews.llvm.org/D158735
