This is a new attempt at #69320.
The transform dialect stores a "library module" that the preload pass
can populate. Until now, each pass registered an additional module by
simply pushing it to a vector; however, the interpreter only used the
first of them. This commit turns the registration into "loading", i.e.,
each newly added module gets merged into the existing one. This allows
the loading to be split into several passes, and using the library in
the interpreter now takes all of them into account. While this design
avoids repeated merging every time the library is accessed, it requires
that the implementation of merging modules lives in the
TransformDialect target (since it at the dialect depend on each
other).
This resolves https://github.com/llvm/llvm-project/issues/69111.
This adds a flag to the `TransformDialectInterpreter` that relaxes the
requirement for only a single top-level transform op.
This is useful for supporting transforms that take transform IR as
payload.
This also aligns the function `findTopLevelTransform`
[here](7b0f4c9db5 (diff-551f92bb609487ccf981daf9571f0f1b1703ab2330560a388a5f0d133e520be4L59))
with its documentation:
In the presence of multiple top-level transform ops it now correctly
returns the first of them after reporting the error instead of returning
a `nullptr`.
This revision provides the ability to use an arbitrary named sequence op
as
the entry point to a transform dialect strategy.
It is also a step towards better transform dialect usage in pass
pipelines
that need to preload a transform library rather thanparse it on the fly.
The interpreter itself is significantly simpler than its testing
counterpart
by avoiding payload/debug root tags and multiple shared modules.
In the process, the NamedSequenceOp::apply function is adapted to allow
it
being an entry point.
NamedSequenceOp is **not** extended to take the PossibleTopLevelTrait at
this
time, because the implementation of the trait is specific to allowing
one
top-level dangling op with a region such as SequenceOp or
AlternativesOp.
In particular, the verifier of PossibleTopLevelTrait does not allow for
an
empty body, which is necessary to declare a NamedSequenceOp that gets
linked
in separately before application.
In the future, we should dispense with the PossibleTopLevelTrait
altogether
and always enter the interpreter with a NamedSequenceOp.
Lastly, relevant TD linking utilities are moved to
TransformInterpreterUtils
and reused from there.
The transfrom interpreter accepts an argument to a "library" file with
named sequences. This patch exteneds this functionality such that (1)
several such individual files are accepted and (2) folders can be passed
in, in which all `*.mlir` files are loaded.
Until now, the interpreter would only load those symbols from the
provided library files that were declared in the main transform module.
However, sequences in the library may include other sequences on their
own. Until now, if such sequences were not *also* declared in the main
transform module, the interpreter would fail to resolve them. Forward
declaring all of them is undesirable as it defeats the purpose of
encapsulation into library modules.
This PR implements a kind of linker for transform scripts to solve this
problem. The linker merges all symbols of the library module into the
main module before interpreting the latter. Symbols whose names collide
are handled as follows: (1) if they are both functions (in the sense of
`FunctionOpInterface`) with compatible signatures, one is external, and
the other one is public, then they are merged; (2) of one of them is
private, that one is renamed; and (3) an error is raised otherwise.
One consequence of this change is that the loading of the library files
in the interpreter pass is not idempotent anymore, i.e., subsequent
interpreter passes cannot (and need not) load the same library files again
since would lead to doubly defined symbols.
This commit renames the arguments of several static implementation
functions of the transform interpreter base class to match the names of
the corresponding member variables in order to clarify their intent.
Similarly, it renames some local variables to reflect their relationship
with corresponding member variables. Finally, this commit also asserts
in `interpreterBaseRunOnOperationImpl` that at most one of shared and
library module are set (which the initialization function guarantees)
and simplifies some related `if` conditions.
Functions are always callable operations and thus every operation
implementing the `FunctionOpInterface` also implements the
`CallableOpInterface`. The only exception was the FuncOp in the toy
example. To make implementation of the `FunctionOpInterface` easier,
this commit lets `FunctionOpInterface` inherit from
`CallableOpInterface` and merges some of their methods. More precisely,
the `CallableOpInterface` has methods to get the argument and result
attributes and a method to get the result types of the callable region.
These methods are always implemented the same way as their analogues in
`FunctionOpInterface` and thus this commit moves all the argument and
result attribute handling methods to the callable interface as well as
the methods to get the argument and result types. The
`FuntionOpInterface` then does not have to declare them as well, but
just inherits them from the `CallableOpInterface`.
Adding the inheritance relation also required to move the
`FunctionOpInterface` from the IR directory to the Interfaces directory
since IR should not depend on Interfaces.
Reviewed By: jpienaar, springerm
Differential Revision: https://reviews.llvm.org/D157988
Add a TransformInterpreterPassBase capability to generate the (shared)
module containing the transform script during the pass initialization.
This is helpful to programmatically generate the script as opposed to
parsing it from the textual module.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D152185
The pass attaches attributes to operations for repro generation
purposes, but never removes them. This is not desirable when the pass
actually succeeds.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D151791
Use the argument attribute mechanism for function-like operations to
annotate the arguments of named transform sequences as consuming or only
reading the handles passed as arguments. This makes it possible to
correctly specify handle invalidation for external named sequences by
requiring their declarations to always provide such annotations.
Additionally, these annotations remove the need to analyze the body of
a named sequence to understand its effects on the arguments. Make them
required for named sequences that are called from the same file, in
addition to external sequences.
Provide a convenience pass that infers annotations by analyzing bodies
of named sequences provided they are not called from the same file.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D147223
Introduce support for external definitions of named sequences in the
transform dialect by letting the TransformInterpreterPassBase read a
"library" MLIR file. This file is expected to contain definitions for
named sequences that are only declared in the main transformation
script. This allows for sharing non-trivial transform combinations
without duplication.
This patch provides only the minimal plumbing for a single textual IR
file. Further changes are possible to support multiple libraries and
bytecode files.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D146961
Named sequences introduce an additional abstraction and reuse capability
to the transform dialect. They can be though of as macros parameterized
with handles that can be invoked in places where a transform dialect
operation is expected. Such reuse was previously not possible in the
dialect and required dynamic construction of the transform IR from the
client language. Named sequences are intentionally restricted to
disallow recursion, as it could make the dialect accidentally
Turing-complete, which isn't desired at this point.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D146433
Introduce support for the third kind of values in the transform dialect:
value handles. Similarly to operation handles, value handles are
pointing to a set of values in the payload IR. This enables
transformation to be targeted at specific values, such as individual
results of a multi-result payload operation without indirecting through
the producing op or block arguments that previously could not be easily
addressed. This is expected to support a broad class of memory-oriented
transformations such as selective bufferization, buffer assignment, and
memory transfer management.
Value handles are functionally similar to operation handles and require
similar implementation logic. The most important change concerns the
handle invalidation mechanism where operation and value handles can
affect each other.
This patch includes two cleanups that make it easier to introduce value
handles:
- `RaggedArray` structure that encapsulates the SmallVector of
ArrayRef backed by flat SmallVector logic, frequently used in the
transform interfaces implementation;
- rewrite the tests that associated payload handles with an integer
value `reinterpret_cast`ed as a pointer, which were a frequent
source of confusion and crashes when adding more debugging
facilities that can inspect the payload.
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D143385
The transform dialect infrastructure does not provide a default
interpreter pass and instead expects users to create their own to ensure
all relevant extensions and dependent dialects are loaded. Provide a
base class for implementing such passes that includes the additional
facilities for debugging and is aware of the multithreaded nature of
pass execution.
Reviewed By: pifon2a, nicolasvasilache
Differential Revision: https://reviews.llvm.org/D142729
