138 Commits

Author SHA1 Message Date
Nicolas Vasilache
bb69de3f41 [mlir][Linalg] Add a vectorization pattern for linalg::PadTensorOp
The new pattern is exercised from the TestLinalgTransforms pass.

Differential Revision: https://reviews.llvm.org/D96410
2021-02-10 14:13:49 +00:00
Uday Bondhugula
fdfd647837 [MLIR] NFC Fix vector transforms build warnings
Fix build warnings from VectorTransforms.cpp.
2021-02-10 10:42:56 +05:30
Tres Popp
c2c83e97c3 Revert "Revert "Reorder MLIRContext location in BuiltinAttributes.h""
This reverts commit 511dd4f4383b1c2873beac4dbea2df302f1f9d0c along with
a couple fixes.

Original message:
Now the context is the first, rather than the last input.

This better matches the rest of the infrastructure and makes
it easier to move these types to being declaratively specified.

Phabricator: https://reviews.llvm.org/D96111
2021-02-08 10:39:58 +01:00
Tres Popp
511dd4f438 Revert "Reorder MLIRContext location in BuiltinAttributes.h"
This reverts commit 7827753f9810e846fb702f3e8dcff0bfb37344e1.
2021-02-08 09:32:42 +01:00
Tres Popp
7827753f98 Reorder MLIRContext location in BuiltinAttributes.h
Now the context is the first, rather than the last input.

This better matches the rest of the infrastructure and makes
it easier to move these types to being declaratively specified.

Differential Revision: https://reviews.llvm.org/D96111
2021-02-08 09:28:09 +01:00
Lei Zhang
7630520ae3 [mlir][vector] Add pattern to shuffle bitcast ops
These patterns move vector.bitcast ops to be before
insert ops or after extract ops where suitable.
With them, bitcast will happen on smaller vectors
and there are more chances to share extract/insert
ops.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D96040
2021-02-05 17:52:49 -05:00
Lei Zhang
8dae90997a [mlir][vector] Add constant folding for fp16 to fp32 bitcast
Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D96041
2021-02-05 09:12:50 -05:00
Lei Zhang
874ce9b80f [mlir][vector] Add patterns to cast away leading 1-dim
This patch adds patterns to use vector.shape_cast to cast
away leading 1-dimensions from a few vector operations.
It allows exposing more canonical forms of vector.transfer_read,
vector.transfer_write, vector_extract_strided_slice, and
vector.insert_strided_slice. With this, we can have more
opportunity to cancelling extract/insert ops or forwarding
write/read ops.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D95873
2021-02-05 09:02:15 -05:00
Nicolas Vasilache
05d5125d8a [mlir] Generalize OpFoldResult usage in ops with offsets, sizes and operands.
This revision starts evolving the APIs to manipulate ops with offsets, sizes and operands towards a ValueOrAttr abstraction that is already used in folding under the name OpFoldResult.

The objective, in the future, is to allow such manipulations all the way to the level of ODS to avoid all the genuflexions involved in distinguishing between values and attributes for generic constant foldings.

Once this evolution is accepted, the next step will be a mechanical OpFoldResult -> ValueOrAttr.

Differential Revision: https://reviews.llvm.org/D95310
2021-01-25 14:17:03 +00:00
Aart Bik
046612d29d [mlir][vector] verify memref of vector memory ops
This ensures the memref base + indices expression is well-formed

Reviewed By: ThomasRaoux, ftynse

Differential Revision: https://reviews.llvm.org/D94441
2021-01-11 13:32:39 -08:00
Thomas Raoux
3d693bd0bd [mlir][vector] Add memory effects to transfer_read transfer_write ops
This allow more accurate modeling of the side effects and allow dead code
elimination to remove dead transfer ops.

Differential Revision: https://reviews.llvm.org/D94318
2021-01-11 09:25:37 -08:00
Aart Bik
6728af16cf [mlir][vector] modified scatter/gather syntax, pass_thru mandatory
This change makes the scatter/gather syntax more consistent with
the syntax of all the other memory operations in the Vector dialect
(order of types, use of [] for index, etc.). This will make the MLIR
code easier to read. In addition, the pass_thru parameter of the
gather has been made mandatory (there is very little benefit in
using the implicit "undefined" values).

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D94352
2021-01-09 11:41:37 -08:00
Aart Bik
a57def30f5 [mlir][vector] generalized masked l/s and compressed l/s with indices
Adding the ability to index the base address brings these operations closer
to the transfer read and write semantics (with lowering advantages), ensures
more consistent use in vector MLIR code (easier to read), and reduces the
amount of code duplication to lower memrefs into base addresses considerably
(making codegen less error-prone).

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D94278
2021-01-08 13:59:34 -08:00
Kazuaki Ishizaki
f88fab5006 [mlir] NFC: fix trivial typos
fix typo under include and lib directories

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D94220
2021-01-08 02:10:12 +09:00
Thomas Raoux
080943f752 [mlir][vector] Support transfer op on tensor optimizations
Support store to load forwarding and dead store transformations for transfer op
on tensor.

Differential Revision: https://reviews.llvm.org/D94148
2021-01-06 15:09:03 -08:00
Thomas Raoux
efd05040e1 [mlir] Add hoisting transformation for transfer ops on tensor
Add same hoisting transformation existing for transfer ops on buffers for
transfer_ops on tensor. The logic is significantly different so this is done as
a separate transformation and it is expect that user would know which
transformation to use based on the flow.

Differential Revision: https://reviews.llvm.org/D94115
2021-01-06 14:23:59 -08:00
Thomas Raoux
f9190c8681 [mlir][vector] Support unrolling for transfer ops using tensors
Differential Revision: https://reviews.llvm.org/D93904
2021-01-06 13:28:04 -08:00
Thomas Raoux
cf216670a0 [mlir][linalg] Add vectorization for linalg on tensor ops
Support vectorization of linalg ops using tensor inputs/outputs.

Differential Revision: https://reviews.llvm.org/D93890
2020-12-29 09:02:23 -08:00
Chris Lattner
9eb3e564d3 [ODS] Make the getType() method on a OneResult instruction return a specific type.
Implement Bug 46698, making ODS synthesize a getType() method that returns a
specific C++ class for OneResult methods where we know that class.  This eliminates
a common source of casts in things like:

   myOp.getType().cast<FIRRTLType>().getPassive()

because we know that myOp always returns a FIRRTLType.  This also encourages
op authors to type their results more tightly (which is also good for
verification).

I chose to implement this by splitting the OneResult trait into itself plus a
OneTypedResult trait, given that many things are using `hasTrait<OneResult>`
to conditionalize various logic.

While this changes makes many many ops get more specific getType() results, it
is generally drop-in compatible with the previous behavior because 'x.cast<T>()'
is allowed when x is already known to be a T.  The one exception to this is that
we need declarations of the types used by ops, which is why a couple headers
needed additional #includes.

I updated a few things in tree to remove the now-redundant `.cast<>`'s, but there
are probably many more than can be removed.

Differential Revision: https://reviews.llvm.org/D93790
2020-12-26 13:52:40 -08:00
Thomas Raoux
74186880ba [mlir][vector] Add more vector Ops canonicalization
Add canonicalization for BroadcastOp, ExtractStrideSlicesOp and ShapeCastOp

Differential Revision: https://reviews.llvm.org/D93120
2020-12-23 11:25:01 -08:00
Thomas Raoux
7c7b55b985 [mlir][vector] Extend vector unroll to all element-wise ops
Extend unroll to support all element-wise ops and allow unrolling for ops with
vector operands of with the same shape as the destination but different element
type (like Cmp or Select).

Differential Revision: https://reviews.llvm.org/D93121
2020-12-21 13:31:22 -08:00
Thomas Raoux
26c8f9081b [mlir[[vector] Extend Transfer read/write ops to support tensor types.
Transfer_ops can now work on both buffers and tensor. Right now, lowering of
the tensor case is not supported yet.

Differential Revision: https://reviews.llvm.org/D93500
2020-12-21 08:55:04 -08:00
River Riddle
1b97cdf885 [mlir][IR][NFC] Move context/location parameters of builtin Type::get methods to the start of the parameter list
This better matches the rest of the infrastructure, is much simpler, and makes it easier to move these types to being declaratively specified.

Differential Revision: https://reviews.llvm.org/D93432
2020-12-17 13:01:36 -08:00
Christian Sigg
1ffc1aaa09 [mlir] Use mlir::OpState::operator->() to get to methods of mlir::Operation.
This is a preparation step to remove those methods from OpState.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D93098
2020-12-13 09:58:16 +01:00
Christian Sigg
0bf4a82a5a [mlir] Use mlir::OpState::operator->() to get to methods of mlir::Operation. This is a preparation step to remove the corresponding methods from OpState.
Reviewed By: silvas, rriddle

Differential Revision: https://reviews.llvm.org/D92878
2020-12-09 12:11:32 +01:00
River Riddle
e66c2e259f [mlir][NFC] Remove Function.h and Module.h in favor of BuiltinOps.h
The definitions of ModuleOp and FuncOp are now within BuiltinOps.h, making the individual files obsolete.

Differential Revision: https://reviews.llvm.org/D92622
2020-12-03 18:02:10 -08:00
Max Kudryavtsev
636db7f87c [MLIR] Fix vector::TransferWriteOp builder losing permutation map
Supervectorizer pass uses this builder and loses the permutation map.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D92145
2020-12-03 09:53:08 -08:00
Christian Sigg
c4a0405902 Add Operation* OpState::operator->() to provide more convenient access to members of Operation.
Given that OpState already implicit converts to Operator*, this seems reasonable.

The alternative would be to add more functions to OpState which forward to Operation.

Reviewed By: rriddle, ftynse

Differential Revision: https://reviews.llvm.org/D92266
2020-12-02 15:46:20 +01:00
Thomas Raoux
369c51a74b [mlir][vector] Add transfer_op LoadToStore forwarding and deadStore optimizations
Add transformation to be able to forward transfer_write into transfer_read
operation and to be able to remove dead transfer_write when a transfer_write is
overwritten before being read.

Differential Revision: https://reviews.llvm.org/D91321
2020-11-20 11:59:01 -08:00
River Riddle
65fcddff24 [mlir][BuiltinDialect] Resolve comments from D91571
* Move ops to a BuiltinOps.h
* Add file comments
2020-11-19 11:12:49 -08:00
River Riddle
73ca690df8 [mlir][NFC] Remove references to Module.h and Function.h
These includes have been deprecated in favor of BuiltinDialect.h, which contains the definitions of ModuleOp and FuncOp.

Differential Revision: https://reviews.llvm.org/D91572
2020-11-17 00:55:47 -08:00
Aart Bik
9ddb464d37 [mlir] refactor common idiom into AffineMap method
motivated by a refactoring in the new sparse code (yet to be merged), this avoids some lengthy code dup

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D91465
2020-11-13 19:18:13 -08:00
Thomas Raoux
6ad31c0f4a [mlir][vector] Support N-D vector in InsertMap/ExtractMap op
Support multi-dimension vector for InsertMap/ExtractMap op and update the
transformations. Currently the relation between IDs and dimension is implicitly
deduced from the types. We can then calculate an AffineMap based on it. In the
future the AffineMap could be part of the operation itself.

Differential Revision: https://reviews.llvm.org/D90995
2020-11-13 12:40:17 -08:00
Thomas Raoux
36480657d8 [mlir][vector] Add canonicalization patterns for ExtractStride/ShapeCast + Splat constant
Differential Revision: https://reviews.llvm.org/D90567
2020-11-03 11:29:54 -08:00
Thomas Raoux
9081e7594d [mlir][vector] Address post-commit review comments on vector ops folding patterns
Differential Revision: https://reviews.llvm.org/D90183
2020-11-02 10:57:32 -08:00
Thomas Raoux
5d45f758f0 [mlir][vector] Improve vector distribute integration test and fix block distribution
Fix semantic in the distribute integration test based on offline feedback. This
exposed a bug in block distribution, we need to make sure the id is multiplied
by the stride of the vector. Fix the transformation and unit test.

Differential Revision: https://reviews.llvm.org/D89291
2020-10-29 14:54:53 -07:00
Kazuaki Ishizaki
41b09f4eff [mlir] NFC: fix trivial typos
fix typos in comments and documents

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D90089
2020-10-29 04:05:22 +09:00
Thomas Raoux
bd07be4f3f [mlir][vector] Update doc strings for insert_map/extract_map and fix insert_map semantic
Based on discourse discussion, fix the doc string and remove examples with
wrong semantic. Also fix insert_map semantic by adding missing operand for
vector we are inserting into.

Differential Revision: https://reviews.llvm.org/D89563
2020-10-26 10:47:01 -07:00
Thomas Raoux
ea6a60a9a6 [mlir][vector] Add folder for ExtractStridedSliceOp
Add folder for the case where ExtractStridedSliceOp source comes from a chain
of InsertStridedSliceOp. Also add a folder for the trivial case where the
ExtractStridedSliceOp is a no-op.

Differential Revision: https://reviews.llvm.org/D89850
2020-10-23 12:18:09 -07:00
Thomas Raoux
8c72eea9a0 [mlir][vector] Add folding for ExtractOp with ShapeCastOp source
Differential Revision: https://reviews.llvm.org/D89853
2020-10-23 12:06:18 -07:00
Thomas Raoux
edbdea7466 [mlir][vector] Add unrolling patterns for Transfer read/write
Adding unroll support for transfer read and transfer write operation. This
allows to pick the ideal size for the memory access for a given target.

Differential Revision: https://reviews.llvm.org/D89289
2020-10-15 15:17:36 -07:00
Thomas Raoux
cf402a1987 [mlir][vector] Add unit test for vector distribute by block
When distributing a vector larger than the given multiplicity, we can
distribute it by block where each id gets a chunk of consecutive element
along the dimension distributed. This adds a test for this case and adds extra
checks to make sure we don't distribute for cases not multiple of multiplicity.

Differential Revision: https://reviews.llvm.org/D89061
2020-10-08 14:44:03 -07:00
Thomas Raoux
92e83afe44 [mlir][vector] Fold extractOp coming from broadcastOp
Combine ExtractOp with scalar result with BroadcastOp source. This is useful to
be able to incrementally convert degenerated vector of one element into scalar.

Differential Revision: https://reviews.llvm.org/D88751
2020-10-06 10:27:39 -07:00
Benjamin Kramer
6e2b267d1c Promote transpose from linalg to standard dialect
While affine maps are part of the builtin memref type, there is very
limited support for manipulating them in the standard dialect. Add
transpose to the set of ops to complement the existing view/subview ops.
This is a metadata transformation that encodes the transpose into the
strides of a memref.

I'm planning to use this when lowering operations on strided memrefs,
using the transpose to remove the stride without adding a dependency on
linalg dialect.

Differential Revision: https://reviews.llvm.org/D88651
2020-10-05 10:58:20 +02:00
Thomas Raoux
d1c8e179d8 [mlir][vector] Add canonicalization patterns for extractMap/insertMap
Add basic canonicalization patterns for the extractMap/insertMap to allow them
to be folded into Transfer ops.
Also mark transferRead as memory read so that it can be removed by dead code.

Differential Revision: https://reviews.llvm.org/D88622
2020-10-02 10:13:11 -07:00
Geoffrey Martin-Noble
d4e889f1f5 Remove Ops suffix from dialect library names
Dialects include more than just ops, so this suffix is outdated. Follows
discussion in
https://llvm.discourse.group/t/rfc-canonical-file-paths-to-dialects/621

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D88530
2020-09-30 18:00:44 -07:00
Thomas Raoux
dd14e58252 [mlir][vector] First step of vector distribution transformation
This is the first of several steps to support distributing large vectors. This
adds instructions extract_map and insert_map that allow us to do incremental
lowering. Right now the transformation only apply to simple pointwise operation
with a vector size matching the multiplicity of the IDs used to distribute the
vector.
This can be used to distribute large vectors to loops or SPMD.

Differential Revision: https://reviews.llvm.org/D88341
2020-09-30 13:14:55 -07:00
Jakub Lichman
14088a6f5d [mlir] Added support for rank reducing subviews
This commit adds support for subviews which enable to reduce resulting rank
by dropping static dimensions of size 1.

Differential Revision: https://reviews.llvm.org/D88534
2020-09-30 11:15:18 +00:00
Aart Bik
e9628955f5 [mlir] [VectorOps] Relaxed restrictions on vector.reduction types even more
Recently, restrictions on vector reductions were made more relaxed by
accepting any width signless integer and floating-point. This CL relaxes
the restriction even more by including unsigned and signed integers.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D88442
2020-09-28 13:38:03 -07:00
Benjamin Kramer
2d76274b99 [mlir][VectorOps] Loosen restrictions on vector.reduction types
LLVM can deal with any integer or float type, don't arbitrarily restrict
it to f32/f64/i32/i64.

Differential Revision: https://reviews.llvm.org/D88010
2020-09-21 12:45:23 +02:00