88 Commits

Author SHA1 Message Date
Ramiro Leal-Cavazos
44baa65589 Revert "Revert "Fix handling of special and large vals in expand pattern for round" and "Add pattern that expands math.roundeven into math.round + arith""
This reverts commit 87cef78fa1c7bf6efc544e990894a6062d56abec.

The issue in the original revert is that a lit test expecting a `-nan`
as an output was failing on M2. Since the IEEE 754-2008 standard does
not require the sign to be printed when displaying a `nan`, this
commit changes the `CHECK` for `-nan` to one that checks the result
value bitcasted to an `i32` to ensure that input is being left
unchanged. This check should now be independent of platform being used
to run test.

Reviewed By: jpienaar, mehdi_amini

Differential Revision: https://reviews.llvm.org/D148941
2023-04-22 07:15:40 -07:00
Mehdi Amini
87cef78fa1 Revert "Fix handling of special and large vals in expand pattern for round" and "Add pattern that expands math.roundeven into math.round + arith"
This reverts commit 8d2bae9abdc30e104bab00a4dd0f9d39f5bdda6e and
commit ab2fc9521ec606603412645d4a4b3cf456acd153.

Tests are broken on Mac M2
2023-04-21 00:16:32 -06:00
Ramiro Leal-Cavazos
8d2bae9abd Add pattern that expands math.roundeven into math.round + arith
This commit adds a pattern that expands `math.roundeven` into
`math.round` + some ops from `arith`. This is needed to be able to run
`math.roundeven` in a vectorized manner.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D148285
2023-04-20 12:48:12 -07:00
Ramiro Leal-Cavazos
ab2fc9521e Fix handling of special and large vals in expand pattern for round
The current expand pattern for `math.round` does not handle the
special values -0.0, +-inf, and +-nan correctly. It also does not
properly handle values with magnitude |x| >= 2^23. Lastly, the pattern
generates invalid IR when the input to `math.round` is a vector. This
patch fixes these issues.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D148398
2023-04-20 18:08:19 +00:00
Balaji V. Iyer
2d4e856709 [mlir][math] Expand math.powf to exp, log and multiply
Powf functions are pushed directly to libm. This is problematic for
situations where libm is not available. This patch will decompose the
powf function into log of exponent multiplied by log of base and raise
it to the exp.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D148164
2023-04-14 14:04:19 +00:00
Balaji V. Iyer
be9115788c [mlir][math] Expand math.round to truncate, compare and increment.
Round functions are pushed directly to libm. This is problematic for
situations where libm is not available. This patch will decompose the
roundf function by adding 0.5 to positive number to input
(subtracting for negative) following by a truncate.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D148026
2023-04-13 18:02:10 +00:00
Balaji V. Iyer
4da96515ea [mlir][math] Expand math.exp2 to use math.exp.
Exp2 functions are pushed directly to libm. This is problematic for
situations where libm is not available. This patch will expand the exp2
function to use exp2 with the input multiplied by ln2 (natural log).

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D148064
2023-04-13 16:06:04 +00:00
Balaji V. Iyer
2217888d2c [mlir][math] Expand math.ceilf to truncate, compares and increments
Ceilf are pushed directly to libm. This is problematic for
situations where libm is not available. This patch will break down
a ceilf function to truncate followed by an increment if the
truncated value is smaller than the input value.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D147974
2023-04-11 13:52:45 +00:00
Alex Zinenko
76c3908273 [mlir] rename Math/dependent-dialect.mlir to depends-on-arith.mlir 2023-04-11 12:35:46 +02:00
Alex Zinenko
56a275b999 [mlir] make Math dialect depend on Arith dialect
Ops from the Math dialect use fastmath attributes defined in Arith.
Therefore Math dialect must declare a dependency on Arith for proper
construction and parsing.

Reviewed By: tpopp

Differential Revision: https://reviews.llvm.org/D147999
2023-04-11 12:34:51 +02:00
Balaji V. Iyer
af9eb1e384 [mlir][math] Expand math.floorf to truncate, compares and increments
Floorf are pushed directly to libm. This is problematic for
situations where libm is not available. This patch will break down
a floorf function to truncate followed by an increment for negative
values, if necessary.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D147966
2023-04-10 21:04:27 +00:00
Balaji V. Iyer
a7c2102d98 [mlir][math]Expand Fused math.fmaf to a multiply-add
Fused multiply and add are being pushed directly to the libm. This is problematic
for situations where libm is not available. This patch will break down a fused multiply and
add into a multiply followed by an add.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D147811
2023-04-07 22:14:56 +00:00
Robert Suderman
711c58938f [mlir][math] Update math arith expansions for vectorization
The math arithmetic expansions do not support vectorized types.
Updated the lowerings so that they support vectorized types. This
includes a different implementation for `math.ctlz` to be a binary
search and not have variable termination time.

Reviewed By: jpienaar, NatashaKnk

Differential Revision: https://reviews.llvm.org/D147289
2023-04-06 18:42:01 +00:00
Robert Suderman
57e1943e8f [mlir] Add support for non-f32 polynomial approximation
Polynomial approximations assume F32 values. We can convert all non-f32
cases to operate on f32s with intermediate casts.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D146677
2023-03-27 21:57:57 +00:00
Robert Suderman
6b53881048 [mlir][math] Add math.cbrt polynomial approximation
Cbrt can be approximated with some relatively simple polynomial
operators. This includes a lit test validating the implementation
and some run tests that validate numerical correct.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D145019
2023-03-06 13:29:49 -08:00
Robert Suderman
740e2e908c [mlir][math] Math expansion for math.tan
We can implement a polynomial approximation of math.tan by
decomposing to `math.sin` and `math.cos`. While it is not
technically a polynomial approximation it should be the most
straight forward approximation.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D144980
2023-03-01 01:13:54 +00:00
Matthias Springer
ed9194be6d [mlir] GreedyPatternRewriter: Add ancestors to worklist
When adding an op to the worklist, also add its ancestors to the worklist. This allows for RewritePatterns to match an op `a` based on what is inside of the body of `a`.

This change fixes a problem that became apparent with `vector.warp_execute_on_lane_0`, but could probably be triggered with similar patterns. The pattern extracts an op `b` with `eligible = true` from the body of an op `a`:
```
test.a {
  %0 = test.b() {eligible = true}
  yield %0
}
```

Afterwards:
```
%0 = test.b() {eligible = true}
test.a {
  yield %0
}
```

The pattern is an `OpRewritePattern<OpA>`. For some reason, `test.a` is not on the GreedyPatternRewriter's worklist. E.g., because no pattern could be applied and it was removed. Now, another pattern updates `test.b`, so that `eligible` is changed from `true` to `false`. The `OpRewritePattern<OpA>` could now be applied, but (without this revision) `test.a` is still not on the worklist.

Note: In the above example, an `OpRewritePattern<OpB>` could have been used instead of an `OpRewritePattern<OpA>`. With such a design, we can run into the same problem (when the `eligible` attr is on `test.a` and `test.b` is removed from the worklist because no patterns could be applied).

Note: This change uncovered an unrelated bug in TestSCFUtils.cpp that was triggered due to a change in the order in which ops are processed. A TODO is added to the broken code and test cases are adapted so that the bug is no longer triggered.

Differential Revision: https://reviews.llvm.org/D140304
2023-01-13 10:51:28 +01:00
Matthias Springer
e7790fbed3 [mlir] Add test-convergence option to Canonicalizer tests
This new option is set to `false` by default. It should  be set only in Canonicalizer tests to detect faulty canonicalization patterns. I.e., patterns that prevent the canonicalizer from converging. The canonicalizer should always convergence on such small unit tests that we have in `canonicalize.mlir`.

Two faulty canonicalization patterns were detected and fixed with this change.

Differential Revision: https://reviews.llvm.org/D140873
2023-01-04 12:02:21 +01:00
Johannes Reifferscheid
998a3a3894 Add a math.cbrt instruction and lowering to libm.
There's currently no way to get accurate cube roots in the math dialect.
powf(x, 1/3.0) is too inaccurate in some cases.

Reviewed By: akuegel

Differential Revision: https://reviews.llvm.org/D140842
2023-01-03 08:44:12 +01:00
Slava Zakharin
07a4c4d601 [mlir][math] Added arith::FastMathAttr support for math::FPowI.
Differential Revision: https://reviews.llvm.org/D139805
2022-12-13 20:47:20 -08:00
Slava Zakharin
095ce655ec [mlir][math] Simplify pow(x, 0.75) into sqrt(sqrt(x)) * sqrt(x).
Trivial simplification for CPU2017/503.bwaves resulting in 3.89%
speed-up on icelake.

Differential Revision: https://reviews.llvm.org/D137351
2022-11-04 10:48:19 -07:00
Slava Zakharin
589764a382 [mlir][math] Initial support for fastmath flag attributes for Math dialect.
Added arith::FastMathAttr and ArithFastMathInterface support for Math dialect
floating point operations.

This change-set creates ArithCommon conversion utils that currently
provide classes and methods to aid with arith::FastMathAttr conversion
into LLVM::FastmathFlags. These utils are used in ArithToLLVM and
MathToLLVM convertors, but may eventually be used by other converters
that need to convert fast math attributes.

Since Math dialect operations use arith::FastMathAttr, MathOps.td now
has to include enum and attributes definitions from Arith dialect.
To minimize the amount of TD code included from Arith dialect,
I moved FastMathAttr definition into ArithBase.td.

Differential Revision: https://reviews.llvm.org/D136312
2022-11-04 10:41:56 -07:00
jacquesguan
a4ace22c05 [mlir][Math] Change regex to match fp value on different target.
Link: https://github.com/llvm/llvm-project/issues/58048

Reviewed By: ftynse, Mogball

Differential Revision: https://reviews.llvm.org/D134850
2022-10-12 15:08:21 +08:00
jacquesguan
6eebdc46e4 [mlir][Math] Add constant folder for ErfOp.
This patch adds constant folder for ErfOp by using erf/erff of libm.

Reviewed By: ftynse, Mogball

Differential Revision: https://reviews.llvm.org/D134017
2022-09-19 10:55:16 +08:00
jacquesguan
71e52a125c [mlir][Math] Add constant folder for SinOp.
This patch adds constant folder for SinOp by using sin/sinf of libm.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D133915
2022-09-16 14:30:05 +08:00
Jeff Niu
3108249dea [MLIR][math] Use approximate matches for folded ops
LibM implementations differ, so the folders can have different results
on different platforms. For instance, the `cos` folder was failing on M1
mac. I chose to match the constant floats to 2(.5) significant digits.

Reviewed By: jacquesguan

Differential Revision: https://reviews.llvm.org/D133797
2022-09-14 08:39:41 -07:00
jacquesguan
9d0b90e933 [mlir][Math] Add TruncOp.
This patch adds TruncOp for Math, it returns the operand rounded to the nearest integer not larger in magnitude than the operand. And this patch also adds the correspond llvm intrinsic op.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D133342
2022-09-09 10:01:28 +08:00
Kai Sasaki
5bb621056b [mlir][math] Canonicalization for math.floor op
Support constant folding for math.floor op as well as math.ceil.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D133398
2022-09-09 10:21:48 +09:00
jacquesguan
b53bccb18d [mlir][Math] Add constant folder for RoundOp.
This patch uses round/roundf of libm to fold RoundOp of constant.

Differential Revision: https://reviews.llvm.org/D133401
2022-09-08 14:51:17 +08:00
jacquesguan
ac66d87c4b [mlir][Math] Add constant folder for RoundEvenOp.
This patch uses roundeven/roundevenf of libm to fold RoundEvenOp of constant.

Differential Revision: https://reviews.llvm.org/D133344
2022-09-07 11:13:00 +08:00
jacquesguan
e3434a8627 [mlir][Math] Add constant folder for CosOp.
This patch adds constant folder for CosOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D131233
2022-09-07 10:54:08 +08:00
Tres Popp
4639a85f94 [mlir] Add math.roundeven and llvm.intr.roundeven
This is similar to math.round, but rounds to even instead of rounding away from
zero in the case of halfway values. This CL also adds lowerings to libm and
to the LLVM intrinsic.

Differential Revision: https://reviews.llvm.org/D132375
2022-08-25 13:39:01 +02:00
Slava Zakharin
f9d988f1ac [mlir][math] Added basic support for FPowI operation.
The operation computes pow(b, p), where 'b' is floating point
and 'p' is a signed integer. The result's type matches 'b' type.
The operands must have the same shape.

Differential Revision: https://reviews.llvm.org/D129811
2022-08-16 09:24:01 -07:00
Slava Zakharin
2dde4ba639 [mlir][math] Added algebraic simplification for IPowI operation.
Differential Revision: https://reviews.llvm.org/D130390
2022-08-15 11:55:05 -07:00
Slava Zakharin
cd3a234fa9 [mlir][math] Added constant folding for IPowI operation.
Differential Revision: https://reviews.llvm.org/D130389
2022-08-15 11:31:14 -07:00
Slava Zakharin
08b4cf3620 [mlir][math] Added basic support for IPowI operation.
The operation computes pow(b, p), where 'b' and 'p' are signed integers
of the same width. The result's type matches the operands' type.

Differential Revision: https://reviews.llvm.org/D129809
2022-08-10 10:11:53 -07:00
Jeff Niu
00f7096d31 [mlir][math] Rename math.abs -> math.absf
To make room for introducing `math.absi`.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131325
2022-08-08 11:04:58 -04:00
jacquesguan
40d74fcb55 [mlir][Math] Add constant folder for Atan2Op.
This patch adds constant folder for Atan2Op which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D131050
2022-08-05 10:30:58 +08:00
jacquesguan
752c9d0dab [mlir][Math] Add constant folder for AtanOp.
This patch adds constant folder for AtanOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130983
2022-08-03 14:10:02 +08:00
jacquesguan
008ea1c201 [mlir][Math] Add constant folder for TanhOp.
This patch adds constant folder for TanhOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130960
2022-08-02 19:37:27 +08:00
jacquesguan
f1033a3f47 [mlir][Math] Add constant folder for TanOp.
This patch adds constant folder for TanOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130873
2022-08-02 11:20:53 +08:00
Mehdi Amini
0a17692148 [MLIR] Reduce precision check for expm1 folder: this is dependent on libm (NFC)
We noticed this failing depending on the platform, checking the last few
digit isn't necessary for this test anyway.
2022-08-01 21:26:51 +00:00
jacquesguan
16cb6ce554 [mlir][Math] Add constant folder for ExpM1Op.
This patch adds constant folder for ExpM1Op which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130567
2022-08-01 14:40:50 +08:00
jacquesguan
78015047b2 [mlir][Math] Add constant folder for Exp2Op.
This patch adds constant folder for Exp2Op which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130472
2022-07-26 08:48:48 +08:00
jacquesguan
9e241c70f7 [mlir][Math] Add constant folder for ExpOp.
This patch adds constant folder for ExpOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130318
2022-07-25 16:26:48 +08:00
jacquesguan
9c22853ec4 [mlir][Math] Add constant folder for LogOp.
This patch adds constant folder for LogOp which only supports single and double precision floating-point.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D130148
2022-07-21 10:58:32 +08:00
jacquesguan
c3d856bf58 [mlir][Math] Add constant folder for Log1pOp.
This patch adds constant folder for Log1pOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D129979
2022-07-20 14:59:35 +08:00
jacquesguan
a5cae20bdb [mlir][Math] Add constant folder for Log10Op.
This patch adds constant folder for Log10Op which only support single and double precision floating-point.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D129740
2022-07-18 10:19:25 +08:00
Slava Zakharin
451e5e2b28 [mlir][math] Added math::tan operation.
Differential Revision: https://reviews.llvm.org/D129539
2022-07-15 09:17:06 -07:00
jacquesguan
4d7d5c5f00 [mlir][Math] Support fold SqrtOp with constant dense.
This patch uses constFoldUnaryOpConditional to replace current folder in order to support constant dense.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D129459
2022-07-12 16:46:19 +08:00