Preserve the alignment of the original atomicrmw, rather than using
the ABI alignment.
The same problem exists for loads, but that code is being removed
in D141277 anyway.
While demanded bits constant shrinking appears to prevent this in
practice right now, it is principally possible for C2 to have
set bits that are known not-needed (zeroable). See: D140858
`+` will overflow here, `|` will get the right logic.
Differential Revision: https://reviews.llvm.org/D141089
Follow-up to:
6c39a3aae1dc
That converted a pattern with ashr directly to icmp+zext, and
this updates the pattern that we used to convert to.
This canonicalizes to icmp for better analysis in the minimum case
and shortens patterns where the source type is not the same as dest type:
https://alive2.llvm.org/ce/z/tpXJ64https://alive2.llvm.org/ce/z/dQ405O
This requires an adjustment to an icmp transform to avoid infinite looping.
Use deduction guides instead of helper functions.
The only non-automatic changes have been:
1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).
Per reviewers' comment, some useless makeArrayRef have been removed in the process.
This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.
Differential Revision: https://reviews.llvm.org/D140955
We shouldn't penalize instructions that have extra flags.
Drop the poison-generating flags if needed instead of bailing out.
This makes canonicalization/optimization more uniform.
There is a chance that dropping flags will cause some
other transform to not fire, but we added a preliminary
patch to avoid that with:
f0faea571403
See D140665 for more details.
~sext(A) & Op1 --> A ? 0 : Op1
With no extra uses, this pattern is already reduced,
but we would miss it in examples such as issue #59773.
https://alive2.llvm.org/ce/z/WGLcSR
EmitGEPOffset() supports vector GEPs nowadays, so we don't need
any further code changes.
compare_gep_with_base_vector1 shows a weakness in folding the
resulting comparison if an index splat has to be performed.
As discussed in issue #59279, we want fneg/fabs to conform to the
IEEE-754 spec for signbit operations - quoting from section 5.5.1
of IEEE-754-2008:
"negate(x) copies a floating-point operand x to a destination in
the same format, reversing the sign bit"
"abs(x) copies a floating-point operand x to a destination in the
same format, setting the sign bit to 0 (positive)"
"The operations treat floating-point numbers and NaNs alike."
So we gate this transform with "nnan" in addition to "nsz":
(X > 0.0) ? X : -X --> fabs(X)
Without that restriction, we could have for example:
(+NaN > 0.0) ? +NaN : -NaN --> -NaN
(because an ordered compare with NaN is always false)
That would be different than fabs(+NaN) --> +NaN.
More fabs/fneg patterns demonstrated here:
https://godbolt.org/z/h8ecc659d
(without any FMF, these are correct independently of this patch -
no fabs should be created)
The code change is a one-liner, but we have lots of tests diffs
because there are many variations of the basic pattern.
Differential Revision: https://reviews.llvm.org/D139785
If we go through the generic EmitGEPOffset code, the resulting
expression can be (and is) reduced in the same way this code did
manually. There are no changes in lit tests or llvm-test-suite.
This fold predates the time where we started adding nsw to the adds
created by EmitGEPOffset, so it was likely needed back then.
This might not actually be NFC due to worklist order changes etc.
This mirrors a similar shufflevector transformation so the same
effect is obtained for scalable vectors. The transformation is
only performed when it can be proven the number of resulting
reversals is not increased. By bubbling the reversals from operand
to result this should typically be the case and ideally leads to
back-back shuffles that can be elimitated entirely.
Differential Revision: https://reviews.llvm.org/D139342
This mirrors a similar shufflevector transformation so the same
effect is obtained for scalable vectors. The transformation is
only performed when it can be proven the number of resulting
reversals is not increased. By bubbling the reversals from operand
to result this should typically be the case and ideally leads to
back-back shuffles that can be elimitated entirely.
Differential Revision: https://reviews.llvm.org/D139339
This mirrors a similar shufflevector transformation so the same
effect is obtained for scalable vectors. The transformation is
only performed when it can be proven the number of resulting
reversals is not increased. By bubbling the reversals from operand
to result this should typically be the case and ideally leads to
back-back shuffles that can be elimitated entirely.
Differential Revision: https://reviews.llvm.org/D139340
In the X == C ? f(X) : Y -> X == C ? f(C) : Y fold, perform the
replacement in f(X) recursively. For now, this just goes two
instructions up rather than one instruction up.
As per post-commit feedback - we generally do not like Constant Expressions,
and trying to deal with them leads to inconsistent results
that may very well be non-optimal. So just don't.
One of these two changes is exposing (or causing) some more miscompiles.
A reproducer is in progress, so reverting until resolved.
This reverts commit 9ddff66d0c9c3e18d56e6b20aa26a2a8cdfb6d2b.
`canFreelyInvertAllUsersOf()`, in general, does not make sense
for constants, and constant expressions are likely even more problematic.
For those, we just want to create a simple constant expression and be done.
Fixes https://github.com/llvm/llvm-project/issues/59613
and (fcmp ord x, 0), (fcmp u* x, inf) -> fcmp o* x, inf
and (fcmp ord x, 0), (fcmp u* fabs(x), inf) -> fcmp o* x, inf
Clang emits this peculiar pattern as an isfinite check in
__builtin_isnormal which can be simplified. We should fix clang to
emit this in the first place, but should also fold it here.
The important bit here is that we gracefully handle other uses,
iff they can be adapted to inversion.
I'll note, the previous logic was actively bad,
it increased instruction count since it didn't actually ensure
that the inversions happened.
Matches what we do for binary operations, but a special care needs
is needed to preserve operand order, as the logical operations
are not strictly commutative!
It's a bigger match than usual, but I have not found any
sub-patterns that reduce:
(X / DivC) + sext ((X & (SMin | (DivC - 1)) >u SMin) --> X >>s log2(DivC)
https://alive2.llvm.org/ce/z/MJzlhl
Fixes issue #55741
This puts lower insert indexes before higher. This is independent
of endian, so it requires an adjustment to a fold added with
4446f71ce392, but it makes that fold more robust.
That's also where this patch was suggested - D139668.
This matches what we already do in DAGCombiner, but there is one
more constraint because there's an existing canonicalization for
insert-of-scalar-constant. I'm not sure if that is still needed,
so it may be adjusted/removed as a follow-up.
value() has undesired exception checking semantics and calls
__throw_bad_optional_access in libc++. Moreover, the API is unavailable without
_LIBCPP_NO_EXCEPTIONS on older Mach-O platforms (see
_LIBCPP_AVAILABILITY_BAD_OPTIONAL_ACCESS).
The class in InstCombineInternal.h inherits from InstCombiner.h.
I think this split was created when target specific InstCombines
were moved to go through TTI.
I had to update some of the code in InstCombiner.h to match changes
that had been made to InstCombineInternal.h.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D140230
Currently InstCombine folds using the
`return replaceInstUsesWith(V, Builder.CreateFoo())`
pattern do not preserve the original name of the instruction.
To preserve the name, you either have to use something like
`return FooInst::Create(...)` which is usually less nice, or go
out of the way to preserve the name with takeName(). We often
don't do that.
This patch instead preserves the name in replaceInstUsesWith()
when replacing a named instruction with an unnamed instruction.
To be conservative, I also added a zero-use check, which is a
proxy for the case where the instruction was just created, rather
than an existing one reused. Possibly we could drop that part.
As InstCombine tests are robust against renames this does not
cause any test diffs, so I regenerated a random test to show the
effects.
Differential Revision: https://reviews.llvm.org/D140192
It was annoying to write the check for this in the one case I added,
and I'm planning on adding another, so add a convenient PatternMatch
like for other special case values.
I have no idea what is going on in the DoubleAPFloat case, I reversed
this from the makeSmallestNormalized test. Also could implement this
as *this == getSmallestNormalized() for less code, but this avoids the
construction of a temporary APFloat copy and follows the style of the
other functions.
This transform was added with 4446f71ce392. However, as noted in
the post-commit feedback, the transform is not safe with an
arbitrary base vector because we may leak poison from a narrow
element into an adjacent element when bitcasting.
I made the least invasive code change in case we do figure out
a way to make this safe.
