This reverts commit 3bb32c61b2f1f5d14dd056dd198dc898dce5a44e.
Use InsertionPt for DT to handle non-memory access dominators
Differential Revision: https://reviews.llvm.org/D155406
New memory accesses are usually inserted by using one of the
createMemoryAccessXYZ() methods followed by insertUse() or
insertDef(). createMemoryAccessXYZ() accepts a defining access,
however this defining access will always be overwritten by
insertUse() / insertDef().
Update the documentation to clarify this, and stop passing
Definition to createMemoryAccessXYZ() if it's followed by
insertUse/insertDef.
Alternatively, we could also make insertUse / insertDef keep the
defining access if it is specified, and only recompute it if it's
missing.
Differential Revision: https://reviews.llvm.org/D157979
This reverts commit 36a6eb7d12a9f827bf3d5d4e5fdc68b8a62807b2.
[MemCpyOpt] check that load/store and dest/src alloca are all in the same bb
Differential Revision: https://reviews.llvm.org/D153453
Co-authored-by: serge-sans-paille <sguelton@mozilla.com>
Make sure the code comments in processMemSetMemCpyDependence match
with the actual transform. They indicated that the memset being
rewritten was sunk to after a memcpy, while it actually is inserted
just before the memcpy.
Also make sure we use the debug location of the original memset
when creating the new simplified memset. In the past we've been
using the debug location for the memcpy which could be a bit
confusing.
Differential Revision: https://reviews.llvm.org/D135574
The llvm.memcpy.inline intrinsic must be expanded into code that
does not contain any function calls because it is intended for
the implementation of low-level functions like memcpy. Currently the
MemCpyOpt might covert llvm.memcpy.inline into llvm.memmove in
certain circumstances. This patch fixes the issue.
Fixes https://github.com/llvm/llvm-project/issues/61791.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D147162
When fetching allocation sizes, we almost always want to have the
size in bytes, but we were only providing an InBits API. Also add
the corresponding byte-based conjugate to save some *8 and /8
juggling everywhere.
Inspecting the downstream use of the cpyAlign, it is clear that
`performCallSlotOptzn` is expecting it to represent the alignment
of the copy destination, not the minimum of the src and dest
alignments. This patch renames the parameter to make this more
obvious.
I believe this change is NFC, because the downstream code has
alignment checks such that it all works out in the end. I have not
been able to construct a test case that actually triggers a change
in output.
Reviewed By: nikic
Differential Revision: https://reviews.llvm.org/D140603
While we can't use a single BatchAA instance for the entire
MemCpyOpt run without further justification, we can use BatchAA
while performing the queries related to a single instruction
(these will first perform some AA-based checks, and then modify
the IR only afterwards).
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated. The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.
This is part of an effort to migrate from llvm::Optional to
std::optional:
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
The Assignment Tracking debug-info feature is outlined in this RFC:
https://discourse.llvm.org/t/
rfc-assignment-tracking-a-better-way-of-specifying-variable-locations-in-ir
Maintain and propagate DIAssignID attachments in memcpyopt.
Reviewed By: jmorse
Differential Revision: https://reviews.llvm.org/D133312
Currently call slot optimization may be prevented because the
lifetime markers for the destination only start after the call.
In this case, rather than aborting the transform, we should move
the lifetime.start before the call to enable the transform.
Differential Revision: https://reviews.llvm.org/D135886
This was already handled correctly below, but not checked for the
original store pointer operand. Encountered when converting tests
to opaque pointers, where the intermediate bitcast goes away.
`commonAlignment` is a shortcut to pick the smallest of two `Align`
objects. As-is it doesn't bring much value compared to `std::min`.
Differential Revision: https://reviews.llvm.org/D128345
As discussed on https://github.com/llvm/llvm-project/issues/54682,
MemorySSA currently has a bug when computing the clobber of calls
that access loop-varying locations. I think a "proper" fix for this
on the MemorySSA side might be non-trivial, but we can easily work
around this in MemCpyOpt:
Currently, MemCpyOpt uses a location-less getClobberingMemoryAccess()
call to find a clobber on either the src or dest location, and then
refines it for the src and dest clobber. This was intended as an
optimization, as the location-less API is cached, while the
location-affected APIs are not.
However, I don't think this really makes a difference in practice,
because I don't think anything will use the cached clobbers on
those calls later anyway. On CTMark, this patch seems to be very
mildly positive actually.
So I think this is a reasonable way to avoid the problem for now,
though MemorySSA should also get a fix.
Differential Revision: https://reviews.llvm.org/D122911
The search for the clobbering call is fairly expensive if uses are not optimized at construction. Defer the clobber walk to the point in the implementation we need it; there are a bunch of bailouts before that point. (e.g. If the source pointer is not an alloca, we can't do callslotopt.)
On a test case which involves a bunch of copies from argument pointers, this switches memcpyopt from > 1/2 second to < 10ms.
We have the same code repeated in both callers, sink it into callee.
The motivation here isn't just code style, we can also defer the relatively expensive aliasing checks until the cheap structural preconditions have been validated. (e.g. Don't bother aliasing if src is not an alloca.) This helps compile time significantly.
