413 Commits

Author SHA1 Message Date
Nikita Popov
874c0327e7 [Attributor] Use ConstantFoldLoadFromConst()
When determining the initial value of the object, use the constant
folding API to load a given type at a given offset in the global
initializer. This makes it work for cases where the load doesn't
directly correspond to an aggregate member.

Differential Revision: https://reviews.llvm.org/D135435
2022-10-10 10:17:37 +02:00
Johannes Doerfert
e18736149c [Attributor] Teach AAPointerInfo about atomic cmxchg and rmw
The atomic operations behave similar to a store except that we don't
know the new value and we read the result first.
2022-10-05 06:48:00 -07:00
Johannes Doerfert
93e51fa444 [Attributor] AAPointerInfo can model non-escaping call uses
If a call base use will not capture a pointer we can approximate the
effects. This is important especially for readnone/only uses. Even
may-write uses are not too bad with reachability in place. Capturing
is the problem as we loose track of update sides.
2022-10-05 06:29:14 -07:00
Johannes Doerfert
477e8e10f0 [Attributor] Teach AAPointerInfo to look into aggregates
If we have a constant aggregate, e.g., as an initializer, we usually
failed to extract the proper value/type from it. This patch provides the
size and offset information necessary to extract the right part of the
constant.
2022-10-05 06:19:47 -07:00
Johannes Doerfert
4040c5c2f4 [Attributor][NFC] Re-run update_test_checks on all Attributor tests 2022-10-05 06:19:43 -07:00
Nikita Popov
6e504d637d [ValueTracking] Handle constant exprs in isKnownNonZero()
Handle constant expressions by falling through to the general
operator-based code. In particular, this adds support for bitcast
and GEP expressions.
2022-10-04 11:58:07 +02:00
Sameer Sahasrabuddhe
3f078b308b [AAPointerInfo] OffsetInfo: Unassigned is distinct from Unknown
A User like the PHINode may be visited multiple times for the same pointer along
different def-use edges. The uninitialized state of OffsetInfo at the first
visit needs to be distinct from the Unknown value that may be assigned after
processing the PHINode. Without that, a PHINode with all inputs Unknown is never
followed to its uses. This results in incorrect optimization because some
interfering accessess are missed.

Differential Revision: https://reviews.llvm.org/D134704
2022-09-28 20:31:36 +05:30
Nikita Popov
846709b287 [Attribute] Clean up test prefixes (NFC)
Now that the legacy PM is no longer tested, the huge matrix of
test prefixes used by attributor tests is no longer needed and very
confusing for the casual reader. Reduce the prefixes down to just
CHECK, TUNIT and CGSCC.
2022-09-23 11:08:11 +02:00
Sebastian Peryt
99c9b37d11 [NFC][1/n] Remove -enable-new-pm=0 flags from lit tests
This is the first patch in a series intended for removing flag
-enable-new-pm=0 from lit tests. This is part of a bigger
effort of completely removing legacy code related to legacy
pass manager in favor of currently default new pass manager.

In this patch flag has been removed only from tests where no significant
change has been required because checks has been duplicated for
both PMs.

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D134150
2022-09-19 09:57:37 -07:00
Johannes Doerfert
c922cac868 Revert "[Attributor] AAPointerInfo should allow "harmless" uses"
Revert "[Attributor] Teach AAPointerInfo to look into aggregates"

This reverts commit 844f6c5d03d58e7ac0c6b838e4a7834ac575ab9b and
4ed0a88cd8a77370073feb270d77a9e8b27bd68c as they broke the buildbots
that run openmp/libomptarget/test/offloading/bug49021.cpp.
2022-09-11 21:37:54 -07:00
Johannes Doerfert
844f6c5d03 [Attributor] AAPointerInfo should allow "harmless" uses
If a call base use will not capture a pointer we can approximate the
effects. This is important especially for readnone/only uses.
2022-09-11 20:16:11 -07:00
Johannes Doerfert
4ed0a88cd8 [Attributor] Teach AAPointerInfo to look into aggregates
If we have a constant aggregate, e.g., as an initializer, we usually
failed to extract the proper value/type from it. This patch provides the
size and offset information necessary to extract the right part of the
constant.
2022-09-11 20:16:11 -07:00
Johannes Doerfert
b046ebdc01 [Attributor][FIX] Conservatively handle ptr2int, don't crash
If a pointer-2-int cast is found we give up on AAPointerInfo for now.
This caused a crash before.

Reported by John Tramm (@jtramm).
2022-09-11 20:16:11 -07:00
Fangrui Song
c2a3888793 [IR] Use Min behavior for module flag "PIC Level"
Using Max for both "PIC Level" and "PIE Level" is inconsistent. PIC imposes less
restriction while PIE imposes more restriction. The result generally
picks the more restrictive behavior: Min for PIC.

This choice matches `ld -r`: a non-pic object and a pic object merge into a
result which should be treated as non-pic.

To allow linking "PIC Level" using Error/Max from old bitcode files, upgrade
Error/Max to Min.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D130531
2022-08-18 16:28:55 -07:00
Johannes Doerfert
b65471d715 [Attributor][FIX] Visit same instructions with different scopes
If we collect potential values we need to visit a value even if we have
seen it before if the scope is different. The scope is part of the
result after all. Test included.

Fixes https://github.com/llvm/llvm-project/issues/56753

Differential Revision: https://reviews.llvm.org/D131597
2022-08-10 16:02:12 -05:00
Augie Fackler
85063090e9 MemoryBuiltins: remove malloc-family funcs from list
We no longer need specialized knowledge of these allocator functions in
this file since we have the correct attributes available now.

As far as I can tell the changes in the attributor tests are due to
things getting more consistent on alloc-family once we remove the static
list entries.

The two test changes in NewGVN merit extra scrutiny: NewGVN appears to
be _extremely_ sensitive to the inaccessiblememonly for reasons that
are beyond me. As a result, I had-enumerated all the attributes on
allocation functions in those two tests instead of using -inferattrs.
I assumed that the two -disable-simplify-libcalls tests there no
longer are sensible since the function declaration now includes all the
relevant attributes.

Differential Revision: https://reviews.llvm.org/D130107
2022-07-25 17:29:01 -04:00
Augie Fackler
12c0bf8ba9 tests: add attributes that would normally come from inferattrs
As my goal is to remove at least _some_ functions from the static list
in MemoryBuiltins.cpp, these tests either need to run inferattrs or
statically declare these attributes to keep passing. A couple of tests
had alternate cases which are no longer meaningful, e.g.
`malloc-load-removal.ll`.

Differential Revision: https://reviews.llvm.org/D123087
2022-07-25 17:29:00 -04:00
Johannes Doerfert
6b7eae11f1 [Attributor][FIX] HasBeenWrittenTo logic should only be used for reads
If we look at a write, we should not enact the "has been written to"
logic introduced to avoid spurious write -> read dependences. Doing so
lead to elimination of stores we needed, which is obviously bad.
2022-07-22 23:57:57 -05:00
Johannes Doerfert
a50b9f9f1f [Attributor][FIX] Handle non-recursive but re-entrant functions properly
If a function is non-recursive we only performed intra-procedural
reasoning for reachability (via AA::isPotentiallyReachable). However,
if it is re-entrant that doesn't mean we can't reach. Instead of this
problematic logic in the reachability reasoning we utilize logic in
AAPointerInfo. If a location is for sure written by a function it can
be re-entrant or recursive we know only intra-procedural reasoning is
sufficient.
2022-07-22 00:00:56 -05:00
Johannes Doerfert
62f7888d6d [Attributor] Dominating must-write accesses allow unknown initial values
If we have a dominating must-write access we do not need to know the
initial value of some object to perform reasoning about the potential
values. The dominating must-write has overwritten the initial value.
2022-07-21 23:08:43 -05:00
Johannes Doerfert
dfac030271 [Intrinsics] Add nocallback to the memset/cpy/move intrinsics
These were forgotten when D118680 was applied. Similar to D125937.

Differential Revision: https://reviews.llvm.org/D129516
2022-07-21 22:52:46 -05:00
Johannes Doerfert
ad98ef8be4 [Attributor] Deal with complex PHI nodes better during AAPointerInfo
We were quite conservative when it came to PHI node handling to avoid
recursive reasoning. Now we check more direct if we have seen a PHI
already or not. This allows non-recursive PHI chains to be handled.

This also exposed a bug as we did only model the effect of one loop
traversal. `phi_no_store_3` has been adapted to show how we would have
used `undef` instead of `1` before. With this patch we don't replace
it at all, which is expected as we do not argue about loop iterations
(or alignments).
2022-07-20 17:34:50 -05:00
Johannes Doerfert
142897dd7d [Attributor] Only non-exact accesses require a uniform bit-pattern (=0)
If we only have exact accesses we should never require the bit-pattern
to be uniform (in this case 0). Only a non-exact access should force us
to require only 0 values.
2022-07-20 17:34:50 -05:00
Johannes Doerfert
f84712f0b8 [Attributor] Teach checkForAllUses to follow returns into callers
If we can determine all call sites we can follow a use in a return
instruction into the caller. AAPointerInfo utilizes this feature.
2022-07-19 18:17:40 -05:00
Johannes Doerfert
bf789b1957 [Attributor] Replace AAValueSimplify with AAPotentialValues
For the longest time we used `AAValueSimplify` and
`genericValueTraversal` to determine "potential values". This was
problematic for many reasons:
- We recomputed the result a lot as there was no caching for the 9
  locations calling `genericValueTraversal`.
- We added the idea of "intra" vs. "inter" procedural simplification
  only as an afterthought. `genericValueTraversal` did offer an option
  but `AAValueSimplify` did not. Thus, we might end up with "too much"
  simplification in certain situations and then gave up on it.
- Because `genericValueTraversal` was not a real `AA` we ended up with
  problems like the infinite recursion bug (#54981) as well as code
  duplication.

This patch introduces `AAPotentialValues` and replaces the
`AAValueSimplify` uses with it. `genericValueTraversal` is folded into
`AAPotentialValues` as are the instruction simplifications performed in
`AAValueSimplify` before. We further distinguish "intra" and "inter"
procedural simplification now.

`AAValueSimplify` was not deleted as we haven't ported the
re-materialization of instructions yet. There are other differences over
the former handling, e.g., we may not fold trivially foldable
instructions right now, e.g., `add i32 1, 1` is not folded to `i32 2`
but if an operand would be simplified to `i32 1` we would fold it still.

We are also even more aware of function/SCC boundaries in CGSCC passes,
which is good even if some tests look like they regress.

Fixes: https://github.com/llvm/llvm-project/issues/54981

Note: A previous version was flawed and consequently reverted in
      6555558a80589d1c5a1154b92cc3af9495f8f86c.
2022-07-19 16:24:42 -05:00
Johannes Doerfert
f6e0c05e3d Revert "[Attributor] Replace AAValueSimplify with AAPotentialValues"
This reverts commit f17639ea0cd30f52ac853ba2eb25518426cc3bb8 as three
AMDGPU tests haven't been updated. Will need to verify the changes are
not regressions we should avoid.
2022-07-08 00:53:38 -05:00
Johannes Doerfert
f17639ea0c [Attributor] Replace AAValueSimplify with AAPotentialValues
For the longest time we used `AAValueSimplify` and
`genericValueTraversal` to determine "potential values". This was
problematic for many reasons:
- We recomputed the result a lot as there was no caching for the 9
  locations calling `genericValueTraversal`.
- We added the idea of "intra" vs. "inter" procedural simplification
  only as an afterthought. `genericValueTraversal` did offer an option
  but `AAValueSimplify` did not. Thus, we might end up with "too much"
  simplification in certain situations and then gave up on it.
- Because `genericValueTraversal` was not a real `AA` we ended up with
  problems like the infinite recursion bug (#54981) as well as code
  duplication.

This patch introduces `AAPotentialValues` and replaces the
`AAValueSimplify` uses with it. `genericValueTraversal` is folded into
`AAPotentialValues` as are the instruction simplifications performed in
`AAValueSimplify` before. We further distinguish "intra" and "inter"
procedural simplification now.

`AAValueSimplify` was not deleted as we haven't ported the
re-materialization of instructions yet. There are other differences over
the former handling, e.g., we may not fold trivially foldable
instructions right now, e.g., `add i32 1, 1` is not folded to `i32 2`
but if an operand would be simplified to `i32 1` we would fold it still.

We are also even more aware of function/SCC boundaries in CGSCC passes,
which is good even if some tests look like they regress.

Fixes: https://github.com/llvm/llvm-project/issues/54981

Note: A previous version was flawed and consequently reverted in
      6555558a80589d1c5a1154b92cc3af9495f8f86c.
2022-07-08 00:38:27 -05:00
Johannes Doerfert
efe8c581ff [Attributor][NFC] Improve heap2stack result readability and code style 2022-07-07 16:49:22 -05:00
Johannes Doerfert
07766f4070 [Attributor] Move heap2stack allocas to the entry block if possible
If we are certainly not in a loop we can directly emit the heap2stack
allocas in the function entry block. This will help to get rid of them
(SROA) and avoid stacksave/restore intrinsics when the function is
inlined.
2022-07-01 21:34:12 -05:00
Evgenii Stepanov
5011b4ca0e Revert "[Attributor] Ensure to use the proper liveness AA"
Reason: memory leaks

This reverts commit 083010312aa4a0ba0cd5299bd3b039af8fb6d58f.
2022-06-22 13:40:45 -07:00
Johannes Doerfert
b7cc3b10c5 [Attributor][FIX] Avoid empty bin in AAPointerInfo
This avoid creating empty bins in AAPointerInfo which can lead to
segfaults. Also ensure we do not try to translate from callee to caller
except if we really take the argument state and move it to the call site
argument state.

Fixes: https://github.com/llvm/llvm-project/issues/55726
2022-06-21 21:30:57 -05:00
Johannes Doerfert
083010312a [Attributor] Ensure to use the proper liveness AA
When determining liveness via Attributor::isAssumedDead(...) we might
end up without a liveness AA or with one pointing into another function.
Neither is helpful and we will avoid both from now on.

Reapplied after fixing the ASAN error which caused the revert:
db68a25ca9
2022-06-21 21:28:26 -05:00
Mitch Phillips
db68a25ca9 Revert "[Attributor] Ensure to use the proper liveness AA"
This reverts commit a3273c0c06003b9e5f53a856844fee42a17e6b7b.

Reason: Broke the ASan buildbots with a memory leak. See
https://reviews.llvm.org/rG94841c713fdd2bce3276015d1e946d414bb74ee8 for
more information.
2022-06-10 14:05:09 -07:00
Johannes Doerfert
6555558a80 Revert "[Attributor] Replace AAValueSimplify with AAPotentialValues"
This reverts commit da50dab1ae111e9e6cb0248a47a038b17f798705.

Patch broke AMD GPU OpenMP offload buildbots.
https://lab.llvm.org/buildbot/#/builders/193/builds/13246
2022-06-09 17:04:01 +02:00
Johannes Doerfert
da50dab1ae [Attributor] Replace AAValueSimplify with AAPotentialValues
For the longest time we used `AAValueSimplify` and
`genericValueTraversal` to determine "potential values". This was
problematic for many reasons:
- We recomputed the result a lot as there was no caching for the 9
  locations calling `genericValueTraversal`.
- We added the idea of "intra" vs. "inter" procedural simplification
  only as an afterthought. `genericValueTraversal` did offer an option
  but `AAValueSimplify` did not. Thus, we might end up with "too much"
  simplification in certain situations and then gave up on it.
- Because `genericValueTraversal` was not a real `AA` we ended up with
  problems like the infinite recursion bug (#54981) as well as code
  duplication.

This patch introduces `AAPotentialValues` and replaces the
`AAValueSimplify` uses with it. `genericValueTraversal` is folded into
`AAPotentialValues` as are the instruction simplifications performed in
`AAValueSimplify` before. We further distinguish "intra" and "inter"
procedural simplification now.

`AAValueSimplify` was not deleted as we haven't ported the
re-materialization of instructions yet. There are other differences over
the former handling, e.g., we may not fold trivially foldable
instructions right now, e.g., `add i32 1, 1` is not folded to `i32 2`
but if an operand would be simplified to `i32 1` we would fold it still.

We are also even more aware of function/SCC boundaries in CGSCC passes,
which is good.

Fixes: https://github.com/llvm/llvm-project/issues/54981
2022-06-09 16:48:53 +02:00
Johannes Doerfert
94841c713f [Attributor] Try to delete stores and simplify stored values
By default we should try to eliminate unused stores and simplify values
stored while we are at it.
2022-06-09 16:48:53 +02:00
Johannes Doerfert
a3273c0c06 [Attributor] Ensure to use the proper liveness AA
When determining liveness via Attributor::isAssumedDead(...) we might
end up without a liveness AA or with one pointing into another function.
Neither is helpful and we will avoid both from now on.
2022-06-09 16:48:53 +02:00
Johannes Doerfert
393be12b74 [Attributor] Look at base values for align, nonnull, and deref
Stripping bitcasts and 0-geps helps normalization and minimizes the
impact of a follow up change.
2022-06-09 13:41:23 +02:00
Johannes Doerfert
cb8adf76f7 [Attributor] Simplify loads from constant globals
If a global is constant and the initializer is known we can simplify
loads from it as the value has to be the initializer.
2022-06-09 13:41:23 +02:00
Johannes Doerfert
1df6e171c3 [Attributor] Simplify (integer range) state handling
We used to be very conservative when integer states were merged.
Instead of adding the known range (which is large due to uncertainty)
into the assumed range (which is hopefully small), we can also only
allow to merge in both at the same time into their respective
counterpart. This will ensure we keep the invariant that assumed is part
of known.
2022-06-09 12:00:26 +02:00
Johannes Doerfert
4277c1be88 [Attributor][FIX] Avoid metadata and duplicate replication assertion
When we recreate instructions as part of simplification we need to take
care of debug metadata and replacing the value multiple times. For now,
we handle both conservatively.
2022-06-09 12:00:26 +02:00
Phoebe Wang
7c04454227 [ArgPromotion][Attributor] Update min-legal-vector-width when do promotion
X86 codegen uses function attribute `min-legal-vector-width` to select the proper ABI. The intention of the attribute is to reflect user's requirement when they passing or returning vector arguments. So Clang front-end will iterate the vector arguments and set `min-legal-vector-width` to the width of the maximum for both caller and callee.

It is assumed any middle end optimizations won't care of the attribute expect inlining and argument promotion.
- For inlining, we will propagate the attribute of inlined functions because the inlining functions become the newer caller.
- For argument promotion, we check the `min-legal-vector-width` of the caller and callee and refuse to promote when they don't match.

The problem comes from the optimizations' combination, as shown by https://godbolt.org/z/zo3hba8xW. The caller `foo` has two callees `bar` and `baz`. When doing argument promotion, both `foo` and `bar` has the same `min-legal-vector-width`. So the argument was promoted to vector. Then the inlining inlines `baz` to `foo` and updates `min-legal-vector-width`, which results in ABI mismatch between `foo` and `bar`.

This patch fixes the problem by expanding the concept of `min-legal-vector-width` to indicator of functions arguments. That says, any passes touch functions arguments have to set `min-legal-vector-width` to the value reflects the width of vector arguments. It makes sense to me because any arguments modifications are ABI related and should response for the ABI compatibility.

Differential Revision: https://reviews.llvm.org/D123284
2022-05-02 14:13:05 +08:00
Johannes Doerfert
e87f10a771 [Attributor] CGSCC pass should not recompute results outside the SCC (reapply)
When we run the CGSCC pass we should only invest time on the SCC. We can
initialize AAs with information from the module slice but we should not
update those AAs. We make an exception for are call site of the SCC as
they are helpful providing information for the SCC.

Minor modifications to pointer privatization allow us to perform it even
in the CGSCC pass, similar to ArgumentPromotion.
2022-04-17 12:48:49 -05:00
Johannes Doerfert
39a68cc016 Revert "[Attributor] CGSCC pass should not recompute results outside the SCC"
This reverts commit 0d7f81e31315f8cda56ce6fde5ff5145e0325c51, it caused
the AMDGPU tests that use the Attributor to fail.
2022-04-15 15:29:51 -05:00
Johannes Doerfert
0d7f81e313 [Attributor] CGSCC pass should not recompute results outside the SCC
When we run the CGSCC pass we should only invest time on the SCC. We can
initialize AAs with information from the module slice but we should not
update those AAs.
2022-04-15 14:56:09 -05:00
Johannes Doerfert
2d8e7834b0 [Attributor][NFC] Rename AAPotentialValues to AAPotentialConstantValues 2022-04-15 14:56:09 -05:00
Johannes Doerfert
af30de7788 [Attributor] Introduce AAInstanceInfo
The Attributor, as many other parts in LLVM, uses pointer equivalence
for `llvm::Value`s. This only works as long as `llvm::Value`s are
dynamically unique, or, to be exact, we will never end up with the same
`llvm::Value` representing two dynamic instances. We already provided a
helper to check the former, namely `AA::isDynamicallyUnique`, however we
could not check the latter. In this patch we move the logic into a
separate AA which helps with the growing complexity and use cases. We
also extend the interface to answer the second question rather than the
first. So we do not determine dynamically uniqueness but if we might end
up with the `llvm::Value` describing a different dynamic instance. Note
that the latter is very much tied to the Attributor capabilities to look
through memory, recursion, etc. so we need to update the logic as we go.
2022-04-05 23:07:13 -05:00
Johannes Doerfert
c42aa1be74 [Attributor] Keep loads feeding in llvm.assume if stores stays
If a load is only used by an `llvm.assume` and the stores feeding into
the load are not removable, keep the load.
2022-04-05 23:07:12 -05:00
Johannes Doerfert
857bf306d7 [Attributor] Remove broken and duplicated load simplification
We look through loads in the "generic value traversal" and we
consequently don't need to look through them again in AAValueSimplify*.
The test changes stem from the fact that we allowed any simplified
value, incl. non-dynamically unique ones, as long as the underlying
memory was an alloca. This doesn't seem to make sense as allocas do not
protect against dynamically non-unique values. We need to make the
unique check better rather than excluding allocas. That in mind, we can
remove a lot of code by simply relying on the generic value traversal
load look through.

To soften the blow some minor adjustments have been made that allow more
simplification through the now used scheme and some tests have been
given a `norecurse` for now.
2022-04-05 20:49:03 -05:00
Johannes Doerfert
3e8c4366e2 [Attributor] Visit droppable uses in AAIsDead
If we ignore droppable users everything only used in llvm.assume (among
other things) is going to be deleted as dead. This is not helpful.
Instead we want to only delete things we actually don't need anymore. A
follow up will deal with loads in a smarter way.
2022-04-05 18:20:45 -05:00