161449 Commits

Author SHA1 Message Date
Fangrui Song
2417618d5c [Verifier] Reject dllexport with non-default visibility
Add a visibility check for dllimport and dllexport. Note: dllimport with a
non-default visibility (implicit dso_local) is already rejected, but with a less
clear dso_local error.

The MC level visibility `MCSA_Exported` (D123951) is mapped from IR level
default visibility when dllexport is specified. The D123951 error is now very
difficult to trigger (needs to disable the IR verifier).

Reviewed By: mstorsjo

Differential Revision: https://reviews.llvm.org/D133267
2022-09-05 10:53:41 -07:00
LiaoChunyu
456c7ef68e [InstSimplify][NFC] shortened the code 2022-09-05 23:57:53 +08:00
LiaoChunyu
5e9522c311 [InstSimplify] Odd - X ==/!= X -> false/true
Reviewed By: spatel

Differential Revision: https://reviews.llvm.org/D132989
2022-09-05 23:51:45 +08:00
Momchil Velikov
078899cd64 [SimplifyCFG] Allow SimplifyCFG hoisting to skip over non-matching instructions
SimplifyCFG does some common code hoisting, which is limited
to hoisting a sequence of identical instruction in identical
order and stops at the first non-identical instruction.

This patch allows hoisting instruction pairs over
same-length sequences of non-matching instructions. The
linear asymptotic complexity of the algorithm stays the
same, there's an extra parameter
`simplifycfg-hoist-common-skip-limit` serving to limit
compilation time and/or the size of the hoisted live ranges.

The patch improves SPECv6/525.x264_r by about 10%.

Reviewed By: nikic, dmgreen

Differential Revision: https://reviews.llvm.org/D129370
2022-09-05 15:13:46 +01:00
Daniil Fukalov
99d364d1f4 [MachineVerifier] Fix crash on early clobbered subreg operands.
MachineVerifier tried to checkLivenessAtDef() ignoring it is actually a subreg.

The issue was with processing two subregs of the same reg are used in the same
instruction (e.g. inline asm): "def early-clobber" and other just "def".

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D126661
2022-09-05 17:08:21 +03:00
David Sherwood
ffa6267300 [CodeGen] Support extracting fixed-length vectors from illegal scalable vectors
For some indices we can simply extract the fixed-length subvector from the
low half of the scalable vector, for example when the index is less than the
minimum number of elements in the low half. For all other cases we can
expand the operation through the stack by storing out the vector and
reloading the fixed-length part we need.

Fixes https://github.com/llvm/llvm-project/issues/55412

Tests added here:

  CodeGen/AArch64/sve-extract-fixed-from-scalable-vector.ll

Differential Revision: https://reviews.llvm.org/D117499
2022-09-05 15:05:14 +01:00
Tian Zhou
8fa432be4f [InstCombine] reduce test-for-overflow of shifted value
Fixes #57338.

The added code makes the following transformations:

For unsigned predicates / eq / ne:
icmp pred (x << 1), x --> icmp getSignedPredicate(pred) x, 0
icmp pred x, (x << 1) --> icmp getSignedPredicate(pred) 0, x

Some examples:
https://alive2.llvm.org/ce/z/ckn4cj
https://alive2.llvm.org/ce/z/h-4bAQ

Differential Revision: https://reviews.llvm.org/D132888
2022-09-05 09:51:51 -04:00
Ivan Kosarev
5db8d6fd2b [AMDGPU][CodeGen] Support (base | offset) SMEM loads.
Prevents generation of unnecessary s_or_b32 instructions.

Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D132552
2022-09-05 14:22:06 +01:00
Simon Pilgrim
bd0801cddf [X86] Cleanup SLM SSE shift and CMPGTQ scheduler model numbers
These were causing weird mismatches for the D103695 script report as I'm trying to enable cost kinds support for vector shift and integer comparisons.

The SSE shifts by (non-constant) scalar are half-rate but still only 1uop and PCMPGT is half-rate and only on Pipe0 (although not as slow as PCMPEQQ which we already handle).
2022-09-05 13:44:05 +01:00
Ivan Kosarev
f33645301e [AMDGPU][CodeGen] Support (soffset + offset) s_buffer_load's.
Reviewed By: foad

Differential Revision: https://reviews.llvm.org/D130263
2022-09-05 12:53:05 +01:00
Amara Emerson
fb60e50c78 [GlobalISel] Fix a combine crash due to a negative G_INSERT_VECTOR_ELT idx.
These should really be folded away to undef but we shouldn't crash in any case.
2022-09-05 12:10:17 +01:00
Simon Pilgrim
4e6783f866 [DAG] getFreeze()/getNode() - account for operand depth when calling isGuaranteedNotToBeUndefOrPoison (PR57554)
Similar to #57402 - we were calling isGuaranteedNotToBeUndefOrPoison on the freeze operand (with Depth = 0), but wasn't accounting for the fact that a later isGuaranteedNotToBeUndefOrPoison assertion will call from the new node (with Depth = 0 as well) - which will then recursively call isGuaranteedNotToBeUndefOrPoison for its operands with Depth = 1

Fixes #57554
2022-09-05 11:46:46 +01:00
Florian Hahn
408ebe5e3a
[VPlan] Move VPWidenCallRecipe to VPlanRecipes.cpp (NFC).
Depends on D132585.

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D132586
2022-09-05 10:48:29 +01:00
Nikita Popov
388b684354 [LICM] Separate check for writability and thread-safety (NFCI)
This used a single check to make sure that the object is both
writable and thread-local. Separate them out to make the
deficiencies in the current code more obvious.
2022-09-05 09:43:17 +02:00
wanglei
bf47954703 [LoongArch] Add more fixups and relocations
This patch makes the assembler support all modifiers defined in gnu-as.
Also changes some diagnostic information.

Differential Revision: https://reviews.llvm.org/D132633
2022-09-05 14:55:18 +08:00
Craig Topper
e529c0a2a0 [TargetLowering] Use ComputeMaxSignificantBits instead of ComputeNumSignBits in expandMUL_LOHI. NFC
The way ComputeNumSignBits was being used was only correct if
OuterBitSize is exactly 2x InnerBitSize. Which is always true,
but not obviously so. Comparing ComputeMaxSignificantBits to
InnerBitSize feels more correct.
2022-09-04 22:35:16 -07:00
Craig Topper
45e2809f71 [TargetLowering] Use getShiftAmountConstant. NFC 2022-09-04 20:22:32 -07:00
gonglingqin
bc743bf666 [LoongArch] Add codegen support for fcopysign
Differential Revision: https://reviews.llvm.org/D133185
2022-09-05 11:03:54 +08:00
Vitaly Buka
6c52736e02 Revert "[llvm] Use range-based for loops (NFC)"
range-based loop should not be used here, as
fixupImmediateBr push_backs into the container.

http://lab.llvm.org/buildbot/#/builders/168
http://lab.llvm.org/buildbot/#/builders/74
http://lab.llvm.org/buildbot/#/builders/5
http://lab.llvm.org/buildbot/#/builders/239
http://lab.llvm.org/buildbot/#/builders/237
http://lab.llvm.org/buildbot/#/builders/236

This reverts commit fedc59734a44ef7b62c5f389b0cdffd02264b2a9.
2022-09-04 15:28:53 -07:00
Florian Hahn
ba3d29f871
[LCSSA] Update unreachable uses with poison.
Users of LCSSA may not expect non-phi uses when checking the uses
outside a loop, which may cause crashes. This is due to the fact that we
do not update uses in unreachable blocks.

To ensure all reachable uses outside the loop are phis, update uses in
unreachable blocks to use poison in dead code.

Fixes #57508.
2022-09-04 22:26:18 +01:00
Simon Pilgrim
8534f51474 [CostModel][X86] Add CostKinds handling for sqrt intrinsicc
This was achieved using the 'cost-tables vs llvm-mca' script from D103695

Some of the znver1/znver2 latency/throughput numbers were really weird (some copy+paste afaict) - I've used the numbers from the AMD SoG, which roughly match the 'worst case' range value from Agner
2022-09-04 18:39:21 +01:00
Simon Pilgrim
626a84db47 [CostModel][X86] getTypeBasedIntrinsicInstrCost - convert to CostKindTblEntry
Begin the refactoring to use CostKindTblEntry and return real latency/codesize/sizelatency costs instead of reusing the throughput numbers
2022-09-04 17:59:08 +01:00
Simon Pilgrim
80d4b3a275 Revert rG06e73626cf0fc33b025a0f98f1eee4a302279982 "[CostModel][X86] getTypeBasedIntrinsicInstrCost - convert to CostKindTblEntry"
Some arm buildbots are complaining about a phase ordering test failure in unsigned-multiply-overflow-check.ll - I guess this test needs making x86 specific first
2022-09-04 17:51:11 +01:00
Simon Pilgrim
06e73626cf [CostModel][X86] getTypeBasedIntrinsicInstrCost - convert to CostKindTblEntry
Begin the refactoring to use CostKindTblEntry and return real latency/codesize/sizelatency costs instead of reusing the throughput numbers
2022-09-04 17:28:45 +01:00
Benjamin Kramer
95db37320d [OpenMP] Silence unused variable warning in release builds
llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp:3397:27: warning: unused variable 'SV' [-Werror]
  for (const llvm::Value *SV : StoreValues)
                            ^
2022-09-04 15:19:48 +02:00
Simon Pilgrim
59dbd6a0cf [CostModel][X86] Remove redundant AVX512 v64i8 shift costs
These are handled earlier (and more accurately) in AVX512BWShiftCostTable
2022-09-04 14:06:26 +01:00
Simon Pilgrim
c444af1c20 [CostModel][X86] Add CostKinds handling for mul ops
This was achieved using the 'cost-tables vs llvm-mca' script D103695

Also fix a missing pmullw v16i16 half-rate throughput as znver1 double-pumps - matches numbers from AMD SoG + Agner
2022-09-04 11:59:05 +01:00
Kazu Hirata
7d8c2d17eb [llvm] Use range-based for loops (NFC)
Identified with modernize-loop-convert.
2022-09-03 23:27:25 -07:00
Kazu Hirata
03c3c2db10 [llvm] Use std::remove_reference_t (NFC) 2022-09-03 23:27:22 -07:00
Fangrui Song
9fc679b87c [SanitizerCoverage] Simplify pc-table and improve test. NFC 2022-09-03 14:29:21 -07:00
Kazu Hirata
86e8164a8f [llvm] Qualify auto in range-based for loops (NFC)
Identified with readability-qualified-auto.
2022-09-03 11:17:49 -07:00
Kazu Hirata
32aa35b504 Drop empty string literals from static_assert (NFC)
Identified with modernize-unary-static-assert.
2022-09-03 11:17:47 -07:00
Kazu Hirata
9eca5ed790 [llvm] Use std::enable_if_t (NFC) 2022-09-03 11:17:44 -07:00
Kazu Hirata
fedc59734a [llvm] Use range-based for loops (NFC) 2022-09-03 11:17:40 -07:00
Kazu Hirata
ff2fe7b829 Use llvm::upper_bound (NFC) 2022-09-03 11:17:39 -07:00
Kazu Hirata
89f1433225 Use llvm::lower_bound (NFC) 2022-09-03 11:17:37 -07:00
Kazu Hirata
3850edd9e0 Use llvm::count_if (NFC) 2022-09-03 11:17:35 -07:00
Kazu Hirata
bc96b36a41 [CodeGen] Use std::lcm (NFC) 2022-09-03 11:17:33 -07:00
Simon Pilgrim
444685de06 [CostModel][X86] Adjust mul v4i32/v8i32 throughput cost
Based off the numbers from AMD SoG + Agner - vXi32 are both half-rate, and znver1 double-pumps the v8i32 op

We should have caught this earlier as many Intel models have half-rate pmulld already :-(
2022-09-03 18:45:08 +01:00
Simon Pilgrim
114b7762a9 [CostModel][X86] Add CostKinds handling for add/sub ops
This was achieved using the 'cost-tables vs llvm-mca' script D103695
2022-09-03 18:45:08 +01:00
Simon Pilgrim
5aee2726d8 [CostModel][X86] Add CostKinds handling for fdiv ops
This was achieved with an updated version of the 'cost-tables vs llvm-mca' script D103695

As we're using 'typical' worst case values, not all cost entries come from a single CPU - e.g. the latency/throughput from haswell but the size-latency(uops) from zen1/alderlake-e due to 'double pumping'

As the uop count (used for TCK_SizeAndLatency) for divss/divps is typically so low, we need to override isExpensiveToSpeculativelyExecute to ensure we keep fdiv calls behind branches - although for some very recent cpu targets it might not be necessary any more and could be relaxed.
2022-09-03 15:48:39 +01:00
Sanjay Patel
22e1f66f26 [SCCP] add helper function for replacing signed operations; NFC
Preliminary refactoring for planned enhancement in D133198.
2022-09-03 10:30:10 -04:00
Simon Pilgrim
bddbd408b7 [X86] Fix fdiv throughput/latency/uops counts
Matches znver1/2 numbers from AMD SoG + Agner - no additional uops for folded instructions and znver1 double pumps 256-bit vectors

Matches skylake/icelake throughput numbers from Intel AoM + Agner/instlatx64

Noticed while adding fdiv CostKinds support
2022-09-03 15:23:46 +01:00
Sanjay Patel
5c759edc57 [InstCombine] reduce another or-xor bitwise logic pattern
~(A & ?) | (A ^ B) --> ~((A & ?) & B)
https://alive2.llvm.org/ce/z/mxex6V

This is similar to 9d218b61cc50 where we peeked through
another logic op to find a common operand.
2022-09-03 09:32:08 -04:00
Simon Pilgrim
1c12e12111 [CostModel][X86] Add fdiv(double) throughput x87 costs for 2022-09-03 14:08:25 +01:00
Simon Pilgrim
62cdfdab4d [DAG] canCreateUndefOrPoison - add freeze(insert_subvector(x,y,c)) -> insert_subvector(freeze(x),freeze(y),c) support
We already have plenty of assertions in place to ensure that the insertion index is constant and inrange
2022-09-03 13:41:33 +01:00
Simon Pilgrim
e2d140e9c3 [TTI] Add isExpensiveToSpeculativelyExecute wrapper
CGP uses a raw `getInstructionCost(I, TargetTransformInfo::TCK_SizeAndLatency) >= TCC_Expensive` check to see if its better to move an expensive instruction used in a select behind a branch instead.

This is causing issues with upcoming improvements to TCK_SizeAndLatency costs on X86 as we need to use TCK_SizeAndLatency as an uop count (so its compatible with various target-specific buffer sizes - see D132288), but we can have instructions that have a low TCK_SizeAndLatency value but should still be treated as 'expensive' (FDIV for example) - by adding a isExpensiveToSpeculativelyExecute wrapper we can keep the current behaviour but still add an x86 override in a future patch when the cost tables are updated to compensate.
2022-09-03 13:12:22 +01:00
Simon Pilgrim
bd956b7db3 [X86] Fix fmul throughput/latency/uops counts
Matches numbers from AMD SoG + Agner - should always be on FPU Pipes 0+1, no additional uops for folded instructions and znver1 double pumps 256-bit vectors and is always latency = 4cy for f64 multiplies

Noticed while adding fmul CostKinds support to the x86 cost models in rG0735200e3f50 and znver1 wasn't being flagged as requiring 2uop for 256-bit vectors
2022-09-03 11:10:51 +01:00
Simon Pilgrim
0735200e3f [CostModel][X86] Add CostKinds handling for fmul ops
This was achieved with an updated version of the 'cost-tables vs llvm-mca' script D103695

As we're using 'typical' worst case values, not all cost entries come from a single CPU - e.g. the latency/throughput from haswell but the size-latency(uops) from zen1/alderlake-e due to 'double pumping'
2022-09-03 10:42:20 +01:00
Simon Pilgrim
82090cb85e [CostModel][X86] Remove unused float x87 costs
We only need the double costs for SSE1 fallback
2022-09-03 09:59:20 +01:00