2382 Commits

Author SHA1 Message Date
Boyao Wang
697beb3f17
[TargetLowering] Change getOptimalMemOpType and findOptimalMemOpLowering to take LLVM Context (#147664)
Add LLVM Context to getOptimalMemOpType and findOptimalMemOpLowering. So
that we can use EVT::getVectorVT to generate EVT type in
getOptimalMemOpType.

Related to [#146673](https://github.com/llvm/llvm-project/pull/146673).
2025-07-10 11:11:09 +08:00
Matt Arsenault
deade03910
ARM: Unconditionally set eabi libcall calling convs in RuntimeLibcalls (#146083)
This fully consolidates all the calling convention configuration into
RuntimeLibcallInfo. I'm assuming that __aeabi functions have a universal
calling convention, and on other ABIs just don't use them. This will
enable splitting of RuntimeLibcallInfo into the ABI and lowering component.
2025-07-09 17:16:48 +09:00
Matt Arsenault
dc69b00b0a
RuntimeLibcalls: Remove table of soft float compare cond codes (#146082)
Previously we had a table of entries for every Libcall for
the comparison to use against an integer 0 if it was a soft
float compare function. This was only relevant to a handful of
opcodes, so it was wasteful. Now that we can distinguish the
abstract libcall for the compare with the concrete implementation,
we can just directly hardcode the comparison against the libcall
impl without this configuration system.
2025-07-09 17:13:58 +09:00
Matt Arsenault
dd9646565e ARM: Move sjlj libcall configuration to RuntimeLibcalls
Manually submitting, closes #147227
2025-07-08 13:52:32 +09:00
Matt Arsenault
591b0d0fdf
RuntimeLibcalls: Associate calling convention with libcall impls (#144979)
Instead of associating the libcall with the RTLIB::Libcall, put it
into a table indexed by the RTLIB::LibcallImpl. The LibcallImpls
should contain all ABI details for a particular implementation, not
the abstract Libcall. In the future the wrappers in terms of the
RTLIB::Libcall should be removed.
2025-07-08 10:20:52 +09:00
Matt Arsenault
2bd31edd57 ARM: Remove subtarget field tracking SjLj
This is a module level property that needs to be globally
consistent, so it does not belong in the subtarget.

Now that the Triple knows the default exception handling type,
consolidate the interpretation of None as select target default
exception handling in TargetMachine and use that. This enables
moving the configuration of UNWIND_RESUME to RuntimeLibcalls.

Manually submitting, closes #147226
2025-07-08 09:56:34 +09:00
Jay Foad
17d6aa01ec
[ARM] Fix expansion of ABS in a call sequence (#147270)
Fixes #147162
2025-07-07 15:52:37 +01:00
Matt Arsenault
d8ef156379
DAG: Remove verifyReturnAddressArgumentIsConstant (#147240)
The intrinsic argument is already marked with immarg so non-constant
values are rejected by the IR verifier.
2025-07-07 16:28:47 +09:00
AZero13
7d65cb1952
[ARM] Copy (SELECT_CC setgt, iN lhs, -1, 1, -1) -> (OR (ASR lhs, N-1), 1 from AArch64 to ARM (#146561)
It works perfectly for ARM too.
2025-07-05 18:17:33 +01:00
David Green
9fcea2e465 [ARM] Add neon vector support for roundeven
As per #142559, this marks froundeven as legal for Neon and upgrades the
existing arm.neon.vrintn intrinsics.
2025-07-04 15:27:33 +01:00
David Green
ec35065789 [ARM] Add neon vector support for rint
As per #142559, this marks frint as legal for Neon and upgrades the existing
arm.neon.vrintx intrinsics.
2025-07-03 21:27:48 +01:00
David Green
1f8f477bd0 [ARM] Add neon vector support for trunc
As per #142559, this marks ftrunc as legal for Neon and upgrades the existing
arm.neon.vrintz intrinsics.
2025-07-03 07:41:13 +01:00
David Green
5332534b9c [ARM] Add neon vector support for ceil
As per #142559, this marks fceil as legal for Neon and upgrades the existing
arm.neon.vrintp intrinsics.
2025-07-01 15:41:10 +01:00
David Green
6bd9ff04af [ARM] Add neon vector support for round
As per #142559, this marks fround as legal for Neon and upgrades the existing
arm.neon.vrinta intrinsics.
2025-06-30 17:15:26 +01:00
David Green
dcc9e36b18
[ARM] Add neon vector support for floor (#142559)
This marks ffloor as legal providing that armv8 and neon is present (or
fullfp16 for the fp16 instructions). The existing arm_neon_vrintm
intrinsics are auto-upgraded to llvm.floor.

If this is OK I will update the other vrint intrinsics.
2025-06-29 11:37:16 +01:00
Jie Fu
feb61f5b05 [Target] Prevent copying in loop variables (NFC)
/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.cpp:2769:19: error: loop variable '[Reg, N]' creates a copy from type 'std::pair<unsigned int, llvm::SDValue> const' [-Werror,-Wrange-loop-construct]
  for (const auto [Reg, N] : RegsToPass) {
                  ^
/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.cpp:2769:8: note: use reference type 'std::pair<unsigned int, llvm::SDValue> const &' to prevent copying
  for (const auto [Reg, N] : RegsToPass) {
       ^~~~~~~~~~~~~~~~~~~~~
                  &
/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.cpp:2954:19: error: loop variable '[Reg, N]' creates a copy from type 'std::pair<unsigned int, llvm::SDValue> const' [-Werror,-Wrange-loop-construct]
  for (const auto [Reg, N] : RegsToPass)
                  ^
/llvm-project/llvm/lib/Target/ARM/ARMISelLowering.cpp:2954:8: note: use reference type 'std::pair<unsigned int, llvm::SDValue> const &' to prevent copying
  for (const auto [Reg, N] : RegsToPass)
       ^~~~~~~~~~~~~~~~~~~~~
                  &
2 errors generated.
2025-06-28 19:29:00 +08:00
Kazu Hirata
094a7087b8
[Target] Use range-based for loops (NFC) (#146198) 2025-06-27 22:07:58 -07:00
Matt Arsenault
4243e502c1
ARM: Add runtime libcall definitions for aebi memory functions (#144974)
Fix bypassing ordinary RuntimeLibcalls APIs for cases handled
in ARMSelectionDAGInfo
2025-06-27 17:43:46 +09:00
Matt Arsenault
b88e1f6a79
TableGen: Generate enum for runtime libcall implementations (#144973)
Work towards separating the ABI existence of libcalls vs. the
lowering selection. Set libcall selection through enums, rather
than through raw string names.
2025-06-27 17:40:43 +09:00
David Green
888f84f72c
[ARM] Return the correct chain when expanding READ_REGISTER (#145237)
This prevents it CSEing multiple nodes together from "volatile"
registers as they would end up with the same chain. The new chain out
should be the chain from the new READ_REGISTER node.

Fixes #144845
2025-06-25 07:08:46 +01:00
Matt Arsenault
73e4f8a71f
ARM: Use member initializer list (#145459) 2025-06-24 17:47:34 +09:00
Matt Arsenault
48155f93dd
CodeGen: Emit error if getRegisterByName fails (#145194)
This avoids using report_fatal_error and standardizes the error
message in a subset of the error conditions.
2025-06-23 16:33:35 +09:00
Matt Arsenault
91439817e8
ARM: Avoid using isTarget wrappers around Triple predicates (#144705)
These are module level properties, and querying them through
a function-level subtarget context is confusing. Plus we don't
need an aliased name. This doesn't avoid all the uses, just the
ones in the TargetLowering constructor.
2025-06-20 09:34:35 +09:00
Matt Arsenault
5bee2c34bd
RuntimeLibcalls: Pass in FloatABI and EABI type (#144691)
We need the full set of ABI options to accurately compute
the full set of libcalls. This partially resolves missing
information required to compute the set of ARM calls.
2025-06-19 19:02:42 +09:00
Matt Arsenault
874a02f05b
ARM: Move ABI helpers from Subtarget to TargetMachine (#144680)
These are module level concepts, and attaching them to the
function level subtarget is confusing. Similarly these other
helpers that only operate on the triple should also be removed
from the subtarget.
2025-06-19 09:38:22 +09:00
Matt Arsenault
97bfb936af
DAG: Move soft float predicate management into RuntimeLibcalls (#142905)
Work towards making RuntimeLibcalls the centralized location for
all libcall information. This requires changing the encoding from
tracking the ISD::CondCode to using CmpInst::Predicate.
2025-06-17 09:42:53 +09:00
Matt Arsenault
aac603c478
ARM: Avoid repeating hardcoded windows division libcall names (#143834)
This is properly set in the runtime libcall info, so query
the name.
2025-06-12 21:20:45 +09:00
Matt Arsenault
3550662c04
ARM: Avoid using getTargetLowering in TargetLowering (#143833)
This is this.
2025-06-12 21:17:48 +09:00
Matt Arsenault
4079ed3c9e
ARM: Move setting of more runtime libcalls to RuntimeLibcallInfo (#143826)
These are the easy cases that do not really depend on the subtarget,
other than for the deceptive predicates on the subtarget class. Most
of the rest of the cases here also do not, but this is obscured by
going through helper predicates added onto the subtarget which hide
dependence on TargetOptions.
2025-06-12 17:35:55 +09:00
Matt Arsenault
505c550e4c
DAG: Assert fcmp uno runtime calls are boolean values (#142898)
This saves 2 instructions in the ARM soft float case for fcmp ueq.

This code is written in an confusingly overly general way. The point
of getCmpLibcallCC is to express that the compiler-rt implementations
of the FP compares are different aliases around functions which may
return -1 in some cases. This does not apply to the call for unordered,
which returns a normal boolean.

Also stop overriding the default value for the unordered compare for ARM.
This was setting it to the same value as the default, which is now assumed.
2025-06-10 10:46:29 +09:00
Matt Arsenault
a082f665f8
ARM: Start moving runtime libcall configuration out of TargetLowering (#142617)
These Module level triple checks implemented in the Subtarget are kind of
a pain and we should probably get rid of all of them in across all targets,
and stick to a consistent set of names.
2025-06-10 10:00:04 +09:00
Matt Arsenault
4455d9d3e3
RuntimeLibcalls: Use iterable enum for libcall types (#143075) 2025-06-06 20:58:18 +09:00
Matt Arsenault
18e94550e1
ARM: Remove unused CondCode field from libcall table (#142616) 2025-06-03 23:02:01 +02:00
David Green
7a688c080f [ARM] Add vector vrint tests and fix FP16 to expand. 2025-05-31 12:21:46 +01:00
Folkert de Vries
46b389218b
[ARM]: codegen llvm.roundeven.v* (#141786)
fixes https://github.com/llvm/llvm-project/issues/73588

The aarch64 version of `frintn.ll` notes the intention to auto-upgrade
`frintn` to `roundeven`. I haven't been able to figure out how to make
that happen though (either for arm or aarch64).

The original issue came up in
https://github.com/rust-lang/stdarch/pull/1807
2025-05-30 19:36:29 +01:00
Marius Kamp
10647685ca
[SDAG] Make Select-with-Identity-Fold More Flexible; NFC (#136554)
This change adds new parameters to the method
`shouldFoldSelectWithIdentityConstant()`. The method now takes the
opcode of the select node and the non-identity operand of the select
node. To gain access to the appropriate arguments, the call of
`shouldFoldSelectWithIdentityConstant()` is moved after all other checks
have been performed. Moreover, this change adjusts the precondition of
the fold so that it would work for `SELECT` nodes in addition to
`VSELECT` nodes.
    
No functional change is intended because all implementations of
`shouldFoldSelectWithIdentityConstant()` are adjusted such that they
restrict the fold to a `VSELECT` node; the same restriction as before.
    
The rationale of this change is to make more fine grained decisions
possible when to revert the InstCombine canonicalization of
`(select c (binop x y) y)` to `(binop (select c x idc) y)` in the
backends.
2025-05-29 09:46:39 +02:00
Justin Bogner
b7bb256703
Warn on misuse of DiagnosticInfo classes that hold Twines (#137397)
This annotates the `Twine` passed to the constructors of the various
DiagnosticInfo subclasses with `[[clang::lifetimebound]]`, which causes
us to warn when we would try to print the twine after it had already
been destructed.

We also update `DiagnosticInfoUnsupported` to hold a `const Twine &`
like all of the other DiagnosticInfo classes, since this warning allows
us to clean up all of the places where it was being used incorrectly.
2025-05-28 12:26:39 -07:00
Rahul Joshi
52c2e45c11
[NFC][CodeGen] Adopt MachineFunctionProperties convenience accessors (#141101) 2025-05-23 08:30:29 -07:00
Craig Topper
dcd62f3674
[SelectionDAG] Rename MemSDNode::getOriginalAlign to getBaseAlign. NFC (#139930)
This matches the underlying function in MachineMemOperand and how it is
printed when BaseAlign differs from Align.
2025-05-16 09:37:02 -07:00
Kazu Hirata
18ecff4f65
[llvm] Use llvm::stable_sort (NFC) (#140067) 2025-05-15 12:18:18 -07:00
Kazu Hirata
58014a506d
[IR] Teach getConstraintString to return StringRef (NFC) (#139401)
With this change, some callers get to use StringRef::starts_with.

I'm planning to teach getAsmString to return StringRef also, but
I'ld like to keep that separate from this patch.
2025-05-10 13:00:47 -07:00
Kazu Hirata
aa15596b5f
[llvm] Remove unused local variables (NFC) (#138478) 2025-05-04 21:33:54 -07:00
Sergei Barannikov
7af555e524
[ARM][RISCV] Partially revert #101786 (#137120)
The change as is breaks the Linux kernel build as pointed out in the
comments.
2025-04-24 10:13:05 +03:00
Sergei Barannikov
11a3de7e98
[SDag][ARM][RISCV] Allow lowering CTPOP into a libcall (#101786)
This is a reland of #99752 with the bug fixed (see test diff in the
third commit in this PR).
All `popcount` libcalls return `int`, but `ISD::CTPOP` returns the type
of the argument, which can be wider than `int`. The fix is to make DAG
legalizer pass the correct return type to `makeLibCall` and sign-extend
the result afterwards.

Original commit message:
The main change is adding CTPOP to `RuntimeLibcalls.def` to allow
targets to use LibCall action for CTPOP. DAG legalizers are changed
accordingly.

Pull Request: https://github.com/llvm/llvm-project/pull/101786
2025-04-23 12:43:05 +03:00
Rahul Joshi
74b7abf154
[IRBuilder] Add new overload for CreateIntrinsic (#131942)
Add a new `CreateIntrinsic` overload with no `Types`, useful for
creating calls to non-overloaded intrinsics that don't need additional
mangling.
2025-03-31 08:10:34 -07:00
David Green
2e54b4f9ea [ARM] Silence signed comparison warning. NFC
After f4ec179bf5295f92aa0346392a58fad54f9b458e, AbsImm is no longer signed and cannot be < 0.
2025-03-31 13:59:58 +01:00
Kazu Hirata
fac8fe9cf9
[Target] Use *Set::insert_range (NFC) (#132879)
We can use *Set::insert_range to collapse:

  for (auto Elem : Range)
    Set.insert(E);

down to:

  Set.insert_range(Range);

In some cases, we can further fold that into the set declaration.
2025-03-24 22:42:04 -07:00
Kazu Hirata
1b189cab5e
[llvm] Use *Set::insert_range (NFC) (#132509)
DenseSet, SmallPtrSet, SmallSet, SetVector, and StringSet recently
gained C++23-style insert_range.  This patch uses insert_range in
conjunction with llvm::{predecessors,successors} and
MachineBasicBlock::{predecessors,successors}.
2025-03-22 08:07:33 -07:00
Simon Tatham
f4ec179bf5
[ARM] Fix undefined behavior in isLegalAddImmediate (#132219)
Building clang under UBsan, it reported an integer overflow in this
function when the input value was -2^63, because it's UB to pass the
maximum negative value of an integer type to `std::abs`.

Fixed by adding a new absolute-value function in `MathExtras.h` whose
return type is the unsigned version of the argument type, and using that
instead.

(This seems like the kind of thing C++ should have already had, but
apparently it doesn't, and I couldn't find an existing one in LLVM
Support either.)
2025-03-21 09:03:50 +00:00
Craig Topper
d7879e524f
[ARM] Use DenseSet instead of DenseMap. NFC (#131978)
The value in the map is set to "true" when something is added to the
map.

Techncally this:

  if (!DefRegs.contains(Reg))

will set the value in the map to false if it didn't already exist, and
this is used to indicate the value wasn't in the map. This only occurs
after all the "true" values have already been added to the map.
2025-03-19 08:32:09 -07:00