To deduce whether the optimization is legal we need to compare the target
features between caller and callee versions. The criteria for bypassing
the resolver are the following:
* If the callee's feature set is a subset of the caller's feature set,
then the callee is a candidate for direct call.
* Among such candidates the one of highest priority is the best match
and it shall be picked, unless there is a version of the callee with
higher priority than the best match which cannot be picked from a
higher priority caller (directly or through the resolver).
* For every higher priority callee version than the best match, there
is a higher priority caller version whose feature set availability
is implied by the callee's feature set.
Example:
Callers and Callees are ordered in decreasing priority.
The arrows indicate successful call redirections.
Caller Callee Explanation
=========================================================================
mops+sve2 --+--> mops all the callee versions are subsets of the
| caller but mops has the highest priority
|
mops --+ sve2 between mops and default callees, mops wins
sve sve between sve and default callees, sve wins
but sve2 does not have a high priority caller
default -----> default sve (callee) implies sve (caller),
sve2(callee) implies sve (caller),
mops(callee) implies mops(caller)
This commit relands the changes from "[LV]: Teach LV to recursively
(de)interleave. #89018"
Reason for revert:
- The patch exposed a bug in the IA pass, the bug is now fixed and landed by commit: #122643
This is already defined for each register class in AArch64RegisterInfo,
not hardcoding it here makes these values easier to change (perhaps
based on hardware mode).
Should the operands of `omp.atomic.read` differ, emit an implicit cast.
In case of `struct` arguments, extract the 0-th index, emit an implicit
cast if required, and store at the destination.
Fixes https://github.com/llvm/llvm-project/issues/112908
This commit add relax relocations for `tls_le` code sequence.
Handwritten assembly and generating source code by clang are both
affected.
Scheduled `tls_le` code sequence can be relaxed normally and we can add
relax relocs when code emitting according to their relocs. Other
relaxable macros' code sequence cannot simply add relax relocs according
to their relocs, such as `PCALA_{HI20/LO12}`, we do not want to add
relax relocs when code model is large. This will be implemented in later
commit.
Emit `R_LARCH_RELAX` relocations when expanding some macros, including:
- `la.tls.ie`, `la.tls.ld`, `la.tls.gd`, `la.tls.desc`,
- `call36`, `tail36`.
Other macros that need to emit `R_LARCH_RELAX` relocations was
implemented in https://github.com/llvm/llvm-project/pull/72961, including:
- `la.local`, `la.pcrel`, `la.pcrel` expanded as `la.abs`, `la`,
`la.global`, `la/la.global` expanded as `la.pcrel`, `la.got`.
Note: `la.tls.le` macro can be relaxed when expanded with
`R_LARCH_TLS_LE_{HI20/ADD/LO12}_R` relocations. But if we do so,
previously handwritten assembly code will occur error due to the
redundant `add.{w/d}` followed by `la.tls.le`. So `la.tls.le` keeps to
expands with `R_LARCH_TLS_LE_{HI20/LO12}`.
When iterating over function records, filtered by file name, currently,
the iteration goes over all the function records, repeatedly for each
source file, essentially giving quadratic behavior.
413647d730972eac9675f695c2ea63fb393a5531 sped up some cases by keeping
track of the indices of the function records corresponding to each file
name. This change expands the use of that map to FunctionRecordIterator.
On a test case with Firefox's libxul.so and a 2.5MB profile, this brings
down the runtime of `llvm-cov export $lib --instr-profile $prof -t lcov`
from 12 minutes with 90% spent in skipOtherFiles to 19 seconds with no
samples in skipOtherFiles at all under a sampling profiler (with a
sampling interval of 1ms).
Fixes#62079
This patch fixes:
llvm/lib/Target/AMDGPU/SIISelLowering.cpp:13908:46: error:
comparison of integers of different signs: 'uint32_t' (aka 'unsigned
int') and 'int' [-Werror,-Wsign-compare]
The intention is to use a "copy" instead of a "sub" to handle the high
parts of 64-bit multiply for this specific case.
This unlocks copy prop use cases where the copy can be reused by later
multiply+add sequences if possible.
Fixes: SWDEV-487672, SWDEV-487669
The proposed patch, in general, tries to transform the below code
sequence:
x = 1.0 / sqrt (a);
r1 = x * x; // same as 1.0 / a
r2 = a / sqrt(a); // same as sqrt (a)
TO
(If x, r1 and r2 are all used further in the code)
r1 = 1.0 / a
r2 = sqrt (a)
x = r1 * r2
The transform tries to make high latency sqrt and div operations
independent and also saves on one multiplication.
The patch was tested with SPEC17 suite with cpu=neoverse-v2. The
performance uplift achieved was:
544.nab_r ~4%
No other regressions were observed. Also, no compile time differences
were observed with the patch.
Closes#54652
This avoids regressions in a future AMDGPU commit. Previously we
would have a build_vector (extract_vector_elt x), undef with free
access to the elements bloated into a shuffle of one element + undef,
which has much worse combine support than the extract.
Alternatively could check aggressivelyPreferBuildVectorSources, but
I'm not sure it's really different than isExtractVecEltCheap.
Once again we have excessive TLI hooks with bad defaults. Permit this
for 32-bit element vectors, which are just use-different-register.
We should permit 16-bit vectors as cheap with legal packed instructions,
but I see some mixed improvements and regressions that need investigation.
This avoids some of the pending regressions after AMDGPU implements
isExtractVecEltCheap.
In a case like shl <value, undef>, splat k, because the second operand
was fully defined, we would fall through and use the splat value for the
first operand, losing the undef high bits. This would result in an additional
instruction to handle the high bits. Add some reduced testcases for different
opcodes for one of the regressions.
The motivation for this is to allow us to match strided accesses that
are emitted from the loop vectorizer with EVL tail folding (see #122232)
In these loops the step isn't loop invariant and is based off of
@llvm.experimental.get.vector.length.
We can relax this as long as we make sure to construct the updates after
the definition inside the loop, instead of the preheader.
I presume the restriction was previously added so that the step would
dominate the insertion point in the preheader. I can't think of why it
wouldn't be safe to calculate it in the loop otherwise.
InstrMaps is a helper data structure that maps scalars to vectors and
the reverse. This is used by the vectorizer to figure out which vectors
it can extract scalar values from.
I have a particular user downstream who likes to write shuffles in terms
of unions involving _BitInt(128) types. This isn't completely crazy
because there's a bunch of code in the wild which was written with SSE
in mind, so 128 bits is a common data fragment size.
The problem is that generic lowering scalarizes this to ELEN, and we end
up with really terrible extract/insert sequences if the i128 shuffle is
between other (non-i128) operations.
I explored trying to do this via generic lowering infrastructure, and
frankly got lost. Doing this a target specific DAG is a bit ugly -
really, there's nothing hugely target specific here - but oh well. If
reviewers prefer, I could probably phrase this as a generic DAG combine,
but I'm not sure that's hugely better. If reviewers have a strong
preference on how to handle this, let me know, but I may need a bit of
help.
A couple notes:
* The argument passing weirdness is due to a missing combine to turn a
build_vector of adjacent i64 loads back into a vector load. I'm a bit
surprised we don't get that, but the isel output clearly has the
build_vector at i64.
* The splat case I plan to revisit in another patch. That's a relatively
common pattern, and the fact I have to scalarize that to avoid an
infinite loop is non-ideal.
When replacing load with a select on the address with a select and 2
loads of the values, copy poison-generating metadata from the original
load to the newly created loads, which are placed at the same place as
the original loads. We cannot copy metadata that may trigger UB.
PR: https://github.com/llvm/llvm-project/pull/115605
Support true16 format for v_cndmask_b16 in MC and CodeGen in true16 and
fake16 flow.
Since we are replacing `v_cndmask_b16` to `v_cndmask_b16_t16/fake16`, we
have to at least update the fake16 codeGen to get codeGen test passing.
For this case, we have to update the true16 and with fake16 together,
otherwise some of the true16 tests will fail
In 1e53f9523d3d5fcb2993b4b6540f1ed8d743380b, the FileSystem.h header was
changed to always include <sys/stat.h>, while it previously only was
included if HAVE_SYS_STAT_H was defined.
HAVE_SYS_STAT_H was defined in llvm/Config/config.h, while FileSystem.h
only included llvm/Config/llvm-config.h. Thus, <sys/stat.h> was only
being included in some but not all cases.
The change to always include <sys/stat.h> broke compiling LLDB for MinGW
targets, because the MinGW <sys/stat.h> header adds an "#define fstat
_fstat64" define, which breaks LLDBs use of a struct with a member named
"fstat".
Remove the include of <sys/stat.h> in FileSystem.h, as it seems to not
be necessary in practice, fixing compilation of LLDB for MinGW targets.
Change one instance of defined(_MSC_VER) into defined(_WIN32) in
ErrorHandling.cpp to get <io.h> included; this source file did include
config.h before transitively including FileSystem.h. The include of
<sys/stat.h> in FileSystem.h would bring in <io.h> (needed for
::write()), explaining why this ifdef didn't need to cover MinGW before.
- Add support for `@llvm.exp2()`:
- LLVM: `float` -> PTX: `ex2.approx{.ftz}.f32`
- LLVM: `half` -> PTX: `ex2.approx.f16`
- LLVM: `<2 x half>` -> PTX: `ex2.approx.f16x2`
- LLVM: `bfloat` -> PTX: `ex2.approx.ftz.bf16`
- LLVM: `<2 x bfloat>` -> PTX: `ex2.approx.ftz.bf16x2`
- Any operations with non-native vector widths are expanded. On
targets not supporting f16/bf16, values are promoted to f32.
- Add *CONDITIONAL* support for `@llvm.log2()` [^1]:
- LLVM: `float` -> PTX: `lg2.approx{.ftz}.f32`
- Support for f16/bf16 is emulated by promoting values to f32.
[1]: CUDA implements `exp2()` with `ex2.approx` but `log2()` is
implemented differently, so this is off by default. To enable, use the
flag `-nvptx-approx-log2f32`.
``` - add clang builtin to Builtins.td
- link builtin in hlsl_intrinsics
- add codegen for spirv intrinsic and two directx intrinsics to retain
signedness information of the operands in CGBuiltin.cpp
- add semantic analysis in SemaHLSL.cpp
- add lowering of spirv intrinsic to spirv backend in
SPIRVInstructionSelector.cpp
- add lowering of directx intrinsics to WaveActiveOp dxil op in
DXIL.td
- add test cases to illustrate passespendent pr merges.
```
Resolves#70106
---------
Co-authored-by: Finn Plummer <canadienfinn@gmail.com>
Add Apple M4 host detection, which fixes
https://github.com/rust-lang/rust/issues/133414.
Also add support for older ARM families (this is likely never going to
get used, since only macOS is officially supported as host OS, but nice
to have for completeness sake). Error handling (checking
`CPUFAMILY_UNKNOWN`) is also included here.
Finally, add links to extra documentation to make it easier for others
to update this in the future.
NOTE: These values are taken from `mach/machine.h` the Xcode 16.2 SDK,
and has been confirmed on an M4 Max in
https://github.com/rust-lang/rust/issues/133414#issuecomment-2499123337.
This restores the functionality of AsmPrinterHandlers to what it was
prior to https://github.com/llvm/llvm-project/pull/96785. The attempted
hack there of adding a duplicate DebugHandlerBase handling added a lot
of hidden state and assumptions, which just segfaulted when we tried to
continuing using this API. Instead, this just goes back to the old
design, but adds a separate array for the basic EH handles. The
duplicate array is identical to the other array of handler, but which
doesn't get their begin/endInstruction callbacks called. This still
saves the negligible but measurable amount of virtual function calls as
was the goal of #96785, while restoring the API to the pre-LLVM-19
status quo.
For "returned" attribute arguments, the physical register is really a
virtual register which shouldn't be stored in an MCRegister. This
patch moves the conversion from Register to MCRegister into the derived
classes of IncomingArgHandler. The derived class
ReturnedArgCallReturnHandler
does not use the register so no MCRegister is created in that case.
The function and argument have been renamed to remove "Phys".
