Required for mingw-w64, which uses the alias attribute in its CRT.
Follows ARM64EC mangling rules by mangling the alias symbol and emitting
an unmangled anti-dependency alias. Since metadata is not allowed on
GlobalAlias objects, extend arm64ec_unmangled_name to support multiple
unmangled names and attach the alias anti-dependency name to the target
function's metadata.
This MIPS behavior from edb9d84dcc4824865e86f963e52d67eb50dde7f5 (2010)
is obsoleted and misleading. This caused confusion in
https://reviews.llvm.org/D123702 ([NVPTX] Disable parens for identifiers
starting with '$')
Note: $tmp was rejected by AsmParser before
https://reviews.llvm.org/D75111 (2020)
This is a follow-up patch of
https://github.com/llvm/llvm-project/pull/125756
In this PR, static-data-splitter pass produces the aggregated profile
counts of constants for constant pools in a global state
(`StateDataProfileInfo`), and asm printer consumes the profile counts to
produce `.hot` or `.unlikely` prefixes.
This implementation covers both x86 and aarch64 asm printer.
After outputting block scalar string, the indent will be wrong.
This patch fixes Padding after block scalar string to ensure the correct
format of yaml.
The new added ut will fail in main.
```diff
@@ -3,4 +3,4 @@
Just a block
scalar doc
-scalar: a
+ scalar: a
...\n
```
GCC, correctly, doesn't vectorize in this case. Absence of direct
instructions to convert larger fixed point to lower floating point
precision inadvertently causes rounding leading to subtle differences
across ISAs.
https://godbolt.org/z/ssEchMWrE
Co-authored by: @echristo
This pass is designed to increase ILP by performing accumulation into
multiple registers. It currently supports only the S/UABAL accumulation
instruction, but can be extended to support additional instructions.
Reland of #126060 which was reverted due to a conflict with #131272.
This caused ZT0 to be preserved around `__arm_tpidr2_save` in functions
with "aarch64_new_zt0". The block in which `__arm_tpidr2_save` is called
is added by the SMEABIPass and may be reachable in cases where ZA has
not been enabled* (so using `str zt0` is invalid).
* (when za_save_buffer is null and num_za_save_slices is zero)
This reapplies original commit e122483762b44c7f4386165099ff2a404705d7d4
Second try, but with win64 removed from the scope, since it was somehow
broken by original commit.
### Original PR description
If vaargs are not used there is no need to save them. LLVM already
implements such optimization for x86, as well as gcc [1].
Some ABI tests are kept almost as-is, except for stack offsets, by just
adding llvm.va_start. Only laapcs_vararg_frame.ll test was rewritten to
match new behavior.
[1] https://godbolt.org/z/GWWKr8xMd
Previously, the `SHF_AARCH64_PURECODE` section flag wasn't added to the
implicitly created `.text` section if the module didn't contain any
functions, because no other section had the flag set.
Now, the `SHF_AARCH64_PURECODE` is always added if the "+execute-only"
target feature is set for the module during compilation.
This pattern shows up often in media libraries. The optimization should only
kick in for O3. Currently only supports a single family of accumulation
instructions, but can easily be expanded to support additional
instructions in the future.
There was a case where `normalizeNonPipelinedInstructions` didn't
schedule unpipelineable instructions correctly, which could generate
illegal code. This patch fixes this issue by rejecting the schedule if
we fail to insert the unpipelineable instructions at stage 0.
Here is a part of the debug output for `sms-unpipeline-insts3.mir`
before applying this patch.
```
SU(0): %27:gpr32 = PHI %21:gpr32all, %bb.3, %28:gpr32all, %bb.4
Successors:
SU(14): Data Latency=0 Reg=%27
SU(15): Anti Latency=1
...
SU(14): %41:gpr32 = ADDWrr %27:gpr32, %12:gpr32common
Predecessors:
SU(0): Data Latency=0 Reg=%27
SU(16): Ord Latency=0 Artificial
Successors:
SU(15): Data Latency=1 Reg=%41
SU(15): %28:gpr32all = COPY %41:gpr32
Predecessors:
SU(14): Data Latency=1 Reg=%41
SU(0): Anti Latency=1
SU(16): %30:ppr = WHILELO_PWW_S %27:gpr32, %15:gpr32, implicit-def $nzcv
Predecessors:
SU(0): Data Latency=0 Reg=%27
Successors:
SU(14): Ord Latency=0 Artificial
...
Do not pipeline SU(16)
Do not pipeline SU(1)
Do not pipeline SU(0)
Do not pipeline SU(15)
Do not pipeline SU(14)
SU(0) is not pipelined; moving from cycle 19 to 0 Instr: ...
SU(1) is not pipelined; moving from cycle 10 to 0 Instr: ...
SU(15) is not pipelined; moving from cycle 28 to 19 Instr: ...
SU(16) is not pipelined; moving from cycle 19 to 0 Instr: ...
Schedule Found? 1 (II=10)
...
cycle 9 (1) (14) %41:gpr32 = ADDWrr %27:gpr32, %12:gpr32common
cycle 9 (1) (15) %28:gpr32all = COPY %41:gpr32
```
The SUs are traversed in the order of the original basic block, so in
this case a new cycle of each instruction is determined in the order of
`SU(0)`, `SU(1)`, `SU(14)`, `SU(15)`, `SU(16)`. Since there is an
artificial dependence from `SU(16)` to `SU(14)`, which is contradict to
the original SU order, the new cycle of `SU(14)` must be greater than or
equal to the cycle of `SU(16)` at that time. This results in the failure
of scheduling `SU(14)` at stage 0. For now, we reject the schedule for
such cases.
The aarch64-cond-br-tuning pass transforms a CBZX instruction into a
conditional branch (B.cond). One of the by products of the
transformation is that the source instruction of the CBZX, which is an
ANDXri instruction, gets transformed into a ANDSXri instruction, however
this transformation doesn't preserve it's debug instruction number.
This patch fixes that issue.
The general predicates of the PTEST and PTEST_like instructions may
belong to different register classes. This can lead to the insertion of
a COPY instruction, making them appear different. However, for the
purpose of PTEST removal, such copies are irrelevant, and we can look
through them to improve the likelihood of finding a match.
Similar to other operations, s8, s16 s32 and s64 vector elements are
clamped to legal vector sizes, odd number of elements are widened to the
next power-2 and s128 is scalarized.
This helps legalize cttz as well as ctpop.
Like other operations larger than i64, we scalarize i128 and allow them to
legalize from there. This also helps with v2i64 udiv by constant, which needs a
legalize a umulh.
Similar to other operations, s8, s16 and s32 vector elements are clamped
to legal vector sizes, but in this case s64 are scalarized to use the
gpr instructions. This allows vector types to split as opposed to
scalarizing.
Following the discussion on #131174, update generate-tests.py script to
emit atomicrmw tests where the result is unused and add a note to
explain why these do use ST[F]ADD.
When attempting to perform complex deinterleaving on an unrolled loop
containing a reduction, the complex deinterleaving pass would fail to
accommodate the wider types when accumulating the unrolled paths.
Instead of trying to alter the incoming IR to fit expectations, the pass
should instead decide against processing any reduction that results in a
non-complex or non-vector value.
The pattern
```LLVM
%shl = shl i32 %x, 31
%ashr = ashr i32 %shl, 31
```
would be combined to `SIGN_EXTEND_INREG %x, ValueType:ch:i1` by
SelectionDAG.
However InstCombine normalizes this pattern to:
```LLVM
%and = and i32 %x, 1
%neg = sub i32 0, %and
```
This adds matching code to DAGCombiner to catch this variant as well.
These date back to when the non-intrinsic format of variable locations
was still being tested and was behind a compile-time flag, so not all
builds / bots would correctly run them. The solution at the time, to get
at least some test coverage, was to have tests opt-in to non-intrinsic
debug-info if it was built into LLVM.
Nowadays, non-intrinsic format is the default and has been on for more
than a year, there's no need for this flag to exist.
(I've downgraded the flag from "try" to explicitly requesting
non-intrinsic format in some places, so that we can deal with tests that
are explicitly about non-intrinsic format in their own commit).
This is meant as a preparation for PR #130988 "[AMDGPU] Implement IR
expansion for frem instruction" which implements the expansion of
another instruction in this pass. The more general name seems more
appropriate given this change and quite reasonable even without it.
EnableTailMerge is false by default and is handled by the pass builder.
Passes are independent of target pipeline options.
This completes the generic `MachineLateOptimization` passes for the NPM
pipeline.
There were two issues when `/guard:ehcont` is enabled with the SEH
personality on Windows:
1. As @namazso correctly identified, we bail out of
`WinException::endFunction` early for `MSVC_TableSEH` with funclets,
expecting the exception data to be emitted in `endFunclet`, but
`endFunclet` didn't copy the EHCont metadata from the function to the
module.
2. The SEH personality requires that the basic block containing the
`catchpad` is the target, not the `catchret`.
Fixes#64585
When we have a floating-point operation that a target doesn't support
for a given type, but does support for a wider type, then there are two
ways this can be handled:
* If the target doesn't have any registers at all of this type then
LegalizeTypes will convert the operation.
* If we do have registers but no operation for this type, then the
operation action will be Promote and it's handled in PromoteNode.
In both cases the operation at the wider type, and the conversion
operations to and from that type, should have the same fast math flags
as the original operation.
This is being done in preparation for a DAGCombine patch which makes use
of these fast math flags.
When compiling VLS SVE, the compiler often replaces VL-based offsets
with immediate-based ones. This leads to a mismatch in the allowed
addressing modes due to SVE loads/stores generally expecting immediate
offsets relative to VL. For example, given:
```c
svfloat64_t foo(const double *x) {
svbool_t pg = svptrue_b64();
return svld1_f64(pg, x+svcntd());
}
```
When compiled with `-msve-vector-bits=128`, we currently generate:
```gas
foo:
ptrue p0.d
mov x8, #2
ld1d { z0.d }, p0/z, [x0, x8, lsl #3]
ret
```
Instead, we could be generating:
```gas
foo:
ldr z0, [x0, #1, mul vl]
ret
```
Likewise for other types, stores, and other VLS lengths.
This patch achieves the above by extending `SelectAddrModeIndexedSVE`
to let constants through when `vscale` is known.
ld64 doesn't currently support the PAGEOFF relocations on anything but load/stores
so we need to bail-out here to fix the build failures on greendragon.
rdar://145495288
