Model zext i1 %x to in as select i1 %x, in 1, in 0 in case, if there are
other select instructions, which can be combined into a bundle.
Fixes#178403
Recommit after revert in 993e1f66afcfe9da03bd813e669eada341b11d2f
Reviewers: hiraditya, RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/180635
My initial goal was to name bits {19-15} as either rs1, vs1, or imm as
appropriate.
The VCIX instructions effectively have 3 properties that are combined:
the type of bits 19-15 (determined by funct3), how many vector registers
are sources (determined by funct6), and whether vd is a written or not
(determined by vm).
I found the VCIXInfo class very hard to read and understand. This patch
breaks it up into simpler steps and moves some properties to flags in
the existing VCIXType classs. A new VCIXRs1Info class is added to
contain the properties for bits 19-15. The VCIXInfo now combines the
properties from these 2 classes and the HaveOutputDst flag to create the
various things needed to build the final instruction. To reduce the
number of template arguments, the VCIXInfo is passed all the way down to
the base class.
Much of the refactoring in this patch was accomplished with AI. I have
looked at the diffs in the output tablegen files to verify it works as
expected. RISCVGenInstrInfo.inc only changes line number comments.
RISCVGenDisassembler.inc doesn't change at all as expected.
RISCVGenMCCodeEmitter.inc changes due to the different field names and
orders in the record.
optimizeCondBranch is looking for immediates that are 1 apart to rewrite
the branch by sharing a constant. BSETI x0, 11 can be used to produce
2048 which is one more than the largest positive constant produced by
addi.
Previously, kernel symbols in offload binaries for SYCL had to be stored
as newline-separated strings and we had to use llvm::join and
line_iterator. It was needed because Offload Binary v1 format did not
store string value sizes. It is not necessary with Offload Binary v2
format, which stores string value size and hence eliminates the need for
newline separators.
This change adds initial support to libObject for reading compact
imports and support for writing compact imports in the linker.
There is minimal testing here since to tools like lllvm-readobj, and
obj2yaml don't currently report compact imports any differently.
See https://github.com/WebAssembly/compact-import-section
Previous commit message:
>Previous commit message:
>
>> Original commit message:
>>
>>>When users explicitly specify a PTX version via -mattr=+ptxNN that's
insufficient for their target SM, we now emit a fatal error. Previously,
we silently upgraded the PTX version to the minimum required for the
target SM.
>>>
>>>When no SM or PTX version is specified, we now use PTX 3.2 (the
minimum for the default SM 3.0) instead of PTX 6.0.
>>
>>The following commits should fix the failures that arose when I
previously tried to land this commit:
>>
>>
>>9fc5fd0ad6
should address the llvm-nvptx*-nvidia-* build failures:
https://github.com/llvm/llvm-project/pull/174834#issuecomment-3742242651
>>
>>
>>600514a637
should address the MLIR failures
>
>The previous commit was reverted with
d23cb79ba4
because the
[mlir-nvidia](https://lab.llvm.org/buildbot/#/builders/138/builds/24797)
and
[mlir-nvidia-gcc7](https://lab.llvm.org/buildbot/#/builders/116/builds/23929)
Buildbots were failing.
>
>Those tests failed because MLIR's default SM was 5.0, which caused
NVPTX
to target PTX ISA v4.0, which did not support the intrinsics used in the
failing tests.
>
>243f011577
should address this by bumping MLIR's default SM to 7.5. Now, using
MLIR's new default SM, NVPTX
targets the PTX ISA v6.3, which supports the intrinsics used in the
failing tests.
---
The previous commit was reverted with
e9b578a4d77025e18318efedd0f3f3764338d859
[because](https://github.com/llvm/llvm-project/pull/179304#issuecomment-3856301333)
the clang driver set the default PTX ISA version to v4.2 when no CUDA
installation is found. However, given our patch, we should not set a
default; instead, let the LLVM backend select the appropriate PTX ISA
version for the target SM.
Windows supports short 8.3 form filenames (for example,
`compile_commands.json` -> `COMPIL~1.JSO`) for legacy reasons. See:
https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file#short-vs-long-names
Such paths are not unusual because, on Windows, the system temporary
directory is commonly derived from the `TMP`/`TEMP` environment
variables. For historical compatibility reasons, these variables are
often set to short 8.3 form paths on systems where user names exceed
eight characters.
Introduce `windows::makeLongFormPath()` to convert paths to their long
form by expanding any 8.3 components via `GetLongPathNameW`.
As part of this change:
- Extended-length path prefix handling is centralized by adding
`stripExtendedPrefix()` and reusing it in `realPathFromHandle()`.
- `widenPath()` is cleaned up to use shared prefix constants.
This was split out from #178303 at the request of the codeowner so that
the Windows support parts can be reviewed separately.
This fixes a miscompile in #180005 where we didn't check that the first
incoming value isn't poison.
We should use the first non-poison incoming value if it exists, or just
poison if all the incoming values are poison.
Since TargetTransformInfo::enableAggressiveInterleaving(bool
HasReductions) takes the HasReductions argument, the LoopVectorizer
should save its returned value in a variable called AggressivelyInterleave
instead of AggressivelyInterleaveReductions.
urem x, n: result < n (remainder is always less than divisor)
urem x, n: result <= x (remainder is at most the dividend)
udiv x, n: result <= x (quotient is at most the dividend)
https://alive2.llvm.org/ce/z/ezzsjQ
Model zext i1 %x to in as select i1 %x, in 1, in 0 in case, if there are
other select instructions, which can be combined into a bundle.
Fixes#178403
Reviewers: hiraditya, RKSimon
Pull Request: https://github.com/llvm/llvm-project/pull/180635
When a global variable has a size that exceeds the size of the address
space it resides in, the verifier should fail as the variable can
neither be materialized nor fully accessed. This patch adds a check to
the verifier to enforce it.
---------
Signed-off-by: Steffen Holst Larsen <HolstLarsen.Steffen@amd.com>
Co-authored-by: Steffen Holst Larsen <HolstLarsen.Steffen@amd.com>
Sub-reductions can be implemented in two ways:
(1) negate the operand in the vector loop (the default way).
(2) subtract the reduced value from the init value in the middle block.
Note that both ways keep the reduction itself as an 'add' reduction,
which is necessary because only llvm.vector.partial.reduce.add exists.
The ISD nodes for partial reductions don't support folding the
sub/negation into its operands because the following is not a valid
transformation:
```
sub(0, mul(ext(a), ext(b)))
-> mul(ext(a), ext(sub(0, b)))
```
It can therefore be better to choose option (2) such that the partial
reduction is always positive (starting at '0') and to do a final
subtract in the middle block.
For AArch64 there are no dot-product instructions that can
do a `partial.reduce.sub(acc, mul(ext(a), ext(b)))` operation.
I'm not sure if such instructions exist for other targets.
(If so then we may want to make this decision a target option)
This PR also increases the AArch64 cost of a partial sub-reduction
when this exists in an 'add-sub' reduction chain.
Fixes https://github.com/llvm/llvm-project/issues/178703
The first param of LoopRotatePass is EnableHeaderDuplication. The value
'true' means 'enable the header duplication'.
`LoopRotatePass(bool EnableHeaderDuplication, bool PrepareForLTO)`
---------
Co-authored-by: Pengcheng Wang <wangpengcheng.pp@bytedance.com>
The heuristic for deciding which scope line to use for a continuation
funclet relies on iterating on the instructions of the first BB of the
continuation. Often, this contains a single unconditional branch, which
is skipped by the heuristic. However, in coro-retcon, two such
"jump-only" BBs are generated. This patch amends the heuristic to
account for that.
Adds rules for G_ATOMICRMW_{MAX, MIN, UMAX, UMIN, UINC_WRAP, UDEC_WRAP}.
Each of these generic opcode are supported for S32 and S64 types
on flat, global and local address spaces.
This patch adds support in Clang for the PRFM IR instruction, by adding
the following builtin:
void __pldir(void const *addr);
This builtin is described in the following ACLE proposal:
https://github.com/ARM-software/acle/pull/406
This patch extends the support added in #158088 to loops where the
assignment is non-speculatable (e.g. a conditional load or divide).
For example, the following loop can now be vectorized:
```
int simple_csa_int_load(
int* a, int* b, int default_val, int N, int threshold)
{
int result = default_val;
for (int i = 0; i < N; ++i)
if (a[i] > threshold)
result = b[i];
return result;
}
```
It does this by extending the recurrence matching from only looking for
selects, to include phis where all operands are the header phi, except
for one which can be an arbitrary value outside the recurrence.
---
Reverts llvm/llvm-project#180275 (original PR: #178862)
Additional type legalization for `ISD::VECTOR_FIND_LAST_ACTIVE` was
added in #180290, which should resolve the backend crashes on x86.
Start trying to use SimplifyDemandedFPClass on instructions, starting
with fmul. This subsumes the old transform on multiply of 0. The
main change is the introduction of nnan/ninf. I do not think anywhere
was systematically trying to introduce fast math flags before, though
a few odd transforms would set them.
Previously we only called SimplifyDemandedFPClass on function returns
with nofpclass annotations. Start following the pattern of
SimplifyDemandedBits, where this will be called from relevant root
instructions.
I was wondering if this should go into InstCombineAggressive, but that
apparently does not make use of InstCombineInternal's worklist.
Now that getModRefInfo for calls handles read and write effects by
examining both calls, the clobbering query no longer needs to treat
reads as clobbers. Update the check to consider writes only, aligning
call handling with other instructions
Same as https://github.com/llvm/llvm-project/pull/177988, but for
fminimum_num/fmaximum_num. Directly canonicalize these to the
corresponding intrinsics, and let the shrinking happen directly on the
intrinsics.
This lowers the splitting as:
```
any_active(hi_mask)
? (find_last_active(hi_mask) + lo_mask.getVectorElementCount())
: find_last_active(lo_mask)
```
And trivially lowers `<1 x i1>` scalarization to returning zero. Which
is a natural result of the splitting (and the lack of a sentinel
"none-active" result value).
The lowerings likely can be improved. This patch is for completeness.
Should fix:
https://github.com/llvm/llvm-project/pull/178862#issuecomment-3862310334Fixes#180212
Make it possible to use `s_alloc_vgpr` at the IR level. This is a huge
footgun and use for anything other than compiler internal purposes is
heavily discouraged. The calling code must make sure that it does not
allocate fewer VGPRs than necessary - the intrinsic is NOT a request to
the backend to limit the number of VGPRs it uses (in essence it's not so
different from what we do with the dynamic VGPR flags of the
`amdgcn.cs.chain` intrinsic, it just makes it possible to use this
functionality in other scenarios).
When comparing multi-word integers with Zicond, we generate:
(or (czero_eqz (lo1 < lo2), (hi1 == hi2)),
(czero_nez (hi1 < hi2), (hi1 == hi2)))
The czero_nez is redundant because when hi1 == hi2 is true, hi1 < hi2 is
already 0. This patch adds a DAG combine to recognize:
czero_nez (setcc X, Y, CC), (setcc X, Y, eq) -> (setcc X, Y, CC)
when CC is a strict inequality (lt, gt, ult, ugt).
This saves one instruction in 128-bit comparisons on RV64 with Zicond.
Note the czero_nez becomes a czero.eqz in the final assembly because the
seteq is replaced by an xor that produces 0 when the values are equal.
Part of #179584
Assisted-by: claude
When a type test has two phases and is used by llvm.cond.loop to
implement a conditional trap, it is more efficient for two infinite
loops to be generated. Arrange for this by having the pass detect the
typical IR pattern used for conditional CFI traps and generate the second
llvm.cond.loop if found.
Part of this RFC:
https://discourse.llvm.org/t/rfc-optimizing-conditional-traps/89456
Reviewers: fmayer, vitalybuka
Reviewed By: vitalybuka
Pull Request: https://github.com/llvm/llvm-project/pull/177687
This patch introduces `get(T)` and `set(T, Val)` functions for Waitcnt
and removes getCounterRef() and getWait(). For this to work we also need
to move InstrCounterType to AMDGPUBaseInfo.h.
Please note that the member variables are still public to keep this
patch small.
They will be replaced in the follow-up patch.
This adds atomicrmw `uinc_wrap` and `udec_wrap` operations support for
SPIR-V. Since SPIR-V doesn't provide dedicated instructions for those
two operations, we have to use the `AtomicExpand` pass to expand the
operations into CAS forms.
Closes#177204.
