Before llvm20, (void)__sync_fetch_and_add(...) always generates locked
xadd insns. In linux kernel upstream discussion [1], it is found that
for arm64 architecture, the original semantics of
(void)__sync_fetch_and_add(...), i.e., __atomic_fetch_add(...), is
preferred in order for jit to emit proper native barrier insns.
In llvm commits [2] and [3], (void)__sync_fetch_and_add(...) will
generate the following insns:
- for cpu v1/v2: locked xadd insns to keep backward compatibility
- for cpu v3/v4: __atomic_fetch_add() insns
To ensure proper barrier semantics for (void)__sync_fetch_and_add(...),
cpu v3/v4 is recommended.
This patch enables cpu=v3 as the default cpu version. For users wanting
to use cpu v1, -mcpu=v1 needs to be explicitly added to clang/llc
command line.
[1]
https://lore.kernel.org/bpf/ZqqiQQWRnz7H93Hc@google.com/T/#mb68d67bc8f39e35a0c3db52468b9de59b79f021f
[2] https://github.com/llvm/llvm-project/pull/101428
[3] https://github.com/llvm/llvm-project/pull/106494
collectInstsToScalarize may decide to scalarize a call. If so, we have
to update the widening decision for the call, otherwise the call won't
be scalarized as expected during VPlan construction.
This issue was uncovered by f82543d509.
Don't just leave the result as unknown. I think this currently
works out thanks to undef resolution, but the correct thing to
do is set it to overdefined explicitly.
This patch adds a common lower action for `G_FABS`, which generates `and
x8, x8, #0x7fffffffffffffff` to reset the sign bit. The action does not
support vectors since `G_AND` does not support fp128.
This approach is different than what SDAG is doing. SDAG stores the
value onto stack, clears the sign bit in the most significant byte, and
loads the value back into register. This involves multiple memory ops
and sounds slower.
Currently, the testing infrastructure for SPIR-V is based on FileCheck.
Those tests are great to check some level of codegen, but when the test
needs check both the CFG layout and the content of each basic-block,
things becomes messy.
- Because the CHECK/CHECK-DAG/CHECK-NEXT state is limited, it is
sometimes hard to catch the good block: if 2 basic blocks have similar
instructions, FileCheck can match the wrong one.
- Cross-lane interaction can be a bit difficult to understand, and
writting a FileCheck test that is strong enough to catch bad CFG
transforms while not being broken everytime some unrelated codegen part
changes is hard.
And lastly, the spirv-val tooling we have checks that the generated
SPIR-V respects the spec, not that it is correct in regards to the
source IR.
For those reasons, I believe the best way to test the structurizer is
to:
- run spirv-val to make sure the CFG respects the spec.
- simulate the function to validate result for each lane, making sure
the generated code is correct.
This simulator has no other dependencies than core python. It also only
supports a very limited set of instructions as we can test most features
through control-flow and some basic cross-lane interactions.
As-is, the added tests are just a harness for the simulator itself. If
this gets merged, the structurizer PR will benefit from this as I'll be
able to add extensive testing using this.
---------
Signed-off-by: Nathan Gauër <brioche@google.com>
The LegalizeDAG expansion will go through memory since i16 isn't a legal
type. Avoid this by using FMV nodes.
Similar to what we did for #106886 for FNEG and FABS. Special care is
needed to handle the Sign operand being a different type.
Without this patch, {ImportMapTy,SortedImportList}::{begin,end} make
unnecessary copies of ImportIDTable via:
map_iterator(Imports.begin(), IDs);
The second parameter, IDs, is passed by value, so we make a copy of
MapVector inside ImportIDTable every time we call begin and end.
These begin and end show up as time-consuming functions in the
performance profile.
This patch fixes the problem by passing IDs by reference with
std::cref.
While we are at it, this patch deletes the copy constructor and
assignment operator. I cannot think of any legitimate need reason to
make a copy of the deduplication table.
Apparently DragonFly BSD and Solaris/illumos call these APIs
`pthread_get_name_np` / `pthread_set_name_np` (with an extra
underscore) instead of `pthread_getname_np` / `pthread_setname_np`.
… instructions.
This is a fix I stumbled upon while working on something else. I decided
to break it out since it seems like a good "first issue" to submit. I
updated the comments in the "wrong error" test files to indicate that
the messages are no longer incorrect, but I left the names of the test
files alone. I was not sure what to do with those, so I would appreciate
thoughts or guidance.
This moves the logic to create simplified operands using SCEV to MUL
recipe creation. This is needed to match the behavior of the legacy's cost
model. TODOs are to extend to other opcodes and move to a transform.
Note that this also restricts the number of SCEV simplifications we
apply to more precisely match the cases handled by the legacy cost
model.
Fixes https://github.com/llvm/llvm-project/issues/107015.
Follow-up to 9ccf825, adjust computeCost to also pass IntrinsicInst to
TTI if available, as there are multiple places in TTI which use the
IntrinsicInst.
Fixes https://github.com/llvm/llvm-project/issues/107016.
I don't think we need this node. We can isel fp_extend directly.
fp_extend to f64 requires two instructions, but we can emit them with an
isel pattern.
I have not removed RISCVISD::FP_ROUND_BF16 because f64->bf16 needs more
work to fix the double rounding.
Fix intrinsic function attributes to not generate attribute sets that
are empty in `getIntrinsicFnAttributeSet`. Refactor the code to use
helper functions to get effective memory effects for an intrinsic and to
check if it has non-default attributes.
This eliminates one case statement in `getIntrinsicFnAttributeSet` that
we generate today for the case when intrinsic attributes are default
ones.
Also rename `Intrinsic` to `Int` to follow the naming convention used in
this file and adjust emission code to not emit unnecessary empty line
between cases generated.
This patch introduces lowering of the partial add reduction intrinsic to
a udot or svdot for AArch64. This also involves adding a
`shouldExpandPartialReductionIntrinsic` target hook, which AArch64 will
return false from in the cases that it can be lowered.
Due to a reviewer request on PR #88385 I have created this patch
to add a getPredicatedExitCount function, which is similar to
getExitCount except that it uses the predicated backedge taken
information. With PR #88385 we will start to care about more
loops with multiple exits, and want the ability to query exit
counts for a particular exiting block. Such loops may require
predicates in order to be vectorised.
New tests added here:
Analysis/ScalarEvolution/predicated-exit-count.ll
because that doesn't work (results in `LINK : error LNK2001: unresolved
external symbol malloc`).
Based on the title of #91862 it was only intended for use in 64-bit
builds.
The LoopIdiomVectorize pass already creates calls to the intrinsic
experimental_cttz_elts, but PR #88385 will start calling this more
too so I've created a helper for it.
We weren't taking account of the space we require in the stubs for
things that are dllimported, and as a result we could hit the assertion
failure for running out of stub space. Fix that.
Also add a couple of `override` specifiers that were missing last time
(#102586).
rdar://133473673
When serialising to textual IR, there can be constant Values referred to
by DbgRecords that don't appear anywhere else, and have types hidden
even deeper in side them. Enumerate these when enumerating all types.
Test by Mikael Holmén.
