If the divisor is 1, the magic algorithm does not return a correct
result and we end up using a select to pick the numerator for those
elements at the end.
Therefore we can use undef for that element of the earlier operations
when the divisor is 1. We sometimes get this through SimplifyDemandedVectorElts,
but not always. Definitely seems like we don't if the NPQ fixup is used.
Unfortunately, DAGCombiner is unable to fold srl X, <0, undef> to X so
I had to add flags to avoid emitting the srl unless one of the shift
amounts is non-zero.
Reviewed By: lebedev.ri
Differential Revision: https://reviews.llvm.org/D141022
These appear to have had 32 bit check lines manually deleted - presumably since the checks are verbose. Please don't do this! Split the test file if you want, but manually deleting test lines makes the diffs for later autogen changes really confusing.
While we have great handling for UNDEF operands,
FREEZE-UNDEF operands are effectively normal operands.
We are better off "interleaving" such BUILD_VECTORS into a blend
between a splat of FREEZE-UNDEF, and "thawed" source BUILD_VECTOR,
both of which are more natural for us to handle.
Refs. f738ab9075 (r95017306)
The original computation was both making assumptions that do not hold
in practice, and being overly pessimistic. We should just check
every possible split factor, and pick the best one.
Fixes https://github.com/llvm/llvm-project/issues/59781
Single-element vectors are legalized by splitting,
so the the memory operations would also get scalarized.
While we do have some support to reconstruct scalarized loads,
we clearly don't catch everything.
The comment for the affected AArch64 store suggests that
having two stores was the desired outcome in the first place.
This was showing as a source of *many* regressions
with more aggressive ZERO_EXTEND_VECTOR_INREG recognition.
This kind of pattern seems to come up as regressions
with better ZERO_EXTEND_VECTOR_INREG recognition.
For initial implementation, this is quite restricted
to the minimal viable transform, otherwise there are
too many regressions to be dealt with.
Some of these even crash instruction selection for AVX512.
This is one of the patterns that comes up as regressions
with more aggressive ZERO_EXTEND_VECTOR_INREG recognition.
https://godbolt.org/z/x88aqfrT5
This mainly cleans up a few patterns that are legalized by scalarization
from a wide-element vector, but then are further split apart to build
a more narrow-sized-element vector. In particular this happens in some
cases for illegal ISD::ZERO_EXTEND_VECTOR_INREG.
Given a ISD::EXTRACT_VECTOR_ELT, which is a glorified bit sequence extract,
recursively analyse all of it's users. and try to model themselves as
bit sequence extractions. If all of them agree on the new, narrower element
type, and all of them can be modelled as ISD::EXTRACT_VECTOR_ELT's of that
new element type, do that, but only if unmodelled users are ISD::BUILD_VECTOR.
If the dividend has leading zeros, we can use them to reduce the
size of the multiplier and avoid the fixup cases.
This patch is for scalars only, but we might be able to do this
for vectors in a follow up.
Differential Revision: https://reviews.llvm.org/D140750
It should be possible to deduplicate AVX2 and AVX512F checklines,
but i'm not sure which combination of check prefixes would do that.
https://godbolt.org/z/sndT9n1nz
Currently, the medium code model for x86_64 emits position-dependent relocations (R_X86_64_64) for local functions, regardless of PIC or no-PIC mode. (This means generically that code compiled with the medium model cannot be linked into a position-independent executable.)
Example:
```
static int g(int n) {
return 2 * n + 3;
}
void f(int(**p)(int)) {
*p = g;
}
```
This results in:
```
Disassembly of section .text:
0000000000000000 <f>:
0: 48 b8 00 00 00 00 00 00 00 00 movabs rax, 0x0
a: 48 89 07 mov qword ptr [rdi], rax
d: c3 ret
```
```
Relocation section '.rela.text' at offset 0xf0 contains 1 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend
0000000000000002 0000000200000001 R_X86_64_64 0000000000000000 .text + 10
```
This patch changes the behaviour to unconditionally emit a RIP-relative access, both in PIC and non-PIC mode. This fixes PIC mode, and is perhaps an improvement in non-PIC mode, too, since it results in a shorter instruction. A 32-bit relocation should suffice since the medium memory model demands that all code fit within 2GiB.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D140593
We might have sunk a bitcast into shuffle, and now it might be operating
on more fine-grained elements than what we'd match, so we must not be
dependent on whatever the granularity the shuffle happened to be in,
but transform it into the one canonical for us - with widest elements.
Depending on the particular DAG, we might either create a `freeze`,
or not. And only in the former case, the cycle would be formed.
It would be nicer to have `ReplaceAllUsesOfValueWithIf()`,
like we have in IR, but we don't have that.
Fixes https://github.com/llvm/llvm-project/issues/59677
The original code was confusing. It was stripping poison-generating flags,
but the comments were saying that doing so was a TODO.
If the poison-generating flags are present, then even if all operands
are guaranteed not to be undef or poison, the whole operation may still
produce undef or poison. We can still deal with that case,
and we already do deal with it in fact, by also dropping those flags.
Refs. https://github.com/llvm/llvm-project/issues/59676
Lack of such operands implies that the op might be poison-producing due to
it's flags. We seem to drop them already, but the comments are confusing.
Fixes https://github.com/llvm/llvm-project/issues/59676
32-byte ones are for consistency only, we really only care about
up to 16-byte on 64-bit and maybe up to 8-byte on 32-bit.
In 16byte ones, we are still having some redundant vec<->scalar traffic.
https://reviews.llvm.org/D140493
This may allow us to further simplify the vector,
and freezing the extracted result is still fine:
```
----------------------------------------
define i8 @src(<2 x i8> %src, i64 %idx) {
%0:
%i1 = freeze <2 x i8> %src
%i2 = extractelement <2 x i8> %i1, i64 %idx
ret i8 %i2
}
=>
define i8 @tgt(<2 x i8> %src, i64 %idx) {
%0:
%i1 = extractelement <2 x i8> %src, i64 %idx
%i2 = freeze i8 %i1
ret i8 %i2
}
Transformation seems to be correct!
```
BUT, there must not be other uses of that freeze,
see `@freeze_extractelement_extra_use`.
Also, looks like we are missing some ISEL-level handling for freeze.
The option from D116070 does not work as intended and will not be needed when
hidden visibility is used. A function needs ENDBR if it may be reached
indirectly. If we make ThinLTO combine the address-taken property (close to
`!GV.use_empty() && !GV.hasAtLeastLocalUnnamedAddr()`), then the condition can
be expressed with:
`AddressTaken || (!F.hasLocalLinkage() && (VisibleToRegularObj || !F.hasHiddenVisibility()))`
The current `F.hasAddressTaken()` condition does not take into acount of
address-significance in another bitcode file or ELF relocatable file.
For the Linux kernel, it uses relocatable linking. lld/ELF uses a
conservative approach by setting all `VisibleToRegularObj` to true.
Using the non-relocatable semantics may under-estimate
`VisibleToRegularObj`. As @pcc mentioned on
https://github.com/ClangBuiltLinux/linux/issues/1737#issuecomment-1343414686
, we probably need a symbol list to supply additional
`VisibleToRegularObj` symbols (not part of the relocatable LTO link).
Reviewed By: samitolvanen
Differential Revision: https://reviews.llvm.org/D140363
One of these two changes is exposing (or causing) some more miscompiles.
A reproducer is in progress, so reverting until resolved.
This reverts commit 428f36401b1b695fd501ebfdc8773bed8ced8d4e.
