The previous `attempt to fix [CGData][MachineOutliner] Global Outlining
(#90074) #108037` was incomplete because the
`ImmutableModuleSummaryIndexWrapperPass` is now optional for the
MachineOutliner pass.
With this fix, the test file `CodeGen/AArch64/O3-pipeline.ll` shows no
changes compared to its state before `[CGData][MachineOutliner] Global
Outlining (#90074)`.
Co-authored-by: Kyungwoo Lee <kyulee@fb.com>
Instead of visiting call sites in Attribute::checkForAllUses, we now
keep track of returns in AAPointerInfo and use the call site return
information as required. This way, the user of
AAPointerInfo(CallSite)Argument can determine if the call return should
be visited. We do not collect them as "may accesses" in the
AAPointerInfo(CallSite)Argument itself in case a return user is found.
In the `lowerPack` transform, there is a special case for lowering into
a simple `tensor.pad` + `tensor.insert_slice`, but the destination
becomes a newly created `tensor.empty`. This PR fixes the transform to
reuse the original destination of the `tensor.pack`.
MacroAnnotations has three std::optional fields.
Functions makeDeprecation, makeRestrictExpansion, and makeFinal
construct an instance of MacroAnnotations with one field initialized
with a non-default value (that is, some value other than
std::nullopt).
Functions addMacroDeprecationMsg, addRestrictExpansionMsg, and
addFinalLoc either create a new map entry with one field initialized
with a non-default value or replaces one field of an existing map
entry.
We can do all this with a simple statement of the form:
AnnotationInfos[II].FieldName = NonDefaultValue;
which takes care of default initialization of the fields with
std::nullopt when a requested map entry does not exist.
This patch refactors the procedure of getting the register number from a
register name to LLVMState rather than having individual users get the
values themselves by getting a reference to the map from LLVMState. This
is primarily intended to make some downstream usage in Gematria simpler,
but also cleans up a little bit upstream by pulling the actual map
searching out and just leaving error handling to the clients.
The original getter is left to enable downstream migration in Gematria,
particularly before it gets imported into google internal.
This commit introduces support for outlining functions across modules
using codegen data generated from previous codegen. The codegen data
currently manages the outlined hash tree, which records outlining
instances that occurred locally in the past.
The machine outliner now operates in one of three modes:
1. CGDataMode::None: This is the default outliner mode that uses the
suffix tree to identify (local) outlining candidates within a module.
This mode is also used by (full)LTO to maintain optimal behavior with
the combined module.
2. CGDataMode::Write (`-codegen-data-generate`): This mode is identical
to the default mode, but it also publishes the stable hash sequences of
instructions in the outlined functions into a local outlined hash tree.
It then encodes this into the `__llvm_outline` section, which will be
dead-stripped at link time.
3. CGDataMode::Read (`-codegen-data-use-path={.cgdata}`): This mode
reads a codegen data file (.cgdata) and initializes a global outlined
hash tree. This tree is used to generate global outlining candidates.
Note that the codegen data file has been post-processed with the raw
`__llvm_outline` sections from all native objects using the
`llvm-cgdata` tool (or a linker, `LLD`, or a new ThinLTO pipeline
later).
This depends on https://github.com/llvm/llvm-project/pull/105398. After
this PR, LLD (https://github.com/llvm/llvm-project/pull/90166) and Clang
(https://github.com/llvm/llvm-project/pull/90304) will follow for each
client side support.
This is a patch for
https://discourse.llvm.org/t/rfc-enhanced-machine-outliner-part-2-thinlto-nolto/78753.
The resolution of LWG2593 didn't require the standard library
implementation to change. It merely strengthened requirements on
user-defined allocator types and allowed the implementation to make
stronger assumptions. The status is tentatively set to Nothing To Do.
However, `test_allocator` in libc++'s test suit needs to be fixed to
conform to the strengthened requirements.
Closes#100220.
The goal of this PR is to facilitate integration of SPIRV Backend into
misc 3rd party tools and libraries by means of exposing an API call that
translate LLVM module to SPIR-V and write results into a string as
binary SPIR-V output, providing diagnostics on fail and means of
configuring translation in a style of command line options.
An example of a use case may be Khronos Translator that provides
bidirectional translation LLVM IR <=> SPIR-V, where LLVM IR => SPIR-V
step may be substituted by the call to SPIR-V Backend API, implemented
by this PR.
When we introduced the machinery for transitive includes validation, at
some point we stopped including the full set of transitive includes in
the CSV files and instead only tracked the set of public headers
included *directly* by a top-level header.
The reason for doing that was so that the CSV files containing
"transitive" includes could be used to draw the dependency graph of
libc++ headers. However, the downside was that it made the contents of
the CSV files much harder to interpret.
In particular, many changes that modify the CSV files do not in fact
modify the effective set of transitive includes, which is confusing.
This patch goes back to storing the full set of transitive includes in
the CSV files and removes the ability to graph the libc++ includes
directly from those CSV files, which we never actually used.
Mostly NFC, I was bothered by the declaration that were always made even
if unsued, and I think using LLVM Ops is nicer anyway with regards to
side effects here.
```
func.func private @llvm.stacksave.p0() -> !fir.ref<i8>
func.func private @llvm.stackrestore.p0(!fir.ref<i8>)
```
There are other places in lowering that are using the calls instead of
the LLVM intrinsics, but I will deal with them another time (the issue
there is mostly to get the proper address space for the llvm.ptr type).
When AArch64LoadStoreOptimizer pass merges an SP update with a
load/store instruction and needs to adjust unwind information either:
* create the merged instruction at the location of the SP update
(so no CFI instructions are moved), or
* only move a CFI instruction if the move would not reorder it across
other CFI instructions
If neither of the above is possible, don't perform the optimisation.
In AArch64, the endianness of instruction encodings is always little,
whereas the endianness of data swaps between LE and BE modes. So
getImplicitAddend must use the right one of read32() and read32le(), for
data and code respectively. It was using read32() throughout, causing
instructions to be read as big-endian in BE mode, getting the wrong
addend.
Fixed, and updated the existing test to check both endiannesses. The
expected results for data must be byte-swapped, but the ones for code
need no adjustment.
This intrinsic is meant to be used in functions that have a "tail" that
needs to be run with all the lanes enabled. The "tail" may contain
complex control flow that makes it unsuitable for the use of the
existing WWM intrinsics. Instead, we will pretend that the function
starts with all the lanes enabled, then branches into the actual body of
the function for the lanes that were meant to run it, and then finally
all the lanes will rejoin and run the tail.
As such, the intrinsic will return the EXEC mask for the body of the
function, and is meant to be used only as part of a very limited pattern
(for now only in amdgpu_cs_chain functions):
```
entry:
%func_exec = call i1 @llvm.amdgcn.init.whole.wave()
br i1 %func_exec, label %func, label %tail
func:
; ... stuff that should run with the actual EXEC mask
br label %tail
tail:
; ... stuff that runs with all the lanes enabled;
; can contain more than one basic block
```
It's an error to use the result of this intrinsic for anything
other than a branch (but unfortunately checking that in the verifier is
non-trivial because SIAnnotateControlFlow will introduce an amdgcn.if
between the intrinsic and the branch).
The intrinsic is lowered to a SI_INIT_WHOLE_WAVE pseudo, which for now
is expanded in si-wqm (which is where SI_INIT_EXEC is handled too);
however the information that the function was conceptually started in
whole wave mode is stored in the machine function info
(hasInitWholeWave). This will be useful in prolog epilog insertion,
where we can skip saving the inactive lanes for CSRs (since if the
function started with all the lanes active, then there are no inactive
lanes to preserve).
This allows e.g. DWARFDIE::GetName() to return the name of the type when
looking at its declaration (which contains only
DW_AT_declaration+DW_AT_signature). This is similar to how we recurse
through DW_AT_specification when looking for a function name. Llvm dwarf
parser has obtained the same functionality through #99495.
This fixes a bug where we would confuse a type like NS::Outer::Struct
with NS::Struct (because NS::Outer (and its name) was in a type unit).
This patch creates a simple RAII wrapper class for `SymMap` to make it
easier to use and prevent a missing matching `popScope()` for a
`pushScope()` call on simple use cases.
Some push-pop pairs are replaced with instances of the new class by this
patch.
Update the predicate protecting bfloat instructions to only reference
FEAT_SVE_B16B16, which matches the specification.
Rename and move instruction classes to match the names of the encoding
groups the bfloat arithmetic instructions belong.
GCC compiles the built-in function `__builtin_bswap16`, to the ARM
instruction rev16, which reverses the byte order of 16-bit data. On the
other Clang compiles the same built-in function to e.g.
```
rev w8, w0
lsr w0, w8, #16
```
i.e. it performs a byte reversal of a 32-bit register, (which moves the
lower half, which contains the 16-bit data, to the upper half) and then
right shifts the reversed 16-bit data back to the lower half of the
register.
We can improve Clang codegen by generating `rev16` instead of `rev` and
`lsr`, like GCC.
This reapplies commits 462251b80b7 and 9b67c99dc5b, which were reverted in
53d35c4e86b due to bot failures for the wrapper_function_utils_test.cpp unit
test.
This PR makes tosa.negate op for integer types to use the simplified
calculation branch if input_zp and output_zp values are also zero.
Signed-off-by: Dmitriy Smirnov <dmitriy.smirnov@arm.com>
Similar to VFxUF, also add a VF VPValue to VPlan and use it to get the
runtime VF in VPWidenIntOrFpInductionRecipe. Code for VF is only
generated if there are users of VF, to avoid unnecessary test changes.
PR: https://github.com/llvm/llvm-project/pull/95305
Check that `binop(zext(value)`, other) is possible and profitable to transform
into: `zext(binop(value, trunc(other)))`.
When CPU architecture has illegal scalar type iX, but vector type <N * iX> is
legal, scalar expressions before vectorisation may be extended to a legal
type iY. This extension could result in underutilization of vector lanes,
as more lanes could be used at one instruction with the lower type.
Vectorisers may not always recognize opportunities for type shrinking, and
this patch aims to address that limitation.
