In [1], a few new insns are proposed to expand BPF ISA to
. fixing the limitation of existing insn (e.g., 16bit jmp offset)
. adding new insns which may improve code quality
(sign_ext_ld, sign_ext_mov, st)
. feature complete (sdiv, smod)
. better user experience (bswap)
This patch implemented insn encoding for
. sign-extended load
. sign-extended mov
. sdiv/smod
. bswap insns
. unconditional jump with 32bit offset
The new bswap insns are generated under cpu=v4 for __builtin_bswap.
For cpu=v3 or earlier, for __builtin_bswap, be or le insns are generated
which is not intuitive for the user.
To support 32-bit branch offset, a 32-bit ja (JMPL) insn is implemented.
For conditional branch which is beyond 16-bit offset, llvm will do
some transformation 'cond_jmp' -> 'cond_jmp + jmpl' to simulate 32bit
conditional jmp. See BPFMIPeephole.cpp for details. The algorithm is
hueristic based. I have tested bpf selftest pyperf600 with unroll account
600 which can indeed generate 32-bit jump insn, e.g.,
13: 06 00 00 00 9b cd 00 00 gotol +0xcd9b <LBB0_6619>
Eduard is working on to add 'st' insn to cpu=v4.
A list of llc flags:
disable-ldsx, disable-movsx, disable-bswap,
disable-sdiv-smod, disable-gotol
can be used to disable a particular insn for cpu v4.
For example, user can do:
llc -march=bpf -mcpu=v4 -disable-movsx t.ll
to enable cpu v4 without movsx insns.
References:
[1] https://lore.kernel.org/bpf/4bfe98be-5333-1c7e-2f6d-42486c8ec039@meta.com/
Differential Revision: https://reviews.llvm.org/D144829
Extended BPFCheckAndAdjustIR pass with sinkMinMax() transformation
that undoes LICM hoistMinMax pass.
The undo transformation converts the following patterns:
x < min(a, b) -> x < a && x < b
x > min(a, b) -> x > a || x > b
x < max(a, b) -> x < a || x < b
x > max(a, b) -> x > a && x > b
Where 'a' or 'b' is a constant.
Also supports `sext min(...) ...` and `zext min(...) ...`.
~~~
This was previously commited as 09feee559a29 and reverted in
0bf9bfeacc8c because of the testbot memory leak report:
https://lab.llvm.org/buildbot/#/builders/5/builds/34931
The memory leak issue was caused by incorrect instruction removal
sequence in skinMinMaxBB():
I->dropAllReferences(); --------> I->eraseFromParent();
I->removeFromParent(); fixed to
Differential Revision: https://reviews.llvm.org/D147990
There are plans to add some BTF processing to tools like objdump and
readelf. This commit moves BTF.{h,def} files from BPF target specific
location to include/llvm/DebugInfo/* to avoid tools including headers
from lib/Target/*.
Reviewed By: yonghong-song, MaskRay
Differential Revision: https://reviews.llvm.org/D149501
Extended BPFCheckAndAdjustIR pass with sinkMinMax() transformation
that undoes LICM hoistMinMax pass.
The undo transformation converts the following patterns:
x < min(a, b) -> x < a && x < b
x > min(a, b) -> x > a || x > b
x < max(a, b) -> x < a || x < b
x > max(a, b) -> x > a && x > b
Where 'a' or 'b' is a constant.
Also supports `sext min(...) ...` and `zext min(...) ...`.
Differential Revision: https://reviews.llvm.org/D147990
Remove overrides of parseDirective that unconditionally return NoMatch.
This is what the base implementation does.
This is a follow-up to D154101 based on post-commit review feedback.
Conventionally, parsing methods return false on success and true on
error. However, directive parsing methods need a third state: the
directive is not target specific. AsmParser::parseStatement detected
this case by using a fragile heuristic: if the target parser did not
consume any tokens, the directive is assumed to be not target-specific.
Some targets fail to follow the convention: they return success after
emitting an error or do not consume the entire line and return failure
on successful parsing. This was partially worked around by checking for
pending errors in parseStatement.
This patch tries to improve the situation by introducing parseDirective
method that returns ParseStatus -- three-state class. The new method
should eventually replace the old one returning bool.
ParseStatus is intentionally implicitly constructible from bool to allow
uses like `return Error(Loc, "message")`. It also has a potential to
replace OperandMatchResulTy as it is more convenient to use due to the
implicit construction from bool and more type safe.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D154101
`NoMerge` attribute on machine instructions prevents certain
transformations from merging these instructions.
One of such transformations is 'llvm/lib/CodeGen/BranchFolding.cpp'.
This attribute should be copied from IR `call` instructions to machine
level instructions. See `X86TargetLowering::LowerCall` as another
example.
Differential Revision: https://reviews.llvm.org/D152987
SubtargetFeature.h is currently part of MC while it doesn't depend on
anything in MC. Since some LLVM components might have the need to work
with target features without necessarily needing MC, it might be
worthwhile to move SubtargetFeature.h to a different location. This will
reduce the dependencies of said components.
Note that I choose TargetParser as the destination because that's where
Triple lives and SubtargetFeatures feels related to that.
This issues came up during a JITLink review (D149522). JITLink would
like to avoid a dependency on MC while still needing to store target
features.
Reviewed By: MaskRay, arsenm
Differential Revision: https://reviews.llvm.org/D150549
D137796 made these passes unused.
`opt --bpf-ir-peephole` is specified in one test. Add a `registerPipelineParsingCallback`
so that we can use change the test to use `opt --passes=bpf-ir-peephole` instead.
The declaration was added without a corresponding function definition
by:
commit dc1dbf6ef320175acbdc1206da4b0a176b304449
Author: Yonghong Song <yhs@fb.com>
Date: Wed Aug 23 04:25:57 2017 +0000
The corresponding function definition was removed by:
commit ec51851026a55e1cfc7f006f0e75f0a19acb32d3
Author: Jiong Wang <wong.kwongyuan.tools@gmail.com>
Date: Wed Oct 16 15:27:59 2019 +0000
The term "next stack offset" is misleading because the next argument is
not necessarily allocated at this offset due to alignment constrains.
It also does not make much sense when allocating arguments at negative
offsets (introduced in a follow-up patch), because the returned offset
would be past the end of the next argument.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D149566
This reduces dependencies on `llvm-tblgen` so much.
`CodeGenTypes` depends on `Support` at the moment.
Be careful to append deps on this, since Targets' tablegens
depend on this.
Depends on D149024
Differential Revision: https://reviews.llvm.org/D148769
With LLVM patch https://reviews.llvm.org/D148269, we hit a linux kernel
bpf selftest compilation failure like below:
...
progs/test_xdp_noinline.c:739:8: error: too many args to t8: i64 = GlobalAddress<ptr @encap_v4> 0, progs/test_xdp_noinline.c:739:8
if (!encap_v4(xdp, cval, &pckt, dst, pkt_bytes))
^
...
progs/test_xdp_noinline.c:321:6: error: defined with too many args
bool encap_v4(struct xdp_md *xdp, struct ctl_value *cval,
^
...
Note that bpf selftests are compiled with -O2 which is
the recommended flag for bpf community.
The bpf backend calling convention is only allowing 5
parameters in registers and does not allow pass arguments
through stacks. In the above case, ArgumentPromotionPass
replaced parameter '&pckt' as two parameters, so the total
number of arguments after ArgumentPromotion pass becomes 6
and this caused later compilation failure during instruction
selection phase.
This patch added a TargetTransformInfo hook getMaxNumArgs()
which returns 5 for BPF and UINT_MAX for other targets.
Differential Revision: https://reviews.llvm.org/D148551
With the emergence of TP 4091 for NVMe, eBPF can be used to offload programs to computational storage processors.
This change introduces the possibility of overriding the default stack size of 512 bytes for non-kernel runtime environments.
Reviewed By: yonghong-song, eddyz87
Differential Revision: https://reviews.llvm.org/D147707
`BPF.td` is used to generate (among other things) `MCSubtargetInfo`
setup function for BPF target.
Specifically, the `BPFGenSubtargetInfo.inc` file:
enum {
ALU32 = 0,
...
};
...
extern const llvm::SubtargetSubTypeKV BPFSubTypeKV[] = {
{ "generic", { { { 0x0ULL, ... } } }, ... },
{ "probe", { { { 0x0ULL, ... } } }, ... },
{ "v1", { { { 0x0ULL, ... } } }, ... },
{ "v2", { { { 0x0ULL, ... } } }, ... },
{ "v3", { { { 0x1ULL, ... } } }, ... },
};
...
static inline MCSubtargetInfo *createBPFMCSubtargetInfoImpl(...) {
return new BPFGenMCSubtargetInfo(..., BPFSubTypeKV, ...);
}
The `SubtargetSubTypeKV` is defined in `MCSubtargetInfo.h` as:
/// Used to provide key value pairs for feature and CPU bit flags.
struct SubtargetSubTypeKV {
const char *Key; ///< K-V key string
FeatureBitArray Implies; ///< K-V bit mask
FeatureBitArray TuneImplies; ///< K-V bit mask
const MCSchedModel *SchedModel;
...
}
The first bit array specifies features enabled by default for a
specific CPU. This commit makes sure that this information is
communicated to `tablegen` and correct `BPFSubTypeKV` table is
generated. This allows tools like `objdump` to detect available
features when `--mcpu` flag is specified.
Differential Revision: https://reviews.llvm.org/D148037
Fixes BPF assembler parsing errors for the following instructions:
- atomic_fetch_add
- atomic_fetch_and
- atomic_fetch_xor
- atomic_fetch_or
- cmpxchg32_32
- cmpxchg_64
- xchg32_32
- xchg_64
Also add a test to verify that all instructions could be assembled and disassembled.
Differential Revision: https://reviews.llvm.org/D147421
Commit 3671bdbcd214("[BPF] Fix a BTF type pruning bug") fixed a
pruning bug to allow generate more types. But the commit has a bug
which permits to generate more types than necessary. The following
is an example to illustrate the problem.
struct t1 {
int a;
};
struct t2 {
struct t1 *p1;
struct t1 *p2;
int b;
};
int foo(struct t2 *arg) {
return arg->b;
}
The following is the part of BTF generation sequence:
(1). 'struct t2 *arg' -> 'struct t1 *p1'
In this step, the type 'struct t1' will be generated as
a forward decl and the ptr type (to 'struct t1') will
be stored in the internal type table.
(2). now the second field 'struct t1 *p2' will be processed.
Since the ptr type (to 'struct t1') already in the type
table, the existing logic strips out ptr modifier and
is able to generate BTF type for 'struct t1'.
In the above step (2), if CheckPointer is true (the type traversal
chain including a struct member), 'ptr' modifier should be checked
and the subsequent type generation should be skipped since
the same case has been processed in visitDerivedType().
The issue is exposed when I am trying to use llvm15 to compile
some internal bpf programs. The bpf skeleton put the whole
ELF section (after striping some sections like dwarf) as a string.
The large BTF section triggered the following error:
bpf_object_with_struct_ops_test_prog_bpf/BpfObjectWithStructOpsTestProg.skel.h:222:23:
error: string literal of length 140144 exceeds maximum length 65536 that C++ compilers
are required to support [-Werror,-Woverlength-strings]
return (const void *)"\
^~
1 error generated.
Although adding -Wno-overlength-strings could workaround the issue,
improving llvm BTF generation sounds better esp. for users using vmlinux.h.
Differential Revision: https://reviews.llvm.org/D145816
The forwarding header is left in place because of its use in
`polly/lib/External/isl/interface/extract_interface.cc`, but I have
added a GCC warning about the fact it is deprecated, because it is used
in `isl` from where it is included by Polly.
After frontend changes in the following commit:
"BPF: preserve btf_decl_tag for parameters of extern functions"
same mechanics could be used to get the list of function parameters
and associated btf_decl_tag entries for both extern and non-extern
functions.
This commit extracts this mechanics as a separate auxiliary function
BTFDebug::processDISubprogram(). The function is called for both
extern and non-extern functions in order to generated corresponding
BTF_DECL_TAG records.
Differential Revision: https://reviews.llvm.org/D140971
Use function TargetLoweringObjectFile::SectionForGlobal() to compute
section names for globals described in BTF_KIND_DATASEC records.
This fixes a discrepancy in section name computation between
BTFDebug::processGlobals and the rest of the LLVM pipeline.
Specifically, the following example illustrates the discrepancy
before this commit:
struct Foo {
int i;
} __attribute__((aligned(16)));
struct Foo foo = { 0 };
The initializer for 'foo' looks as follows:
%struct.Foo { i32 0, [12 x i8] undef }
TargetLoweringObjectFile::SectionForGlobal() classifies 'foo' as
a part of '.bss' section, while BTFDebug::processGlobals
classified it as a part of '.data' section because of the
following expression:
SecName = Global.getInitializer()->isZeroValue() ? ".bss" : ".data"
The isZeroValue() returns false because of the undef tail of the
initializer, while SectionForGlobal() allows such patterns in '.bss'.
Differential Revision: https://reviews.llvm.org/D140505
Follow up to the series:
1. https://reviews.llvm.org/D140161
2. https://reviews.llvm.org/D140349
3. https://reviews.llvm.org/D140331
4. https://reviews.llvm.org/D140323
Completes the work from the previous two for remaining targets.
This creates the following named passes that can be run via
`llc -{start|stop}-{before|after}`:
- arc-isel
- arm-isel
- avr-isel
- bpf-isel
- csky-isel
- hexagon-isel
- lanai-isel
- loongarch-isel
- m68k-isel
- msp430-isel
- mips-isel
- nvptx-isel
- ppc-codegen
- riscv-isel
- sparc-isel
- systemz-isel
- ve-isel
- wasm-isel
- xcore-isel
A nice way to write tests for SelectionDAGISel might be to use a RUN:
line like:
llc -mtriple=<triple> -start-before=<arch>-isel -stop-after=finalize-isel -o -
Fixes: https://github.com/llvm/llvm-project/issues/59538
Reviewed By: asb, zixuan-wu
Differential Revision: https://reviews.llvm.org/D140364
This is a fairly large changeset, but it can be broken into a few
pieces:
- `llvm/Support/*TargetParser*` are all moved from the LLVM Support
component into a new LLVM Component called "TargetParser". This
potentially enables using tablegen to maintain this information, as
is shown in https://reviews.llvm.org/D137517. This cannot currently
be done, as llvm-tblgen relies on LLVM's Support component.
- This also moves two files from Support which use and depend on
information in the TargetParser:
- `llvm/Support/Host.{h,cpp}` which contains functions for inspecting
the current Host machine for info about it, primarily to support
getting the host triple, but also for `-mcpu=native` support in e.g.
Clang. This is fairly tightly intertwined with the information in
`X86TargetParser.h`, so keeping them in the same component makes
sense.
- `llvm/ADT/Triple.h` and `llvm/Support/Triple.cpp`, which contains
the target triple parser and representation. This is very intertwined
with the Arm target parser, because the arm architecture version
appears in canonical triples on arm platforms.
- I moved the relevant unittests to their own directory.
And so, we end up with a single component that has all the information
about the following, which to me seems like a unified component:
- Triples that LLVM Knows about
- Architecture names and CPUs that LLVM knows about
- CPU detection logic for LLVM
Given this, I have also moved `RISCVISAInfo.h` into this component, as
it seems to me to be part of that same set of functionality.
If you get link errors in your components after this patch, you likely
need to add TargetParser into LLVM_LINK_COMPONENTS in CMake.
Differential Revision: https://reviews.llvm.org/D137838
The value will be RAUWd, make sure the reference in CallInfo gets
updated.
It seems like this was not a problem without opaque pointers due
to the bitcast in between.
With D134950, targets get notified when a virtual register is created and/or
cloned. Targets can do the needful with the delegate callback. AMDGPU propagates
the virtual register flags maintained in the target file itself. They are useful
to identify a certain type of machine operands while inserting spill stores and
reloads. Since RegAllocFast spills the physical register itself, there is no way
its virtual register can be mapped back to retrieve the flags. It can be solved
by passing the virtual register as an additional argument. This argument has no
use when the spill interfaces are called during the greedy allocator or even the
PrologEpilogInserter and can pass a null register in such cases.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D138656
Previously we had a shared ID in SelectionDAGISel. AMDGPU has an
initializePass function for its subclass of SelectionDAGISel. No
other target does.
This causes all target specific SelectionDAGISel passes to be known
as "amdgpu-isel".
I'm not sure what would happen if another target tried to implement
an initializePass function too since the ID is already claimed.
This patch gives all targets their own ID and passes it down to
SelectionDAGISel constructor to MachineFunctionPass's constructor.
Unfortunately, I think this causes most targets to lose
print-before/after-all support for their SelectionDAGISel pass.
And they probably no longer support start/stop-before/after. We
can add initializePass functions to fix this as a follow up. NOTE:
This was probably also broken if the AMDGPU target isn't compiled in.
Step 1 to fixing PR59538.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D140161
These targets use `MCInst`, but don't explicitly link
to the library providing it (MC), and just rely on it
being pulled transitively through e.g. MCDisassembler,
but that only pulls includes, and does not link to it.
Case in point, when i add explicit destructor to `MCInst`,
defined in `.cpp`, these targets were failing to link.
Since opt no longer supports to run default (O0/O1/O2/O3/Os/Oz)
pipelines using the legacy PM, there are no in-tree uses of
TargetMachine::adjustPassManager remaining. This patch removes the
no longer used adjustPassManager functions.
Reviewed By: aeubanks
Differential Revision: https://reviews.llvm.org/D137796
Use a dyn_cast<> to IntrinsicInst and an enum compare instead.
While touching this code also re-generate the test to use positive check
lines instead of negative ones and remove some unneeded metadata.
Reviewed By: yonghong-song
Differential Revision: https://reviews.llvm.org/D138565
All in-tree targets pass pointer-sized ConstantSDNodes to the
method. This overload reduced amount of boilerplate code a bit. This
also makes getCALLSEQ_END consistent with getCALLSEQ_START, which
already takes uint64_ts.
