030b8cb156
This commit adds a new BPF specific structure attribte
`__attribute__((preserve_static_offset))` and a pass to deal with it.
This attribute may be attached to a struct or union declaration, where
it notifies the compiler that this structure is a "context" structure.
The following limitations apply to context structures:
- runtime environment might patch access to the fields of this type by
updating the field offset;
BPF verifier limits access patterns allowed for certain data
types. E.g. `struct __sk_buff` and `struct bpf_sock_ops`. For these
types only `LD/ST <reg> <static-offset>` memory loads and stores are
allowed.
This is so because offsets of the fields of these structures do not
match real offsets in the running kernel. During BPF program
load/verification loads and stores to the fields of these types are
rewritten so that offsets match real offsets. For this rewrite to
happen static offsets have to be encoded in the instructions.
See `kernel/bpf/verifier.c:convert_ctx_access` function in the Linux
kernel source tree for details.
- runtime environment might disallow access to the field of the type
through modified pointers.
During BPF program verification a tag `PTR_TO_CTX` is tracked for
register values. In case if register with such tag is modified BPF
programs are not allowed to read or write memory using register. See
kernel/bpf/verifier.c:check_mem_access function in the Linux kernel
source tree for details.
Access to the structure fields is translated to IR as a sequence:
- `(load (getelementptr %ptr %offset))` or
- `(store (getelementptr %ptr %offset))`
During instruction selection phase such sequences are translated as a
single load instruction with embedded offset, e.g. `LDW %ptr, %offset`,
which matches access pattern necessary for the restricted
set of types described above (when `%offset` is static).
Multiple optimizer passes might separate these instructions, this
includes:
- SimplifyCFGPass (sinking)
- InstCombine (sinking)
- GVN (hoisting)
The `preserve_static_offset` attribute marks structures for which the
following transformations happen:
- at the early IR processing stage:
- `(load (getelementptr ...))` replaced by call to intrinsic
`llvm.bpf.getelementptr.and.load`;
- `(store (getelementptr ...))` replaced by call to intrinsic
`llvm.bpf.getelementptr.and.store`;
- at the late IR processing stage this modification is undone.
Such handling prevents various optimizer passes from generating
sequences of instructions that would be rejected by BPF verifier.
The __attribute__((preserve_static_offset)) has a priority over
__attribute__((preserve_access_index)). When preserve_access_index
attribute is present preserve access index transformations are not
applied.
This addresses the issue reported by the following thread:
https://lore.kernel.org/bpf/CAA-VZPmxh8o8EBcJ=m-DH4ytcxDFmo0JKsm1p1gf40kS0CE3NQ@mail.gmail.com/T/#m4b9ce2ce73b34f34172328f975235fc6f19841b6
This is a second attempt to commit this change, previous reverted
commit is: cb13e9286b6d4e384b5d4203e853d44e2eff0f0f.
The following items had been fixed:
- test case bpf-preserve-static-offset-bitfield.c now uses
`-triple bpfel` to avoid different codegen for little/big endian
targets.
- BPFPreserveStaticOffset.cpp:removePAICalls() modified to avoid
use after free for `WorkList` elements `V`.
Differential Revision: https://reviews.llvm.org/D133361
65 lines
1.8 KiB
C++
65 lines
1.8 KiB
C++
//===- BPFCORE.h - Common info for Compile-Once Run-EveryWhere -*- C++ -*-===//
|
|
//
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
//
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
#ifndef LLVM_LIB_TARGET_BPF_BPFCORE_H
|
|
#define LLVM_LIB_TARGET_BPF_BPFCORE_H
|
|
|
|
#include "llvm/ADT/StringRef.h"
|
|
#include "llvm/IR/Instructions.h"
|
|
|
|
namespace llvm {
|
|
|
|
class BasicBlock;
|
|
class Instruction;
|
|
class Module;
|
|
|
|
class BPFCoreSharedInfo {
|
|
public:
|
|
enum BTFTypeIdFlag : uint32_t {
|
|
BTF_TYPE_ID_LOCAL_RELOC = 0,
|
|
BTF_TYPE_ID_REMOTE_RELOC,
|
|
|
|
MAX_BTF_TYPE_ID_FLAG,
|
|
};
|
|
|
|
enum PreserveTypeInfo : uint32_t {
|
|
PRESERVE_TYPE_INFO_EXISTENCE = 0,
|
|
PRESERVE_TYPE_INFO_SIZE,
|
|
PRESERVE_TYPE_INFO_MATCH,
|
|
|
|
MAX_PRESERVE_TYPE_INFO_FLAG,
|
|
};
|
|
|
|
enum PreserveEnumValue : uint32_t {
|
|
PRESERVE_ENUM_VALUE_EXISTENCE = 0,
|
|
PRESERVE_ENUM_VALUE,
|
|
|
|
MAX_PRESERVE_ENUM_VALUE_FLAG,
|
|
};
|
|
|
|
/// The attribute attached to globals representing a field access
|
|
static constexpr StringRef AmaAttr = "btf_ama";
|
|
/// The attribute attached to globals representing a type id
|
|
static constexpr StringRef TypeIdAttr = "btf_type_id";
|
|
|
|
/// llvm.bpf.passthrough builtin seq number
|
|
static uint32_t SeqNum;
|
|
|
|
/// Insert a bpf passthrough builtin function.
|
|
static Instruction *insertPassThrough(Module *M, BasicBlock *BB,
|
|
Instruction *Input,
|
|
Instruction *Before);
|
|
static void removeArrayAccessCall(CallInst *Call);
|
|
static void removeStructAccessCall(CallInst *Call);
|
|
static void removeUnionAccessCall(CallInst *Call);
|
|
};
|
|
|
|
} // namespace llvm
|
|
|
|
#endif
|