eBPF Overview

What eBPF Actually Is

eBPF is a sandboxed virtual machine inside the Linux kernel. It lets you run custom bytecode at kernel hook points without modifying kernel source or loading kernel modules. Programs are verified at load time for safety (no unbounded loops, no invalid memory access), then JIT-compiled to native machine code.

The "extended" part matters: classic BPF (cBPF, 1992) was a simple packet filter with 2 registers and 32-bit ops. eBPF (2014+) is a general-purpose in-kernel execution engine with 11 registers (R0-R10), 64-bit ops, maps (key-value stores), helper functions, kfuncs, and tail calls.

Programs attach to hook points: kprobes (any kernel function), uprobes (any userspace function), tracepoints (stable kernel instrumentation), XDP (network driver ingress), tc (traffic control), cgroup hooks, LSM hooks, and since Linux 6.12 even the CPU scheduler itself. Each hook type defines what context the program receives and what it can do.

Kernel Execution Pipeline

Every eBPF program, regardless of frontend, travels the same road: source to bytecode, bytecode through the bpf() syscall, past the verifier, through the JIT, onto a hook.

USERSPACE
  BCC Python/Lua    bpftrace       libbpf CO-RE / ebpf-go / aya
       |                 |                   |
   C source        DSL script        pre-compiled .o (BTF-relocated)
       |                 |                   |
       v                 v                   v
  LLVM/Clang compiles to BPF bytecode   (or loaded directly)
       |
------ bpf() syscall --------------------------------------------
       |
KERNEL
       v
  VERIFIER   rejects unsafe programs: bounds checks, termination
       |        proof, stack <= 512 bytes, no wild pointer derefs *
       v
  JIT COMPILER   bytecode -> native (x86_64, arm64, riscv64, ...)
       |
       v
  ATTACH to hook point (kprobe, tracepoint, XDP, LSM, ...)
       |
       v
  BPF MAPS <-- shared state between kernel prog & userspace
  (hash, array, ringbuf, perf_event, LRU, stack_trace, arena, ...)
       |
       v
  Userspace reads maps / ring buffer / perf buffer for output

* bounded loops allowed since 5.3 (verifier proves termination);
  bpf_loop since 5.17, open-coded iterators since 6.4

The six steps

1. Write the program C for BCC/libbpf, a DSL for bpftrace, Rust for aya. The program declares its hook type, maps, and logic.
2. Compile to BPF bytecode LLVM's BPF backend (clang -target bpf). BCC does this at runtime on the target machine; libbpf-style toolchains do it once at build time.
3. Load via the bpf() syscall Bytecode, map definitions, and BTF type info enter the kernel. Requires CAP_BPF (or root; delegable via BPF token since 6.9).
4. Verification The verifier walks all code paths and proves the program is safe. No proof, no load. This is the gate everything else depends on.
5. JIT compilation Accepted bytecode becomes native machine code. Effectively free at runtime; enabled by default on all major architectures.
6. Attach The program is bound to its hook point and starts firing. Maps carry data back to userspace.

Verifier & Maps

The BPF verifier

Every BPF program passes through the verifier before execution. It walks all possible code paths and proves:

1. No unreachable instructions
2. No out-of-bounds or unchecked memory access
3. All paths terminate (loops must be provably bounded)
4. Stack usage <= 512 bytes per frame
5. Only helpers/kfuncs allowed for the program type are called
6. R0 holds a valid return value before exit
7. Map lookup results are null-checked before dereference

The complexity budget is 1 million verified instructions per program (since 5.2; it was 4096 in the early days). The verifier is your biggest adversary when writing BPF C: R1 invalid mem access 'scalar' means you dereferenced a raw kernel pointer without bpf_probe_read_kernel(). In kprobe context, every struct field read from a kernel pointer needs an explicit probe read (BCC rewrites most of them for you; see BCC Anatomy).

BPF maps

Maps are the shared data structures between kernel-side BPF programs and userspace. They persist across program invocations and are the primary mechanism for collecting data.

Where BPF Programs Attach

The hook point determines the program type, the context struct it receives, and which helpers it may call. Three broad families: tracing, networking, security.

Tracing

Networking

Security

BCC Anatomy

BCC (BPF Compiler Collection) provides a Python (and Lua) frontend for writing BPF programs. The kernel-side C is embedded as a string, compiled at runtime via LLVM, and loaded into the kernel. This is the fastest path from zero to working instrumentation, and it is what every tool in this repo uses.

A complete BCC program

BCC Python
#!/usr/bin/env python3
from bcc import BPF

# ---- kernel side (C) ----
prog = """
struct event_t {
    u32 pid;
    char comm[16];
    char fname[64];
    u64 size;
};

BPF_PERF_OUTPUT(events);              // declare perf buffer

int trace_read_entry(struct pt_regs *ctx,
                     struct file *file,
                     char __user *buf,
                     size_t count) {
    struct event_t evt = {};
    evt.pid = bpf_get_current_pid_tgid() >> 32;
    bpf_get_current_comm(&evt.comm, sizeof(evt.comm));
    bpf_probe_read_kernel_str(&evt.fname, sizeof(evt.fname),
                               file->f_path.dentry->d_name.name);
    evt.size = count;
    events.perf_submit(ctx, &evt, sizeof(evt));
    return 0;
}
"""

# ---- userspace side (Python) ----
b = BPF(text=prog)                           # compile + load
b.attach_kprobe(event="vfs_read",
                fn_name="trace_read_entry") # attach to hook

def handle_event(cpu, data, size):         # callback for events
    evt = b["events"].event(data)
    print(f"{evt.pid}  {evt.comm}  {evt.fname}  {evt.size}")

b["events"].open_perf_buffer(handle_event)
while True:
    b.perf_buffer_poll()                     # blocks, calls handle_event

The Lab Tools in This Repo

Every script in this repository is a runnable BCC program that demonstrates one or two of the concepts above. Run them with root (or CAP_BPF + CAP_PERFMON), roughly in this order.

Reference Tables

BPF helpers you will actually use

The old bpf_probe_read() still exists for compatibility but has been split into the _kernel/_user variants since 5.5; new code should always say which address space it means.

BCC macros and Python API

C side (embedded string)
BPF_HASH(name, key_t, val_t)   // hash map
BPF_ARRAY(name, val_t, max)    // array map
BPF_PERF_OUTPUT(name)          // perf buffer
BPF_PERCPU_ARRAY(name)         // per-cpu array
BPF_RINGBUF_OUTPUT(name, pgs)  // ring buffer
TRACEPOINT_PROBE(cat, name)    // tracepoint handler

Python side
b.attach_kprobe(event, fn_name)
b.attach_kretprobe(event, fn_name)
b.attach_uprobe(name, sym, fn_name)
b.get_syscall_fnname("execve")  # arch-portable
b["map"].open_perf_buffer(cb)
b.perf_buffer_poll()             # block + dispatch
b["map"][key]                    # direct map read

Common kprobe targets

Verifier Errors & Fixes

The verifier's error messages are terse and the line numbers refer to bytecode, not your C. These are the ones you will actually hit, and what they mean.

Toolchains Compared

Five realistic ways to write eBPF in 2026. The split that matters is runtime compilation (BCC) versus compile-once-run-everywhere (everything else, via BTF + CO-RE).

ebpf-go: The Cilium Library

cilium/ebpf (ebpf-go) is the pure-Go library for loading and managing pre-compiled BPF programs. Write BPF C, compile it with clang to a .o at build time, then load, attach, and read maps from Go. No LLVM on the target machine; CO-RE + BTF handles cross-kernel portability.

workflow
# 1. write the kernel side
probe.bpf.c

# 2. generate Go bindings + embed bytecode (wraps clang -target bpf)
$ go generate ./...        # runs bpf2go

# 3. load and attach from Go
objs := probeObjects{}
loadProbeObjects(&objs, nil)
link.Kprobe("tcp_v4_connect", objs.TraceConnect, nil)

# 4. read events
rd, _ := ringbuf.NewReader(objs.Events)
record, _ := rd.Read()

This is what Cilium itself uses for Kubernetes networking, service mesh, and observability (Hubble), and what Cloudflare, Meta, and Datadog build production eBPF tooling with. If you are building an interactive eBPF tool in Go, this is the load-bearing dependency; the engine loads compiled programs, and bpf2go keeps the C and Go sides in sync at build time.

bpftrace One-Liners

bpftrace is the awk of kernel tracing: probes, predicates, and actions in a one-line DSL, compiled to eBPF under the hood. Since 0.21 it is built on libbpf and uses your kernel's BTF, so no headers are needed on a modern distro.

bpftrace
# count syscalls by process
$ bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'

# trace file opens with path
$ bpftrace -e 'tracepoint:syscalls:sys_enter_openat
  { printf("%s %s\n", comm, str(args.filename)); }'

# histogram of read() sizes
$ bpftrace -e 'tracepoint:syscalls:sys_exit_read /args.ret > 0/
  { @bytes = hist(args.ret); }'

# TCP connect with destination IP
$ bpftrace -e 'kprobe:tcp_v4_connect { printf("%s -> %s\n", comm,
    ntop(((struct sock *)arg0)->__sk_common.skc_daddr)); }'

# slow syscalls (>1ms)
$ bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @start[tid] = nsecs; }
  tracepoint:raw_syscalls:sys_exit /@start[tid]/
  { $d = nsecs - @start[tid];
    if ($d > 1000000) { printf("%s %dms\n", comm, $d / 1000000); }
    delete(@start[tid]); }'

# list every probe you could attach to
$ bpftrace -l 'tracepoint:syscalls:*'

Note the parallels: the slow-syscalls one-liner is slow_syscalls.py from Lab Tools in three lines, and the openat trace is the core of file_watcher.py. bpftrace is the fastest way to test a hypothesis before writing the full tool.

Modern eBPF: 5.x to 6.x

Everything earlier in this guide worked in 2020. This is what the kernel added since, roughly in order of appearance. If your kernel is 6.12 or newer (this repo's reference machine runs 6.17), all of it is available.

eBPF Evolution

Further Rabbit Holes

Where to go once the lab tools feel small. Each of these leads to better questions.