eBPF: What's in it?

If you work with cloud apps, you've probably noticed a growing trend to use eBPF for profiling, observability, security and network tasks. To fully understand the potential and limitations of this technology, it's good to know how it works under the hood.

Let's look at how applications are executed from a Linux system perspective. In simple terms, everything operates in three layers:
1. User Space. It's where our applications run. This is the non-privileged part of the OS.
2. Kernel space. The privileged part of the OS that handles low-level operations. These operations usually provide access to the system hardware (file system, network, memory, etc.). Applications interact with it through system calls (syscalls).
3. Hardware. The physical device layer.

eBPF is a technology that allows to embed a program on Kernel OS level, where this program is triggered on particular system events like opening file, reading file, establishing a network connection, etc. In other words, eBPF approach allows to monitor what's going on with your applications on a system level without code instrumentation. One of the earliest and most well-known tools based on this technology is tcpdump.

Some interesting ways companies use eBPF now:
- Netflix introduced bpftop, a tool to measure how long processes spend in the CPU scheduled state. If processes take too long, it often points to CPU bottlenecks like throttling or over-allocation.
- Datadog shared their experience using eBPF for chaos testing via ChaosMesh.
- Confluent adopted Cilium, an eBPF-based CNI plugin for networking and security in Kubernetes.

Over the past few years I've seen more and more adoption of eBPF-based tools across the industry. And looks like trend will continue to grow especially in the area of observability and profiling.

#engineering