A methodology for using eBPF to efficiently monitor network behavior in Linux Kubernetes clusters

Abstract

With the rise of container orchestration systems, such as Kubernetes and microservice based application architectures there has been a corresponding growth in tools aimed at monitoring these systems. As monitoring approaches have evolved the implementation of instrumentation has shifted from the application level to the platform level. The extended Berkeley Packet Filter (eBPF) can enable high performance and low overhead collection for platform level monitoring. Existing commercial eBPF monitoring systems are often tightly integrated systems with large dependencies and little flexibility in integration into alternative monitoring systems. This thesis presents a methodology for developing modular self-contained eBPF monitoring systems which are portable across various kernel versions, Container Network Interface (CNI) plugins, and cluster configurations. The choice of stable hook points and the BPF CO-RE approach to development using the libbpf or Cilium/ebpf loaders is recommended in this methodology. A proof of concept monitor was developed which captures network traffic on a cluster using the stable Traffic Control direct-action hook point. Packet capture at pod virtual ethernet network interfaces was selected to allow for CNI independent correlation of packets to cluster workloads. The prototype developed provides a suitable platform for implementing additional monitoring functionality on top of and was integrated with an existing NetApp cloud monitoring system.