Reverse Engineering the Intel Cascade Lake Mesh Interconnect

Abstract

The rising core counts of multicore processors have driven the move to more scalable on-chip networks to allow for efficient communication between the cores, memory subsystem, and other processor peripherals. Intel’s latest line of Scalable processors introduced the mesh interconnect in the form of a two-dimensional network of rings that reduces the bandwidth and latency bottlenecks of prior ring interconnects. Much is still unknown about the mesh interconnect, and details from the official documentation are sparse. In this thesis, we perform the first in-depth reverse-engineering of the mesh interconnect on an Intel Cascade Lake server. We use performance counters to determine the layouts of the cores on the die and reverse-engineer the traffic scheduling policy for packet routing on the interconnect. In addition, we develop tools to generate and monitor cross-core cache coherence traffic. We then apply these tools to determine the precise conditions required for traffic contention on the network. This information is combined with publicly available documentation and prior work to provide an unprecedented understanding of the new Intel mesh interconnect. Further, this work paves the way for future investigation into the use of the network-on-chip as a potential hardware side channel.