SuperSONIC: Cloud-Native Infrastructure for ML Inferencing
Name
3708035.3736049.pdf
Size
2.16 MB
Format
Adobe PDF
Checksum (MD5)
9a306c0b4f37ee322f90ddd4745856ee
Author(s)
Kondratyev, Dmitry
Riedel, Benedikt
Chou, Yuan-Tang
Cochran-Branson, Miles
Paladino, Noah
Schultz, David
Liu, Mia
Duarte, Javier
Harris, Philip
Hsu, Shih-Chieh
Date Issued
July 18, 2025
Publisher
ACM|Practice and Experience in Advanced Research Computing
Citation
Dmitry Kondratyev, Benedikt Riedel, Yuan-Tang Chou, Miles Cochran-Branson, Noah Paladino, David Schultz, Mia Liu, Javier Duarte, Philip Harris, and Shih-Chieh Hsu. 2025. SuperSONIC: Cloud-Native Infrastructure for ML Inferencing. In Practice and Experience in Advanced Research Computing 2025: The Power of Collaboration (PEARC '25). Association for Computing Machinery, New York, NY, USA, Article 29, 1–5.
Version
Final published version
Abstract
The increasing computational demand from growing data rates and complex machine learning (ML) algorithms in large-scale scientific experiments has driven the adoption of the Services for Optimized Network Inference on Coprocessors (SONIC) approach. SONIC accelerates ML inference by offloading it to local or remote coprocessors to optimize resource utilization. Leveraging its portability to different types of coprocessors, SONIC enhances data processing and model deployment efficiency for cutting-edge research in high energy physics (HEP) and multi-messenger astrophysics (MMA). We developed the SuperSONIC project, a scalable server infrastructure for SONIC, enabling the deployment of computationally intensive tasks to Kubernetes clusters equipped with graphics processing units (GPUs). Using NVIDIA Triton Inference Server, SuperSONIC decouples client workflows from server infrastructure, standardizing communication, optimizing throughput, load balancing, and monitoring. SuperSONIC has been successfully deployed for the CMS and ATLAS experiments at the CERN Large Hadron Collider (LHC), the IceCube Neutrino Observatory (IceCube), and the Laser Interferometer Gravitational-Wave Observatory (LIGO) and tested on Kubernetes clusters at Purdue University, the National Research Platform (NRP), and the University of Chicago. SuperSONIC addresses the challenges of the Cloud-native era by providing a reusable, configurable framework that enhances the efficiency of accelerator-based inference deployment across diverse scientific domains and industries.
Description
PEARC ’25, Columbus, OH, USA
MIT Department
Massachusetts Institute of Technology. Department of Physics
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3708035.3736049
