AI Challenge for Satellite Pattern-of-Life Identification: Dataset, Design and Results

Abstract

Despite the availability of extensive historical data on Earth-orbiting objects, artificial intelligence (AI) adoption in space domain awareness remains limited. To address this gap, the 2024 MIT ARCLab Prize for AI Innovation in Space challenged participants to develop AI models for characterizing satellite pattern-of-life (PoL) in Geostationary Earth Orbit. The challenge focused on developing machine learning models capable of classifying behavioral patterns and detecting key transition events in multivariate time-series data. The challenge dataset comprised of 2402 satellite trajectories spanning six months with a two-hour temporal resolution. The data are generated using high-fidelity satellite propagators based on simulated trajectories, Vector Covariance Message data, and two-line elements. This dataset features diverse operational behaviors and propulsion systems, providing a robust foundation for AI analysis. The challenge attracted over 100 teams worldwide, with more than 350 submissions showcasing a diverse range of AI approaches, including deep learning architectures (CNNs, LSTMs, transformers), gradient-boosting techniques (XGBoost, CatBoost), and hybrid models. The top performing teams demonstrated AI’s effectiveness in PoL characterization, with Hawaii2024 achieving an F2 score of 0.952 on the partial test set using a CNN-LSTM hybrid approach, followed closely by Millennial-IUP and QR_Is that utilized XGBoost with tailored transition-labeling and gradient-boosted decision tree with a model-stacking strategy, respectively. This paper presents an analysis of the competition’s dataset, evaluation methodology, and top-performing solutions.