Spacecraft Autonomy through Computer Vision and Onboard Planning

Abstract

Earth observation (EO) from satellite platforms has experienced widespread growth since the commercialization and widespread availability of data, and has had large impacts on applications such as agriculture, disaster monitoring, and defense and intelligence. Observing unpredictable phenomena is still challenging for EO missions due to long lead times from scheduling, uplinking, and executing the image capture onboard the spacecraft. This delay between planning and execution means that conditions can change in between them, causing a task to become unobservable and missed in the meantime, for example due to cloud cover obscuring a target. Dynamic tasking (DT) is a mission concept that aims to mitigate this unpredictability by moving autonomy onboard the spacecraft and quickly reacting to conditions as observed, using several potential perception sources. In this work, we consider DT as applied to a tasked Earth-observing satellite, whose goal is to image Earth’s landmass at predefined targets. The considered goal in this work for DT and onboard autonomy is avoidance of cloud cover, which can cause up to 66% of imaging tasks to be occluded, but factoring real-world constraints on operationalization and onboard edge computing. Instead of using end-to-end learned methods, we build upon existing work on spacecraft scheduling, incorporating a mixed-integer linear program (MILP) scheduler as the primary scheduling algorithm. Rather than directly incorporating DT into a global problem, we instead develop a set of heuristics which can estimate the utility of lookahead actions. We construct these heuristics from two directions: one from a simplified and constrained version of the scheduling problem with order statistics, and a second using a convolutional neural network with large amounts of synthetically generated data. We also consider DT using information from meteorological satellites in geostationary Earth orbit (GEO), parameterizing information delay rather than performing detailed analysis of data pipelines. In cases where all tasks are equally valued, all DT cases tested, including both meteorological and vision cases, outperform the conventional scheduler across all trials, ranging between 40% and 100% increase in total schedule utility based on cloud-free captures, depending on the DT method used. In cases where tasks have Pareto-distributed utility, the gap between the omniscient and conventional schedule shrinks drastically, to within 4% of total utility, and only a single DT method outperforms the conventional schedule consistently, as the environment becomes significantly more challenging due to asymmetric upside and downside risk. We also present methods by which to fractionate global state such that data can be efficiently stored and updated across a satellite constellation, allowing these heuristics to continue working across the constellation with minimal modification.