Optimizing MQ-9 Training Schedules in Highly Constrained Environments using Reinforcement Learning
Name
Kenyon_Final V2.pdf
Size
358.48 KB
Format
Adobe PDF
Checksum (MD5)
e5c8afe6bae6aa5fc54e306882ad929b
Author(s)
Kenyon, Aaron
Underhill, Ngaire
Date Issued
September 11, 2026
Abstract
Generating MQ-9 training schedules is a highly constrained planning problem that requires schedulers to balance student progression, instructor qualifications, aircraft and simulator availability, shift windows, appointments, syllabus dependencies, and changing unit priorities. In many units, this process remains heavily manual and spreadsheet-based, placing a daily administrative burden on small scheduling teams. This paper presents the Optimized Aircrew Scheduling and Integrated System (OASIS), a human-in-the-loop scheduling management system designed to support MQ-9 Formal Training Unit (FTU) scheduling through automated schedule generation, validation, review, and publication. The OASIS model's daily schedule generation is a priority driven resource-allocation problem of selecting and ordering Candidate Events (CEs) that prioritize student training while accounting for event pairings, qualification requirements, crew-position support, availability windows, and resource constraints. This paper compares two optimization approaches: a deterministic greedy scheduler and a Reinforcement Learning (RL) event-ordering policy. The greedy scheduler implements a deterministic schedule-construction and validation pipeline while the RL policy uses a masked-action environment to reorder CEs before utilizing the same deterministic scheduleconstruction method from the greedy scheduler. Compared to the greedy baseline, results showed that the RL policy increased the number of CEs scheduled but did not consistently improve syllabus completions or reduce syllabus completion times. These results suggest that RL is promising for increasing the number of CEs scheduled, although additional testing and reward selection may be necessary to convert additional scheduled activity into faster syllabi completion. OASIS reduces manual scheduling burden, exposes schedule conflicts, and provides a foundation for future learning-based optimization of MQ-9 training pipelines.
MIT Department
Lincoln Laboratory
Persistent DSpace Link
