The design of a HSMM-based operator state monitoring display

Abstract

This thesis presents the development of and the findings from the design and evaluation of a hidden semi- Markov model (HSMM)-based operator state monitoring display. This operator state monitoring display is designed to function as a decision support tool (DST) for the supervisor of a small team of operators (between 2 and 4 team members) that are each monitoring and controlling multiple highly autonomous heterogeneous unmanned vehicles (UVs). Such displays are important in real-time, mission-critical complex systems because both operator and vehicle state monitoring are difficult, and failure to appropriately handle emerging situations could have life or mission-critical consequences. Recent research has shown that HSMM-based models can be used to model the time-sensitive behavior of operators controlling multiple heterogeneous UVs. Because this method requires substantial knowledge in probability theory to understand, the difficulty lies in the accurate, useful display of the HSMM information to a team commander in the field. It must be assumed that the team commander generally does not have the required background in stochastic processes to understand the method and may be biased in interpreting probabilistic functions. This further increases the difficulty of the proposed method. In this thesis, a cognitive task analysis (CTA) was performed to determine the functional and information requirements of the DST, and a human systems engineering design process was used to develop a prototype display. A human subject experiment was then conducted to test the effectiveness of the DST. The DST was shown to improve team supervisor performance in terms of increased decision accuracy, decreased incorrect interventions, and decreased response times in single alert scenarios. The DST was also shown to decrease the number of incorrect interventions, while having no affect on decision accuracy and total response time scenarios when the supervisor faced multiple simultaneous alerts. Additionally, the DST was not shown to increase operator mental workload, as measured by a secondary task, for any of the scenarios. Overall, the results suggest that HSMM-based operator state modeling can be effectively utilized in a real-time DST for team supervisors. While this research was focused on a team supervisor of multiple operators each independently controlling multiple heterogeneous UVs, the results are generalizable, and any research in time-critical team HSC domains may benefit from this work.