Modeling the effect of trend information on human failure detection and diagnosis in spacecraft systems

Abstract

Systems are performing increasingly complicated tasks, made possible by significant advances in hardware and software technology. This task complexity is reflected in the system design, with a corresponding increased demand on comprehensive design efforts. Fundamental to the safety and mission success of these systems is the tradeoffs between human tasking and system tasking, and the resultant human interface. The research presented in this thesis was motivated by the development of an early-stage system design tool. This tool includes models of human decision making in order to evaluate system design tradeoffs with regard to human performance. An experiment was conducted to evaluate the effect of trend information displays on human decision making performance. Decision latency and accuracy were examined as performance metrics. To elicit information regarding the subjects' decision making process, the Lens model was used to gather metrics on achievement and decision consistency. The experimental results showed that both detection latency and diagnosis accuracy improved when trend information about dynamic system parameters is explicitly available to operators of spacecraft systems. The presence of this additional information also improved decision consistency. However, it made no significant difference for subjects' detection accuracy, diagnosis latency or achievement. Other predictors of latency and accuracy included the type of failure and the spacecraft trajectory. This was expected as both of these factors are important contributors to an operator's mental model of normal system behavior, which is critical to detecting and identifying failures. From these results, it can be concluded that operators of spacecraft systems could benefit from the inclusion of trend information, since it improves failure detection and diagnosis performance which can improve overall mission safety and success.