Hidden Markov Models for analysis of pilot instrument scanning and attention switching

Abstract

(cont.) high workload. The results of another flight simulation experiment demonstrated how the pilots' attention budgeting among these tasks estimated by HMM analysis, combined with the pilots' eye-movement statistical results, could enhance a cockpit display format study. The experiments demonstrated what additional insights can be obtained by incorporating HMM analysis into the analysis of pilots' eye movements.

Pilots' eye movements provide researchers rich information about the pilots' cognitive process during flight. Indeed, many researchers have included pilots' eye-movement measures in their flight simulator experiments. Currently, however, due to the lack of a reasonable model of pilots' scanning process, most researchers must rely on simple statistical analysis of eye-movement data, such as mean fixation durations on each instrument. The problem is that such statistical analyses often involve time-averaging operations, and so the information regarding the sequence of instrument scanning, the richest part of the data that reflects the pilot's moment-to-moment thought and attention processes, often has been lost or not fully utilized. The thesis proposes a new analysis tool based on Hidden Markov Models (HMMs). This analysis exploits pilots' instrument-crosschecking eye movements within an instrument group related to the vertical-, horizontal-, or airspeed-tracking task. From the pilots' eye-movement data, the HMM estimates the most likely sequence of underlying tracking tasks that the pilot attended to. HMM analysis is especially useful when some instruments overlap among multiple tracking tasks (e.g., the attitude indicator overlaps among all three tracking tasks) because it can utilize the sequential information from the instrument scanning to compute the likelihood of each of the possible tracking tasks. The actual pilot eye-movements data collected during ILS approach simulation experiments indicated that some experienced pilots may attend to more than three tasks during flight, with the fourth one being a monitoring task, while some inexperienced pilots may attend to only two, dropping one of the tracking tasks probably due to