Developing an abstraction layer for the visualization of HSMM-based predictive decision support

Abstract

Hidden semi-Markov models (HSMMs) have been previously proposed as real-time operator behavior prediction models that could be used by a supervisor to detect future anomalous behaviors. Because of the disconnect between HSMM prediction results and the data format anticipated by the decision support visualization (DSV) display designer, an abstraction layer was developed to transform HSMM results into data in the anticipated format. In order to transform the raw HSMM results, a model accuracy scoring metric was created to assess HSMM prediction data and produce model performance trend data with a graphical depiction of variance and lower bounds. A prediction-generating (PG) algorithm was devised to utilize the model accuracy scoring metric and the HSMM library functions to generate multi-step ahead predictions up to 3 minutes into the future. In order to implement a responsive decision support system monitoring up to 10 operators simultaneously, original design requirements constrained maximum latency at 500ms, as suggested by previous research. However, the PG algorithm yielded significant system latency, and thus, computational enhancements were put in place to speed up the algorithm. Moreover, trade-offs were made between the length of input to the PG algorithm and the length of predictions generated. Both parameters were linearly proportional to latency. Other research has shown that a maximum latency of less than 200ms may be more desirable, and thus, the total number of operators supported would be down to 4 per the given system. The resulting proof-of-concept system operates in real-time, providing a team supervisor the most up-to-date supervision of up to 4 UV operators simultaneously. A pilot study was conducted to test the usability of the system where no major issues were found, and the study proved that the system operates as per the design requirements.