Process-oriented evaluation of user interactions in integrated system analysis tools

Abstract

When computer-based tools are used for analysis of complex systems, the design of user interactions and interfaces becomes an essential part of development that determines the overall quality. The objective of this study is to investigate the processes and results of user interactions with integrated analysis tools to synthesize design implications for future tool development. In this study, two space exploration logistics tools are compared in a controlled user experiment. Through a comparative usability analysis, this study evaluated user performance and perception to provide design implications for future integrated analysis tools. For a comprehensive evaluation, multiple methods were used for data collection, including observation, questionnaire and interview. In addition to a result-oriented performance analysis, a process-oriented approach was used for analyzing patterns in user behaviors and errors. Results are presented with reference to the related features embedded in the interfaces of the two tools. Based on the comparative results, synthesized design insights for hierarchical structure, model transparency, automation, and visualization and feedback are discussed for integrated analysis tools in general.