Evaluation Criteria for Human-Automation Performance Metrics
Name
Cummings_Evaluation Criteria.pdf
Size
522.66 KB
Format
Adobe PDF
Checksum (MD5)
9c0ba69591f4f1dbd4ef4d83305d4dfd
Author(s)
Pina, Patricia Elena
Cummings, M. L.
Donmez, Birsen
Date Issued
January 2008
Journal
ACM Workshop on Performance Metrics for Intelligent Systems
Publisher
Association for Computing Machinery
Citation
Donmez, Birsen, Patricia E. Pina, and M. L. Cummings. “Evaluation criteria for human-automation performance metrics.” Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems. Gaithersburg, Maryland: ACM, 2008. 77-82. c2008 Association for Computing Machinery
Version
Author's final manuscript
Abstract
Previous research has identified broad metric classes for human-automation performance to facilitate metric selection, as well as understanding and comparison of research results. However, there is still lack of an objective method for selecting the most efficient set of metrics. This research identifies and presents a list of evaluation criteria that can help determine the quality of a metric in terms of experimental constraints, comprehensive understanding, construct validity, statistical efficiency, and measurement technique efficiency. Future research will build on these evaluation criteria and existing generic metric classes to develop a cost-benefit analysis approach that can be used for metric selection.
Subjects
Metric Quality
Human Supervisory Control
Validity
Statistics
Experiments
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/1774674.1774687
