Determining thresholds using adaptive procedures and psychometric fits: evaluating efficiency using theory, simulations, and human experiments
Name
221_2015_Article_4501.pdf
Size
849.02 KB
Format
Adobe PDF
Checksum (MD5)
284608ecd61aa7adebc157bf4a7b26ad
Author(s)
Karmali, Faisal
Yi, Yongwoo
Merfeld, Daniel M.
Chaudhuri, Shomesh Ernesto
Date Issued
December 2015
Journal
Experimental Brain Research
Publisher
Springer Berlin Heidelberg
Citation
Karmali, Faisal et al. “Determining Thresholds Using Adaptive Procedures and Psychometric Fits: Evaluating Efficiency Using Theory, Simulations, and Human Experiments.” Experimental Brain Research 234.3 (2016): 773–789.
Version
Author's final manuscript
Abstract
When measuring thresholds, careful selection of stimulus amplitude can increase efficiency by increasing the precision of psychometric fit parameters (e.g., decreasing the fit parameter error bars). To find efficient adaptive algorithms for psychometric threshold (“sigma”) estimation, we combined analytic approaches, Monte Carlo simulations, and human experiments for a one-interval, binary forced-choice, direction-recognition task. To our knowledge, this is the first time analytic results have been combined and compared with either simulation or human results. Human performance was consistent with theory and not significantly different from simulation predictions. Our analytic approach provides a bound on efficiency, which we compared against the efficiency of standard staircase algorithms, a modified staircase algorithm with asymmetric step sizes, and a maximum likelihood estimation (MLE) procedure. Simulation results suggest that optimal efficiency at determining threshold is provided by the MLE procedure targeting a fraction correct level of 0.92, an asymmetric 4-down, 1-up staircase targeting between 0.86 and 0.92 or a standard 6-down, 1-up staircase. Psychometric test efficiency, computed by comparing simulation and analytic results, was between 41 and 58 % for 50 trials for these three algorithms, reaching up to 84 % for 200 trials. These approaches were 13–21 % more efficient than the commonly used 3-down, 1-up symmetric staircase. We also applied recent advances to reduce accuracy errors using a bias-reduced fitting approach. Taken together, the results lend confidence that the assumptions underlying each approach are reasonable and that human threshold forced-choice decision making is modeled well by detection theory models and mimics simulations based on detection theory models.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s00221-015-4501-8
