Confidently Comparing Estimates with the c-value
Name
Confidently Comparing Estimates with the c-value.pdf
Description
Published version
Size
1.99 MB
Format
Adobe PDF
Checksum (MD5)
95019f58ab8553dc311ac94c989d8a2a
Author(s)
Trippe, Brian L
Deshpande, Sameer K
Broderick, Tamara
Date Issued
February 24, 2023
Journal
Journal of the American Statistical Association
Publisher
Taylor & Francis
Citation
Trippe, B. L., Deshpande, S. K., & Broderick, T. (2023). Confidently Comparing Estimates with the c-value. Journal of the American Statistical Association, 119(546), 983–994.
Version
Final published version
Abstract
Modern statistics provides an ever-expanding toolkit for estimating unknown parameters. Consequently, applied statisticians frequently face a difficult decision: retain a parameter estimate from a familiar method or replace it with an estimate from a newer or more complex one. While it is traditional to compare estimates using risk, such comparisons are rarely conclusive in realistic settings. In response, we propose the “c-value” as a measure of confidence that a new estimate achieves smaller loss than an old estimate on a given dataset. We show that it is unlikely that a large c-value coincides with a larger loss for the new estimate. Therefore, just as a small p-value supports rejecting a null hypothesis, a large c-value supports using a new estimate in place of the old. For a wide class of problems and estimates, we show how to compute a c-value by first constructing a data-dependent high-probability lower bound on the difference in loss. The c-value is frequentist in nature, but we show that it can provide validation of shrinkage estimates derived from Bayesian models in real data applications involving hierarchical models and Gaussian processes. Supplementary materials for this article are available online.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-NonCommercial-NoDerivatives
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1080/01621459.2022.2153688
