Empirical Bayes via ERM and Rademacher complexities: the Poisson model
Name
teh-anzoteh-sm-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
863.95 KB
Format
Adobe PDF
Checksum (MD5)
8d866316163d7311c5de4607ec358c2b
Author(s)
Teh, Anzo Zhao Yang
Advisor(s)
Polyanskiy, Yury
Date Issued
September 2023
Publisher
Massachusetts Institute of Technology
Abstract
We consider the problem of empirical Bayes estimation for (multivariate) Poisson means. Existing solutions that have been shown theoretically optimal for minimizing the regret (excess risk over the Bayesian oracle that knows the prior) have several shortcomings. For example, the classical Robbins estimator does not retain the monotonicity property of the Bayes estimator and performs poorly under moderate sample size. Estimators based on
the minimum distance and non-parametric maximum likelihood (NPMLE) methods correct these issues, but are computationally expensive with complexity growing exponentially with
dimension. Extending the approach of Barbehenn and Zhao (2022), in this work we construct monotone estimators based on empirical risk minimization (ERM) that retain similar theoretical guarantees and can be computed much more efficiently. Adapting the idea of offset Rademacher complexity Liang et. al (2015) to the non-standard loss and function class in empirical Bayes, we show that the shape-constrained ERM estimator attains the minimax regret within constant factors in one dimension and within logarithmic factors in multiple dimensions.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
