Optimally tackling covariate shift in RKHS-based nonparametric regression
Name
Download (8).pdf
Description
Published version
Size
322.12 KB
Format
Adobe PDF
Checksum (MD5)
3c342e908214b8c891df4e08910ec7a3
Author(s)
Ma, Cong
Pathak, Reese
Wainwright, Martin J
Date Issued
April 2023
Journal
The Annals of Statistics
Publisher
Institute of Mathematical Statistics
Citation
Cong Ma. Reese Pathak. Martin J. Wainwright. "Optimally tackling covariate shift in RKHS-based nonparametric regression." Ann. Statist. 51 (2) 738 - 761, April 2023.
Version
Final published version
Abstract
We study the covariate shift problem in the context of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We focus on two natural families of covariate shift problems defined using the likelihood ratios between the source and target distributions. When the likelihood ratios are uniformly bounded, we prove that the kernel ridge regression (KRR) estimator with a carefully chosen regularization parameter is minimax rate-optimal (up to a log factor) for a large family of RKHSs with regular kernel eigenvalues. Interestingly, KRR does not require full knowledge of the likelihood ratio apart from an upper bound on it. In striking contrast to the standard statistical setting without covariate shift, we also demonstrate that a naïve estimator, which minimizes the empirical risk over the function class, is strictly suboptimal under covariate shift as compared to KRR. We then address the larger class of covariate shift problems where likelihood ratio is possibly unbounded yet has a finite second moment. Here, we propose a reweighted KRR estimator that weights samples based on a careful truncation of the likelihood ratios. Again, we are able to show that this estimator is minimax optimal, up to logarithmic factors.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1214/23-AOS2268
