Universal regression with adversarial responses
Name
Download (10).pdf
Description
Published version
Size
318.94 KB
Format
Adobe PDF
Checksum (MD5)
3daab0a8a41936bcb768e1d993657fd5
Author(s)
Blanchard, Moïse
Jaillet, Patrick
Date Issued
June 2023
Journal
The Annals of Statistics
Publisher
Institute of Mathematical Statistics
Citation
Moïse Blanchard. Patrick Jaillet. "Universal regression with adversarial responses." Ann. Statist. 51 (3) 1401 - 1426, June 2023.
Version
Final published version
Abstract
We provide algorithms for regression with adversarial responses under large classes of non-i.i.d. instance sequences, on general separable metric spaces, with provably minimal assumptions. We also give characterizations of learnability in this regression context. We consider universal consistency, which asks for strong consistency of a learner without restrictions on the value responses. Our analysis shows that such an objective is achievable for a significantly larger class of instance sequences than stationary processes, and unveils a fundamental dichotomy between value spaces: whether finite-horizon mean estimation is achievable or not. We further provide optimistically universal learning rules, that is, such that if they fail to achieve universal consistency, any other algorithms will fail as well. For unbounded losses, we propose a mild integrability condition under which there exist algorithms for adversarial regression under large classes of non-i.i.d. instance sequences. In addition, our analysis also provides a learning rule for mean estimation in general metric spaces that is consistent under adversarial responses without any moment conditions on the sequence, a result of independent interest.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1214/23-AOS2299
