Sharp thresholds in inference of planted subgraphs
Name
Download (13).pdf
Description
Published version
Size
516.12 KB
Format
Adobe PDF
Checksum (MD5)
fecc191f16b6cb165a3f02a9303b0634
Author(s)
Mossel, Elchanan
Niles-Weed, Jonathan
Sohn, Youngtak
Sun, Nike
Zadik, Ilias
Date Issued
February 2025
Journal
The Annals of Applied Probability
Publisher
Institute of Mathematical Statistics
Citation
Elchanan Mossel. Jonathan Niles-Weed. Youngtak Sohn. Nike Sun. Ilias Zadik. "Sharp thresholds in inference of planted subgraphs." Ann. Appl. Probab. 35 (1) 523 - 563, February 2025.
Version
Final published version
Abstract
We connect the study of phase transitions in high-dimensional statistical inference to the study of threshold phenomena in random graphs.
A major question in the study of the Erdős–Rényi random graph G(nⓜ,p)is to understand the probability, as a function of p, that G(nⓜ,p)contains a given subgraph H=H_nH, starting with classical work of Erdős and Rényi (1960). More recent work studies this question for general H, both in building a general theory of sharp versus coarse transitions (Friedgut and Bourgain (1999); Hatami (2012)) and in results on the location of the transition (Kahn and Kalai (2007); Talagrand (2010); Frankston, Kahn, Narayanan, Park (2019); Park and Pham (2022)).
In inference problems, one often studies the optimal accuracy of inference as a function of the amount of noise. In a variety of sparse recovery problems, an “all-or-nothing (AoN) phenomenon” has been observed: Informally, as the amount of noise is gradually increased, at some critical threshold the inference problem undergoes a sharp jump from near-perfect recovery to near-zero accuracy (Gamarnik and Zadik (2017); Reeves, Xu, Zadik (2021)). We can regard AoN as the natural inference analogue of the sharp threshold phenomenon in random graphs. In contrast with the general theory developed for sharp thresholds of random graph properties, the AoN phenomenon has only been studied so far in specific inference settings, and a general theory behind its appearance remains elusive.
In this paper we study the general problem of inferring a graph H=H_nplanted in an Erdős–Rényi random graph, thus naturally connecting the two lines of research mentioned above. We show that questions of AoN are closely connected to first moment thresholds, and to a generalization of the so-called Kahn–Kalai expectation threshold that scans over subgraphs of H of edge density at least q. In a variety of settings we characterize AoN, by showing that AoN occurs if and only if this “generalized expectation threshold” is roughly constant in q. Our proofs combine techniques from random graph theory and Bayesian inference.
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1214/24-aap2120
