Notice

This is not the latest version of this item. The latest version can be found at:https://dspace.mit.edu/handle/1721.1/138223.5

Show simple item record

dc.contributor.authorGamarnik, David
dc.date.accessioned2021-11-29T16:27:32Z
dc.date.available2021-11-29T15:19:58Z
dc.date.available2021-11-29T16:27:32Z
dc.date.issued2021-09-27
dc.identifier.issn0178-8051
dc.identifier.urihttps://hdl.handle.net/1721.1/138223.2
dc.description.abstractAbstract We study support recovery for a $$k \times k$$ k × k principal submatrix with elevated mean $$\lambda /N$$ λ / N , hidden in an $$N\times N$$ N × N symmetric mean zero Gaussian matrix. Here $$\lambda >0$$ λ > 0 is a universal constant, and we assume $$k = N \rho $$ k = N ρ for some constant $$\rho \in (0,1)$$ ρ ∈ ( 0 , 1 ) . We establish that there exists a constant $$C>0$$ C > 0 such that the MLE recovers a constant proportion of the hidden submatrix if $$\lambda {\ge C} \sqrt{\frac{1}{\rho } \log \frac{1}{\rho }}$$ λ ≥ C 1 ρ log 1 ρ , while such recovery is information theoretically impossible if $$\lambda = o( \sqrt{\frac{1}{\rho } \log \frac{1}{\rho }} )$$ λ = o ( 1 ρ log 1 ρ ) . The MLE is computationally intractable in general, and in fact, for $$\rho >0$$ ρ > 0 sufficiently small, this problem is conjectured to exhibit a statistical-computational gap. To provide rigorous evidence for this, we study the likelihood landscape for this problem, and establish that for some $$\varepsilon >0$$ ε > 0 and $$\sqrt{\frac{1}{\rho } \log \frac{1}{\rho } } \ll \lambda \ll \frac{1}{\rho ^{1/2 + \varepsilon }}$$ 1 ρ log 1 ρ ≪ λ ≪ 1 ρ 1 / 2 + ε , the problem exhibits a variant of the Overlap-Gap-Property (OGP). As a direct consequence, we establish that a family of local MCMC based algorithms do not achieve optimal recovery. Finally, we establish that for $$\lambda > 1/\rho $$ λ > 1 / ρ , a simple spectral method recovers a constant proportion of the hidden submatrix.en_US
dc.publisherSpringer Berlin Heidelbergen_US
dc.relation.isversionofhttps://doi.org/10.1007/s00440-021-01089-7en_US
dc.rightsCreative Commons Attribution-Noncommercial-Share Alikeen_US
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/4.0/en_US
dc.sourceSpringer Berlin Heidelbergen_US
dc.titleThe overlap gap property in principal submatrix recoveryen_US
dc.typeArticleen_US
dc.identifier.citationGamarnik, David, Jagannath, Aukosh and Sen, Subhabrata. 2021. "The overlap gap property in principal submatrix recovery."en_US
dc.relation.journalProbability theory and related fieldsen_US
dc.eprint.versionAuthor's final manuscripten_US
dc.type.urihttp://purl.org/eprint/type/JournalArticleen_US
eprint.statushttp://purl.org/eprint/status/PeerRevieweden_US
dc.date.updated2021-11-25T05:00:55Z
dc.language.rfc3066en
dc.rights.holderThe Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature
dspace.embargo.termsY
dspace.date.submission2021-11-25T05:00:54Z
mit.journal.volume181en_US
mit.licenseOPEN_ACCESS_POLICY
mit.metadata.statusReady for final reviewen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

VersionItemDateSummary

*Selected version