A large-deviation analysis for the maximum likelihood learning of tree structures
Name
Anandkumar-2009-A large-deviation analysis for the maximum likelihood learning of tree structures.pdf
Size
992.09 KB
Format
Adobe PDF
Checksum (MD5)
93ee7178d909eb5e96034b4bb296738e
Author(s)
Tong, Lang
Tan, Vincent Yan Fu
Anandkumar, Animashree
Willsky, Alan S.
Date Issued
August 2009
Journal
IEEE International Symposium on Information Theory, 2009. ISIT 2009.
Citation
Tan, V.Y.F. et al. “A large-deviation analysis for the maximum likelihood learning of tree structures.” Information Theory, 2009. ISIT 2009. IEEE International Symposium on. 2009. 1140-1144. Web.
Version
Final published version
Abstract
The problem of maximum-likelihood learning of the structure of an unknown discrete distribution from samples is considered when the distribution is Markov on a tree. Large-deviation analysis of the error in estimation of the set of edges of the tree is performed. Necessary and sufficient conditions are provided to ensure that this error probability decays exponentially. These conditions are based on the mutual information between each pair of variables being distinct from that of other pairs. The rate of error decay, or error exponent, is derived using the large-deviation principle. The error exponent is approximated using Euclidean information theory and is given by a ratio, to be interpreted as the signal-to-noise ratio (SNR) for learning. Numerical experiments show the SNR approximation is accurate.
Subjects
Error exponents
Euclidean Information Theory
Large-deviations
Tree structure learning
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Stochastic Systems Group
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ISIT.2009.5206012
