A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures
Name
Willsky_A large-deviation.pdf
Size
485.79 KB
Format
Adobe PDF
Checksum (MD5)
612b41fe17daa89bea0b6b9e9167ba33
Author(s)
Tan, Vincent Yan Fu
Willsky, Alan S.
Anandkumar, Animashree
Tong, Lang
Date Issued
March 2011
Journal
IEEE Transactions on Information Theory
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Tan, Vincent Y. F. et al. “A Large-Deviation Analysis of the Maximum-Likelihood Learning of Markov Tree Structures.” IEEE Transactions on Information Theory 57.3 (2011): 1714–1735. © Copyright 2011 IEEE
Version
Author's final manuscript
Abstract
The problem of maximum-likelihood (ML) estimation of discrete tree-structured distributions is considered. Chow and Liu established that ML-estimation reduces to the construction of a maximum-weight spanning tree using the empirical mutual information quantities as the edge weights. Using the theory of large-deviations, we analyze the exponent associated with the error probability of the event that the ML-estimate of the Markov tree structure differs from the true tree structure, given a set of independently drawn samples. By exploiting the fact that the output of ML-estimation is a tree, we establish that the error exponent is equal to the exponential rate of decay of a single dominant crossover event. We prove that in this dominant crossover event, a non-neighbor node pair replaces a true edge of the distribution that is along the path of edges in the true tree graph connecting the nodes in the non-neighbor pair. Using ideas from Euclidean information theory, we then analyze the scenario of ML-estimation in the very noisy learning regime and show that the error exponent can be approximated as a ratio, which is interpreted as the signal-to-noise ratio (SNR) for learning tree distributions. We show via numerical experiments that in this regime, our SNR approximation is accurate.
Description
November 21, 2010
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/tit.2011.2104513
