Necessary and sufficient conditions for high-dimensional salient feature subset recovery
Name
Willsky_Necessary and sufficient.pdf
Size
156.55 KB
Format
Adobe PDF
Checksum (MD5)
fac63b9260c77901d9fdbc364f1276d8
Author(s)
Tan, Vincent Yan Fu
Johnson, Matthew James
Willsky, Alan S.
Date Issued
July 2010
Journal
Proceedings of the IEEE International Symposium on Information Theory Proceedings (ISIT), 2010
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Tan, Vincent Y. F., Matthew Johnson, and Alan S. Willsky. “Necessary and Sufficient Conditions for High-dimensional Salient Feature Subset Recovery.” IEEE International Symposium on Information Theory Proceedings (ISIT), 2010. 1388–1392. ©2010 IEEE
Version
Final published version
Abstract
We consider recovering the salient feature subset for distinguishing between two probability models from i.i.d. samples. Identifying the salient set improves discrimination performance and reduces complexity. The focus in this work is on the high-dimensional regime where the number of variables d, the number of salient variables k and the number of samples n all grow. The definition of saliency is motivated by error exponents in a binary hypothesis test and is stated in terms of relative entropies. It is shown that if n grows faster than max{ck log((d-k)/k), exp(c'k)} for constants c, c', then the error probability in selecting the salient set can be made arbitrarily small. Thus, n can be much smaller than d. The exponential rate of decay and converse theorems are also provided. An efficient and consistent algorithm is proposed when the distributions are graphical models which are Markov on trees.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ISIT.2010.5513598
