Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks
Author(s)Varshney, Kush R.; Willsky, Alan S.; Willsky, Alan
MetadataShow full item record
Low-dimensional statistics of measurements play an important role in detection problems, including those encountered in sensor networks. In this work, we focus on learning low-dimensional linear statistics of high-dimensional measurement data along with decision rules defined in the low-dimensional space in the case when the probability density of the measurements and class labels is not given, but a training set of samples from this distribution is given. We pose a joint optimization problem for linear dimensionality reduction and margin-based classification, and develop a coordinate descent algorithm on the Stiefel manifold for its solution. Although the coordinate descent is not guaranteed to find the globally optimal solution, crucially, its alternating structure enables us to extend it for sensor networks with a message-passing approach requiring little communication. Linear dimensionality reduction prevents overfitting when learning from finite training data. In the sensor network setting, dimensionality reduction not only prevents overfitting, but also reduces power consumption due to communication. The learned reduced-dimensional space and decision rule is shown to be consistent and its Rademacher complexity is characterized. Experimental results are presented for a variety of datasets, including those from existing sensor networks, demonstrating the potential of our methodology in comparison with other dimensionality reduction approaches.
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science; Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
IEEE Transactions on Signal Processing
Institute of Electrical and Electronics Engineers
Varshney, Kush R., and Alan S. Willsky. Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks. IEEE Transactions on Signal Processing 59, no. 6 (June 2011): 2496-2512.
Author's final manuscript