Simultaneous learning of trees and representations for extreme classification and density estimation
Name
jernite17a.pdf
Description
Published version
Size
354.09 KB
Format
Adobe PDF
Checksum (MD5)
ff402c229c3475f2d8fda547f25572c8
Author(s)
Sontag, David Alexander
Date Issued
August 2017
Journal
Proceedings of Machine Learning Research
Publisher
International Machine Learning Society
Citation
Jernite, Yacine et al. “Simultaneous learning of trees and representations for extreme classification and density estimation.” Paper in the Proceedings of Machine Learning Research, 70, 34th International Conference on Machine Learning, Sydney, Australia, August 6-11 2017, International Machine Learning Society © 2017 The Author(s)
Version
Final published version
Abstract
We consider multi-class classification where the predictor has a hierarchical structure that allows for a very large number of labels both at train and test time. The predictive power of such models can heavily depend on the structure of the tree, and although past work showed how to learn the tree structure, it expected that the feature vectors remained static. We provide a novel algorithm to simultaneously perform representation learning for the input data and learning of the hierarchical predictor. Our approach optimizes an objective function which favors balanced and easily-separable multi-way node partitions. We theoretically analyze this objective, showing that it gives rise to a boosting style property and a bound on classification error. We next show how to extend the algorithm to conditional density estimation. We empirically validate both variants of the algorithm on text classification and language modeling, respectively, and show that they compare favorably to common baselines in terms of accuracy and running time.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://proceedings.mlr.press/v70/
