Hierarchical optimal transport for document representation
Name
NeurIPS-2019-hierarchical-optimal-transport-for-document-representation-Paper.pdf
Description
Published version
Size
2.3 MB
Format
Unknown
Checksum (MD5)
6d6d3e2a11580b316eea69e6aed9dc21
Author(s)
Yurochkin, Mikhail
Claici, Sebastian
Chien, Edward
Mirzazadeh, Farzaneh
Solomon, Justin
Date Issued
December 2019
Journal
Advances in Neural Information Processing Systems
Citation
2019. "Hierarchical optimal transport for document representation." Advances in Neural Information Processing Systems, 32.
Version
Final published version
Abstract
© 2019 Neural information processing systems foundation. All rights reserved. The ability to measure similarity between documents enables intelligent summarization and analysis of large corpora. Past distances between documents suffer from either an inability to incorporate semantic similarities between words or from scalability issues. As an alternative, we introduce hierarchical optimal transport as a meta-distance between documents, where documents are modeled as distributions over topics, which themselves are modeled as distributions over words. We then solve an optimal transport problem on the smaller topic space to compute a similarity score. We give conditions on the topics under which this construction defines a distance, and we relate it to the word mover's distance. We evaluate our technique for k-NN classification and show better interpretability and scalability with comparable performance to current methods at a fraction of the cost.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
MIT-IBM Watson AI Lab
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
