Molding CNNs for text: Non-linear, non-consecutive convolutions
Name
Barzilay_Molding cnns.pdf
Size
544.94 KB
Format
Adobe PDF
Checksum (MD5)
e71dfaaae2367470f4255a9b2c35973c
Author(s)
Lei, Tao
Barzilay, Regina
Jaakkola, Tommi S.
Date Issued
September 2015
Journal
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics
Citation
Lei, Tao, Regina Barzilay, and Tommi Jaakkola. "Molding CNNs for text: Non-linear, non-consecutive convolutions." Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1565–1575, Lisbon, Portugal, 17-21 September 2015., pp.1565-1575.
Version
Original manuscript
Abstract
The success of deep learning often derives from well-chosen operational building blocks. In this work, we revise the temporal convolution operation in CNNs to better adapt it to text processing. Instead of concatenating word representations, we appeal to tensor algebra and use
low-rank n-gram tensors to directly exploit interactions between words already at the convolution stage. Moreover, we extend the n-gram convolution to non-consecutive words to recognize patterns with intervening words. Through a combination of low-rank tensors, and pattern weighting, we
can efficiently evaluate the resulting convolution operation via dynamic programming. We test the resulting architecture on standard sentiment classification and news categorization tasks.Our model achieves state-of-the-art performance both in terms of accuracy and training speed. For instance, we obtain 51.2% accuracy on the fine-grained sentiment classification task.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
http://www.emnlp2015.org/proceedings/EMNLP/
