Neural network architectures for Prepositional Phrase attachment disambiguation

Abstract

This thesis addresses the problem of Prepositional Phrase (PP) attachment disambiguation, a key challenge in syntactic parsing. In natural language sentences, a PP may often be attached to several possible candidates. While humans can usually identify the correct candidate successfully, syntactic parsers are known to have high error rated on this kind of construction. This work explores the use of compositional models of meaning in choosing the correct attachment location. The compositional model is defined using a recursive neural network. Word vector representations are obtained from large amounts of raw text and fed into the neural network. The vectors are first forward propagated up the network in order to create a composite representation, which is used to score all possible candidates. In training, errors are back-propagated down the network such that the composition matrix is updated from the supervised data. Several possible neural architectures are designed and experimentally tested in both English and Arabic data sets. As a comparative system, we offer a learning-to-rank algorithm based on an SVM classifier which has access to a wide range of features. The performance of this system is compared to the compositional models.