Augmenting Transformers for Open Domain Procedural Text Comprehesion
Name
Pei-ypei-meng-eecs-2021-thesis.pdf
Description
Thesis PDF
Size
824.83 KB
Format
Adobe PDF
Checksum (MD5)
83f786c1bfecada635b45349b8694e78
Author(s)
Pei, Yixuan
Advisor(s)
Shrobe, Howard
Date Issued
September 2021
Publisher
Massachusetts Institute of Technology
Abstract
Recent advances in deep learning model architectures have permitted state-of-the-art results in various fields such as NLP and CV. Although these systems have matched and, in some cases, surpassed human performance, many of them are still treated as black boxes, with sometimes unpredictable results. To try and shed some light on behaviors of natural language generation models, we examine the task of procedural text comprehension using neuro-symbolic techniques. We use this task as a testbed for exploring the limitations of state-of-the-art systems such as GPT on the task of predicting the resulting state changes from the text description of a procedure. We also experiment with whether and how symbolic augmentations may help these systems with understanding language. We see some promising results in concept-net knowledge injection, and note that other augmentations provide more natural generations.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
