Towards abstractive captioning of infographics
Author(s)Landman, Nathan, M. Eng. Massachusetts Institute of Technology
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
MetadataShow full item record
Machine understanding of text-based narratives have predominantly focused on documents with rigid hierarchical structures and sequentially ordered inputs. These inputs include documents such as news stories, encyclopedia entries, books, and many others. However, little research has focused on understanding text-based information without this structure. Current text understanding models fail when information is presented in less structured ways, without a clear and pre-defined spatial arrangement of the content. This thesis explores a subset of components required for understanding infographics -- documents whose structure is not necessarily linear and whose content may involve a variety of images. We expand on state-of-the-art methodologies in character recognition and text summarization in order to better understand how to process content without a pre-determined spatial arrangement, and subsequently generate captions for given infographics automatically. To shine light at the reasoning behind the captions being generated, we develop a graphical user interface that helps visualize the portions of a document being used when generating specific parts of a caption.
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 91-94).
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology
Electrical Engineering and Computer Science.