Repository logo
Log in(current)
Repository logoMIT Open ScholarshipDSpace@MIT
  1. Home
  2. Center for Brains, Minds & Machines
  3. Publications
  4. CBMM Memo Series
  5. Transformer Module Networks for Systematic Generalization in Visual Question Answering

Transformer Module Networks for Systematic Generalization in Visual Question Answering

Thumbnail Image
Download
Name

CBMM-Memo-121.pdf

Size

1.06 MB

Format

Adobe PDF

Checksum (MD5)

f8a58e082f4366739963791126d54def

Author(s)
Yamada, Moyuru
•
D'Amario, Vanessa
•
Takemoto, Kentaro
•
Boix, Xavier
•
Sasaki, Tomotake
Date Issued
February 3, 2022
Publisher
Center for Brains, Minds and Machines (CBMM)
Series/Report no.
CBMM Memo;121
Abstract
Transformer-based models achieve great performance on Visual Question Answering (VQA). How- ever, when we evaluate them on systematic generalization, i.e., handling novel combinations of known concepts, their performance degrades. Neural Module Networks (NMNs) are a promising approach for systematic generalization that consists on composing modules, i.e., neural networks that tackle a sub-task. Inspired by Transformers and NMNs, we propose Transformer Module Network (TMN), a novel Transformer-based model for VQA that dynamically composes modules into a question-specific Transformer network. TMNs achieve state-of-the-art systematic generalization performance in three VQA datasets, namely, CLEVR-CoGenT, CLOSURE and GQA-SGL, in some cases improving more than 30% over standard Transformers.
Persistent DSpace Link
https://hdl.handle.net/1721.1/139843
Repository logo
PrivacyPermissionsAccessibilityContact us
Repository logo
Notify us about copyright concerns.