A machine learning approach to molecular structure recognition in chemical literature

Abstract

Reaction diagrams in chemistry papers contain essential reaction information that is not available in the text. In order to extract comprehensive reaction information from chemistry literature, it is vital to convert these diagrams into a format compatible with searchable cheminformatic databases. Existing methods rely on rule-based procedures that have difficulty generalizing to noisy or different styled images. In this thesis, I implement a deep learning pipeline for identifying molecules in chemical diagrams and 'translating' the images into their corresponding SMILES strings. Diagram segmentation is performed using Mask R-CNN trained on an automatically generated set of diagrams. Translation to SMILES strings is performed using a neural machine translation model augmented with domain adaptation. Experimental results suggest that this model outperforms both rule-based and machine learning based models on diagrams extracted from real chemical literature.