Multimodal Data Fusion for Deep Learning Applications in Intracoronary Image Segmentation

Abstract

This thesis describes steps towards the construction of a multi-anatomical, multimodal segmentation and co-registration platform for intracoronary images. Although manual annotation and co-registration of intracoronary images from different modalities remain the gold standard today for facilitating the use of intravascular image analysis and morphological component extraction in guiding clinical decision-making, building automated pipelines for these tasks is of increasing interest to optimize these processes. This thesis consists of the construction of an optimized and robust multianatomical segmentation model, with experimentation detailed on different possible modes of pre-training. We also contribute to the process of creating a flexible, reliable platform that can segment and co-register intracoronary images of different imaging modalities, improving upon an in-house non-rigid registration procedure for co-registering coronary computed tomography angiography (CCTA) and optical coherence tomography (OCT) frames with the initialization of a new hybrid model that uses user-inputted fiduciary bifurcations as landmarks to guide the non-rigid registration process for intermediary frames in multimodal pullbacks. We hypothesize that this will enable the co-registration model to account for the global environment when aligning corresponding frames rather than relying solely on local optimization. The simultaneous development of both intravascular image segmentation and co-registration processes is conducted to contribute towards a greater ambition of creating a platform that can segment and co-register images from multiple modalities, pre and post-intervention.