Language-Centric Medical Image Understanding

Abstract

This thesis advances medical image understanding by leveraging the multifaceted roles of language: as supervision, prior knowledge, and a medium for communication. We introduce three main contributions: (1) a weakly supervised framework that uses language in clinical reports to guide fine-grained alignment between image regions and textual descriptions, (2) an adaptive debiasing method that uses language prior to improve the robustness of learning algorithms under noisy supervision, and (3) a novel approach for calibrating linguistic expressions of diagnostic certainty, enabling more reliable communication of clinical findings. Together, these methods lead to more accurate, robust, and reliable machine learning systems, ultimately streamlining clinical workflows and improving patient care.