Noisy with a Chance of Mislabels: A Local and Training Dynamics Perspective on Detecting Label Noise in Deep Classification

Abstract

Noisy labels are a pervasive challenge in modern supervised learning, especially in highstakes domains such as healthcare, where model reliability is critical. Detecting and mitigating the influence of mislabeled data is essential to improving both performance and interpretability. Building on insights from training dynamics, we propose Local Consistency across Training Epochs (LoCaTE), a class of data-filtering methods that leverages over-parameterized and over-trained neural networks to distinguish clean samples from mislabeled ones. Our approach integrates both local neighborhood information and the behavior of samples across training epochs to identify noise and enhance model robustness. We evaluate our method on real (human) and synthetic label noise across three classification datasets, finding that it achieves competitive F₁ of label error detection and improved downstream accuracy using a lightweight classifier with low added computational cost.