Investigation on ImageNet Remaining Errors with TRAK

Abstract

The Imagenet dataset is an important benchmark and test bed for computer vision models. Two of its most important characteristics are the size and difficulty, which were what motivated the breakthrough deep learning model Alexnet a decade ago. As researches progress and computation power grows, the best models nowadays can achieve accuracy as high as 90% on Imagenet. With such high accuracy, model predictions are usually of high precision and the causes of this long tail of error are unknown. Many studies have suggested that reassessing Imagenet, a nontrivial amount of label error and noise is found and effort had been made to fix this label noise in the test set, mainly through manual review. However, not many studies have dived into fixing labels for the training set, largely due to its large scale. The proposed thesis aims to understand the remaining errors that models are still making on the ImageNet dataset and investigate the labeling problems in the Imagenet training set, utilizing TRAK- a recently developed efficient data attribution method to help identify problematic images among the 1.4 million images in Imagenet training set.