Investigating Different Image Representations for Image Retrieval

Abstract

With image and video databases becoming more prevalent and larger, so too does the tools used for navigating and searching them. Image retrieval, or the process of finding one particular image from a set of images, is one such process. Image retrieval relies on some innate searching of the images in the database being retrieved from. Rather than doing this on the full images, image retrieval systems use image representations to allow the searching to happen faster while maintaining the image information to retrieve the correct image. The MIT Data Systems Group (DSG) created Seesaw, a system for interactive ad-hoc searches in image data sets with no assumption of pre-defined search queries made in advance. This thesis focuses on investigating different image representations to see which can improve image retrieval in the case of Seesaw. This project looks at different segmentation models and region proposal networks from Mask R-CNNs to see if these trained models can provide any comparable or better performance in Seesaw compared to the non-trained image representation system it currently uses. What is found is that the region proposal network representation performs on par to the representation found in Seesaw, but additionally that segmentation models can be used to eliminate some vectors in the Seesaw representation without compromising on performance.