OSPC: Multimodal Harmful Content Detection using Fine-tuned Language Models

Abstract

The Online Safety Prize Challenge (OSPC) presented several challenges: (1) the lack of a training or sample dataset, and limited interactions with the submission portal, (2) limitations in hardware, software package size and processing time. In this report, we present our method that was consistently able to achieve AUROC score of above 0.74 (within top 3 of submissions). The following factors improved AUROC score significantly: (1) use of multilingual optical character recognition (OCR) models (+0.024), (2) exact logit scores instead of sampled decoding (+0.040), (3) fine-tuning of pretrained models on synthetically generated datasets (+0.076 to +0.106). We outline key implementation details in this report including the use of model quantization, robust integration testing including GPU memory leak checks and inference time restrictions.