OSPC: Multimodal Harmful Content Detection using Fine-tuned Language Models
Name
3589335.3665997.pdf
Size
960.29 KB
Format
Adobe PDF
Checksum (MD5)
5586306a7753352a0f69382b9b1f3f4b
Author(s)
Cai, Bill
Date Issued
May 13, 2024
Publisher
ACM|Companion Proceedings of the ACM Web Conference 2024
Citation
Cai, Bill. 2024. "OSPC: Multimodal Harmful Content Detection using Fine-tuned Language Models."
Version
Final published version
Abstract
The Online Safety Prize Challenge (OSPC) presented several challenges: (1) the lack of a training or sample dataset, and limited interactions with the submission portal, (2) limitations in hardware, software package size and processing time. In this report, we present our method that was consistently able to achieve AUROC score of above 0.74 (within top 3 of submissions). The following factors improved AUROC score significantly: (1) use of multilingual optical character recognition (OCR) models (+0.024), (2) exact logit scores instead of sampled decoding (+0.040), (3) fine-tuning of pretrained models on synthetically generated datasets (+0.076 to +0.106). We outline key implementation details in this report including the use of model quantization, robust integration testing including GPU memory leak checks and inference time restrictions.
Description
WWW ’24 Companion, May 13–17, 2024, Singapore, Singapore
MIT Department
Massachusetts Institute of Technology. Computation for Design and Optimization Program
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3589335.3665997
