Detecting check-worthy claims

Abstract

An automated fact-checking system aims to check the factuality of published information, such as news articles or blog posts. It is a time-consuming process and contains several steps: (i ) detecting check-worthy claims, (ii ) extracting a set of evidence for a given worthy claim from reliable sources, (iii ) predicting the stances of the set of evidence with respect to the claim, (iv ) integrating all information from (ii ) and (iii ) to detect whether the claim is factually true or false. In this thesis, we focus on the first step that is detecting check-worthy claims. It aims to facilitate manual fact-checking efforts by prioritizing the claims that fact-checkers should consider first. Previous works [Jaradat et al., 2018; Gencheva et al., 2017] for detecting checkworthy claims mainly focused on a debate domain, and used a set of hand-crafted features and simple semantic representations at the sentence level based on averaged word2vec embeddings [Mikolov et al., 2013]. Their limitations are (i ) less attention to the context and text representation, and (ii ) the limited size of the developed debate dataset that is not enough to train the neural network models. To address these limitations, in this thesis, we investigate different approaches to incorporate context both at word and sentence levels using recurrent neural networks (RNNs), Embeddings from Language Models (ELMo) [Peters et al., 2018] and Bidirectional Encoder Representations from Transformers (BERT) [Devlin et al., 2018]. We show our model based on BERT can outperform the state-of-the-art baseline on the presidential debate dataset [Gencheva et al., 2017]. In addition, we extend the Debate dataset and create a new dataset that uses Wikipedia inline citations as a proxy for check-worthiness. The dataset contains millions of check-worthy claims and covers various domains. Our experiments on this dataset show the useful features correlated with check-worthiness can vary across domains.