Analyzing the Usability of Natural Language Processing for Detecting Disinformation Tactics, Techniques, and Procedures

Abstract

The purposeful manipulation of information for political gain by powerful state actors is a threat to security and democracy that is challenging to address without infringing upon freedom of expression. By qualitatively analyzing the evolution of Russian media manipulation in the early 21st century, we see that the threat is not a product of, but is exacerbated by, technology such as social media, which increases the speed and reach of malicious information. State strategies for information manipulation co-evolve with internet and communication technology to take advantage of the new platform affordances of social media. We analyze the history of international disinformation policy in the European Union and find that policies fail because they attempt to regulate based on the effect of information manipulation rather than developing tractable definitions and characterizations of illicit information manipulation. As such, this thesis proposes that the persistence of this threat to information security is not primarily a result of technological advancements but rather a failure of policy to adequately define information manipulation. Also, we build a protype, machine learning enabled pipeline to investigate the capabilities and limits of using software techniques to characterize disinformation in a standardized manner. This pipeline offers speed and consistency to process large volumes of disinformation texts. Results indicate that even a prototype of a pipeline can detect important characteristics of disinformation. Standardized characterization of disinformation generated by pipelines such as this prototype could then potentially be used to build legal precedents, supporting a quilt-work policy approach. A technology enabled policy solution is thus a potentially feasible and effective path forward to prevent and combat state-sponsored information manipulation.