Investigating the efficacy of terrorist network visualizations

Abstract

Military intelligence analysts are increasingly tasked to sift through enormous volumes of data to identify the proverbial intelligence "needle in a haystack." One specific domain exemplifying this new intelligence paradigm is network analysis of terrorist organizations. This area of intelligence analysis uses mostly commercially available software applications to leverage the powers of social network theory against large terrorism data sets. An additional challenge is the fast paced development cycle for new sensors, which are capable of collecting data at unmanageable rates. As such, analysts are in dire need of new analytical techniques that give them the ability to effectively and efficiently transform the collected data into intelligible information and, subsequently, intelligence. Therefore, the primary focus of this thesis is to analyze two visualization techniques within social network analysis, with the intent to identify which mode of visualization is most effective for the intelligence tasks of: 1) identifying leaders and 2) identifying clusters. To test the effectiveness of the visualizations, an experiment was conducted in which participants exploited matrix and node-link visualizations constructed from a surrogate terror data set. The objectives of this experiment were to test the effectiveness of the node-link visualization compared to the matrix visualization, based on two criteria: 1) effectiveness at identifying leaders and clusters within a network, and 2) the time it takes to complete each task. Participants in the experiment were all Air Force intelligence analysts and the experiment utilized a 2 (Visualization) x 2 (Task) mixed design study within-subjects on the visualization task factor and between-subjects on the visualization technique factor. The node-link visualization resulted in statistically significantly better performance in all studied scenarios where the objective was identifying leaders. Although node-link also returned a better performance than the matrix for identifying clusters, there was not a statistically significant difference. The same lack of statistical significance holds true for the completion time dependent variable. In all cases, there was not enough difference between the times produced by the node-link and matrix to determine if either offers a statistically significant decrease in the time it takes to complete tasks using either visualization. At this time, the matrix should not be universally integrated into the current methodologies used by analysts to exploit terror network visualizations until more research is conducted into the respective strengths and weaknesses within the intelligence domain. However, analysts should be independently encouraged to explore and adapt new methods of visualization into their current practices and identify new or improved versions of the visualizations identified within this thesis for future testing.