Explaining data in visual analytic systems

Abstract

Data-driven decision making and data analysis has grown in both importance and availability in the past decade, and has seen increasing acceptance in the broader population. Visual tools are needed to help non-technical users explore and make sense of their datasets. However even with existing tools, many common data analysis tasks are still performed using manual, error-prone methods, or simply inaccessible due to non-intuitive interfaces. In this thesis, we addressed a common data analysis task that is ill-served by existing visual analytical tools. Specifically, although visualization tools are well suited to identify patterns in datasets, they do not help users characterize surprising trends or outliers in the visualization and leave that task to the user. We explored the necessary techniques so users can visually explore datasets, specify outliers in the resulting visualizations, and produce explanations that help explain the systematic sources of the outlier values. To this end, we developed three systems: DBWipes, a browser-based visual exploration tool; Scorpion, a set of algorithms that describes the subset of an outlier's input records that "explain away" the anomalous value; and SubZero, a system to track and retrieve the input records that contributed to output records of a complex workflow. From our experiences, we found that existing visual analysis system designs leave a number of program analysis, performance, and functionalities on the table, and proposed an initial design of a data visualization management system (DVMS) that unifies data processing and visualization and can help address these existing issues.