Equitability and dependence

Abstract

Given a high-dimensional data set, we often wish to find the strongest relationships within it. Increasingly, and particularly at modern sample sizes, screening for all non-trivial relationships using an independence test can yield too many results to be a useful approach. What is needed is a way of identifying a smaller set of "strongest" relationships, independent of relationship type (e.g., linear, exponential, etc.). The first goal of this work is to formally present and characterize equitability, a property of measures of dependence that aims to overcome this challenge. We formalize equitability in terms of interval estimates of relationship strength, and then show that under moderate assumptions it is equivalent to requiring that a measure of dependence yield well powered tests not only for distinguishing non-trivial relationships from trivial ones but also for distinguishing stronger relationships from weaker ones. We then show that equitability, to the extent it is achieved, implies that a statistic will be well powered to detect all relationships of a certain minimal strength, across different relationship types. Thus, equitability is a strengthening of power against independence that enables exploration of data sets with a small number of strong, interesting relationships and a large number of weaker, less interesting ones. The second goal of this thesis is to define and theoretically characterize two new statistics that together yield an efficient approach for obtaining both power and equitability. To do this, we first introduce a new population measure of dependence whose goal is equitability and show three equivalent ways that it can be viewed, including as a canonical "smoothing" of mutual information. We then introduce and characterize an efficiently computable consistent estimator of our population measure of dependence, MICe, and we empirically establish its equitability on a large class of noisy functional relationships. Next, we derive a second, related statistic, TICe, whose computation is a trivial side-product of our algorithm and whose goal is powerful independence testing rather than equitability. We prove that this statistic yields a consistent independence test and show in simulations that the test has good power against independence. The third and final goal of this thesis is to present an extensive empirical evaluation of the equitability, power against independence, and runtime of several leading measures of dependence, including MICe and TICe. Our analysis finds that MIC, and TICe achieve state-of-the-art equitability on functional relationships and power against independence, respectively. We also show evidence for a trade-off between power against independence and equitability consistent with our theoretical findings. In the high-dimensional setting, our results suggest that an efficient and practical strategy for achieving a combination of power against independence and equitability is to filter the large set of candidate relationships by TICe and then to rank the remaining ones using MICe.