MIT Libraries homeMIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Theses - Dept. of Electrical Engineering and Computer Sciences
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • View Item
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Theses - Dept. of Electrical Engineering and Computer Sciences
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Equitability and dependence

Author(s)
Reshef, David N
Thumbnail
DownloadFull printable version (20.59Mb)
Other Contributors
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Advisor
Tommi S. Jaakkola and Joshua B. Tenenbaum.
Terms of use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission. http://dspace.mit.edu/handle/1721.1/7582
Metadata
Show full item record
Abstract
Given a high-dimensional data set, we often wish to find the strongest relationships within it. Increasingly, and particularly at modern sample sizes, screening for all non-trivial relationships using an independence test can yield too many results to be a useful approach. What is needed is a way of identifying a smaller set of "strongest" relationships, independent of relationship type (e.g., linear, exponential, etc.). The first goal of this work is to formally present and characterize equitability, a property of measures of dependence that aims to overcome this challenge. We formalize equitability in terms of interval estimates of relationship strength, and then show that under moderate assumptions it is equivalent to requiring that a measure of dependence yield well powered tests not only for distinguishing non-trivial relationships from trivial ones but also for distinguishing stronger relationships from weaker ones. We then show that equitability, to the extent it is achieved, implies that a statistic will be well powered to detect all relationships of a certain minimal strength, across different relationship types. Thus, equitability is a strengthening of power against independence that enables exploration of data sets with a small number of strong, interesting relationships and a large number of weaker, less interesting ones. The second goal of this thesis is to define and theoretically characterize two new statistics that together yield an efficient approach for obtaining both power and equitability. To do this, we first introduce a new population measure of dependence whose goal is equitability and show three equivalent ways that it can be viewed, including as a canonical "smoothing" of mutual information. We then introduce and characterize an efficiently computable consistent estimator of our population measure of dependence, MICe, and we empirically establish its equitability on a large class of noisy functional relationships. Next, we derive a second, related statistic, TICe, whose computation is a trivial side-product of our algorithm and whose goal is powerful independence testing rather than equitability. We prove that this statistic yields a consistent independence test and show in simulations that the test has good power against independence. The third and final goal of this thesis is to present an extensive empirical evaluation of the equitability, power against independence, and runtime of several leading measures of dependence, including MICe and TICe. Our analysis finds that MIC, and TICe achieve state-of-the-art equitability on functional relationships and power against independence, respectively. We also show evidence for a trade-off between power against independence and equitability consistent with our theoretical findings. In the high-dimensional setting, our results suggest that an efficient and practical strategy for achieving a combination of power against independence and equitability is to filter the large set of candidate relationships by TICe and then to rank the remaining ones using MICe.
Description
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.
 
Cataloged from PDF version of thesis.
 
Includes bibliographical references (pages 205-209).
 
Date issued
2017
URI
http://hdl.handle.net/1721.1/112024
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Publisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.

Collections
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.
  • Electrical Engineering and Computer Sciences - Ph.D. / Sc.D.

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries homeMIT Libraries logo

Find us on

Twitter Facebook Instagram YouTube RSS

MIT Libraries navigation

SearchHours & locationsBorrow & requestResearch supportAbout us
PrivacyPermissionsAccessibility
MIT
Massachusetts Institute of Technology
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.