𝑘-Variance: A Clustered Notion of Variance
Name
20m1385895.pdf
Description
Published version
Size
2.33 MB
Format
Adobe PDF
Checksum (MD5)
de1692af428123d36202d418ff70e265
Author(s)
Solomon, Justin
Greenewald, Kristjan
Nagaraja, Haikady
Date Issued
September 2022
Journal
SIAM Journal on Mathematics of Data Science
Publisher
Society for Industrial & Applied Mathematics (SIAM)
Citation
Solomon, Justin, Greenewald, Kristjan and Nagaraja, Haikady. 2022. "𝑘-Variance: A Clustered Notion of Variance." SIAM Journal on Mathematics of Data Science, 4 (3).
Version
Final published version
Abstract
We introduce 𝑘-variance, a generalization of variance built on the machinery of random bipartite matchings. 𝑘-variance measures the expected cost of matching two sets of 𝑘 samples from a distribution to each other, capturing local rather than global information about a measure as 𝑘 increases; it is easily approximated stochastically using sampling and linear programming. In addition to defining 𝑘-variance and proving its basic properties, we provide in-depth analysis of this quantity in several key cases, including one-dimensional measures, clustered measures, and measures concentrated on low-dimensional subsets of ℝ𝑛. We conclude with experiments and open problems motivated by this new way to summarize distributional shape.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
MIT-IBM Watson AI Lab
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1137/20M1385895
