This is not the latest version of this item. The latest version can be found here.
Hierarchical Clustering: Objective Functions and Algorithms
Name
main_openaccess.pdf
Description
Merge of two publications Hierarchical Clustering: Objective Functions and Algorithm [SODA 18] and Hierarchical Clustering Beyond the Worst-Case [NIPS 17]
Size
1.31 MB
Format
Adobe PDF
Checksum (MD5)
892125e83b87717d897cda038e0486fd
Author(s)
Cohen-addad, Vincent
Kanade, Varun
Mallmann-Trenn, Frederik
Mathieu, Claire
Date Issued
June 5, 2019
Journal
Journal of the ACM
Publisher
Association for Computing Machinery (ACM)
Citation
Cohen-Addad, Vincent et al. "Hierarchical Clustering: Objective Functions and Algorithms." Journal of the ACM 66, 4 (June 2019): 26 © 2019 The Authors
Version
Author's final manuscript
Abstract
Hierarchical clustering is a recursive partitioning of a dataset into clusters at an increasingly finer granularity. Motivated by the fact that most work on hierarchical clustering was based on providing algorithms, rather than optimizing a specific objective, Dasgupta framed similarity-based hierarchical clustering as a combinatorial optimization problem, where a “good” hierarchical clustering is one that minimizes a particular cost function [23]. He showed that this cost function has certain desirable properties: To achieve optimal cost, disconnected components (namely, dissimilar elements) must be separated at higher levels of the hierarchy, and when the similarity between data elements is identical, all clusterings achieve the same cost.
We take an axiomatic approach to defining “good” objective functions for both similarity- and dissimilarity-based hierarchical clustering. We characterize a set of admissible objective functions having the property that when the input admits a “natural” ground-truth hierarchical clustering, the ground-truth clustering has an optimal value. We show that this set includes the objective function introduced by Dasgupta.
Equipped with a suitable objective function, we analyze the performance of practical algorithms, as well as develop better and faster algorithms for hierarchical clustering. We also initiate a beyond worst-case analysis of the complexity of the problem and design algorithms for this scenario.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
http://dx.doi.org/10.1145/3321386
