Top-Down SBP: Turning Graph Clustering Upside Down
Name
3731545.3731589.pdf
Size
1.74 MB
Format
Adobe PDF
Checksum (MD5)
31d5c53d24fbbd5cc64b8116d2560ddd
Author(s)
Wanye, Frank
Gleyzer, Vitaliy
Kao, Edward
Feng, Wu-chun
Date Issued
July 20, 2025
Publisher
ACM|The 34th International Symposium on High-Performance Parallel and Distributed Computing
Citation
Wanye, Frank, Gleyzer, Vitaliy, Kao, Edward and Feng, Wu-chun. 2025. "Top-Down SBP: Turning Graph Clustering Upside Down."
Version
Final published version
Abstract
Stochastic block partitioning (SBP) is a statistical inference-based
algorithm for clustering vertices within a graph. It has been shown
to be statistically robust and highly accurate even on graphs with
a complex structure, but its poor scalability limits its usability to
smaller-sized graphs. In this manuscript we argue that one reason
for its poor scalability is the agglomerative, or bottom-up, nature
of SBP’s algorithmic design; the agglomerative computations cause
high memory usage and create a large search space that slows
down statistical inference, particularly in the algorithm’s initial
iterations. To address this bottleneck, we propose Top-Down SBP, a
novel algorithm that replaces the agglomerative (bottom-up) block
merges in SBP with a block-splitting operation. This enables the
algorithm to start with all vertices in one cluster and subdivide
them over time into smaller clusters. We show that Top-Down
SBP is up to 7.7× faster than Bottom-Up SBP without sacrificing
accuracy and can process larger graphs than Bottom-Up SBP on
the same hardware due to an up to 4.1× decrease in memory usage.
Additionally, we adapt existing methods for accelerating BottomUp SBP to the Top-Down approach, leading to up to 13.2× speedup
over accelerated Bottom-Up SBP and up to 403× speedup over
sequential Bottom-Up SBP on 64 compute nodes. Thus, Top-Down
SBP represents substantial improvements to the scalability of SBP,
enabling the analysis of larger datasets on the same hardware.
Description
HPDC ’25, Notre Dame, IN, USA
MIT Department
Lincoln Laboratory
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3731545.3731589
