A Multicore Path to Connectomics-on-Demand
Name
Shavit_A Multicore Path.pdf
Size
6.71 MB
Format
Adobe PDF
Checksum (MD5)
2116e94c4001d35109938d29c05d578e
Author(s)
Matveev, Alexander
Meirovitch, Yaron
Saribekyan, Hayk
Jakubiuk, Wiktor B.
Kaler, Timothy
Odor, Gergely
Budden, David
Zlateski, Aleksandar
Shavit, Nir N.
Date Issued
February 2017
Journal
Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming - PPoPP '17
Publisher
Association for Computing Machinery (ACM)
Citation
Matveev, Alexander, Yaron Meirovitch, Hayk Saribekyan, Wiktor Jakubiuk, Tim Kaler, Gergely Odor, David Budden, Aleksandar Zlateski, and Nir Shavit. “A Multicore Path to Connectomics-on-Demand.” Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming - PPoPP ’17 (2017).
Version
Author's final manuscript
Abstract
The current design trend in large scale machine learning is to use distributed clusters of CPUs and GPUs with MapReduce-style programming. Some have been led to believe that this type of horizontal scaling can reduce or even eliminate the need for traditional algorithm development, careful parallelization, and performance engineering. This paper is a case study showing the contrary: that the benefits of algorithms, parallelization, and performance engineering, can sometimes be so vast that it is possible to solve "cluster-scale" problems on a single commodity multicore machine.
Connectomics is an emerging area of neurobiology that uses cutting edge machine learning and image processing to extract brain connectivity graphs from electron microscopy images. It has long been assumed that the processing of connectomics data will require mass storage, farms of CPU/GPUs, and will take months (if not years) of processing time. We present a high-throughput connectomics-on-demand system that runs on a multicore machine with less than 100 cores and extracts connectomes at the terabyte per hour pace of modern electron microscopes.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Mathematics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3018743.3018766
