Scalable multi-access flash store for big data analytics
Author(s)Jun, Sang-Woo; Liu, Ming; Fleming, Kermin Elliott; Arvind, Arvind
MetadataShow full item record
For many "Big Data" applications, the limiting factor in performance is often the transportation of large amount of data from hard disks to where it can be processed, i.e. DRAM. In this paper we examine an architecture for a scalable distributed flash store which aims to overcome this limitation in two ways. First, the architecture provides a highperformance, high-capacity, scalable random-access storage. It achieves high-throughput by sharing large numbers of flash chips across a low-latency, chip-to-chip backplane network managed by the flash controllers. The additional latency for remote data access via this network is negligible as compared to flash access time. Second, it permits some computation near the data via a FPGA-based programmable flash controller. The controller is located in the datapath between the storage and the host, and provides hardware acceleration for applications without any additional latency. We have constructed a small-scale prototype whose network bandwidth scales directly with the number of nodes, and where average latency for user software to access flash store is less than 70?s, including 3.5?s of network overhead.
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science; Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Materials Science and Engineering
Proceeding FPGA '14 Proceedings of the 2014 ACM/SIGDA international symposium on Field-programmable gate arrays
Association for Computing Machinery (ACM)
Jun, Sang-Woo, Ming Liu, Kermin Elliott Fleming and Arvind. "Scalable Multi-Access Flash Store for Big Data Analytics." In Proceeding FPGA '14 Proceedings of the 2014 ACM/SIGDA international symposium on Field-programmable gate arrays, Monterey, California, USA, February 26-28, 2014, pages 55-64.
Author's final manuscript