A locally encodable and decodable compressed data structure

Abstract

In a variety of applications, ranging from highspeed networks to massive databases, there is a need to maintain histograms and other statistics in a streaming manner. Motivated by such applications, we establish the existence of efficient source codes that are both locally encodable and locally decodable. Our solution is an explicit construction in the form of a (randomized) data structure for storing N integers. The construction uses multi-layered sparse graph codes based on Ramanujan graphs, and has the following properties: (a) the structure utilizes minimal possible space, and (b) the value of any of the integers can be read or updated in near constant time (on average and with high probability). By contrast, data structures proposed in the context of streaming algorithms and compressed sensing in recent years (e.g., various sketches) support local encodability, but not local decodability; and those known as succinct data structures are locally decodable, but not locally encodable.