Randomized Data Structures: New Perspectives and Hidden Surprises

Abstract

This thesis revisits some of the oldest and most basic questions in the theory of randomized data structures—questions such as: How efficient is a linear probing hash table? How fast can you maintain a sorted array of numbers? How big does a pointer have to be? With the help of new techniques, along with a willingness to look beyond conventional wisdom, we are able to achieve much stronger bounds for each of these questions than were previously thought to be possible.

Our results also come with a powerful set of tools that span a wide range of problems and settings. Perhaps the most surprising of these tools is a new paradigm for designing efficient dynamic data structures, in which, by ‘tying our hands behind our back’ (i.e., by artificially restricting ourselves to a special class of privacy-preserving data structures), we are able to circumvent decades-old barriers in time/space efficiency. This technique appears three (completely separate) times in the thesis.

Combined, our results overturn a 60-year-old myth on linear-probing hash tables; refute a 30-year-old conjecture and solve a 40-year-old open problem on dynamic sorting; resolve a 20-year-old open problem on dynamic load balancing; settle some of the most basic and fundamental questions from the theory of space-efficient data structures; and answer a 20-year-old question on memory allocation that was left as the central open problem in the first paper on history independence.