Fast checkpoint and recovery techniques for an in-memory database

Abstract

Multicore in-memory databases for modern machines can support extraordinarily high transaction rates for online transaction processing workloads. A potential weakness of such databases, however, is recovery from crash failures. We show that techniques for disk-based persistence can be ecient enough to keep up with current systems' huge memory sizes and fast transaction rates, be smart enough to avoid additional contention, and provide fast recovery. This thesis presents SiloR, a persistence system built for a very fast multicore database system called Silo. We show that naive logging and checkpoints make normal-case execution slower, but that careful design of the persistence system allows us to keep up with many workloads without negative impact on runtime performance. We design the checkpoint and logging system to utilize multicore's resources to its fullest extent, both during runtime and during recovery. Parallelism allows the system to recover fast. Experiments show that a large database (~~ 50 GB) can be recovered in under five minutes.