Hardware acceleration of NetApp virtual tape library deduplication algorithm

Abstract

In this thesis, I implemented an FPGA version of an algorithm used in the deduplication of backup storage data, to compare its performance to a software implementation of the algorithm. This algorithm flags sections of data with identifiers called "anchors". "Anchors" can be compared across versions of backup data, and if "anchors" are identical, sections of data are unchanged with high probability and might not have to be re-written to the backup disk. I also designed and implemented the environment by which this algorithm would communicate with a host computer. This environment can also easily be adapted for the acceleration of other shifting window algorithms. I tested the algorithm in simulation to verify its results as well as to estimate the speed at which the algorithm could process data.