Error based algorithm selection for differential privacy

Abstract

Differential privacy has recently emerged as a robust framework for delivering privacy guarantees when performing data analysis. Researchers have developed a wide array of algorithms which can provide these privacy guarantees. These algorithms differ in their performance on different datasets. This thesis explores a system for selecting the best algorithm that achieves a target error, given a database and workload: "error-based algorithm selection". In particular, we explore whether this problem can be solved in a private matter, how accurate it can be, and if it can be done competitively. The system we propose approaches the problem by building models for differentially private algorithms which use features from the workload and dataset as well as the target error to predict an epsilon value that will achieve the target error. It then selects the algorithm which predicts the best epsilon and uses it to run the workload. We evaluate the system on 1D and 2D workloads with several different algorithms and datasets. The system predicts the best algorithm a large percentage of the time, usually only being beaten by one algorithm and selects an algorithm that is within the top 2 best algorithms a majority of the time. The individual algorithm models achieve target error within a reasonable margin. The results show this approach is viable for the error-based algorithm selection problem, solving it in a differentially private, algorithm agnostic and competitive manner..