Computational Privacy with Split Learning: Benchmarking of Algorithmic Defenses against Reconstruction Attacks

Abstract

Distributed deep learning has potential for significant impact in preserving data privacy and improving model accuracy by leveraging massive sets of training data. However, passing intermediate weights, gradients, or activations is inherent in current distributed learning techniques, all of which contain information related to input data. This thesis analyzes split learning, a current state of the art distributed deep learning technique, in the context of the private collaborative inference scheme against reconstruction attacks. This is achieved by creating a benchmark and introducing new methods of improving privacy algorithmically. Benchmarking is done by comparing input data reconstruction quality and accuracy of sensitive attribute prediction over the axes of number of activation, input data pairs are leaked, and whether or not model parameters and general data distribution information is known. The proposed privacy improvements involve changes in model training to leak less information that may be used for reconstruction while preserving accuracies for the originally intended model prediction task. These improvements are compared against current state of the art privacy methods in protection over various reconstruction attacks.