Balanced Privacy Budget Allocation for Privacy-Preserving Machine Learning

Abstract

The preservation of privacy during the learning phase of machine learning is challenging. There are two methods to achieve privacy-preserving machine learning: adding noise to machine-learning model parameters, which is often selected for its higher accuracy; and executing learning using noisy data, which is preferred for privacy. Recently, a Scalable Unified Privacy-preserving Machine learning framework ($\mathsf SUPM$) has been proposed, which controls the balance between privacy and accuracy by harmonizing the privacy mechanisms used in dimension reduction, training and testing phases. This paper proposes a novel method that allocates privacy budgets according to their effectiveness that improves the accuracy without sacrificing the number of available attributes. Our privacy budget allocation algorithm can be applied into $\mathsf SUPM$ and improve the accuracy while keeping the privacy. We evaluate its performance using logistic regression and support vector machines as machine learning algorithms. $\mathsf SUPM$ using our privacy budget allocation algorithm is effective in terms of accuracy and the number of available attributes. We also clarify the conditions under which our method is more effective for a given dataset.