Yuqing Zhu is a former research intern in the Media Analytics department of NEC Laboratories America, Inc. while studying at University of California, Santa Barbara.

Posts

Voting Based Approaches For Differentially Private Federated Learning

/in /by

Differentially Private Federated Learning (DPFL) is an emerging field with many applications. Gradient averaging-based DPFL methods require costly communication rounds and hardly work with large capacity models due to the explicit dimension dependence in its added noise. In this work, inspired by knowledge transfer non federated privacy learning from Papernot et al.(2017, 2018), we design two new DPFL schemes, by voting among the data labels returned from each local model, instead of averaging the gradients, which avoids the dimension dependence and significantly reduces the communication cost. Theoretically, by applying secure multi party computation, we could exponentially amplify the (data dependent) privacy guarantees when the margin of the voting scores are large. Extensive experiments show that our approaches significantly improve the privacy utility trade off over the state of the arts in DPFL.

Read more

Private-kNN Practical Differential Privacy for Computer Vision

/in /by

With increasing ethical and legal concerns on privacy for deep models in visual recognition, differential privacy has emerged as a mechanism to disguise membership of sensitive data in training datasets. Recent methods like Private Aggregation of Teacher Ensembles (PATE) leverage a large ensemble of teacher models trained on disjoint subsets of private data, to transfer knowledge to a student model with privacy guarantees. However, labeled vision data is often expensive and datasets, when split into many disjoint training sets, lead to significantly sub-optimal accuracy and thus hardly sustain good privacy bounds. We propose a practically data-efficient scheme based on private release of k-nearest neighbor (kNN) queries, which altogether avoids splitting the training dataset. Our approach allows the use of privacy-amplification by subsampling and iterative refinement of the kNN feature embedding. We rigorously analyze the theoretical properties of our method and demonstrate strong experimental performance on practical computer vision datasets for face attribute recognition and person reidentification. In particular, we achieve comparable or better accuracy than PATE while reducing more than 90% of the privacy loss, thereby providing the “most practical method to-date” for private deep learning in computer vision.

Read more