Efficient Mean Estimation with Pure Differential Privacy via a SumofSquares Exponential Mechanism
Abstract
We give the first polynomialtime algorithm to estimate the mean of a $d$variate probability distribution with bounded covariance from $\tilde{O}(d)$ independent samples subject to pure differential privacy. Prior algorithms for this problem either incur exponential running time, require $\Omega(d^{1.5})$ samples, or satisfy only the weaker concentrated or approximate differential privacy conditions. In particular, all prior polynomialtime algorithms require $d^{1+\Omega(1)}$ samples to guarantee small privacy loss with "cryptographically" high probability, $12^{d^{\Omega(1)}}$, while our algorithm retains $\tilde{O}(d)$ sample complexity even in this stringent setting. Our main technique is a new approach to use the powerful Sum of Squares method (SoS) to design differentially private algorithms. SoS proofs to algorithms is a key theme in numerous recent works in highdimensional algorithmic statistics  estimators which apparently require exponential running time but whose analysis can be captured by lowdegree Sum of Squares proofs can be automatically turned into polynomialtime algorithms with the same provable guarantees. We demonstrate a similar proofs to private algorithms phenomenon: instances of the workhorse exponential mechanism which apparently require exponential time but which can be analyzed with lowdegree SoS proofs can be automatically turned into polynomialtime differentially private algorithms. We prove a metatheorem capturing this phenomenon, which we expect to be of broad use in private algorithm design. Our techniques also draw new connections between differentially private and robust statistics in high dimensions. In particular, viewed through our proofstoprivatealgorithms lens, several wellstudied SoS proofs from recent works in algorithmic robust statistics directly yield key components of our differentially private mean estimation algorithm.
 Publication:

arXiv eprints
 Pub Date:
 November 2021
 arXiv:
 arXiv:2111.12981
 Bibcode:
 2021arXiv211112981H
 Keywords:

 Computer Science  Data Structures and Algorithms;
 Computer Science  Cryptography and Security;
 Computer Science  Information Theory;
 Statistics  Machine Learning