Õptimal Differentially Private Learning of Thresholds and QuasiConcave Optimization
Abstract
The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of $O(\xi^{1} \log(1/\beta))$ (for generalization error $\xi$ with confidence $1\beta$). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size $X$ of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we finally close the gap for approximateDP and provide a nearly tight upper bound of $\tilde{O}(\log^* X)$, which matches a lower bound by Alon et al (that applies even with improper learning) and improves over a prior upper bound of $\tilde{O}((\log^* X)^{1.5})$ by Kaplan et al. We also provide matching upper and lower bounds of $\tilde{\Theta}(2^{\log^*X})$ for the additive error of private quasiconcave optimization (a related and more general problem). Our improvement is achieved via the novel ReorderSliceCompute paradigm for private data analysis which we believe will have further applications.
 Publication:

arXiv eprints
 Pub Date:
 November 2022
 DOI:
 10.48550/arXiv.2211.06387
 arXiv:
 arXiv:2211.06387
 Bibcode:
 2022arXiv221106387C
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Cryptography and Security;
 Computer Science  Data Structures and Algorithms