Lifting uniform learners via distributional decomposition
Abstract
We show how any PAC learning algorithm that works under the uniform distribution can be transformed, in a blackbox fashion, into one that works under an arbitrary and unknown distribution $\mathcal{D}$. The efficiency of our transformation scales with the inherent complexity of $\mathcal{D}$, running in $\mathrm{poly}(n, (md)^d)$ time for distributions over $\{\pm 1\}^n$ whose pmfs are computed by depth$d$ decision trees, where $m$ is the sample complexity of the original algorithm. For monotone distributions our transformation uses only samples from $\mathcal{D}$, and for general ones it uses subcube conditioning samples. A key technical ingredient is an algorithm which, given the aforementioned access to $\mathcal{D}$, produces an optimal decision tree decomposition of $\mathcal{D}$: an approximation of $\mathcal{D}$ as a mixture of uniform distributions over disjoint subcubes. With this decomposition in hand, we run the uniformdistribution learner on each subcube and combine the hypotheses using the decision tree. This algorithmic decomposition lemma also yields new algorithms for learning decision tree distributions with runtimes that exponentially improve on the prior state of the art  results of independent interest in distribution learning.
 Publication:

arXiv eprints
 Pub Date:
 March 2023
 DOI:
 10.48550/arXiv.2303.16208
 arXiv:
 arXiv:2303.16208
 Bibcode:
 2023arXiv230316208B
 Keywords:

 Statistics  Machine Learning;
 Computer Science  Computational Complexity;
 Computer Science  Data Structures and Algorithms;
 Computer Science  Machine Learning
 EPrint:
 To appear in STOC 2023