Mesh-Based Solutions for Nonparametric Penalized Regression
Abstract
It is often of interest to estimate regression functions non-parametrically. Penalized regression (PR) is one statistically-effective, well-studied solution to this problem. Unfortunately, in many cases, finding exact solutions to PR problems is computationally intractable. In this manuscript, we propose a mesh-based approximate solution (MBS) for those scenarios. MBS transforms the complicated functional minimization of NPR, to a finite parameter, discrete convex minimization; and allows us to leverage the tools of modern convex optimization. We show applications of MBS in a number of explicit examples (including both uni- and multi-variate regression), and explore how the number of parameters must increase with our sample-size in order for MBS to maintain the rate-optimality of NPR. We also give an efficient algorithm to minimize the MBS objective while effectively leveraging the sparsity inherent in MBS.
- Publication:
-
arXiv e-prints
- Pub Date:
- December 2021
- DOI:
- 10.48550/arXiv.2112.03428
- arXiv:
- arXiv:2112.03428
- Bibcode:
- 2021arXiv211203428O
- Keywords:
-
- Statistics - Methodology;
- Statistics - Computation;
- Statistics - Machine Learning;
- 62G08;
- 62J07 (Primary);
- 62G20 (Secondary);
- G.3
- E-Print:
- 29 pages, 4 figures