Dual Free Adaptive Mini-batch SDCA for Empirical Risk Minimization
Abstract
In this paper we develop dual free mini-batch SDCA with adaptive probabilities for regularized empirical risk minimization. This work is motivated by recent work of Shai Shalev-Shwartz on dual free SDCA method, however, we allow a non-uniform selection of "dual" coordinates in SDCA. Moreover, the probability can change over time, making it more efficient than fix uniform or non-uniform selection. We also propose an efficient procedure to generate a random non-uniform mini-batch through iterative process. The work is concluded with multiple numerical experiments to show the efficiency of proposed algorithms.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2015
- DOI:
- 10.48550/arXiv.1510.06684
- arXiv:
- arXiv:1510.06684
- Bibcode:
- 2015arXiv151006684H
- Keywords:
-
- Mathematics - Optimization and Control;
- Computer Science - Machine Learning