On Structured FilteringClustering: Global Error Bound and Optimal FirstOrder Algorithms
Abstract
In recent years, the filteringclustering problems have been a central topic in statistics and machine learning, especially the $\ell_1$trend filtering and $\ell_2$convex clustering problems. In practice, such structured problems are typically solved by firstorder algorithms despite the extremely illconditioned structures of difference operator matrices. Inspired by the desire to analyze the convergence rates of these algorithms, we show that for a large class of filteringclustering problems, a \textit{global error bound} condition is satisfied for the dual filteringclustering problems when a certain regularization is chosen. Based on this result, we show that many firstorder algorithms attain the \textit{optimal rate of convergence} in different settings. In particular, we establish a generalized dual gradient ascent (GDGA) algorithmic framework with several subroutines. In deterministic setting when the subroutine is accelerated gradient descent (AGD), the resulting algorithm attains the linear convergence. This linear convergence also holds for the finitesum setting in which the subroutine is the Katyusha algorithm. We also demonstrate that the GDGA with stochastic gradient descent (SGD) subroutine attains the optimal rate of convergence up to the logarithmic factor, shedding the light to the possibility of solving the filteringclustering problems efficiently in online setting. Experiments conducted on $\ell_1$trend filtering problems illustrate the favorable performance of our algorithms over other competing algorithms.
 Publication:

arXiv eprints
 Pub Date:
 April 2019
 arXiv:
 arXiv:1904.07462
 Bibcode:
 2019arXiv190407462H
 Keywords:

 Statistics  Machine Learning;
 Computer Science  Machine Learning;
 Statistics  Computation
 EPrint:
 The first two authors contributed equally to this work. This version greatly improves and expands the results in the previous version