Estimating HigherOrder Moments Using Symmetric Tensor Decomposition
Abstract
We consider the problem of decomposing higherorder moment tensors, i.e., the sum of symmetric outer products of data vectors. Such a decomposition can be used to estimate the means in a Gaussian mixture model and for other applications in machine learning. The $d$thorder empirical moment tensor of a set of $p$ observations of $n$ variables is a symmetric $d$way tensor. Our goal is to find a lowrank tensor approximation comprising $r \ll p$ symmetric outer products. The challenge is that forming the empirical moment tensors costs $O(pn^d)$ operations and $O(n^d)$ storage, which may be prohibitively expensive; additionally, the algorithm to compute the lowrank approximation costs $O(n^d)$ per iteration. Our contribution is avoiding formation of the moment tensor, computing the lowrank tensor approximation of the moment tensor implicitly using $O(pnr)$ operations per iteration and no extra memory. This advance opens the door to more applications of higherorder moments since they can now be efficiently computed. We present numerical evidence of the computational savings and show an example of estimating the means for higherorder moments.
 Publication:

arXiv eprints
 Pub Date:
 November 2019
 arXiv:
 arXiv:1911.03813
 Bibcode:
 2019arXiv191103813S
 Keywords:

 Mathematics  Numerical Analysis