A Dynamic LowRank Fast Gaussian Transform
Abstract
The \emph{Fast Gaussian Transform} (FGT) enables subquadratictime multiplication of an $n\times n$ Gaussian kernel matrix $\mathsf{K}_{i,j}= \exp (  \ x_i  x_j \_2^2 ) $ with an arbitrary vector $h \in \mathbb{R}^n$, where $x_1,\dots, x_n \in \mathbb{R}^d$ are a set of \emph{fixed} source points. This kernel plays a central role in machine learning and random feature maps. Nevertheless, in most modern data analysis applications, datasets are dynamically changing (yet often have low rank), and recomputing the FGT from scratch in (kernelbased) algorithms incurs a major computational overhead ($\gtrsim n$ time for a single source update $\in \mathbb{R}^d$). These applications motivate a \emph{dynamic FGT} algorithm, which maintains a dynamic set of sources under \emph{kerneldensity estimation} (KDE) queries in \emph{sublinear time} while retaining MatVec multiplication accuracy and speed. Assuming the dynamic datapoints $x_i$ lie in a (possibly changing) $k$dimensional subspace ($k\leq d$), our main result is an efficient dynamic FGT algorithm, supporting the following operations in $\log^{O(k)}(n/\varepsilon)$ time: (1) Adding or deleting a source point, and (2) Estimating the ``kerneldensity'' of a query point with respect to sources with $\varepsilon$ additive accuracy. The core of the algorithm is a dynamic data structure for maintaining the \emph{projected} ``interaction rank'' between source and target boxes, decoupled into finite truncation of Taylor and Hermite expansions.
 Publication:

arXiv eprints
 Pub Date:
 February 2022
 DOI:
 10.48550/arXiv.2202.12329
 arXiv:
 arXiv:2202.12329
 Bibcode:
 2022arXiv220212329H
 Keywords:

 Computer Science  Data Structures and Algorithms