Fast and Cheap Covariance Smoothing

We introduce the Tensorized-and-Restricted Krylov (TReK) method, a simple and efficient algorithm for estimating covariance tensors with large observational sizes. TReK extends the conjugate gradient method to incorporate range restrictions, enabling its use in a variety of covariance smoothing applications. By leveraging matrix-level operations, it achieves significant improvements in both computational speed and memory cost, improving over existing methods by an order of magnitude. TReK ensures finite-step convergence in the absence of rounding errors and converges fast in practice, making it well-suited for large-scale problems. The algorithm is also highly flexible, supporting a wide range of forward and projection tensors.