From Oja's Algorithm to the Multiplicative Weights Update Method with Applications
Abstract
Oja's algorithm is a well known online algorithm studied mainly in the context of stochastic principal component analysis. We make a simple observation, yet to the best of our knowledge a novel one, that when applied to a any (not necessarily stochastic) sequence of symmetric matrices which share common eigenvectors, the regret of Oja's algorithm could be directly bounded in terms of the regret of the well known multiplicative weights update method for the problem of prediction with expert advice. Several applications to optimization with quadratic forms over the unit sphere in $\reals^n$ are discussed.
 Publication:

arXiv eprints
 Pub Date:
 October 2023
 DOI:
 10.48550/arXiv.2310.15559
 arXiv:
 arXiv:2310.15559
 Bibcode:
 2023arXiv231015559G
 Keywords:

 Mathematics  Optimization and Control;
 Computer Science  Machine Learning