Frank-Wolfe Algorithms for Saddle Point Problems
Abstract
We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW optimization, we provide the first proof of convergence of a FW-type saddle point solver over polytopes, thereby partially answering a 30 year-old conjecture. We also survey other convergence results and highlight gaps in the theoretical underpinnings of FW-style algorithms. Motivating applications without known efficient alternatives are explored through structured prediction with combinatorial penalties as well as games over matching polytopes involving an exponential number of constraints.
- Publication:
-
arXiv e-prints
- Pub Date:
- October 2016
- DOI:
- 10.48550/arXiv.1610.07797
- arXiv:
- arXiv:1610.07797
- Bibcode:
- 2016arXiv161007797G
- Keywords:
-
- Mathematics - Optimization and Control;
- Computer Science - Machine Learning;
- Statistics - Machine Learning;
- 90C52;
- 90C90;
- 68T05;
- G.1.6;
- I.2.6
- E-Print:
- Appears in: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS 2017). 39 pages