Multi-Agent Training beyond Zero-Sum with Correlated Equilibrium Meta-Solvers
Abstract
Two-player, constant-sum games are well studied in the literature, but there has been limited progress outside of this setting. We propose Joint Policy-Space Response Oracles (JPSRO), an algorithm for training agents in n-player, general-sum extensive form games, which provably converges to an equilibrium. We further suggest correlated equilibria (CE) as promising meta-solvers, and propose a novel solution concept Maximum Gini Correlated Equilibrium (MGCE), a principled and computationally efficient family of solutions for solving the correlated equilibrium selection problem. We conduct several experiments using CE meta-solvers for JPSRO and demonstrate convergence on n-player, general-sum games.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2021
- DOI:
- 10.48550/arXiv.2106.09435
- arXiv:
- arXiv:2106.09435
- Bibcode:
- 2021arXiv210609435M
- Keywords:
-
- Computer Science - Multiagent Systems;
- Computer Science - Artificial Intelligence;
- Computer Science - Computer Science and Game Theory;
- Computer Science - Machine Learning
- E-Print:
- ICML 2021, 9 pages, coded implementation available in https://github.com/deepmind/open_spiel/ (jpsro.py in examples)