Chaos, Extremism and Optimism: Volume Analysis of Learning in Games
Abstract
We present volume analyses of Multiplicative Weights Updates (MWU) and Optimistic Multiplicative Weights Updates (OMWU) in zerosum as well as coordination games. Such analyses provide new insights into these game dynamical systems, which seem hard to achieve via the classical techniques within Computer Science and Machine Learning. The first step is to examine these dynamics not in their original space (simplex of actions) but in a dual space (aggregate payoff space of actions). The second step is to explore how the volume of a set of initial conditions evolves over time when it is pushed forward according to the algorithm. This is reminiscent of approaches in Evolutionary Game Theory where replicator dynamics, the continuoustime analogue of MWU, is known to always preserve volume in all games. Interestingly, when we examine discretetime dynamics, both the choice of the game and the choice of the algorithm play a critical role. So whereas MWU expands volume in zerosum games and is thus Lyapunov chaotic, we show that OMWU contracts volume, providing an alternative understanding for its known convergent behavior. However, we also prove a nofreelunch type of theorem, in the sense that when examining coordination games the roles are reversed: OMWU expands volume exponentially fast, whereas MWU contracts. Using these tools, we prove two novel, rather negative properties of MWU in zerosum games: (1) Extremism: even in games with unique fully mixed Nash equilibrium, the system recurrently gets stuck near purestrategy profiles, despite them being clearly unstable from game theoretic perspective. (2) Unavoidability: given any set of good points (with your own interpretation of "good"), the system cannot avoid bad points indefinitely.
 Publication:

arXiv eprints
 Pub Date:
 May 2020
 arXiv:
 arXiv:2005.13996
 Bibcode:
 2020arXiv200513996K
 Keywords:

 Computer Science  Computer Science and Game Theory;
 Computer Science  Machine Learning;
 Mathematics  Dynamical Systems
 EPrint:
 20 pages, 4 figures