Abstract
While the standard paradigm of non-cooperative game theory focuses on centralized equilibrium analysis and computation, online learning in games is a setting where players employ simple learning dynamics to update their strategies over time. The primary aim of research in this direction is twofold: to either characterize the day-to-day behavior of the dynamics, or conversely to establish convergence results to game-theoretic equilibria. In the former, research into the day-to-day behavior of broad classes of continuous-time dynamics has shown that they exhibit cyclic, and formally Poincaré recurrent, behavior in zero-sum games. However, the robustness of this behavior has been less studied in non-standard game-theoretic settings. In the latter, the focus is primarily on establishing fast theoretical convergence rates of discrete-time algorithms to relevant notions of equilibria. One recent success in this direction has been the introduction of ‘optimistic’ variants of standard online learning algorithms, which speed up convergence while avoiding cyclical behavior. However, it is unclear how widely applicable optimism is, or if there are further modifications that outperform it. In this dissertation, we tackle both of these challenges in various non-cooperative game theoretic settings. First, we establish Poincaré recurrence results of the ubiquitous Follow-The-Regularized-Leader dynamics (and variations thereof) in formulations of time-evolving games and quantum games. These types of games allow us to establish rich connections to the realms of evolutionary biology and quantum information theory respectively. Along the way, we also show that in certain formulations of time-evolving games, time-average convergence to equilibrium actually fails, elucidating a new failure mode of learning dynamics in games. Next, we propose and analyze discrete-time dynamics that can achieve fast convergence to equilibria in multi-player extensive-form games, as well as general-sum normal-form games with an arbitrary number of players and strategies. We establish that in multi-player extensive-form games which admit a network structure, optimistic gradient ascent converges to the set of Nash equilibria at a linear rate. Going beyond optimism, we then present a novel class of ‘clairvoyant’ learning algorithms for learning in general-sum games, which improves upon the state-of-the-art convergence rate to the set of coarse correlated equilibria. In their totality, the results in this dissertation represent a step forward in better understanding the recurrent behavior of continuous-time learning dynamics in zero-sum games, while also providing encouraging equilibrium convergence results in the algorithmic, discrete-time context.