Nash Convergence of Gradient Dynamics in Iterated General-Sum Games

Multi-agent games are becoming an increas­ ingly prevalent formalism for the study of electronic commerce and auctions. The speed at which transactions can take place and the growing complexity of electronic market­ places makes the study of computationally simple agents an appealing direction. In this work, we analyze the behavior of agents that incrementally adapt their strategy through gradient ascent on expected payoff, in the simple setting of two-player, two-action, iter­ ated general-sum games, and present a sur­ prising result. We show that either the agents will converge to a Nash equilibrium, or if the strategies themselves do not converge, then their average payoffs will nevertheless con­ verge to the payoffs of a Nash equilibrium.