We present a new algorithm for polynomial time learning of near optimal behavior in stochastic games. This algorithm incorporates and integrates important recent results of Kearns and Singh [ 1998] in reinforcement learning and of Monderer and Tennenholtz [1997] in repeated games. In stochastic games we face an exploration vs. exploitation dilemma more complex than in Markov decision processes....

We consider stability properties of equilibria in stochastic evolutionary dynamics. In particular, we study the stability of mixed equilibria in strategic form games. In these games, when the populations are small, all strategies may be stable. We prove that when the populations are large, the unique stable outcome of best-reply dynamics in 2 × 2 games with a unique Nash equilibrium that is com...

We study the class of potential games that are also graphical games with respect to a given graph G of connections between the players. We show that, up to strategic equivalence, this class of games can be identified with the set of Markov random fields on G. From this characterization, and from the Hammersley-Clifford theorem, it follows that the potentials of such games can be decomposed to l...

Parrondo's games manifest the apparent paradox where losing strategies can be combined to win and have generated significant multidisciplinary interest in the literature. Here we review two recent approaches, based on the Fokker-Planck equation , that rigorously establish the connection between Parrondo's games and a physical model known as the flashing Brownian ratchet. This gives rise to a ne...

One of the proposed solutions to the equilibrium selection problem for agents learning in repeated games is obtained via the notion of stochastic stability. Learning algorithms are perturbed so that the Markov chain underlying the learning dynamics is necessarily irreducible and yields a unique stable distribution. The stochastically stable distribution is the limit of these stable distribution...

Parrondo’s games manifest the apparent paradox where losing strategies can be combined to win and have generated significant multidisciplinary interest in the literature. Here we review two recent approaches, based on the Fokker-Planck equation, that rigorously establish the connection between Parrondo’s games and a physical model known as the flashing Brownian ratchet. This gives rise to a new...

Markov games, as the generalization of Markov decision processes to the multi-agent case, have long been used for modeling multi-agent systems (MAS). The Markov game view of MAS is considered as a sequence of games having to be played by multiple players while each game belongs to a different state of the environment. In this paper, several learning automata based multiagent system algorithms f...

Complex games such as RTS games are naturally formalized as Markov games. Given a Markov game, it is often possible to hand-code or learn a set of policies that capture the diversity of possible strategies. It is also often possible to hand-code or learn an abstract simulator of the game that can estimate the outcome of playing two strategies against one another from any state. We consider how ...

Logit dynamics [Blume, Games and Economic Behavior, 1993] is a randomized best response dynamics where at every time step a player is selected uniformly at random and she chooses a new strategy according to the “logit choice function”, i.e. a probability distribution biased towards strategies promising higher payoffs, where the bias level corresponds to the degree of rationality of the agents. ...

We consider a two-player zero-sum game given by a Markov chain over a finite set of states K and a family of zero-sum matrix games (G)k∈K . The sequence of states follows the Markov chain. At the beginning of each stage, only player 1 is informed of the current state k, then the game G is played, the actions played are observed by both players and the play proceeds to the next stage. We call su...