The Global Newton Method for games in normal form and in extensive form is shown to have a natural extension to computing Markov-perfect equilibria of stochastic games. GLOBAL NEWTON METHOD FOR STOCHASTIC GAMES SRIHARI GOVINDAN AND ROBERT WILSON Abstract. The Global Newton Method for games in normal form and in extensive form is shown to have a natural extension to computing Markov-perfect equi...

Parrondo’s games are essentially Markov games. They belong to the same class as Snakes and Ladders. The important distinguishing feature of Parrondo’s games is that the transition probabilities may vary in time. It is as though “snakes,” “ladders” and “dice” were being added and removed while the game was still in progress. Parrondo’s games are not homogeneous in time and do not necessarily set...

The Markov game (also called stochastic game (Filar & Vrieze 1997)) has been adopted as a theoretical framework for multiagent reinforcement learning (Littman 1994). In a Markov game, there are n agents, each facing a Markov decision process (MDP). All agents’ MDPs are correlated through their reward functions and the state transition function. As Markov decision process provides a theoretical ...

We recently introduced p-automata, automata that read discrete-time Markov chains and showed they provide an automata-theoretic framework for reasoning about pCTL model checking and abstraction of discrete time Markov chains. We used turn-based stochastic parity games to define acceptance of Markov chains by a special subclass of p-automata. Definition of acceptance required a reduction to a se...

Although variants of value iteration have been proposed for finding Nash or correlated equilibria in general-sum Markov games, these variants have not been shown to be effective in general. In this paper, we demonstrate by construction that existing variants of value iteration cannot find stationary equilibrium policies in arbitrary general-sum Markov games. Instead, we propose an alternative i...