This paper develops a theory of regular Markov perfect equilibria in dynamic stochastic games. We show that almost all dynamic stochastic games have a finite number of locally isolated Markov perfect equilibria that are all regular. These equilibria are essential and strongly stable. Moreover, they all admit purification. JEL classification numbers: C73, C61, C62.

In this paper, we first present a key approximation result for zero-sum, discounted Markov games, providing bounds on the state-wise loss and the loss in the sup norm resulting from using approximate Q-functions. Then we extend the policy rollout technique for MDPs to Markov games. Using our key approximation result, we prove that, under certain conditions, the rollout technique gives rise to a...

Game theory (GT) is an essential formal tool for interacting entities; however computing equilibria in GT is a hard problem. When the same game can be played repeatedly over time, the problem becomes even more complicated. The existence of multiple game states makes the problem of computing equilibria in such games extremely difficult. In this paper, we approach this problem by first proposing ...

Two-player games of billiards, of the sort seen in recent Computer Olympiads held by the International Computer Games Association, are an emerging area with unique challenges for A.I. research. Drawing on the insight gained from our victory in the 2008 ICGA billiards tournament, we define a game-theoretic model of these types of billiards games. The modeling is surprisingly subtle. While sharin...

I consider an alternating offer bargaining game which is played by a risk neutral buyer and seller, where the value of the good to be traded follows a Markov process. For these games the existence of a perfect equilibrium is proved and the set of equilibrium payoffs and strategies are characterised. The main results are: (a) If the buyer is less patient than the seller, then there will be delay...

We study nonzero-sum stopping games with randomized stopping strategies. The existence of Nash equilibrium and ε-equilibrium strategies are discussed under various assumptions on players random payoffs and utility functions dependent on the observed discrete time Markov process. Then we will present a model of a market game in which randomized stopping times are involved. The model is a mixture...

I am going to put forward a program of the analysis of a new class of stochastic games that I call nonlinear Markov games, as they arise as a (competitive) controlled version of nonlinear Markov processes (an emerging field of intensive research, see e.g. [3], [4],[5]). This class of games can model a variety of situation for economics and epidemics, statistical physics, and pursuit evasion pro...

We study perfect information games with an infinite horizon played by an arbitrary number of players. This class of games includes infinitely repeated perfect information games, repeated games with asynchronous moves, games with long and short run players, games with overlapping generations of players, and canonical non-cooperative models of bargaining. We consider two restrictions on equilibri...

We study perfect information games with an infinite horizon played by an arbitrary number of players. This class of games includes infinitely repeated perfect information games, repeated games with asynchronous moves, games with long and short run players, games with overlapping generations of players, and canonical non-cooperative models of bargaining. We consider two restrictions on equilibri...

We define and examine priority mean-payoff games — a natural extension of parity games. By adapting the notion of Blackwell optimality borrowed from the theory of Markov decision processes we show that priority mean-payoff games can be seen as a limit of special multi-discounted games.