We study two-person extensive form games, or “matches,” in which the only possible outcomes (if the game terminates) are that one player or the other is declared the winner. The winner of the match is determined by the winning of points, in “point games.” We call these matches binary Markov games. We show that if a simple monotonicity condition is satisfied, then (a) it is a Nash equilibrium of...

The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called “(decomposable) coarser transition kernels”. This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and stateindependent transitions, and stochastic games with mixtures of constant transition kern...

We study stochastic games with an infinite horizon and sequential moves played by an arbitrary number of players. We assume that social memory is finite—every player, except possibly one, is finitely lived and cannot observe events that are sufficiently far back in the past. This class of games includes games between a long-run player and a sequence of short-run players, and games with overlapp...

Humans exhibit regularities in almost everything they do. We describe a Markov model derived from the behavior patterns of an agent, which is used to determine strategies by predicting which action a user is likely to execute next. We evaluate the predictive accuracy of this approach on a large dataset collected from sample Wumpus World games. We demonstrate from this approach that, the model c...

ION FRAMEWORK FOR MARKOV DECISIONPROCESSES AND PCTL VIA GAMES Mark Kattenbelt Michael Huth

In a Markov game, players engage in a sequence of games determined by a Markov process. In this setting, this paper investigates the impact of varying the informativeness of public information, as defined by Blackwell 1951 and 1953, pertaining to the games that will be played in future periods. In brief, when a curvature condition on payoffs is satisfied, the finding is that, for any fixed disc...

We consider a receding horizon approach as an approximate solution to two-person zero-sum Markov games with infinite horizon discounted cost and average cost criteria. We first present error bounds from the optimal equilibrium value of the game when both players take correlated equilibrium receding horizon policies that are based on exact or approximate solutions of receding finite horizon subg...

In this paper we develop the theory of constrained Markov games. We consider the expected average cost as well as discounted cost. We allow diierent players to have different types of costs. We present suucient conditions for the existence of stationary Nash equilibrium. Our results are based on the theory of sensitivity analysis of mathematical programs developed by Dantzig, Folkman and Shapir...