This paper considers a general class of discounted Markov stochastic games characterized by multidimensional state and action spaces with an order structure, and one-period rewards and state transitions satisfying some complementarity and monotonicity conditions. Existence of pure-strategy Markov (Markov-stationary) equilibria for the …nite (in…nite) horizon game, with nondecreasing –and possib...

We present a new algorithm for polynomial time learning of optimal behavior in stochastic games. This algorithm incorporates and integrates important recent results of Kearns and Singh 5] in reinforcement learning and of Monderer and Tennenholtz 7] in repeated games. In stochastic games, the agent must cope with the existence of an adversary whose actions can be arbitrary. In particular, this a...

We adopt stochastic games as a general framework for dynamic noncooperative systems. This framework provides a way of describing the dynamic interactions of agents in terms of individuals' Markov decision processes. By studying this framework, we go beyond the common practice in the study of learning in games, which primarily focus on repeated games or extensive-form games. For stochastic games...

We introduce the concept of a Markov risk measure and we use it to formulate risk-averse control problems for two Markov decision models: a finite horizon model and a discounted infinite horizon model. For both models we derive risk-averse dynamic programming equations and a value iteration method. For the infinite horizon problem we also develop a risk-averse policy iteration method and we pro...

This paper considers a class of discrete-time reinforcement-learning dynamics and provides a stochasticstability analysis in repeatedly played positive-utility (strategicform) games. For this class of dynamics, convergence to pure Nash equilibria has been demonstrated only for the fine class of potential games. Prior work primarily provides convergence properties through stochastic approximatio...

Computer games can be motivating and engaging experiences that facilitate learning, leading to their increasing use in education and behavioural experiments. For these applications, it is often important to make inferences about the knowledge and cognitive processes of players based on their behaviour. However, designing games that provide useful behavioural data are a difficult task that typic...

One-clock priced timed games is a class of two-player, zero-sum, continuous-time games that was defined and thoroughly studied in previous works. We show that one-clock priced timed games can be solved in time m12n, where n is the number of states and m is the number of actions. The best previously known time bound for solving one-clock priced timed games was 2 2+m), due to Rutkowski. For our i...

In this paper we address the problem of coordination in multi-agent sequential decision problems with infinite statespaces. We adopt a game theoretic formalism to describe the interaction of the multiple decision-makers and propose the novel approximate biased adaptive play algorithm. This algorithm is an extension of biased adaptive play to team Markov games defined over infinite state-spaces....

This paper proposes several statistical tests for finite state Markov games to examine the null hypothesis that the data are generated from a single equilibrium. We formulate tests of (i) the conditional choice and state transition probabilities, (ii) the steady-state distribution, and (iii) the conditional state distribution given an initial state. In a Monte Carlo study we find that the test ...

Interactions between intelligent agents in multiagent systems can be modeled and analyzed by using game theory. The agents select actions that maximize their utility function so that they also take into account the behavior of the other agents in the system. Each agent should therefore utilize some model of the other agents. In this paper, the focus is on the situation which has a temporal stru...