In their standard formulations, stochastic games and Markov decision processes assume a rational opponent or a stationary environment. Online learning algorithms can adapt to arbitrary opponents and non-stationary environments, but do not incorporate the dynamic structure of stochastic games or Markov decision processes. We survey recent approaches that apply online learning to dynamic environm...

We propose two related equilibrium re…nements for voting and agenda-setting games, Sequentially Weakly Undominated Equilibrium (SWUE) and Markov Trembling Hand Perfect Equilibrium (MTHPE), and show how these equilibrium concepts eliminate non-intuitive equilibria that arise naturally in dynamic voting games and games in which random or deterministic sequences of agenda-setters make o¤ers to sev...

Markov games is a framework which can be used to formalise n-agent reinforcement learning (RL). Littman (Markov games as a framework for multi-agent reinforcement learning, in: Proceedings of the 11th International Conference on Machine Learning (ICML-94), 1994.) uses this framework to model two-agent zero-sum problems and, within this context, proposes the minimax-Q algorithm. This paper revie...

We study the complexity of a class of Markov decision processes and, more generally, stochastic games, called 1-exit Recursive Markov Decision Processes (1-RMDPs) and Simple Stochastic Games (1-RSSGs) with strictly positive rewards. These are a class of finitely presented countable-state zero-sum stochastic games, with total expected reward objective. They subsume standard finite-state MDPs and...

We discuss recent work on the algorithmic analysis of systems involving recursion and probability. Recursive Markov chains extend ordinary finite state Markov chains with the ability to invoke other Markov chains in a potentially recursive manner. They offer a natural abstract model of probabilistic programs with procedures, and generalize other classical well-studied stochastic models, eg. Mul...

Markov games are a generalization of Markov decision process to a multi-agent setting. Two-player zero-sum Markov game framework offers an effective platform for designing robust controllers. This paper presents two novel controller design algorithms that use ideas from game-theory literature to produce reliable controllers that are able to maintain performance in presence of noise and paramete...

We consider two-players zero-sum perfect information stochastic games with finitely many states and actions and examine the problem of existence of pure stationary optimal strategies. We show that the existence of such strategies for one-player games (Markov decision processes) implies the existence of such strategies for two-player games. The result is general and holds for any payoff mapping.

A class of stochastic positional games which extend the cyclic games and Markov decision problems with average and discounted optimization costs criteria is formulated and studied. Nash equilibria conditions for considered class of stochastic positional games are derived and some approaches for determining Nash equilibria are described.

We extend the construction of equilibria for linear-quadratic and mean-variance portfolio problems available in literature to a large class mean-field time-inconsistent stochastic control continuous time. Our approach relies on time discretization problem via $n$-person games, which are characterized maximum principle using backward differential equations. The existence is proved by applying we...