We investigate in this paper submodular value functions using complex dynamic programming. In complex dynamic programming (dp) we consider concatenations and linear combinations of standard dp operators, as well as combinations of maximizations and minimizations. These value functions have many applications and interpretations, both in stochastic control (and stochastic zero-sum games) as well ...

Animal behavior and evolution can often be described by game-theoretic models. Although in many situations the number of players is very large, their strategic interactions are usually decomposed into a sum of two-player games. Only recently were evolutionarily stable strategies defined for multi-player games and their properties analyzed [Broom, M., Cannings, C., Vickers, G.T., 1997. Multi-pla...

We consider reinforcement learning algorithms in normal form games. Using two-timescales stochastic approximation, we introduce a modelfree algorithm which is asymptotically equivalent to the smooth fictitious play algorithm, in that both result in asymptotic pseudotrajectories to the flow defined by the smooth best response dynamics. Both of these algorithms are shown to converge almost surely...

We consider reinforcement learning algorithms in normal form games. Using two-time-scales stochastic approximation, we introduce a modelfree algorithm which is asymptotically equivalent to the smooth fictitious play algorithm, in that both result in asymptotic pseudotrajectories to the flow defined by the smooth best response dynamics. Both of these algorithms are shown to converge almost surel...

We consider two-player stochastic games played on a finite state space for an infinite number of rounds. The games are concurrent: in each round, the two players (player 1 and player 2) choose their moves independently and simultaneously; the current state and the two moves determine a probability distribution over the successor states. We also consider the important special case of turn-based ...

Stochastic games generalize Markov decision processes (MDPs) to a multiagent setting by allowing the state transitions to depend jointly on all player actions, and having rewards determined by multiplayer matrix games at each state. We consider the problem of computing Nash equilibria in stochastic games, the analogue of planning in MDPs. We begin by providing a generalization of nite-horizon v...

Stochastic ω-Regular Games