We study non-zero-sum continuous-time stochastic games, also known as continuous-time Markov games, of fixed duration. We concentrate on Markovian strategies. We show by way of example that equilibria need not exist in Markovian strategies, but they always exist in Markovian publicsignal correlated strategies. To do so, we develop criteria for a strategy profile to be an equilibrium via differe...

This paper presents a behavior model of large repeated Bayesian games with an unknown number of players n, correlated types and imperfect monitoring. Each of the n players behaves as if she is negligible in a continuum game, but the game theorist performs a complete Bayesian analysis of what happens with such n players in the real repeated interaction. In addition to performing correct analysis...

We develop an exact dynamic programming algorithm for partially observable stochastic games (POSGs). The algorithm is a synthesis of dynamic programming for partially observable Markov decision processes (POMDPs) and iterative elimination of dominated strategies in normal form games. We prove that it iteratively eliminates very weakly dominated strategies without first forming the normal form r...

This paper deals with an extension of the concept of correlated strategies to Markov stopping games. The Nash equilibrium approach to solving nonzero-sum stopping games may give multiple solutions. An arbitrator can suggest to each player the decision to be applied at each stage based on a joint distribution over the players’ decisions according to some optimality criterion. This is a form of e...