This paper extends the basic work that has been done on tero-sum stochastic games to those that are nonzerosum. Appropriately defined equilibrium points are shown to exist for both the case where the players seek to maximize the total value of their discounted period rewards and the case where they wish to maximize their average reward per period. For the latter case, conditions required on the...

The infinite horizon risk-sensitive discounted-cost and ergodic-cost nonzero-sum stochastic games for controlled Markov chains with countably many states are analyzed. For the discounted-cost game, we prove the existence of Nash equilibrium strategies in the class of Markov strategies under fairly general conditions. Under an additional geometric ergodicity condition and a small cost criterion,...

We extend the “policy rollout” sampling technique for Markov decision processes to Markov games, and provide an approximation result guaranteeing that the resulting sampling-based policy is closer to the Nash equilibrium than the underlying base policy. This improvement is achieved with an amount of sampling that is independent of the state-space size. We base our approximation result on a more...

In this paper we consider the nonparametric identification of Markov dynamic games models in which each firm has its own unobserved state variable, which is persistent over time. This class of models includes most models in the Ericson and Pakes (1995) and Pakes and McGuire (1994) framework. We provide conditions under which the joint Markov equilibrium process of the firms’ observed and unobse...

The Naming Games (NG) are agent-based models for agreement dynamics, peer pressure and herding in social networks, and protocol selection in autonomous ad-hoc sensor networks. By introducing a small noise term to the NG, the resulting Markov Chain model called Noisy Naming Games (NNG) are ergodic, in which all partial consensus states are recurrent. By using Gibbs-Markov equivalence we show how...

Abstract This paper establishes relationships between static Nash equilibria and dynamic Markov perfect equilibria of tariff and quota retaliation games. In supermodular games where tariffs are strategic complements, the steady state of every, symmetric Markov perfect equilibrium must have lower tariffs than in the static equilibrium. If tariffs are strategic substitutes, tariffs in the dynamic...

This paper addresses the problem of learning an equilibrium efficiently in general-sum Markov games through decentralized multi-agent reinforcement learning. Given fundamental difficulty calculating a Nash (NE), we instead aim at finding coarse correlated (CCE), solution concept that generalizes NE by allowing possible correlations among agents’ strategies. We propose algorithm which each agent...

Stochastic (a.k.a. Markov) Games pose many unsolved problems in Game Theory. One class of stochastic games that is better understood is that of Common Interest Stochastic Games (CISG). CISGs form an interesting class of multi-agent settings where the distributed nature of the systems, rather than adverserial behavior, is the main challenge to efficient learning. In this paper we examine three d...