Strategy recovery for stochastic mean payoff games

We prove that to find optimal positional strategies for stochastic mean payoff games when the value of every state of the game is known, in general, is as hard as solving such games tout court. This answers a question posed by Daniel Andersson and Peter Bro Miltersen. In this note, we consider perfect information 0-sum stochastic games, which, for short, we will just call stochastic games. For us, a stochastic game is a finite directed graph whose vertices we call states and whose edges we call transitions, multiple edges and loops are allowed but no state can be a sink. To each state s is associated an owner o(s) which is one of the two players Max and Min. Each transition s A,p −−→t has an action A and a probability p ∈ Q∩ [0, 1], with the condition that, for each state s, the probabilities of the transitions exiting s associated to the same action must sum to 1. We say that the action A is available at state s if one of the transitions exiting s is associated to A. Furthermore to each action A is associated a reward r(A) ∈ Q. A play of a stochastic game G begins in some state s0 and produces an unending sequence of states {si}i∈N and actions {Ai}i∈N. At move i, the owner of the current state si chooses an action Ai among those available at si, then one of the transitions exiting si with action Ai is selected at random according to their respective probabilities, and the next state si+1 is the destination of the chosen transition. A play can be evaluated according to the β-discounted payoff criterion vβ (A0,A1 . . .) = (1−β) ∞ ∑