Cooperative Markov decision processes: time consistency, greedy players satisfaction, and cooperation maintenance

We deal with multi-agent Markov Decision Processes (MDP’s) in which cooperation among players is allowed. We find a cooperative payoff distribution procedure (MDP-CPDP) that distributes in the course of the game the payoff that players would earn in the long run game. We show under which conditions such a MDP-CPDP fulfills a time consistency property, contents greedy players, and strengthen the coalition cohesiveness throughout the game. Finally we refine the concept of Core for Cooperative MDP’s.