DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning

In recent years, multi-agent reinforcement learning (MARL) has presented impressive performance in various applications. However, physical limitations, budget restrictions, and many other factors usually impose constraints on a system (MAS), which cannot be handled by traditional MARL frameworks. Specifically, this paper focuses constrained MASes where agents work cooperatively to maximize the expected team-average return under costs, develops cooperative framework, named DeCOM, for such MASes. particular, DeCOM decomposes policy of each agent into two modules, empowers information sharing among achieve better cooperation. addition, with modularization, training algorithm separates original optimization an unconstrained reward satisfaction problem costs. then iteratively solves these problems computationally efficient manner, makes highly scalable. We also provide theoretical guarantees convergence DeCOM's update algorithm. Finally, we conduct extensive experiments show effectiveness types costs both moderate-scale large-scale (with 500 agents) environments that originate from real-world