OAM: An Option-Action Reinforcement Learning Framework for Universal Multi-Intersection Control

Efficient traffic signal control is an important means to alleviate urban congestion. Reinforcement learning (RL) has shown great potentials in devising optimal plans that can adapt dynamic However, several challenges still need be overcome. Firstly, a paradigm of state, action, and reward design needed, especially for optimality-guaranteed function. Secondly, the generalization RL algorithms hindered by varied topologies physical properties intersections. Lastly, enhancing cooperation between intersections needed large network applications. To address these issues, Option-Action framework universal Multi-intersection (OAM) proposed. Based on well-known cell transmission model, we first define lane-cell-level state better model flow propagation. this queuing dynamics, propose regularized delay as facilitate temporal credit assignment while maintaining equivalence with minimizing average travel time. We then recapitulate phase actions constrained combinations lane options neural structure realize any intersection definition. The multiple-intersection rigorously discussed using potential game theory. test OAM algorithm under four networks different settings, including city-level scenario 2,048 synthetic real-world datasets. results show outperform state-of-the-art controllers reducing