Regularly updated deterministic policy gradient algorithm

Deep Deterministic Policy Gradient (DDPG) algorithm is one of the most well-known reinforcement learning methods. However, this method inefficient and unstable in practical applications. On other hand, bias variance Q estimation target function are sometimes difficult to control. This paper proposes a Regularly Updated (RUD) policy gradient for these problems. theoretically proves that procedure with RUD can make better use new data replay buffer than traditional procedure. In addition, low value more suitable current Clipped Double Q-learning strategy. has designed comparison experiment against previous methods, an ablation original DDPG, analytical experiments Mujoco environments. The experimental results demonstrate effectiveness superiority RUD.