Deep Q‐learning: A robust control approach

This work aims at constructing a bridge between robust control theory and reinforcement learning. Although, learning has shown admirable results in complex tasks, the agent's behavior is opaque. Meanwhile, system several tools for analyzing controlling dynamical systems. article places deep Q-learning into control-oriented perspective to study its dynamics with well-established techniques from control. An uncertain linear time-invariant model formulated by means of neural tangent kernel describe novel approach allows giving conditions stability (convergence) enables analysis frequency-domain. The makes it possible formulate controllers that inject rewards as input loss function achieve better convergence properties. Three output-feedback are synthesized: gain scheduling ℋ 2 $$ {\mathscr{H}}_2 , ∞ {\mathscr{H}}_{\infty } fixed-structure controllers. Compared traditional techniques, which involve heuristics, setting up agent tuning methodology more transparent literature. proposed does not use target network randomized replay memory. role overtaken input, also exploits temporal dependency samples (opposed memory buffer). Numerical simulations different OpenAI Gym environments suggest controlled can converge faster receive higher scores (depending on environment) compared benchmark double Q-learning.