Autonomous Waypoint Generation Strategy for On-Line Navigation in Unknown Environments

This paper introduces a reinforcement learning (RL) based autonomous waypoint generation strategy (AWGS), for on-line path planning in unknown environments. An RL agent intelligently analyzes its surroundings and generates waypoints within the robot’s field of view. The RL agent uses an MDP for waypoint generation that is formulated to be independent of the domain, robot model, and state space dimensionality. The RL agent requires no environment-specific information beyond the robot’s field of view. A path to the selected waypoint is then generated by a path planner. AWGS is applicable to many path or motion planners. However, for brevity, this paper focuses on path planning without the robot’s dynamics constraint. Experiments (i) compare the performance of RL agent’s policies with RRTs and A, and (ii) show that AWGS can: (a) be trained and then used with different robot models, domains, and state-spaces, and (b) successfully navigate in environments with non-convex obstacles.