Planning Irregular Object Packing via Hierarchical Reinforcement Learning

Object packing by autonomous robots is an important challenge in warehouses and logistics industry. Most conventional data-driven planning approaches focus on regular cuboid packing, which are usually heuristic limit the practical use realistic applications with everyday objects. In this paper, we propose a deep hierarchical reinforcement learning approach to simultaneously plan sequence placement for irregular object packing. Specifically, top manager network infers from six principal view heightmaps of all objects, then bottom worker receives next predict position orientation. The two networks trained hierarchically self-supervised Q-Learning framework, where rewards provided results based height, volume stability box. framework repeats iteratively until objects have been packed into box or no space remained unpacked items. We compare our existing robotic methods physics simulator. Experiments show that can pack more less time cost than state-of-the-art also implement manipulator generalization ability real world.