We describe our experiences in trying to implement a hierarchical reinforcement learning system, and follow with conclusions that we have drawn from the difficulties that we encountered. We present our objectives before we started, the problems we encountered along the way, the solutions we devised for some of these problems, and our conclusions afterward about the class of problems for which r...