Training Personal Robots Using Natural Language Instruction

based robot. robots using standard programming methods. Indirect methods, such as learning by reinforcement or imitation, are also inappropriate for acquiring userspecific knowledge. Learning by reinforcement, for example, is a lengthy process and, while suitable for refining low-level motor controls, is impractical for complex tasks. Learning by imitation also has limited scope. Neither method can readily generate knowledge representations that the user can interrogate. We are currently exploring an alternative method: Instruction-Based Learning (IBL), which trains robots using natural language instruction. As the “Related Work” sidebar explains, most previous work in this area has focused on issuing commands or language learning. IBL uses unconstrained language in a real-world robotic application that learns prior to execution. It thus offers several potential advantages. Natural language can concisely express rules and command sequences. Also, because it uses symbols and syntactic rules, it is well suited to interact with robots that represent knowledge at the symbolic level. Such symbolic communication can help robots learn faster1 than those that learn at the sensory-motor association level. Here we describe our initial steps toward realizing an IBL system. Along with an overview of how IBL works, we discuss its key steps in detail, our progress on each, and the challenges we’re encountering along the way.