Towards Rapid Multi-robot Learning from Demonstration at the RoboCup Competition

We describe our previous and current efforts towards achieving an unusual personal RoboCup goal: to train a full team of robots directly through demonstration, on the field of play at the RoboCup venue, how to collaboratively play soccer, and then use this trained team in the competition itself. Using our method, HiTAB, we can train teams of collaborative agents via demonstration to perform nontrivial joint behaviors in the form of hierarchical finite-state automata. We discuss HiTAB, our previous efforts in using it in RoboCup 2011 and 2012, recent experimental work, and our current efforts for 2014, then suggest a new RoboCup Technical Challenge problem in learning from demonstration. Imagine that you are at an unfamiliar disaster site with a team of robots, and are faced with a previously unseen task for them to do. The robots have only rudimentary but useful utility behaviors implemented. You are not a programmer. Without coding them, you have only a few hours to get your robots doing useful collaborative work in this new environment. How would you do this? Our interest lies in rapid, real-time multi-robot training from demonstration. Here a single human trainer teaches a team of robots, via teleoperation, how to collectively perform tasks in previously unforeseen environments. This is difficult for two reasons. First, nontrivial behaviors can present a high-dimensional space to learn, yet one can only provide a few samples, as online training samples are costly to collect. This is a worst case for the so-called “curse of dimensionality”. Second, when training multiple interactive robots, even if you can quantify the emergent macro-level group behavior you wish to achieve, in order to do learning, each agent needs to know the micro-level behavior he is being asked to do. One may have a micro→macro function (a simulator), but it is unlikely that one has the inverse macro→micro function, resulting in what we call the “multiagent inverse problem”. These two challenges mean that real-time multi-robot learning from demonstration has proven very difficult and has a very sparse literature. Over the past several years we have participated in the Kid-Size Humanoid League with a single objective: to successfully do a personal RoboCup-style technical challenge of our own invention, independent of those offered at RoboCup: can we train multiple generic robots, through demonstration on the field, how to play collaborative soccer at RoboCup solely within the preparatory time prior to the competition itself? This is a very high bar: but over the past four years we have made major strides towards achieving it. In RoboCup 2011 we began by replacing a single hard-coded behavior in one attacker with a behavior trained on the field at the venue, and entered that robot into the competition. At RoboCup 2012 we expanded on this by training an attacker to perform all of its soccer behaviors (17 automata, Figure 1), again at the venue. This trained attacker scored our winning goal against Osaka. This year we intend to train multiple robots, and ideally all four robots on the team, to perform collaborative behaviors. Our approach, HiTAB, applies supervised learning to train multiple agents to perform behaviors in the form of decomposed hierarchical finite-state automata. HiTAB uses several tricks, notably task decomposition both per-agent and within a team, to break a complex joint behavior into smaller, very simple ones, and thus radically reduce its dimensionality. Sufficient domain knowledge is involved that HiTAB may fairly be thought of as a form of programming by demonstration. This paper documents our past efforts at applying HiTAB on the field at RoboCup. We also discuss related penalty-kick experiments using the technique, and detail our success so far towards our 2014 goal. Finally, we propose a new RoboCup Technical Challenge in multiagent learning from demonstration.