Towards Experimental Analysis of Challenge Scenarios in Robotics

In this paper, we explore the idea of simulated experimental analysis for challenge scenarios in robotics using the search and secure problem from the Multi Autonomous Ground-robotic International Challenge (MAGIC). The MAGIC problem requires a team of heterogeneous robots to locate, classify, and secure a number of targets in an urban environment with indoor and outdoor areas. We introduce a framework for solving the coordination aspects of the challenge by providing guaranteed clearing strategies (i.e., strategies that ensure coming into contact with any adversarial target). The proposed method allows for repair of the clearing schedule after robot failure, as well as a fall-back strategy if clearing is no longer possible. We analyze scenarios taken directly from the competition, and we utilize repeated simulated trials to validate the hypothesis that strategies designed for locating worstcase targets tend to be more robust to failure than strategies designed for locating average-case targets. Thus, more conservative worst-case methods would tend to perform better if the competition were run many times. However, riskier averagecase strategies may win in a single competition. These results demonstrate how insight can be gained from repeated simulated analysis of challenge scenarios in robotics.