Machine Reconstruction of Human Control Strategies

Complex complex dynamic systems are usually controlled by operators who acquired their skill through years of experience. Typically, such a control skill is sub-cognitive and hard to reconstruct through introspection. The operators cannot completely describe their skill, but can demonstrate it. Therefore an attractive approach to the reconstruction of human control skill involves machine learning from operator's execution traces. The goal is to induce a model of the operator's skill, a control strategy that helps to understand the skill and can be used to control the system. Behavioural cloning is an approach to such skill reconstruction, In the most common approach to behavioural cloning a strategy is induced as a direct mapping from system's states to actions in the form of a decision or regression tree. This thesis develops new ideas to tackle problems that were generally observed with the mentioned approach to human skill reconstruction. One idea is to decompose the learning problem and induce goal-directed strategies that consider learned models of system's dynamics. We introduce a generalized operator's trajectory that can be seen as a continuously changing subgoal. This improves robustness of the resulting controllers. Another idea, that deals with comprehensibility, is to induce qualitative models of human control skill. We show that such qualitative strategies provide an insight into the operator's control skill. On the basis on our experiments, we believe that qualitative strategies can capture important and non-trivial aspects of human control skill. Qualitative strategies open also other new perspectives to the reconstruction of human control skill, such as reconstruction of individual differences in operators's control styles. These ideas were implemented and evaluated in dynamics domains including container crane, a double pendulum referred to as Acrobot, and bicycle ridding. To induce qualitative control strategies we developed algorithm QUIN for learning qualitative constraint tress from numerical examples.