Functional Similarity in Markov Environments

In this paper we discuss the notion of functional similarity between different situations an artificial agent may encounter, and show how it may be used to transfer information across tasks. We say two situations are functionally similar (FS) if there exists an action that has a similar effect in both the situations. So for instance, many situations containing a soccer ball in an open space are FS with respect to the action ”kick ball”. Thus, if we can determine that a novel situation is FS to some previously observed situations with respect to an action, we can use the behavior of the previous situations as an estimate of the behavior of the new situation with respect to that action. In this paper we give a concrete definition of functional similarity for Markov Environments and briefly show how this may be used to construct and use a novel type of forward model, which we call transition prediction model (TPM), for such domains. We also mention some interesting theoretical properties of the TPM. Finally we describe possible avenues of future research and related work.