Biologically Plausible Neural Model for the Recognition of Biological Motion and Actions

The visual recognition of complex movements and actions is crucial for communication and survival in many species. Remarkable sensitivity and robustness of biological motion perception have been demonstrated in psychophysical experiments. In recent years, neurons and cortical areas involved in action recognition have been identified in neurophysiological and imaging studies. However, the detailed neural mechanisms that underlie the recognition of such complex movement patterns remain largely unknown. This paper reviews the experimental results and summarizes them in terms of a biologically plausible neural model. The model is based on the key assumption that action recognition is based on learned prototypical patterns and exploits information from the ventral and the dorsal pathway. The model makes specific predictions that motivate new experiments. We thank I. Bülthoff, E. Curio, Z. Kourtzi, M. Riesenhuber, T. Sejnowski, P. Sinha, I. Thornton, and L. Vaina for very useful comments. We are grateful to A. Benali, Z. Kourtzi, and C. Curio for help with the data acquisition, and to the Max-Planck Institute for Biological Cybernetics, Tübingen, for providing support.We thank M. Fitzgerald for help with the final layout. This report describes research done within the McGovern Institute and the Center for Biological & Computational Learning in the Department of Brain & Cognitive Sciences and in the Artificial Intelligence Laboratory at the Massachusetts Institute of Technology. M. Giese was supported by the Deutsche Forschungsgemeinschaft, Honda R&D Americas Inc., and the Deusche Volkswagen Stiftung. T. Poggio is supported in part by the Whitaker chair. Research at CBCL was sponsored by grants from: Office of Naval Research (DARPA) under contract No. N00014-00-1-0907, National Science Foundation (ITR) under contract No. IIS-0085836, National Science Foundation (KDI) under contract No. DMS-9872936, and National Science Foundation under contract No. IIS-9800032. Additional support was provided by: Central Research Institute of Electric Power Industry, Eastman Kodak Company, DaimlerChrysler AG, Compaq, Komatsu, Ltd., NEC Fund, Nippon Telegraph & Telephone, Siemens Corporate Research, Inc.