Since the mid-1980s, psychologists and neuroscientists have used brain imaging to test hypotheses about human thought processes and their neural instantiation. In just three decades, functional neuroimaging has been transformed from a crude clinical tool to a widely used research method for understanding the human brain and mind. Such rapidly achieved success is bound to evoke skepticism. A deg...

Evaluation of MIR systems requires honesty and skepticism with respect to the selection of test collections and the interpretation of results. We describe three minimal criteria which help to ensure this honesty and skepticism through appropriate selection of test collections, elimination of bias, and objective analysis of results.