Finding and Correcting Flawed Research Literatures

Humanistic psychology has always viewed scientific psychology with skepticism. Good reasons for this skepticism continuously appear. One is then left with the choice, “Is a scientific approach to humans inherently wrongheaded?” or “Is scientific psychology an imperfect but improving enterprise?” This article reviews another domain where research in scientific psychology proves misleading. Suppose a psychologist was asked a question such as, “Is psychotherapy effective?” or “Is remote intercessory prayer effective?” or “Do humans possess psychic powers?” How might a psychologist reply? The most common strategy would be to conduct a meta-analysis over the relevant research literature and report the results. In all 3 cases (i.e., psychotherapy, efficacy of remote intercessory prayer, and telepathic powers) the answer would be a significant, positive effect size, suggesting that all 3 are real, efficacious phenomena. Unfortunately, in at least 2 of the 3 cases, the literature likely gives an incorrect answer to the question. How can one show that some literatures yield “incorrect” answers to research queries, whereas other literatures give “correct” answers? Finally, how should psychology’s publication practices change to avoid flawed literatures?