Psychologically authentic versus inauthentic replication attempts.

What is an authentic replication attempt and what is not? Gerber et al.’s paper (1) gives us the opportunity to reflect on this issue of longstanding concern to us. Gerber et al. (1) attempted to replicate our 2011 research (2), which showed that referring to voting with nouns rather than verbs in a pre-election survey (e.g., “How important is it to you to [be a voter/ vote]. . .”) could increase turnout. Although their study bears a superficial resemblance to ours, it is not an authentic replication. Gerber et al. (1) do not create the psychological context in which the phenomenon could plausibly emerge. As described in our paper (2), noun wording can offer one the opportunity to claim a valued identity by engaging in the relevant behavior. We tested our hypothesis in high-profile elections that received substantial public attention: the 2008 United States presidential election and the 2009 New Jersey gubernatorial election. In these cases, the opportunity to be “a voter” feels like a valued identity that can motivate behavior. Contrast this with the congressional primaries examined by Gerber et al. (1): The outcomes of almost none were in doubt—nearly half were uncontested— and few received any meaningful attention. Consider the 12 of 61 major party House primaries that Gerber et al. call “competitive” [see the Supporting Information to their report (1)]: one was uncontested and six others were decided by huge margins (20.5, 32.8, 33, 39, 47.8, and 58.6 percentage points). In reality, only 4 of those 61 primaries weremeaningfully competitive by even a loose standard (details at https://osf.io/g96sc/). To demonstrate the psychological significance of this difference, we asked 366 online participants to imagine either an election like the New Jersey gubernatorial race or a congressional primary. Participants reported how important and positive the identity “voter” would feel in each scenario. As expected, they judged voting in the high-profile election to have far more important and positive identity implications (t = 8.31, P < 0.0005, d = 0.85; details at https://osf.io/g96sc/). Psychological experiments often seem simple. This is deceptive. They are predicated on a careful analysis of psychological processes and the contextual factors that influence them. An authentic replication begins with this psychological understanding (3). In the best cases it attempts to extend this understanding by, for example, directly comparing contexts in which a phenomenon is likely to emerge to ones in which it is not. In the present case, an electoral context that allows the identity “voter” to feel important is necessary to motivate behavior. Misunderstandings like this may be inevitable as social sciences become more interdisciplinary. To prevent them, we can articulate our theoretical and methodological assumptions in our papers so they are clear to nonspecialists. And we can share expertise across disciplinary lines before attempting replications. Understanding each other’s expertise and pooling that expertise through collaboration is essential. By such understanding and sharing we can pursue useful and rigorous science together.