Syntactic Judgment Experiments

Informal judgments of sentence acceptability have long been the primary source of evidence about grammaticality in syntax, and have been controversial just as long. In the past decade, there has been growing interest in collecting and analyzing acceptability judgments according to the formal protocols of experimental psycholinguistics, an approach sometimes called experimental syntax. This article reviews the major issues relevant to this approach, namely, the relative reliability, validity, sensitivity, and convenience of formal vs. informal methods. 1. From Informal Judgments to Formal Judgment Experimentation Theoretical syntacticians typically test empirical claims with what may be called informal judgments: they create one or two example sentences and decide (perhaps with the help of a colleague or two) whether they seem acceptable in their native language. Informal judgments inspire horror in many scholars, as they so clearly violate the methodological protocols standard in the rest of the empirical cognitive sciences (cognitive psychology and neuroscience). The procedure seems to be a kind of psycholinguistic experiment, with the sentences as stimuli and the judgments as responses (as noted even by critics like Labov 1975), but too few sentences and speakers are tested to assess generality, the experimental participant is usually the linguist himself or herself (risking unrepresentativeness and experimenter bias), and judgments are usually made on a binary good/ bad scale (despite the general recognition that sentence acceptability varies gradiently; Chomsky 1965: 10–1). Moreover, it is not clear how acceptability judgments relate to other data sources like corpora or processing experiments, let alone grammaticality itself (i.e. the status of a sentence accorded by the internal mental grammar). Linguists have been debating the role of acceptability judgments in syntax ever since Chomsky (1957) put judgments at the center of his methodology. In a highly influential review of these debates, Schütze (1996) concludes that syntacticians must learn to collect and analyze judgments in proper psycholinguistic fashion. Today, more attention is being paid to linguistic judgments than ever before. A crude measure of this