Context Effects in Spoken Language Perception

Contextual evidence plays a significant role in perception. It influences the interpretation of sensory evidence and can increase both speed and accuracy. Researchers and clinicians have paid a lot of attention to the auditory sensory evidence available to hearing-impaired subjects and its enhancement by hearing aids and cochlear implants. Much less attention has been paid to the role and use contextual evidence, perhaps because of a shortage of assessment tools and quantitative methods. In this paper, two metrics are introduced. One is the k-factor, which is the ratio of the logarithms of error probabilities when perceiving with and without a particular context. The other is the jfactor, which is the ratio of the logarithms of recognition probabilities for wholes and their constituent parts. Both are derived from probability and information theory. Data from normally hearing adults, and from hearing-impaired children who use cochlear implants, provide useful insights. For example, the contribution of sentence context is shown to be attributable mainly to sentence meaning with little contribution from syntax. There are also substantial individual differences in the ability to take advantage of sentence context. Both types of subject show evidence of increased reliance on sentence context as the available sensory evidence is reduced. When perceiving lowredundancy sentences of known topic, under difficult listening conditions, the contribution of contextual evidence can be as much as 10 times that of the sensory evidence. It is further shown that contextual constraints within consonant-vowelconsonant words lead them to behave as though they consist of between 2 and 2.5 independently perceived phonemes. This was found to be true for both the normally hearing adults and most of the pediatric implantees. A few of the implantees, however appeared not to be taking advantage of lexical context in the perception of phonemes. A model is developed that combines j and k factors so as to predict sentence-level speech perception from the perception of phonetic contrasts. Such models may be useful in predicting possible outcome on the basis of measurements of phonetic contrast perception that can be obtained from very young children. Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 2 Context and perception Perception is the interpretation of sensory evidence that has been generated by the sense organs in response to patterns of physical stimulation originating from objects and events in the external world. Essentially, the perceiver chooses the most likely interpretation from the several possibilities that exist within his internal model of the external world. This internal model, which has been developed as a result of perceptual, cognitive, and social-cognitive development, represents prior knowledge that the perceiver brings to the perceptual process. But perceptual decisions are not based on sensory evidence alone. Every object exists in a context and every event occurs in a context. The context provides the perceiver with additional evidence that can influence the interpretation of sensory evidence. As with sensory evidence, the value of the contextual evidence depends on the perceiver's prior knowledge. To be useful, perception must be fast and accurate. By using contextual evidence effectively, the perceiver can increase both speed and accuracy. If reliance on context is taken to extremes, however, the probability of error increases. Perceptual skill requires an optimal balance between speed and accuracy, an optimal mix in the use of sensory and contextual evidence, and the ability to change both according to the demands of the immediate situation. To summarize, successful perception depends on four factors. The sensory and contextual evidence that are available to the perceiver and the knowledge and skills that the perceiver brings to the task (See, also, Boothroyd, 1994). Context and spoken language perception Although spoken language perception rests on the substrate of general perception, there are several aspects of the process that are unique (see, also, Boothroyd, 1997). For example: 1. The sound-generating events to be perceived are the movement patterns of speech. But these have no inherent value. Rather, they provide evidence about underlying language patterns that are, themselves, evidence about the meaning and purpose of the message being generated by the talker. 2. Similarly, the percepts consist, not of sound-generating events, but of language patterns, and their meanings. 3. There are many situations in which auditory evidence about speech movements is enhanced by visual evidence. In extreme cases of profound hearing loss, vision can even be the only source of sensory evidence as in lipreading . 4.To the physical and social context we must add a linguistic context. Phonemes occur in the context of words. Words occur in the context of sentences. And sentences Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 3 occur in the context of narratives and conversations, usually with a known topic. Contextual evidence, therefore, has a linguistic dimension. 5. To the world and social knowledge that the perceiver brings to the perceptual task, we must add linguistic knowledge phonological, lexical, syntactic, semantic, and pragmatic. The presence of linguistic context effects has been recognized for many years. (See, for example, Rozensweig and Postman (1957) and Broadbent (1967) on the effects of frequency of word occurrence and Miller, Heise, and Lichten (1951) on the effects of sentence context). 6. The speed of spoken language perception is determined, not by the perceiver, but by the talker. As a result, the perceiver is restricted when seeking an appropriate compromise between speed and accuracy or an appropriate balance between sensory and contextual evidence. Anyone trying to understand speech in a second language is familiar with the problems created when the speed at which the talker produces speech exceeds the speed with which it can be perceived and the errors that occur when one deals with this problem by an excessive reliance on context. Prelingual deafness and spoken language perception The child with a congenital or prelingually acquired hearing loss is at a multiple disadvantage in the perception of spoken language. First, the hearing deficit reduces the quality and quantity of sensory evidence available from the speech stimulus. Second, the resulting language deficits reduce the quality and quantity of linguistic contextual evidence. Third, cognitive and intellectual deficits resulting from the language deficit reduce the quality and quantity of non-linguistic contextual evidence. Fourth, in an attempt to deal with the reduced sensory evidence, the child may adopt a perceptual strategy that satisfies short term needs but at the expense of the long-term development of appropriate processing skills. All four aspects of perception are at risk: sensory evidence, contextual evidence, knowledge, and skill. The combined effect of these difficulties is an increased error in spoken language perception. Moreover, the demands of perception with diminished sensory and contextual evidence often lead to rapid fatigue and a reduced attention span for spoken language. Naturally, the severity of the problems just described vary from child to child depending on the severity of the hearing loss, the aptitudes of the child, and the success of sensory and educational intervention. In recent decades, we have learned a lot about the effects of hearing loss, amplification, and cochlear implants on the sensory component of spoken language perception and we have developed a range of assessment tools. But less attention has been paid to understanding and assessing the contextual component. A valuable early contribution to the assessment of context effects was the work of Kalikow, Stevens, and Elliot (1977) which led to the development of the Speech in Noise (SPIN) test, later modified by Bilger (1984). In this test, recognition is measured for the last word in two types of sentence. High-probability sentences carry a lot of Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 4 semantic cues to the identity of the test word (for example, "Zebras have black and white stripes." Low-probability sentences carry no semantic cues (but do retain syntactic cues). For example, "They were talking about the stripes"). Noise is used to reduce sensory evidence. The difference between the two scores provides a measure of the use of semantic contextual evidence. Although these materials were originally developed for the express purpose of assessing the use of context, it is interesting to note that their major application has simply been as a sentence-level performance measure. Word recognition with and without sentence context k-factor theory One problem with the use of a difference between two performance measures as an index of context effects is that the difference is range-dependent. As overall performance approaches 0% or 100% the difference measure automatically approaches zero. A simple difference may not, therefore, accurately quantify the perceiver's use of context. In my own work on this topic, I have sought suitable transforms of the difference scores, using methods based on probability theory and information theory. When listening under difficult conditions, all words in a sentence must be perceived with reduced sensory evidence. But each word need not be perceived independently, as if it were presented in isolation. Once some of the words are recognized, they provide contextual evidence that can increase the recognition probability for other words. This effect can be modeled as proportional increase in the independent channels of information available to the perceiver. The resulting equation is: ps = 1 (1-pi) ................................................................................. (1) Where: ps = word-recognition probability in sentences pi = word-recognition probability in isolation k = a dimensionless factor representing the effect of sentence context . From equation (1) one can derive an expression for k, given two estimates of recognition probability: k = log(1 ps)/log(1pi) ................................................................. (2) In other words, k is the ratio of the logarithms of error probability with and without sentence context. Note, again, that the k-factor can be thought of as a proportional increase in the effective number of channels of independent information available to the perceiver. A k factor of 1 indicates that sentence context has no effect on word recognition. A k factor of 2 indicates that the effect of adding sentence context is equivalent to doubling the number of channels of independent information that are available when the words are Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 5 presented in isolation, and so on. In a sense, one can think of the contextual evidence as multiplying the usefulness of the sensory evidence by the k-factor. Figure 1 shows the predicted effects of sentence context on word recognition for various values of k ranging from 0 to 10. The horizontal axis shows word recognition probability without sentence context, expressed in percent. The vertical axis shows the predicted word recognition with sentence context. Some k-factor data from normal adults Clearly, the effect of sentence context will depend both on the redundancy of the sentence material itself and on the knowledge and skills of the perceiver. By studying adults with normal hearing and language, one can obtain estimates of the contextual information available in the speech materials, as well as data on the normal range of variability in the use of that information. Boothroyd and Nittrouer (1988) attempted to separate the contributions of syntactic and semantic factors to the sentence context effect. Word recognition was measured in three types of sentence: a) random word strings, in which there was no sentence context (e.g., "green like went was"), b) semantically anomalous, or implausible, sentences in which syntax played the dominant role (e.g., "ducks eat old tape"), and c) highly plausible sentences in which both syntax and semantics played a role (e.g., "most birds can fly"). All sentences were four words long so as not to tax short-term memory. Noise was used to reduce sensory evidence for groups of eight young, normally hearing, adults. The results are shown in Figure 2. It will be seen that the k factor for syntax alone was only 1.35. The k factor rose to 2.70 when plausible meaning was added. From these two values, it can be shown that the k factor for semantics alone was 2.0. These data indicate that, although syntax plays a role, most of the sentence context effect is attributable to sentence meaning. There was no evidence from these data that the value of k was range-dependent. Similar values were obtained under both easy and difficult listening conditions. The sentences were, however, extremely short. Boothroyd and Kosky (1991) measured the effect of sentence context in a more realistic situation. The sentence materials were the CUNY topic-related sentence sets. Each set consists of twelve sentences ranging in length from three through fourteen words. Each sentence is about one of twelve topics that remain constant throughout the sets. Before each sentence was heard, subjects were informed of the topic. The purpose was to simulate a conversational setting in which the topic of a sentence is usually known. Recognition of words in isolation was measured using the AB words lists. Each list contains ten consonant-vowel-consonant words. In this study the sensory evidence was reduced, not by noise, but by low-pass filtering. Mean data for six normally hearing adults are shown in Figure 3. Two aspects of these data are noteworthy. First, the k factors are considerably higher than those reported earlier. Three factors may account for the difference. One is the influence of prior knowledge of sentence topic. The second is the use of monosyllables for measuring word recognition in isolation. The sentences Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 6 contained many polysyllables, which are easier to perceive than monosyllables. Third, most of the sentences were longer and more complex than those used in the Boothroyd and Nittrouer study. They would be expected, therefore, to provide more contextual information, including that available from suprasegmental cues. The second noteworthy aspect of the data in Figure 3 is that the value of k does not remain constant across listening conditions. Instead it falls as listening becomes easier. The implication is that the subjects were changing their perceptual strategy to take more advantage of contextual evidence as the amount of sensory evidence was reduced. The fact that this phenomenon was not observed in the Boothroyd and Nittrouer study is probably, again, related to the use of shorter sentences in the earlier study. The heavy line in Figure 3 is a least-squares fit to equation (1) with the added assumption that k falls exponentially from some starting value, when word recognition in isolation is 0%, towards an asymptote of 1. Thus: ki = (1+ a*e )........................................................................................ (3) Where: i = pi = word recognition in isolation ki = value of k when word recognition = i a+1 = value of k for i = 0 (i.e., k0) b = an empirically determined decay constant The best fit to the data of Figure 3 was obtained with k0 = 10.3 and b = 58.4, giving a predicted k100 of 1.9. An important finding from this study was that k values differed significantly among subjects (p<0.05). When recognition probabilities were collapsed across filtering conditions, k values ranged from 4.2 to 9.9. These data support the conclusion that adults with no known difficulties of hearing or language can differ considerably in terms of their ability to take advantage of sentence context under difficult listening conditions. Such differences may well play a role in the wide range of lipreading abilities observed in adults who have lost their hearing. Variability of the k-factor across subjects and listening conditions has also been reported by Grant and Seitz (2000). Some k-factor data from children with hearing loss Figure 4 shows data on word recognition with and without sentence context that were obtained during a study of ability and attainment in a group of eight pediatric implantees (Boothroyd and Boothroyd-Turner, in press). All children used the Nucleus multichannel cochlear implant. All were born with profound hearing loss. The average age at implantation was six years. At the time these data were collected, the average duration of use was five years. All were educated orally in a special unit, with partial mainstreaming, in the Toronto, Canada, Public School system. Performance was Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 7 measured as the percentage of words correctly imitated from recordings of a female talker. As in the Boothroyd and Kosky study, the AB word lists were used to assess word recognition in isolation. The word-in-sentence data were obtained with two types of sentence. One type (grade-5) was intended to be appropriate for the average chronological age of the subjects. The other type (pre-school) was simpler and shorter and designed to be appropriate for much younger children. For both types, the subjects were informed of the topic of a sentence before it was presented. For the pre-school sentences, there were only three topics: washing and bathing, clothes and dressing, and food and eating. For both types of sentence, the k factors decreased as word recognition in isolation increased. In this sense, the data are similar to those reported above for normally hearing adults. They suggest that those children who obtained the least sensory evidence from their implants were compensating by taking grater advantage of linguistic context. The k-values were, on average, twice as high for grade-5 sentences as for pre-school sentences. This observation, again, supports the conclusion that these children were, in general, making effective use of sentence context to compensate for reduced sensory evidence. A disturbing aspect of these findings, however, is the generally poor performance on grade-appropriate sentences. If one were to base an estimate of spoken language perception ability on communication in highly redundant, predictable, situations (as reflected by the pre-school sentence data) one might easily underestimate the difficulties faced by these children in an educational setting, where the material is inevitably less redundant and less predictable (as reflected by the grade-5 sentence data). On the positive side, it should be noted that the data reported here are for auditory perception only. In a classroom, lipreading will often be available as a supplement to hearing. Finally, note the outlier, indicated by the squares. This subject not only had considerable difficulty recognizing words in isolation but also failed to compensate by taking full advantage of sentence context. On the strength of these data she was referred for further evaluation and it was concluded that she had a specific language disorder in addition to her hearing loss. This conclusion was supported by a very low score on the verbal subtests of a standard intelligence test, in spite of a very high score on the non-verbal subtests. Perception of wholes and parts within wholes j-factor theory Information about the use of contextual evidence can also be obtained by comparing recognition probabilities for wholes with recognition probabilities for the constituent parts. One example is the recognition of whole consonant-vowel-consonant words Boothroyd. Context effects............. Saved as Salamanca 2002 paper.doc 8 versus the recognition of the phonemes within the words. If the phonemes must be recognized independently, then the predicted probability of recognition of a whole word is given by: pw = pp .............................................................................................. (4) where: pw = probability that a whole word will be recognized pp = probability that the phonemes within the words will be recognized The exponent "3" reflects the fact that there are 3 phonemes in each word. Equation (4) was validated for young, normally hearing adults in the Boothroyd and Nittrouer (1988) study mentioned earlier, by testing with consonant-vowel-consonant nonsense syllables. But the fact that the phonemes in meaningful words are in a lexical context violates the assumption of independence. For, example, the recognition of two phonemes will increase the probability of recognition of the third. The result is a reduction of the exponent in equation (4). Thus: pw = pp............................................................................................... (5) where j is a dimensionless factor representing the effects of lexical context.