Multimodal Semantic Learning from Child-Directed Input
نویسندگان
چکیده
Children learn the meaning of words by being exposed to perceptually rich situations (linguistic discourse, visual scenes, etc). Current computational learning models typically simulate these rich situations through impoverished symbolic approximations. In this work, we present a distributed word learning model that operates on child-directed speech paired with realistic visual scenes. The model integrates linguistic and extra-linguistic information (visual and social cues), handles referential uncertainty, and correctly learns to associate words with objects, even in cases of limited linguistic exposure.
منابع مشابه
Prosodic Features from Large Corpora of Child-Directed Speech as Predictors of the Age of Acquisition of Words
The impressive ability of children to acquire language is a widely studied phenomenon, and the factors influencing the pace and patterns of word learning remains a subject of active research. Although many models predicting the age of acquisition of words have been proposed, little emphasis has been directed to the raw input children achieve. In this work we present a comparatively large-scale ...
متن کاملA Localist Neural Network Model for Early Child Language Acquistion from Motherese
This paper presents a localist multimodal neural network that uses Hebbian learning to acquire one-word child language from child directed speech (CDS) comprising multiword utterances and queries in addition to one-word utterances. The model implements cross-situational learning between linguistic words used in child directed speech, the accompanying perceptual entities, conceptual relations an...
متن کاملHow can multimodal cues from child-directed interaction reduce learning complexity in robots?
Robots have to deal with an enormous amount of sensory stimuli. One solution in making sense of them is to enable a robot system to actively search for cues that help structuring the information. Studies with infants reveal that parents support the learning-process by modifying their interaction style, dependent on their child’s developmental age. In our study, in which parents demonstrated eve...
متن کاملرشد جنبه معنایی فعل در کودک فارسیزبان: مطالعه طولی
Objective Learning “verb” as one of the main components of sentence, has been always a debatable topics in the process of language learning. One of the important issues in “verb” learning is determining its meaning using syntactic clues and learning its semantic aspects. Therefore, the main objective of this study was to examine the development of the semantic aspect of ...
متن کاملCombining User Modeling and Machine Learning to Predict Users' Multimodal Integration Patterns
Temporal as well as semantic constraints on fusion are at the heart of multimodal system processing. The goal of the present work is to develop useradaptive temporal thresholds with improved performance characteristics over state-of-the-art fixed ones, which can be accomplished by leveraging both empirical user modeling and machine learning techniques to handle the large individual differences ...
متن کامل