Neural Baby Talk
نویسندگان
چکیده
We introduce a novel framework for image captioning that can produce natural language explicitly grounded in entities that object detectors find in the image. Our approach reconciles classical slot filling approaches (that are generally better grounded in images) with modern neural captioning approaches (that are generally more natural sounding and accurate). Our approach first generates a sentence ‘template’ with slot locations explicitly tied to specific image regions. These slots are then filled in by visual concepts identified in the regions by object detectors. The entire architecture (sentence template generation and slot filling with object detectors) is end-to-end differentiable. We verify the effectiveness of our proposed model on different image captioning tasks. On standard image captioning and novel object captioning, our model reaches state-of-the-art on both COCO and Flickr30k datasets. We also demonstrate that our model has unique advantages when the train and test distributions of scene compositions – and hence language priors of associated captions – are different. Code has been made available at: https://github.com/jiasenlu/NeuralBabyTalk.
منابع مشابه
Do baby-talk words reflect biomechanical constraints on speech production?
Baby-talk words are conventionalized lexical items used in infant-directed speech. According to the Frame/Content theory, the structure of baby-talk words reflects biomechanical constraints on speech production associated with the origin of speech, a hypothesis that yields three predictions — compared to adult vocabulary, baby-talk words should exhibit (1) more canonical CV syllables, (2) more ...
متن کاملConceptual Differences Between Children and Adults
In a talk in which I described an extended case study of the young child's acquisition of biological knowledge (Carey 1985), 1 claimed that the preschool child's concepts animal and baby differ from our adult concepts.' One source of evidence for this claim is that 4and 5-year-olds typically do not realize that all animals have babies, indicating a concept linirnal without reproduction as a cor...
متن کاملBreastfeeding with diabetes: yes you can!
Before Your Baby Is Born Taking good care of yourself during pregnancy can help your baby to be healthy. You can also get ready for breastfeeding. Talk with your baby’s doctor about how to monitor your glucose levels after the birth. About half of babies of diabetic mothers may have low blood sugar soon after birth. If you have type 1 diabetes, discuss nighttime insulin dosages, calorie needs, ...
متن کاملBaby Verma Modules for Rational Cherednik Algebras
These are notes for a talk in the MIT-Northeastern Spring 2015 Geometric Representation Theory Seminar. The main source is [G02]. We discuss baby Verma modules for rational Cherednik algebras at t = 0.
متن کاملContact Situations and Foreigner Talk
Ferguson, C. (1971) first introduces the idea of “foreigner talk” (FT) and examines the grammatical simplicity in FT by indicating the absence of the copula in “grammatical machinery for equational clauses” (for instance, in the case of the Japanese language). Evidences of simplified speech in FT is similar to ‘retarded talk’. And, because it can also be directed at children, it is also referre...
متن کامل