Our primary interest in semantic knowledge discovery and use is in the context of natural language understanding. One of the big challenges for language understanding is that much information that text is intended to convey is not explicitly stated. Rather, the reader constructs a mental model of the scene described by the text, including many "obvious" features that were not explicitly mention...

There is ample evidence that human understanding of ordinary language relies in part on a rich capacity for imagistic mental modeling. We argue that genuine language understanding in machines will similarly require an imagistic modeling capacity enabling fast construction of instances of prototypical physical situations and events, whose participants are drawn from a wide variety of entity type...

For natural language understanding systems designed for domains including relatively complex equipment, it is not sufficient to use general knowledge about this equipment. We show problems which can be solved only if the system has access to a detailed equipment model. We discuss features of such models, in particular, their ability to simulate the equipment's behavior. As an illustration, we d...

Embodied approaches to comprehension (Narayanan 1997; Zwaan 1999) propose that understanding language entails performing mental simulations of its content. The evidence, however, is mixed. Action-sentence Compatibility Effect studies (Glenberg and Kaschak 2002) report mental simulation of motor actions during processing of motion language. But the same studies find no evidence that language com...

Object ive To construct a data base (the "Penn Treebank') of written and transcribed spoken American English annotated with detailed grammatical structure. This data base will serve as a national resource, providing training material for a wide variety of approaches to automatic language acquisition, a rei~rence standard for the rigorous evaluation of some components of natural language underst...

During incremental language understanding, comprehenders draw on a rich base of probabilistic cues to efficiently process the noisy perceptual input they receive. One challenge listeners face in employing such cues is that most cues are contextdependent. Here, we present an experiment that investigates the extent to which listeners learn situation-specific adjustments in the information and/or ...

This work is on a previously formalized semantic evaluation task of spatial role labeling (SpRL) that aims at extraction of formal spatial meaning from text. Here, we report the results of initial efforts towards exploiting visual information in the form of images to help spatial language understanding. We discuss the way of designing new models in the framework of declarative learning-based pr...

It is difficult for a natural language understanding system (NLUS) to deal with ambiguities. There is a dilemma: an NLUS must be able to produce plausible interpretations for given sentences, avoiding the combinatorial explosion of possible interpretations. Furthermore, it is desirable for an NLUS to produce several interpretations if they are equally plausible. EXAM, the system described in th...

The generative summarization of textual stories has been one of the goals of computational narratology since attempts at full semantic NLU in the ’70s. Our NLP group has recently created several systems for multidocument news summarization using purely statistical methods. Between these poles, there may be an unexplored avenue where knowledge of story structure can give partial, yet useful sema...