Linguistic Resources for 2013 Knowledge Base Population Evaluations

Knowledge Base Population (KBP) is an evaluation track of the Text Analysis Conference (TAC), a workshop series organized by the National Institute of Standards and Technology (NIST). In 2013, the KBP evaluations included five tasks targeting information extraction and question answering technologies: Entity Linking, Slot Filling, Temporal Slot Filling, Sentiment Slot Filling, and Cold Start. The Sentiment and Temporal Slot Filling tasks were introduced in 2013 in an effort to move the KBP challenges into new domains, specifically beliefs and events. Linguistic Data Consortium (LDC) has supported the TAC KBP evaluation since 2009, each year producing new linguistic resources including data, annotations, system assessments, tools and specifications. This paper describes the resource creation efforts in support of TAC KBP 2013, with an emphasis on procedures and methodologies for query selection, annotation, and assessment.