Research in Question Answering (QA) has been dominated by the TREC methodology of black-box system evaluation. This makes it difficult to evaluate the effectiveness of individual components and requires human involvement. We have collected a set of answer locations within the AQUAINT corpus for a sample of TREC questions, in doing so we also analyse the ability of humans to retrieve answers. Ou...

Community-driven Question Answering (CQA) systems that crowdsource experiential information in the form of questions and answers and have accumulated valuable reusable knowledge. Clustering of QA datasets from CQA systems provides a means of organizing the content to ease tasks such as manual curation and tagging. In this paper, we present a clustering method that exploits the two-part question...

I am in the middle of Manhattan and I am lost. I approach a likely stranger on the road and ask him, pointing to the right: “Is this the way to the railway station?” Discounting for a moment the possibility that I will be ignored and that the stranger will continue on his way (of which the—trivial—game move equivalent would be “I prefer not to answer the question”), I will now get one of three ...

and analysis J. W. Murdock J. Fan A. Lally H. Shima B. K. Boguraev One useful source of evidence for evaluating a candidate answer to a question is a passage that contains the candidate answer and is relevant to the question. In the DeepQA pipeline, we retrieve passages using a novel technique that we call Supporting Evidence Retrieval, in which we perform separate search queries for each candi...

Fine, so what does a population geneticist mean by epistasis? RA Fisher used ‘epistacy’ and later ‘epistasis’ to describe genetic interactions more generally [2]. We think that population geneticists hijacked this term over a decade after its coinage just to confuse the classical geneticists. OK, what does a medical doctor mean by epistasis? A thin film on the surface of a urine specimen. Enoug...

We introduce Question-Answer Meaning Representations (QAMRs), which represent the predicate-argument structure of a sentence as a set of question-answer pairs. We also develop a crowdsourcing scheme to show that QAMRs can be labeled with very little training, and gather a dataset with over 5,000 sentences and 100,000 questions. A detailed qualitative analysis demonstrates that the crowd-generat...