Which Shortcut Solution Do Question Answering Models Prefer to Learn?
نویسندگان
چکیده
Question answering (QA) models for reading comprehension tend to exploit spurious correlations in training sets and thus learn shortcut solutions rather than the intended by QA datasets. that have learned can achieve human-level performance examples where shortcuts are valid, but these same behaviors degrade generalization potential on anti-shortcut invalid. Various methods been proposed mitigate this problem, they do not fully take characteristics of themselves into account. We assume learnability shortcuts, i.e., how easy it is a shortcut, useful problem. Thus, we first examine representative extractive multiple-choice Behavioral tests using biased reveal answer positions word-label preferentially QA, respectively. find more learnable is, flatter deeper loss landscape around solution parameter space. also availability preferred tends make task easier perform from an information-theoretic viewpoint. Lastly, experimentally show be utilized construct effective set; smaller proportion required comparable examples. claim should considered when designing mitigation methods.
منابع مشابه
Which Learning Style Do You Prefer to Improve EFL Learning?
The term "learning styles" refers to the concept that individuals differ in regard to what mode of instruction or study is most effective for them. Proponents of learningstyle assessment contend that optimal instruction requires diagnosing individuals' learning style and tailoring instruction accordingly (pashler, McDaniel, Rohrer, and Bjork, 2009). There are several methods or theories that d...
متن کاملLeveraging Video Descriptions to Learn Video Question Answering
We propose a scalable approach to learn video-based question answering (QA): to answer a free-form natural language question about the contents of a video. Our approach automatically harvests a large number of videos and descriptions freely available online. Then, a large number of candidate QA pairs are automatically generated from descriptions rather than manually annotated. Next, we use thes...
متن کاملInvestigating Embedded Question Reuse in Question Answering
The investigation presented in this paper is a novel method in question answering (QA) that enables a QA system to gain performance through reuse of information in the answer to one question to answer another related question. Our analysis shows that a pair of question in a general open domain QA can have embedding relation through their mentions of noun phrase expressions. We present methods f...
متن کاملWhich Soft Constraints do you Prefer?
Soft constraints are gaining popularity in diverse areas such as orchestration of Web services or optimization of scheduling decisions. However, current approaches to soft constraints preclude them from modelling certain decision problems with multiple preference criteria. We propose a new approach to soft constraints which allows a natural expression of these problems, describe an implementati...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2023
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v37i11.26590