Learning Complex Concepts Using Crowdsourcing: A Bayesian Approach
نویسندگان
چکیده
We develop a Bayesian approach to concept learning for crowdsourcing applications. A probabilistic belief over possible concept definitions is maintained and updated according to (noisy) observations from experts, whose behaviors are modeled using discrete types. We propose recommendation techniques, inference methods, and query selection strategies to assist a user charged with choosing a configuration that satisfies some (partially known) concept. Our model is able to simultaneously learn the concept definition and the types of the experts. We evaluate our model with simulations, showing that our Bayesian strategies are effective even in large concept spaces with many uninformative experts.
منابع مشابه
Bayesian Bias Mitigation for Crowdsourcing
Biased labelers are a systemic problem in crowdsourcing, and a comprehensive toolbox for handling their responses is still being developed. A typical crowdsourcing application can be divided into three steps: data collection, data curation, and learning. At present these steps are often treated separately. We present Bayesian Bias Mitigation for Crowdsourcing (BBMC), a Bayesian model to unify a...
متن کاملA Bayesian Concept Learning Approach to Crowdsourcing
We develop a Bayesian approach to concept learning for crowdsourcing applications. A probabilistic belief over possible concept definitions is maintained and updated according to (noisy) observations from experts, whose behaviors are modeled using discrete types. We propose recommendation techniques, inference methods, and query selection strategies to assist a user charged with choosing a conf...
متن کاملToward Never-Ending Object Learning for Robots
Toward Never-Ending Object Learning for Robots Yuyin SunChair of the Supervisory Committee:Professor Dieter FoxComputer Science and Engineering A household robot usually works in a complex working environment, where it will continuously seenew objects and encounter new concepts in its lifetime. Therefore, being able to learn more objectsis crucial for the robot to be con...
متن کاملLeveraging Crowdsourcing Data For Deep Active Learning An Application: Learning Intents in Alexa
This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expertise contribute an uncontrolled amount (often limited) of annotations. Our fra...
متن کاملUsing a Bayesian Model to Combine LDA Features with Crowdsourced Responses
This paper describes a crowdsourcing system that integrates machine learning techniques with human classifiers, showing how to apply a Bayesian approach to classifier combination to the challenge of crowdsourcing document topic labels. First, we use a number of NLP techniques to extract informative document features. We then screen and select workers using Amazon Mechanical Turk to label select...
متن کامل