Annotated images are required for both supervised model training and evaluation in image classification. Manually annotating is arduous expensive, especially multi-labeled images. A recent trend conducting such laboursome annotation tasks through crowdsourcing, where annotated by volunteers or paid workers online (e.g., of Amazon Mechanical Turk) from scratch. However, the quality crowdsourcing...
