Abstract Majority voting and averaging are common approaches used to resolve annotator disagreements derive single ground truth labels from multiple annotations. However, annotators may systematically disagree with one another, often reflecting their individual biases values, especially in the case of subjective tasks such as detecting affect, aggression, hate speech. Annotator capture importan...