Supplementary Material – Ambiguity Helps: Classification with Disagreements in Crowdsourced Annotations

In this section we give all the necessary details to implement the EP algorithm [1] for the GPCconf method described in the main manuscript. We show how to compute the EP posterior approximation from the product of all approximate factors and how to implement the EP updates to refine each approximate factor. We also show how to compute the EP approximation of the marginal likelihood and its gradients. Recall from the main manuscript that in EP the approximate factors replace the corresponding exact factors of the likelihood in the joint distribution p(y|X,Xconf, f ,g)p(f)p(g). The resulting approximate joint distribution is then normalized to get the EP posterior approximation, and the normalization constant is the approximation to the marginal likelihood. Furthermore, we would like to recall that in the case of GPCconf, the n-th likelihood factor to be approximated by EP is: