On the Calibration and Uncertainty with Pólya-Gamma Augmentation for Dialog Retrieval Models

Deep neural retrieval models have amply demonstrated their power but estimating the reliability of predictions remains challenging. Most dialog response output a single score for on how relevant it is to given question. However, bad calibration deep network results in various uncertainty such that unreliable always misinform user decisions. To investigate these issues, we present an efficient and estimation framework PG-DRR which adds Gaussian Process layer deterministic recovers conjugacy tractable posterior inference by Pólya-Gamma augmentation. Finally, achieves lowest empirical error (ECE) in-domain datasets distributional shift task while keeping R10@1 MAP performance.