Outliers and Bayesian Inference

In this paper we report about an investigation in which we studied the properties of Bayes’ inferred neural network classifiers in the context of outlier detection. The problem of misclassification due to outliers in the test data is seen as a serious problem in safety critical environments. We compare the usual way to deal with uncertainty in the Bayesian framework with a new approach based on the variance of the output layer activations and investigate the utility of both methods for outlier detection. The properties of both methods are visualized on a simple two dimensional classification problem. An investigation comparing both methods on some public data-sets with artificially constructed outlier patterns showed that a combination of the conventional method and the method proposed here should be used. These results where confirmed in a final experiment on real data, where a combination of both methods showed significantly better performance in rejecting outlying observations.