Reliability yields Information Gain

In this paper we prove that the reliability of the classifications of individual instances, provided by a classifier, results in information gain with respect to the accuracy of the classifier. We illustrate this result using our new approach to classification reliability called version space support vector machines.