Learning Noisy Linear Threshold Functions

This papers describes and analyzes algorithms for learning linear threshold function (LTFs) in the presence of classiication noise and monotonic noise. When there is classiication noise, each randomly drawn example is mislabeled (i.e., diiers from the target LTF) with the same probability. For monotonic noise, the probability of mis-labeling an example monotonically decreases with the separation between the target LTF hyperplane and the example. Monotonic noise is a generalization of classiication noise as well as the cases of independent binary features (aka naive Bayes) and normal distributions with equal covariance matrices. Monotonic noise provides a more realistic model of noise because it allows conndence to increase as a function of the distance from the threshold, but it does not impose any artiicial form on the function. This paper shows that LTFs are polynomially PAC-learnable in the presence of classiica-tion noise and monotonic noise if the separation between examples and the target LTF hyperplane is suuciently large, and if the vector length of each example is suuciently small.