Weighted Least Squares Scheme for Reducing Effects of Outliers in Regression based on Extreme Learning Machine

Neural networks have been massively used in regression problems due to their ability to approximate complex nonlinear mappings directly from input patterns. However, collected data for training networks often include outliers which affect final results. This paper presents an approach for training single hidden-layer feedforward neural networks (SLFNs) using weighted least-squares scheme which reduces the effects of outliers. The proposed training method is based on an efficient training algorithm called extreme learning machine (ELM), in which input network weights including hidden layer biases are randomly assigned and output network weights are analytically determined by Moore-Penrose generalized inverse of hidden-layer output matrix. However, instead of being weighted equally, penalties corresponding to training patterns are weighted so that patterns with larger penalty weights contribute more to the detection of output network weights. Experiment results show that this approach can obtain good performance in regression with training sets existing outliers.