Simultaneous input variable and basis function selection for RBF networks

Input selection is advantageous in regression problems. For example, it might decrease the training time of models, reduce measurement costs, and circumvent problems of high dimensionality. Inclusion of useless inputs into the model increases also the likelihood of overfitting. Neural networks provide good generalization in many cases, but their interpretability is usually limited. However, selecting a subset of variables and estimating their relative importances would be valuable in many real world applications. In the present work, a simultaneous input and basis function selection method for the radial basis function (RBF) network is proposed. The selection is performed by minimizing a constrained cost function, in which sparsity of the network is controlled by two continuous valued shrinkage parameters. Each input dimension is weighted and the constraints are imposed on these weights and the output layer coefficients. Direct and alternating optimization procedures are presented to solve the problem. The proposed method is applied to simulated and benchmark data. In the comparison with existing methods, the resulting RBF networks have similar prediction accuracies with the smaller numbers of inputs and basis functions.