Automatic basis selection for RBF networks using Stein’s unbiased risk estimator
نویسندگان
چکیده
The problem of selecting the appropriate number of basis functions is a critical issue for radial basis function neural networks. An RBF network with an overly restricted basis gives poor predictions on new data, since the model has too little flexibility (yielding high bias and low variance). By contrast, an RBF network with too many basis functions also gives poor generalization performance since it is too flexible and fits too much of the noise on the training data (yielding low bias but high variance). Bias and variance are complementary quantities, and it is necessary to assign the number of basis function optimally in order to achieve the best compromise between them. In this paper we derive a theoretical criterion for assigning the appropriate number of basis functions. We use Stein’s unbiased risk estimator (SURE) to derive a generic criterion that defines the optimum number of basis functions to use for a given problem. The efficacy of this criterion is illustrated experimentally.
منابع مشابه
Automatic basis selection techniques for RBF networks
This paper proposes a generic criterion that defines the optimum number of basis functions for radial basis function (RBF) neural networks. The generalization performance of an RBF network relates to its prediction capability on independent test data. This performance gives a measure of the quality of the chosen model. An RBF network with an overly restricted basis gives poor predictions on new...
متن کاملMagnetic Resonance in Medicine 71:1760–1770 (2014) Monte Carlo SURE-Based Parameter Selection for Parallel Magnetic Resonance Imaging Reconstruction
Purpose: Regularizing parallel magnetic resonance imaging (MRI) reconstruction significantly improves image quality but requires tuning parameter selection. We propose a Monte Carlo method for automatic parameter selection based on Stein’s unbiased risk estimate that minimizes the multichannel k-space mean squared error (MSE). We automatically tune parameters for image reconstruction methods th...
متن کامل[hal-00776303, v1] Stein COnsistent Risk Estimator (SCORE) for hard thresholding
In this work, we construct a risk estimator for hard thresholding which can be used as a basis to solve the difficult task of automatically selecting the threshold. As hard thresholding is not even continuous, Stein’s lemma cannot be used to get an unbiased estimator of degrees of freedom, hence of the risk. We prove that under a mild condition, our estimator of the degrees of freedom, although...
متن کاملAdaptive RBF network control for robot manipulators
TThe uncertainty estimation and compensation are challenging problems for the robust control of robot manipulators which are complex systems. This paper presents a novel decentralized model-free robust controller for electrically driven robot manipulators. As a novelty, the proposed controller employs a simple Gaussian Radial-Basis-Function Network as an uncertainty estimator. The proposed netw...
متن کاملOn the “degrees of Freedom” of the Lasso By
We study the effective degrees of freedom of the lasso in the framework of Stein’s unbiased risk estimation (SURE). We show that the number of nonzero coefficients is an unbiased estimate for the degrees of freedom of the lasso—a conclusion that requires no special assumption on the predictors. In addition, the unbiased estimator is shown to be asymptotically consistent. With these results on h...
متن کامل