Finite size scaling of the Bayesian perceptron
نویسندگان
چکیده
We study numerically the properties of the Bayesian perceptron through a gradient descent on the optimal cost function. The theoretical distribution of stabilities is deduced. It predicts that the optimal generalizer lies close to the boundary of the space of ~error-free! solutions. The numerical simulations are in good agreement with the theoretical distribution. The extrapolation of the generalization error to infinite input space size agrees with the theoretical results. Finite size corrections are negative and exhibit two different scaling regimes, depending on the training set size. The variance of the generalization error vanishes for N→` confirming the property of self-averaging. @S1063-651X~97!15406-X#
منابع مشابه
Finite Size Scaling of Perceptron
We study the first-order phase transition in the model of a simple perceptron with continuous weights and large, but finite value of the inputs. Making the analogy with the usual finite-size physical systems, we calculate the shift and the rounding exponents near the transition point. In the case of a general perceptron with larger variety of outputs, the analysis gives only bounds for the expo...
متن کاملFinite Size Scaling in Neural Networks
We demonstrate that the fraction of pattern sets that can be stored in singleand hidden-layer perceptrons exhibits finite size scaling. This feature allows one to estimate the critical storage capacity ac from simulations of relatively small systems. We illustrate this approach by determining ac , together with the finite size scaling exponent n, for storing Gaussian patterns in committee and p...
متن کاملFinite-size effects and optimal test set size in linear perceptrons
Huctuatiolls in the test error are important in thelaming theory of finite-dimensional systems as they represent how well the test error matches the average test emr. By explicitly finding the variance of the test ermr due to randomness present in both the data set and dgonthm for a linear perceptron of dimension n. we are able to address such questions as the optimal test set size. Where exact...
متن کاملEfficient Bayesian inference for learning in the Ising linear perceptron and signal detection in CDMA
Efficient new Bayesian inference technique is employed for studying critical properties of the Ising linear perceptron and for signal detection in code division multiple access (CDMA). The approach is based on a recently introduced message passing technique for densely connected systems. Here we study both critical and non-critical regimes. Results obtained in the non-critical regime give rise ...
متن کاملAccuracy of Genomic Prediction under Different Genetic Architectures and Estimation Methods
The accuracy of genomic breeding value prediction was investigated in various levels of reference population size, trait heritability and the number of quantitative trait locus (QTL). Five Bayesian methods, including Bayesian Ridge regression, BayesA, BayesB, BayesC and Bayesian LASSO, were used to estimate the marker effects for each of 27 scenarios resulted from combining three levels for her...
متن کامل