Convergence of Online Gradient Method for Pi-sigma Neural Networks with Inner-penalty Terms
نویسندگان
چکیده
منابع مشابه
Convergence of Online Gradient Method for Pi-sigma Neural Networks with Inner-penalty Terms
This paper investigates an online gradient method with innerpenalty for a novel feed forward network it is called pi-sigma network. This network utilizes product cells as the output units to indirectly incorporate the capabilities of higherorder networks while using a fewer number of weights and processing units. Penalty term methods have been widely used to improve the generalization performan...
متن کاملConvergence of Online Gradient Method with a Penalty Term for Feedforward Neural Networks with Stochastic
Abstract Online gradient algorithm has been widely used as a learning algorithm for feedforward neural network training. In this paper, we prove a weak convergence theorem of an online gradient algorithm with a penalty term, assuming that the training examples are input in a stochastic way. The monotonicity of the error function in the iteration and the boundedness of the weight are both guaran...
متن کاملConvergence of online gradient method for feedforward neural networks with smoothing L1/2 regularization penalty
Minimization of the training regularization term has been recognized as an important objective for sparse modeling and generalization in feedforward neural networks. Most of the studies so far have been focused on the popular L2 regularization penalty. In this paper, we consider the convergence of online gradient method with smoothing L1/2 regularization term. For normal L1/2 regularization, th...
متن کاملConvergence of an Online Gradient Algorithm with Penalty for Two-layer Neural Networks
Online gradient algorithm has been widely used as a learning algorithm for feedforward neural networks training. Penalty is a common and popular method for improving the generalization performance of networks. In this paper, a convergence theorem is proved for the online gradient learning algorithm with penalty, a term proportional to the magnitude of the weights. The monotonicity of the error ...
متن کاملTraining Pi-Sigma Network by Online Gradient Algorithm with Penalty for Small Weight Update
A pi-sigma network is a class of feedforward neural networks with product units in the output layer. An online gradient algorithm is the simplest and most often used training method for feedforward neural networks. But there arises a problem when the online gradient algorithm is used for pi-sigma networks in that the update increment of the weights may become very small, especially early in tra...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: American Journal of Neural Networks and Applications
سال: 2016
ISSN: 2469-7400
DOI: 10.11648/j.ajnna.20160201.11