In this note we calculate the gradient of the network function in matrix notation.

We describe recurrent neural networks (RNNs), which have attracted great attention on sequential tasks, such as handwriting recognition, speech recognition and image to text. However, compared to general feedforward neural networks, RNNs have feedback loops, which makes it a little hard to understand the backpropagation step. Thus, we focus on basics, especially the error backpropagation to com...

In this paper a topology of neural network intrusion detection system is proposed on which different backpropagation algorithms are benchmarked. The proposed methodology uses sampled data from KddCup99 data set, an intrusion detection attacks database that is a standard for the evaluation of intrusion detection systems. The performance of backpropagation algorithms implemented in batch mode, is...

Most of the real life classification problems have ill defined, imprecise or fuzzy class boundaries. Feedforward neural networks with conventional backpropagation learning algorithm are not tailored to this kind of classification problem. Hence, in this paper, feedforward neural networks, that use backpropagation learning algorithm with fuzzy objective functions, are investigated. A learning al...

In this paper, a variant of Backpropagation algorithm is proposed for feed-forward neural networks learning. The proposed algorithm improve the backpropagation training in terms of quick convergence of the solution depending on the slope of the error graph and increase the speed of convergence of the system. Simulations are conducted to compare and evaluate the convergence behavior and the spee...

Error backpropagation in feedforward neural network models is a popular learning algorithm that has its roots in nonlinear estimation and optimization. It is being used routinely to calculate error gradients in nonlinear systems with hundreds of thousands of parameters. However, the classical architecture for backpropagation has severe restrictions. The extension of backpropagation to networks ...

The backpropagation algorithm is widely used for training multilayer neural networks. In this publication the gain of its activation function(s) is investigated. In speciic, it is proven that changing the gain of the activation function is equivalent to changing the learning rate and the weights. This simpliies the backpropagation learning rule by eliminating one of its parameters. The theorem ...