Fixed Point Analysis for Recurrent Networks
نویسندگان
چکیده
Dana H. Ballard This paper provides a systematic analysis of the recurrent backpropagation (RBP) algorithm, introducing a number of new results. The main limitation of the RBP algorithm is that it assumes the convergence of the network to a stable fixed point in order to backpropagate the error signals. We show by experiment and eigenvalue analysis that this condition can be violated and that chaotic behavior can be avoided. Next we examine the advantages of RBP over the standard backpropagation algorithm. RBP is shown to build stable fixed points corresponding to the input patterns. This makes it an appropriate tool for content addressable memories, one-to-many function learning, and inverse problems.
منابع مشابه
Dynamic Frailty and Change Point Models for Recurrent Events Data
Abstract. We present a Bayesian analysis for recurrent events data using a nonhomogeneous mixed Poisson point process with a dynamic subject-specific frailty function and a dynamic baseline intensity func- tion. The dynamic subject-specific frailty employs a dynamic piecewise constant function with a known pre-specified grid and the baseline in- tensity uses an unknown grid for the piecewise ...
متن کاملIdentifying Fixed Points in Recurrent Neural Networks using Directional Fibers: Supplemental Material on Theoretical Results and Practical Aspects of Numerical Traversal
Fixed points of recurrent neural networks can represent many things, including stored memories, solutions to optimization problems, and waypoints along non-fixed attractors. As such, they are relevant to a number of neurocomputational phenomena, ranging from low-level motor control and tool use to high-level problem solving and decision making. Therefore, global solution of the fixed point equa...
متن کاملNetwork Investigation and Performance Analysis of ZigBee Technology using OPNET
Abstract- Communication has become inevitably part of our day to day activities, in academic, business, banking, and other sectors. It has therefore become so important to implement good and efficient communication system. A reference point according to this research is the wireless sensor networking (WSN) system, and most important thing in communication is to be free from interference, attenu...
متن کاملAttractive Periodic Sets in Discrete-Time Recurrent Networks (with Emphasis on Fixed-Point Stability and Bifurcations in Two-Neuron Networks)
We perform a detailed fixed-point analysis of two-unit recurrent neural networks with sigmoid-shaped transfer functions. Using geometrical arguments in the space of transfer function derivatives, we partition the network state-space into distinct regions corresponding to stability types of the fixed points. Unlike in the previous studies, we do not assume any special form of connectivity patter...
متن کاملA Markovian Study of Recurrent Neural Networks with Stochastic Dynamics
Recurrent neural networks of binary stochastic units with a general distribution function are studied using Markov chains theory. Sufficient conditions for ergodicity are established and under some assumptions, the stationary distribution is determined. The relation between fixed points and absorbing states is studied both theoretically and through simulations. For numerical studies the notion ...
متن کاملFeedforward Approximations to Dynamic Recurrent Network Architectures
Recurrent neural network architectures can have useful computational properties, with complex temporal dynamics and input-sensitive attractor states. However, evaluation of recurrent dynamic architectures requires solving systems of differential equations, and the number of evaluations required to determine their response to a given input can vary with the input or can be indeterminate altogeth...
متن کامل