Eigen Vector Descent and Line Search for Multilayer Perceptron
نویسندگان
چکیده
As learning methods of a multilayer perceptron (MLP), we have the BP algorithm, Newton’s method, quasiNewton method, and so on. However, since the MLP search space is full of crevasse-like forms having a huge condition number, it is unlikely for such usual existing methods to perform efficient search in the space. This paper proposes a new search method which utilizes eigen vector descent and line search, aiming to stably find excellent solutions in such an extraordinary search space. The proposed method is evaluated with promising results through our experiments for MLPs having a sigmoidal or exponential activation function.
منابع مشابه
A Free Line Search Steepest Descent Method for Solving Unconstrained Optimization Problems
In this paper, we solve unconstrained optimization problem using a free line search steepest descent method. First, we propose a double parameter scaled quasi Newton formula for calculating an approximation of the Hessian matrix. The approximation obtained from this formula is a positive definite matrix that is satisfied in the standard secant relation. We also show that the largest eigen value...
متن کاملApplication of multilayer perceptron neural network and support vector machine for modeling the hydrodynamic behavior of permeable breakwaters with porous core
In this research, the application of multilayer perceptron (MLP) neural networks and support vector machine (SVM) for modeling the hydrodynamic behavior of Permeable Breakwaters with Porous Core has been investigated. For this purpose, experimental data have been used on the physical model to relate the reflection and transition coefficients of incident waves as the output parameters to the wid...
متن کاملModeling and analysis of leishmaniasis distribution process using multilayer perceptron neural network and support vector regression (Case study: villages of Isfahan province)
Villages located in Isfahan province are one of the areas prone to the spread of cutaneous leishmaniasis, which is characterized by the occurrence of wounds on the skin. To predict the future prevalence of cutaneous leishmaniasis, Continuous monitoring of the spatial distribution of this disease is essential. Disease modeling was performed using two machine learning algorithms called support ve...
متن کاملA Proposed Optimized and Efficient Intrusion Detection System for Wireless Sensor Network
In recent years intrusions on computer network are the major security threat. Hence, it is important to impede such intrusions. The hindrance of such intrusions entirely relies on its detection, which is primary concern of any security tool like Intrusion detection system (IDS). Therefore, it is imperative to accurately detect network attack. Numerous intrusion detection techniques are availabl...
متن کاملGradient Learning in Structured Parameter Spaces: Adaptive Blind Separation of Signal Sources
The present paper discusses the natural gradient descent learning rules in parameter spaces which have Riemannian geometrical structures. A modi cation is necessary for de ning the steepest descent (gradient) direction in a Riemannian parameter space. Parameter spaces of multilayer perceptrons are good examples of the Riemannian nature. Another example is the space of matrices on which adaptive...
متن کامل