This report presents P scg , a new global optimization method for training mul-tilayered perceptrons. Instead of local minima, global minima of the error function are found. This new method is hybrid in the sense that it combines three very diierent optimization techniques: Random Line Search, Scaled Conjugate Gradient and a 1-dimensional minimization algorithm named P. The best points of each ...

Bayesian optimization depends on solving a global optimization of a acquisition function. However, the acquisition function can be extremely sharp at high dimension having only a few peaks marooned in a large terrain of almost flat surface. Global optimization algorithms such as DIRECT are infeasible at higher dimensions and gradient-dependent methods cannot move if initialized in the flat terr...

A practical algorithm for box-constrained optimization is introduced. The algorithm combines an active-set strategy with spectral projected gradient iterations. In the interior of each face a strategy that deals efficiently with negative curvature is employed. Global convergence results are given. Numerical results are presented.

We demonstrate that a highly efficient global optimization of chirped mirrors can be performed with the memetic algorithm. The inherently high sensitivity of chirped-mirror characteristics to manufacturing errors can be reduced significantly by means of the stochastic quasi-gradient algorithm. The applicability of these algorithms is not limited to chirped mirrors.

this paper proposes a novel hybrid algorithm namely apso-bfo which combines merits of bacterial foraging optimization (bfo) algorithm and adaptive particle swarm optimization (apso) algorithm to determine the optimal pid parameters for control of nonlinear systems. to balance between exploration and exploitation, the proposed hybrid algorithm accomplishes global search over the whole search spa...

We analyze a fast incremental aggregated gradient method for optimizing nonconvex problems of the form minx ∑ i fi(x). Specifically, we analyze the Saga algorithm within an Incremental First-order Oracle framework, and show that it converges to a stationary point provably faster than both gradient descent and stochastic gradient descent. We also discuss a Polyak’s special class of nonconvex pro...

The efficient search of global optimal solutions is an important contemporary subject. Different optimization methods tackle the search in different ways. The gradient based methods are among the fastest optimization methods but the final optimal solution depends on the starting point. The global search using these methods is carried out by providing many starting points. Other optimization met...

In previous work, we have presented a novel global feasibility solver for the large system of quadratic constraints that arise as subproblems in the solving of hard hybrid problems, such as the scheduling of refineries. In this paper we present the Gradient Optimal Constraint Equation Subdivision (GOCES) algorithm, which incorporates a standard NLP solver and the global feasibility solver to fi...

Artificial Neural Networks have been shown to have the potential to perform well for classification problems in many different environments, including business, science and engineering. Studies in the literature often report that the artificial neural network dominates traditional statistical techniques for most problems examined. Since a neural network can embed most traditional techniques as ...

A route planning method based on gradient-field quantum genetic algorithm model was presented in this paper. It introduces the gradient field of a grid map to quantum genetic algorithm model and uses quantum genetic algorithm (QGA) to optimize the cost function of route planning. By combining the quantum characteristics with the capabilities of the large diversity of the population, as well as ...