In maximizing a non-linear function G(0), it is well known that the steepest descent method has a slow convergence rate. Here we propose a systematic procedure to obtain a 1-1 transformation on the variables 0, so that in the space of the transformed variables, the steepest descent method produces the solution faster. The final solution in the original space is obtained by taking the inverse tr...

| In classic backpropagation nets, as introduced by Rumelhart et al. 1], the weights are modiied according to the method of steepest descent. The goal of this weight modiication is to minimise the error in net-outputs for a given training set. Basing upon Jacobs' work 2], we point out drawbacks of steepest descent and suggest improvements on it. These yield a backpropagation net, which adjusts ...

The paper applies abstract optimisation principles in the space of measures within the context of optimal design problems. It is shown that within this framework it is possible to treat various design criteria and constraints in a unified manner providing a “universal” variant of the Kiefer-Wolfowitz theorem and giving a full spectrum of optimality criteria for particular cases. The described s...

This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. Some global convergence conditions of the steepest descent algorithm are obtained by directly analyzing the exact momentum equations for quadratic cost functions. Those conditions can be directly derived from the parameters (different from eigenvalues that are used in the existed ones.) ...

GANs excel at learning high dimensional distributions, but they can update generator parameters in directions that do not correspond to the steepest descent direction of the objective. Prominent examples of problematic update directions include those used in both Goodfellow’s original GAN and the WGAN-GP. To formally describe an optimal update direction, we introduce a theoretical framework whi...

Surface networks capture the topological relations between passes of a continuous surface, the paths of steepest descent and ascent starting at the passes, and the pits and peaks where the steepest paths end. This paper extends the topology of the network in three ways. Objects at the edge of the surface model are introduced. Horizontal areas may represent passes, pits, or peaks, and therefore ...

OBJECTIVE We aimed to evaluate the use of dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) at 3.0 T for differentiating the benign from malignant soft tissue tumors. Also we aimed to assess whether the shorter length of DCE-MRI protocols are adequate, and to evaluate the effect of temporal resolution. MATERIALS AND METHODS Dynamic contrast-enhanced magnetic resonance imaging, at...

We propose a limited memory steepest descent method for solving unconstrained optimization problems. As a steepest descent method, the step computation in each iteration only requires the evaluation of a gradient of the objective function and the calculation of a scalar stepsize. When employed to solve certain convex problems, our method reduces to a variant of the limited memory steepest desce...