Recently, Kundur and Hatzinakos showed that a linear restoration filter designed by using the almost obvious a priori knowledge on the original image, such as (i) nonnegativity of the true image and (ii) the smallest rectangle encompassing the original object, can realize a remarkable performance for a blind image deconvolution problem. In this paper, we propose a new set-theoretic blind image ...

The least mean squares (LMS) method for linear least squares problems differs from the steepest descent method in that it processes data blocks one-by-one, with intermediate adjustment of the parameter vector under optimization. This mode of operation often leads to faster convergence when far from the eventual limit and to slower (sublinear) convergence when close to the optimal solution. We e...

The hybrid steepest-descent method introduced by Yamada 2001 is an algorithmic solution to the variational inequality problem over the fixed point set of nonlinear mapping and applicable to a broad range of convexly constrained nonlinear inverse problems in real Hilbert spaces. Lehdili and Moudafi 1996 introduced the new prox-Tikhonov regularization method for proximal point algorithm to genera...

The paper applies abstract optimisation principles in the space of measures within the context of optimal design problems. It is shown that within this framework it is possible to treat various design criteria and constraints in a unified manner providing a “universal” variant of the Kiefer-Wolfowitz theorem and giving a full spectrum of optimality criteria for particular cases. The described s...

In this paper, we introduce and analyze a hybrid steepest-descent extragradient algorithm for solving triple hierarchical pseudomonotone variational inequalities in a real Hilbert space. The proposed algorithm is based on Korpelevich’s extragradient method, Mann’s iteration method, hybrid steepest-descent method and Halpern’s iteration method. Under mild conditions, the strong convergence of th...

This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. Some global convergence conditions of the steepest descent algorithm are obtained by directly analyzing the exact momentum equations for quadratic cost functions. Those conditions can be directly derived from the parameters (different from eigenvalues that are used in the existed ones.) ...

We propose a variant of the numerical method of steepest descent for oscillatory integrals by using a low-cost explicit polynomial approximation of the paths of steepest descent. A loss of asymptotic order is observed, but in the most relevant cases the overall asymptotic order remains higher than a truncated asymptotic expansion at similar computational effort. Theoretical results based on num...

In maximizing a non-linear function G(0), it is well known that the steepest descent method has a slow convergence rate. Here we propose a systematic procedure to obtain a 1-1 transformation on the variables 0, so that in the space of the transformed variables, the steepest descent method produces the solution faster. The final solution in the original space is obtained by taking the inverse tr...

GANs excel at learning high dimensional distributions, but they can update generator parameters in directions that do not correspond to the steepest descent direction of the objective. Prominent examples of problematic update directions include those used in both Goodfellow’s original GAN and the WGAN-GP. To formally describe an optimal update direction, we introduce a theoretical framework whi...