Optimal transport (OT) plays an essential role in various areas like machine learning and deep learning. However, computing discrete optimal plan for large scale problems with adequate accuracy efficiency is still highly challenging. Recently, methods based on the Sinkhorn algorithm add entropy regularizer to prime problem get a trade off between accuracy. In this paper, we propose novel furthe...

We analyze and evaluate an online gradient descent algorithm with adaptive per-coordinate adjustment of learning rates. Our algorithm can be thought of as an online version of batch gradient descent with a diagonal preconditioner. This approach leads to regret bounds that are stronger than those of standard online gradient descent for general online convex optimization problems. Experimentally,...

Recently, we have demonstrated a new and efficient method to simultaneously reconstruct two unknown interfering wavefronts. A three-dimensional interference pattern was analyzed and then Zernike polynomials and the stochastic parallel gradient descent algorithm were used to expand and calculate wavefronts. In this paper, as one of the applications of this method, the reflected wavefronts from t...

The celebrated Nesterov’s accelerated gradient method offers great speed-ups compared to the classical gradient descend method as it attains the optimal first-order oracle complexity for smooth convex optimization. On the other hand, the popular AdaGrad algorithm competes with mirror descent under the best regularizer by adaptively scaling the gradient. Recently, it has been shown that the acce...

The backpropagation algorithm, which had been originally introduced in the 1970s, is the workhorse of learning in neural networks. This backpropagation algorithm makes use of the famous machine learning algorithm known as Gradient Descent, which is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, ...

For a high-power slab solid-state laser, obtaining high output power and beam quality are the most important indicators. Adaptive optics systems can significantly improve qualities by compensating for phase distortions of laser beams. In this paper, we developed an improved algorithm called Gradient Estimation Stochastic Parallel Descent (AGESPGD) cleanup laser. A second-order gradient search p...

In the last decades, helicopter-borne electromagnetic (HEM) method became a focus of interest in the fields of mineral exploration, geological mapping, groundwater resource investigation and environmental monitoring. As a standard approach, researchers use 1-D inversion of the acquired HEM data to recover the conductivity/resistivity-depth models. Since the relation between HEM data and model ...

In this paper, we propose to train the RBF neural network using a global descent method. Essentially, the method imposes a monotonic transformation on the training objective to improve numerical sensitivity without altering the relative orders of all local extrema. A gradient descent search which inherits the global descent property is derived to locate the global solution of an error objective...

Reactive (memoryless) policies are sufficient in completely observable Markov decision processes (MDPs), but some kind of memory is usually necessary for optimal control of a partially observable MDP. Policies with finite memory can be represented as finite-state automata. In this paper, we extend Baird and Moore’s VAPS algorithm to the problem of learning general finite-state automata. Because...