Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: efficient policy gradient method with stochastic mirror descent. In VRMPO, novel variance-reduced estimator is presented to improve efficiency. We prove that the proposed needs only O(ε−3) trajectories achieve an ε-approximate first-order stationary point, which matches best ...

We show that asymptotically, completely asynchronous stochastic gradient procedures achieve optimal (even to constant factors) convergence rates for the solution of convex optimization problems under nearly the same conditions required for asymptotic optimality of standard stochastic gradient procedures. Roughly, the noise inherent to the stochastic approximation scheme dominates any noise from...

This paper presents a general class of dynamic stochastic optimization problems we refer to as Stochastic Depletion Problems. A number of challenging dynamic optimization problems of practical interest are stochastic depletion problems. Optimal solutions for such problems are difficult to obtain, both from a pragmatic computational perspective as also from a theoretical perspective. As such, si...

Adaptive optics calibration of a novel wide-field scanning microscope is described, comparing relevant parameters for several optimization techniques. Specifically, comparisons of the optimization algorithm, image quality metrics, and the calibration image target are detailed. It is shown that stochastic parallel gradient descent (SPGD) algorithm using image intensity as a metric provides robus...