Descent direction method with line search for unconstrained optimization in noisy environment

A two-phase descent direction method for unconstrained stochastic optimization problem is proposed. A line search method with an arbitrary descent direction is used to determine the step sizes during the initial phase, and the second phase performs the stochastic approximation (SA) step sizes. The almost sure convergence of the proposed method is established, under standard assumption for descent direction and SA methods. The algorithm used for practical implementation combines a line search quasi-Newton method, in particular the BFGS and SR1 methods, with the SA iterations. Numerical results show good performance of the proposed method for different noise levels.