Almost sure convergence of stochastic gradient processes with matrix step sizes
نویسندگان
چکیده
منابع مشابه
Almost sure convergence of stochastic gradient processes with matrix step sizes
We consider a stochastic gradient process, which is a special case of stochastic approximation process, where the positive real step size an is replaced by a random matrix An: Xn+1 = Xn Anrg(Xn) AnVn: We give two theorems of almost sure convergence in the case where the equation rg = 0 has a set of solutions. Key Words and Phrases: Stochastic approximation, stochastic gradient AMS 1991 Subject ...
متن کاملAlmost Sure Convergence to Zero in Stochastic Growth Models
This paper shows that in stochastic one-sector growth models, if the production function does not satisfy the Inada condition at zero, any feasible path converges to zero with probability one provided that the shocks are sufficiently volatile. This result seems significant since, as we argue, the Inada condition at zero is difficult to justify on economic grounds. Our convergence result is exte...
متن کاملAlmost sure convergence of the Bartlett estimator
We study the almost sure convergence of the Bartlett estimator for the asymptotic variance of the sample mean of a stationary weekly dependent process. We also study the a. s. behavior of this estimator in the case of long-range dependent observations. In the weakly dependent case, we establish conditions under which the estimator is strongly consistent. We also show that, after appropriate nor...
متن کاملConvergence diagnostics for stochastic gradient descent with constant step size
Iterative procedures in stochastic optimization are typically comprised of a transient phase and a stationary phase. During the transient phase the procedure converges towards a region of interest, and during the stationary phase the procedure oscillates in a convergence region, commonly around a single point. In this paper, we develop a statistical diagnostic test to detect such phase transiti...
متن کاملJakimovski Methods and Almost-Sure Convergence
The classical summability methods of Borel (B) and Euler (E(λ), λ > 0) play an important role in many areas of mathematics. For instance, in summability theory they are perhaps the most important methods other than the Cesàro (Cα) and Abel (A) methods, and two chapters of the classic book of Hardy (1949) are devoted to them. In probability, the distinction between methods of Cesàro-Abel and Eul...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Statistics & Probability Letters
سال: 2006
ISSN: 0167-7152
DOI: 10.1016/j.spl.2005.09.014