2 Review of probability. 5 2.1 Properties of expectation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Convergence of random variables. . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Convergence in probability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.4 Norms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.5 Informati...

In this paper we study the convergence of online gradient descent algorithms in reproducing kernel Hilbert spaces (RKHSs) without regularization. We establish a sufficient condition and a necessary condition for the convergence of excess generalization errors in expectation. A sufficient condition for the almost sure convergence is also given. With high probability, we provide explicit converge...