Solving Ridge Regression using Sketched Preconditioned SVRG
نویسندگان
چکیده
We develop a novel preconditioning method for ridge regression, based on recent linear sketching methods. By equipping Stochastic Variance Reduced Gradient (SVRG) with this preconditioning process, we obtain a significant speed-up relative to fast stochastic methods such as SVRG, SDCA and SAG.
منابع مشابه
Sketched Ridge Regression: Optimization Perspective, Statistical Perspective, and Model Averaging
We address the statistical and optimization impacts of using classical sketch versus Hessian sketch to solve approximately the Matrix Ridge Regression (MRR) problem. Prior research has considered the effects of classical sketch on least squares regression (LSR), a strictly simpler problem. We establish that classical sketch has a similar effect upon the optimization properties of MRR as it does...
متن کاملSolving large systems arising from fractional models by preconditioned methods
This study develops and analyzes preconditioned Krylov subspace methods to solve linear systems arising from discretization of the time-independent space-fractional models. First, we apply shifted Grunwald formulas to obtain a stable finite difference approximation to fractional advection-diffusion equations. Then, we employee two preconditioned iterative methods, namely, the preconditioned gen...
متن کاملTwo-Parameters Fuzzy Ridge Regression with Crisp Input and Fuzzy Output
In this paper a new weighted fuzzy ridge regression method for a given set of crisp input and triangular fuzzy output values is proposed. In this regard, ridge estimator of fuzzy parameters is obtained for regression model and its prediction error is calculated by using the weighted fuzzy norm of crisp ridge coefficients. . To evaluate the proposed regression model, we introduce the fu...
متن کاملPreconditioned Generalized Minimal Residual Method for Solving Fractional Advection-Diffusion Equation
Introduction Fractional differential equations (FDEs) have attracted much attention and have been widely used in the fields of finance, physics, image processing, and biology, etc. It is not always possible to find an analytical solution for such equations. The approximate solution or numerical scheme may be a good approach, particularly, the schemes in numerical linear algebra for solving ...
متن کاملVariance Reduced Stochastic Gradient Descent with Sufficient Decrease
In this paper, we propose a novel sufficient decrease technique for variance reduced stochastic gradient descent methods such as SAG, SVRG and SAGA. In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of variance reduction algorithms such as SVRG-SD and SAGA-SD as a byproduct. We introduce a c...
متن کامل