Greedy Learning of Generalized Low-Rank Models
نویسندگان
چکیده
Learning of low-rank matrices is fundamental to many machine learning applications. A state-ofthe-art algorithm is the rank-one matrix pursuit (R1MP). However, it can only be used in matrix completion problems with the square loss. In this paper, we develop a more flexible greedy algorithm for generalized low-rank models whose optimization objective can be smooth or nonsmooth, general convex or strongly convex. The proposed algorithm has low per-iteration time complexity and fast convergence rate. Experimental results show that it is much faster than the state-of-the-art, with comparable or even better prediction performance.
منابع مشابه
Learning of Generalized Low-Rank Models: A Greedy Approach
Learning of low-rank matrices is fundamental to many machine learning applications. A state-of-the-art algorithm is the rank-one matrix pursuit (R1MP). However, it can only be used in matrix completion problems with the square loss. In this paper, we develop a more flexible greedy algorithm for generalized low-rank models whose optimization objective can be smooth or nonsmooth, general convex o...
متن کاملFast Multivariate Spatio-temporal Analysis via Low Rank Tensor Learning
Accurate and efficient analysis of multivariate spatio-temporal data is critical in climatology, geology, and sociology applications. Existing models usually assume simple inter-dependence among variables, space, and time, and are computationally expensive. We propose a unified low rank tensor learning framework for multivariate spatio-temporal analysis, which can conveniently incorporate diffe...
متن کاملA Tensor-Based Algorithm for the Optimal Model Reduction of High Dimensional Problems
We propose a method for the approximation of the solution of high-dimensional problems formulated in tensor spaces using low-rank approximation formats. The method can be seen as a perturbation of an ideal minimal residual method with a residual norm corresponding to the error in a solution norm of interest. We introduce and analyze an algorithm for the approximation of the best approximation i...
متن کاملNonparametric Greedy Algorithms for the Sparse Learning Problem
This paper studies the forward greedy strategy in sparse nonparametric regression. For additive models, we propose an algorithm called additive forward regression; for general multivariate models, we propose an algorithm called generalized forward regression. Both algorithms simultaneously conduct estimation and variable selection in nonparametric settings for the high dimensional sparse learni...
متن کاملGreedy low-rank methods for solving general linear matrix equations‡
This work is concerned with the numerical solution of large-scale linear matrix equations A1XB T 1 + · · ·+ AKXB K = C. The most straightforward approach computes X ∈ Rm×n from the solution of an mn×mn linear system, typically limiting the feasible values of m,n to a few hundreds at most. Our new approach exploits the fact that X can often be well approximated by a low-rank matrix. It combines ...
متن کامل