On optimal probabilities in stochastic coordinate descent methods
نویسندگان
چکیده
منابع مشابه
On Optimal Probabilities in Stochastic Coordinate Descent Methods
We propose and analyze a new parallel coordinate descent method—‘NSync— in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities to the sets to optimize the bound. The complexity and practical performance of th...
متن کاملSemi-Stochastic Coordinate Descent
We propose a novel stochastic gradient method—semi-stochastic coordinate descent (S2CD)— for the problem of minimizing a strongly convex function represented as the average of a large number of smooth convex functions: f(x) = 1 n ∑ i fi(x). Our method first performs a deterministic step (computation of the gradient of f at the starting point), followed by a large number of stochastic steps. The...
متن کاملStochastic Coordinate Descent Methods for Regularized Smooth and Nonsmooth Losses
Stochastic Coordinate Descent (SCD) methods are among the first optimization schemes suggested for efficiently solving large scale problems. However, until now, there exists a gap between the convergence rate analysis and practical SCD algorithms for general smooth losses and there is no primal SCD algorithm for nonsmooth losses. In this paper, we discuss these issues using the recently develop...
متن کاملS2CD: Semi-Stochastic Coordinate Descent
We propose a novel reduced variance method—semi-stochastic coordinate descent (S2CD)—for the problem of minimizing a strongly convex function represented as the average of a large number of smooth convex functions: f(x) = 1 n ∑ i fi(x). Our method first performs a deterministic step (computation of the gradient of f at the starting point), followed by a large number of stochastic steps. The pro...
متن کاملStochastic Coordinate Descent for Nonsmooth Convex Optimization
Stochastic coordinate descent, due to its practicality and efficiency, is increasingly popular in machine learning and signal processing communities as it has proven successful in several large-scale optimization problems , such as l1 regularized regression, Support Vector Machine, to name a few. In this paper, we consider a composite problem where the nonsmoothness has a general structure that...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Optimization Letters
سال: 2015
ISSN: 1862-4472,1862-4480
DOI: 10.1007/s11590-015-0916-1