A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer
نویسندگان
چکیده
In this paper, we present a simple analysis of fast rates with high probability of empirical minimization for stochastic composite optimization over a finite-dimensional bounded convex set with exponentially concave loss functions and an arbitrary convex regularization. To the best of our knowledge, this result is the first of its kind. As a byproduct, we can directly obtain the fast rate with high probability for exponentially concave empirical risk minimization with and without any convex regularization, which not only extends existing results of empirical risk minimization but also provides a unified framework for analyzing exponentially concave empirical risk minimization with and without any convex regularization. Our proof is very simple only exploiting the covering number of a finitedimensional bounded set and a concentration inequality of random vectors.
منابع مشابه
Sparse Learning for Large-Scale and High-Dimensional Data: A Randomized Convex-Concave Optimization Approach
In this paper, we develop a randomized algorithm and theory for learning a sparse model from large-scale and high-dimensional data, which is usually formulated as an empirical risk minimization problem with a sparsity-inducing regularizer. Under the assumption that there exists a (approximately) sparse solution with high classification accuracy, we argue that the dual solution is also sparse or...
متن کاملSupport vector machines based on convex risk functions and general norms
This paper studies unified formulations of support vector machines (SVMs) for binary classification on the basis of convex analysis, especially, convex risk functions theory, which is recently developed in the context of financial optimization. Using the notions of convex empirical risk and convex regularizer, a pair of primal and dual formulations of the SVMs are described in a general manner....
متن کاملABCD matrix for reflection and refraction of laser beam at tilted concave and convex elliptic paraboloid interfaces and studying laser beam reflection from a tilted concave parabola of revolution
Studying Gaussian beam is a method to investigate laser beam propagation and ABCD matrix is a fast and simple method to simulate Gaussian beam propagation in different mediums. Of the ABCD matrices studied so far, reflection and refraction matrices at various surfaces have attracted a lot of researches. However in previous work the incident beam and the principle axis of surface are in parallel...
متن کاملStochastic dual averaging methods using variance reduction techniques for regularized empirical risk minimization problems
We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic gradient methods that are based on stochastic dual averaging method with variance reduction. Our methods generate a sparser solution than the existing methods because we do not need to take the average of the history o...
متن کاملStochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization
We consider a generic convex optimization problem associated with regularized empirical risk minimization of linear predictors. The problem structure allows us to reformulate it as a convex-concave saddle point problem. We propose a stochastic primal-dual coordinate method, which alternates between maximizing over one (or more) randomly chosen dual variable and minimizing over the primal variab...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1709.02909 شماره
صفحات -
تاریخ انتشار 2017