Online Alternating Direction Method
نویسندگان
چکیده
Online optimization has emerged as powerful tool in large scale optimization. In this paper, we introduce efficient online optimization algorithms based on the alternating direction method (ADM), which can solve online convex optimization under linear constraints where the objective could be non-smooth. We introduce new proof techniques for ADM in the batch setting, which yields a O(1/T ) convergence rate for ADM and forms the basis for regret analysis in the online setting. We consider two scenarios in the online setting, based on whether an additional Bregman divergence is needed or not. In both settings, we establish regret bounds for both the objective function as well as constraints violation for general and strongly convex functions. We also consider inexact ADM updates where certain terms are linearized to yield efficient updates and show the stochastic convergence rates. In addition, we briefly discuss that online ADM can be used as projection-free online learning algorithm in some scenarios. Preliminary results are presented to illustrate the performance of the proposed algorithms.
منابع مشابه
An inexact alternating direction method with SQP regularization for the structured variational inequalities
In this paper, we propose an inexact alternating direction method with square quadratic proximal (SQP) regularization for the structured variational inequalities. The predictor is obtained via solving SQP system approximately under significantly relaxed accuracy criterion and the new iterate is computed directly by an explicit formula derived from the original SQP method. Under appropriat...
متن کاملAn Alternating Direction Implicit Method for Modeling of Fluid Flow
This research includes of the numerical modeling of fluids in two-dimensional cavity. The cavity flow is an important theoretical problem. In this research, modeling was carried out based on an alternating direction implicit via Vorticity-Stream function formulation. It evaluates different Reynolds numbers and grid sizes. Therefore, for the flow field analysis and prove of the ability of the sc...
متن کاملNew high accuracy super stable alternating direction implicit methods for two and three dimensional hyperbolic damped wave equations
Article history: Received 9 July 2014 Accepted 19 August 2014 Available online xxxx
متن کاملModified Convex Data Clustering Algorithm Based on Alternating Direction Method of Multipliers
Knowing the fact that the main weakness of the most standard methods including k-means and hierarchical data clustering is their sensitivity to initialization and trapping to local minima, this paper proposes a modification of convex data clustering in which there is no need to be peculiar about how to select initial values. Due to properly converting the task of optimization to an equivalent...
متن کاملSolving Two-Dimensional Fuzzy Partial Dierential Equation by the Alternating Direction Implicit Method
متن کامل
A Heterogeneous Alternating-direction Method for a Micro-macro Dilute Polymeric Fluid Model
We examine a heterogeneous alternating-direction method for the approximate solution of the FENE Fokker–Planck equation from polymer fluid dynamics and we use this method to solve a coupled (macro-micro) Navier–Stokes–Fokker–Planck system for dilute polymeric fluids. In this context the Fokker–Planck equation is posed on a high-dimensional domain and is therefore challenging from a computationa...
متن کامل