A Weighted Mirror Descent Algorithm for Nonsmooth Convex Optimization Problem

Large scale nonsmooth convex optimization is a common problem for a range of computational areas including machine learning and computer vision. Problems in these areas contain special domain structures and characteristics. Special treatment of such problem domains, exploiting their structures, can significantly improve the computational burden. We present a weighted Mirror Descent method to solve optimization problems over a Cartesian product of convex sets. The algorithm employs a nonlinear weighted distance in the iterative projection scheme. The convergence analysis identifies optimal weighting parameters that, eventually, lead to the optimal weighted step-size strategy for every projection on a corresponding convex set. We demonstrate the efficiency of the algorithm by solving the Markov Random Fields optimization problem. In particular, we use a weighted log-entropy distance and a weighted Euclidean distance. Promising experimental results demonstrate the effectiveness of the proposed method.