Tight Complexity Bounds for Optimizing Composite Objectives
نویسندگان
چکیده
We provide tight upper and lower bounds on the complexity of minimizing the average of m convex functions using gradient and prox oracles of the component functions. We show a significant gap between the complexity of deterministic vs randomized optimization. For smooth functions, we show that accelerated gradient descent (AGD) and an accelerated variant of SVRG are optimal in the deterministic and randomized settings respectively, and that a gradient oracle is sufficient for the optimal rate. For non-smooth functions, having access to prox oracles reduces the complexity and we present optimal methods based on smoothing that improve over methods using just gradient accesses.
منابع مشابه
Tight adversary bounds for composite functions
The quantum adversary method is a versatile method for proving lower bounds on quantum algorithms. It yields tight bounds for many computational problems, is robust in having many equivalent formulations, and has natural connections to classical lower bounds. A further nice property of the adversary method is that it behaves very well with respect to composition of functions. We generalize the ...
متن کاملImproved Redundancy Bounds for Exponential Objectives
We present new lower and upper bounds for the compression rate of binary prefix codes optimized over memoryless sources according to two related exponential codeword length objectives. The objectives explored here are exponential-average length and exponential-average redundancy. The first of these relates to various problems involving queueing, uncertainty, and lossless communications, and it ...
متن کاملPricing the American put using a new class of tight lower bounds
We present new families of lower bounds for the price of the American put option on a dividend paying stock when the stock follows a log normal process and the option can be exercised continuously to a finite horizon . By put call parity, these bounds can be easily converted to bounds on the price of the Ameican call option on a dividend paying stock. By numerically optimizing these bounds, we ...
متن کاملTight Lower Bounds for Locally Differentially Private Selection
We prove a tight lower bound (up to constant factors) on the sample complexity of any non-interactive local differentially private protocol for optimizing a linear function over the simplex. This lower bound also implies a tight lower bound (again, up to constant factors) on the sample complexity of any non-interactive local differentially private protocol implementing the exponential mechanism...
متن کاملA Quadratically Tight Partition Bound for Classical Communication and Query Complexity
The partition bound introduced in [4] is a way to prove lower bounds in classical communication and query complexity. While the partition bound provides a strong and general way to prove lower bounds, it remains open how tight the bounds obtained from this method are. In this work we give quadratically tight lower bounds via a strengthened version of the partition bound, which we call the publi...
متن کامل