Robust estimation via generalized quasi-gradients

Abstract We explore why many recently proposed robust estimation problems are efficiently solvable, even though the underlying optimization non-convex. study loss landscape of these problems, and identify existence ’generalized quasi-gradients’. Whenever quasi-gradients exist, a large family no-regret algorithms guaranteed to approximate global minimum; this includes commonly used filtering algorithm. For mean distributions under bounded covariance, we show that any first-order stationary point associated problem is an minimum if only corruption level $\epsilon < 1/3$. Consequently, algorithm approaches yields efficient estimator with breakdown $1/3$. With carefully designed initialization step size, improve $1/2$, which optimal. other tasks, including linear regression joint covariance estimation, more rugged: there points arbitrarily far from minimum. Nevertheless, generalized exist construct algorithms. These simpler than previous ones in literature, for error $O(\sqrt{\epsilon })$ optimal rate $O(\epsilon )$ small $ assuming certified hypercontractivity. near-identity simple gradient descent achieves $1/3$ iteration complexity $\tilde{O}(d/\epsilon ^2)$.