Excess risk bounds in robust empirical risk minimization

Abstract This paper investigates robust versions of the general empirical risk minimization algorithm, one core techniques underlying modern statistical methods. Success is based on fact that for a ‘well-behaved’ stochastic process $\left \{ f(X), \ f\in \mathscr F\right \}$ indexed by class functions $f\in F$, averages $\frac{1}{N}\sum _{j=1}^N f(X_j)$ evaluated over sample $X_1,\ldots ,X_N$ i.i.d. copies $X$ provide good approximation to expectations $\mathbb E f(X)$, uniformly large classes F$. However, this might no longer be true if marginal distributions are heavy tailed or contains outliers. We propose version idea replacing proxies and obtain high-confidence bounds excess resulting estimators. In particular, we show estimators can converge $0$ at fast rates with respect size $N$, referring faster than $N^{-1/2}$. discuss implications main results linear logistic regression problems evaluate numerical performance proposed methods simulated real data.