Differentially private SGD with non-smooth losses

In this paper, we are concerned with differentially private stochastic gradient descent (SGD) algorithms in the setting of convex optimization (SCO). Most existing work requires loss to be Lipschitz continuous and strongly smooth, model parameter uniformly bounded. However, these assumptions restrictive as many popular losses violate conditions including hinge for SVM, absolute robust regression, even least square an unbounded domain. We significantly relax establish privacy generalization (utility) guarantees SGD using output perturbations associated non-smooth losses. Specifically, function is relaxed have ?-Hölder (referred smoothness) which instantiates continuity (?=0) strong smoothness (?=1). prove that noisy smooth perturbation can guarantee (?,?)-differential (DP) attain optimal excess population risk O(dlog?(1/?)n?+1n), up logarithmic terms, complexity O(n2??1+?+n). This shows important trade-off between computational statistically performance. particular, our results indicate ??1/2 sufficient (?,?)-DP while achieving a linear O(n).