Fast convergence rates of deep neural networks for classification

Abstract We derive the fast convergence rates of a deep neural network (DNN) classifier with rectified linear unit (ReLU) activation function learned using hinge loss. consider three cases for true model: (1) smooth decision boundary, (2) conditional class probability, and (3) margin condition (i.e., probability inputs near boundary is small). show that DNN loss achieves rate convergences all provided architecture number layers, nodes sparsity) carefully selected. An important implication architectures are very flexible use in various without much modification. In addition, we by minimizing cross-entropy, under conditions noise exponent large. Even though they strong, explain these two not too absurd image classification problems. To confirm our theoretical explanation, present results small numerical study conducted to compare cross-entropy.