Balanced knowledge distillation for long-tailed learning

Deep models trained on long-tailed datasets exhibit unsatisfactory performance tail classes. Existing methods usually modify the classification loss to increase learning focus classes, which unexpectedly sacrifice head In fact, this scheme leads a contradiction between two goals of learning, i.e., generalizable representations and facilitating for work, we explore knowledge distillation in scenarios propose novel framework, named Balanced Knowledge Distillation (BKD), disentangle achieve both simultaneously. Specifically, given teacher model, train student model by minimizing combination an instance-balanced class-balanced loss. The former benefits from sample diversity learns representation, while latter considers class priors facilitates We conduct extensive experiments several benchmark demonstrate that proposed BKD is effective framework scenarios, as well competitive method learning. Our source code available: https://github.com/EricZsy/BalancedKnowledgeDistillation.