FedICT: Federated Multi-task Distillation for Multi-access Edge Computing

The growing interest in intelligent services and privacy protection for mobile devices has given rise to the widespread application of federated learning Multi-access Edge Computing (MEC). Diverse user behaviors call personalized with heterogeneous Machine Learning (ML) models on different devices. Federated Multi-task (FMTL) is proposed train related but ML devices, whereas previous works suffer from excessive communication overhead during training neglect model heterogeneity among MEC. Introducing knowledge distillation into FMTL can simultaneously enable efficient clients, existing methods rely a public dataset, which impractical reality. To tackle this dilemma, MultI-task Distillation CompuTing (FedICT) proposed. FedICT direct local-global aloof bi-directional processes between clients server, aiming multi-task while alleviating client drift derived divergent optimization directions client-side local models. Specifically, includes Prior Knowledge (FPKD) Local Adjustment (LKA). FPKD reinforce clients' fitting data by introducing prior distributions. Moreover, LKA correct loss making transferred better match generalized representation. Experiments three datasets show that significantly outperforms all compared benchmarks various architecture settings, achieving improved accuracy less than 1.2% FedAvg no more 75% round FedGKT.