When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multitimescale Resource Management for Multiaccess Edge Computing in 5G Ultradense Network

Recently, smart cities, healthcare system, and vehicles have raised challenges on the capability connectivity of state-of-the-art Internet-of-Things (IoT) devices, especially for devices in hotspots area. Multiaccess edge computing (MEC) can enhance ability emerging resource-intensive IoT applications has attracted much attention. However, due to time-varying network environments, as well heterogeneous resources it is hard achieve stable, reliable, real-time interactions between their serving servers, 5G ultradense (UDN) scenarios. Ultradense (UDEC) potential fill this gap, era, but still faces its current solutions, such lack of: 1) efficient utilization multiple (e.g., computation, communication, storage, service resources); 2) low overhead offloading decision making resource allocation strategies; 3) privacy security protection schemes. Thus, we first propose an intelligent UDEC (I-UDEC) framework, which integrates blockchain artificial intelligence (AI) into networks. Then, order computation decisions strategies, design a novel two-timescale deep reinforcement learning (2Ts-DRL) approach, consisting fast-timescale slow-timescale process, respectively. The primary objective minimize total delay usage by jointly optimizing offloading, allocation, caching placement. We also leverage federated (FL) train 2Ts-DRL model distributed manner, aiming protect devices’ data privacy. Simulation results corroborate effectiveness both FL I-UDEC framework prove that our proposed algorithm reduce task execution time up 31.87%.