SatEdgeAI: multi-agent federated reinforcement learning for adaptive resource orchestration in satellite–terrestrial integrated edge computing networks

Abstract

Next-generation network architectures are changing with the emerging convergence of Low Earth Orbit (LEO) satellite network constellations and multi-access edge computing (MEC) nodes across the globe, with the aim of providing low-latency computation offloading and global connectivity. Satellite topology, however, is a dynamic topology, with the arrival of various types of tasks on the nodes, and with strict Quality-of-Experience (QoE) constraints. This paper introduces a multi-agent federated reinforcement learning (MAFRL) framework, namely SatEdgeAI to achieve adaptive, decentralized resource orchestration in satellite-terrestrial integrated MEC networks. SatEdgeAI uses distributed agents (one agent per MEC node) sharing updates on their policy gradients by using privacy-preserving Federated Aggregation (FA). A novel Topology-Aware Reward Shaping (TARS) mechanism dynamically assigns weights to individual agent rewards according to quality indicators of the satellite links, allowing coherent optimization of the system despite of asynchronous satellite handovers. Through experiments on a model of 613 walker delta satellites, SatEdgeAI shows a reduction of 38.7% in average task completion latency, 22.4% of MEC resource utilization and 61.3% of task drop rate compared to the best single-agent PPO baseline.