Smart, Sustainable, and Low Latency Computational Offloading From Cellular Network With Cooperating UAVs Mounted RHS Communication: Multi-Agent Deep Reinforcement Learning Based Approach
With the rapid growth in data traffic demand and the increasing number of cellular-connected mobile devices (MDs) over the past decade, future terrestrial networks are expected to deliver high-quality, cost-effective broadband services with ubiquitous access. However, the high costs associated with deploying and maintaining additional base stations (BSs) in congested or hotspot areas constrain terrestrial communication infrastructure from meeting the computing resource demands. Alternatively, computational task offloading via Wi-Fi networks or device-to-device communication links can ensure end-to-end quality of service and experience, particularly during temporary high-traffic events like festivals, concerts, and stadium games. While these ground-based offloading networks depend on robust infrastructure, they are limited by significant propagation delays and low signal-to-interference noise ratios due to limited transmission power. To address these challenges, integrating unmanned aerial vehicles (UAVs) into existing terrestrial cellular networks has emerged as a promising solution by providing seamless connectivity and broad coverage. In particular, mobile edge computing within cellular-UAV networks enhances the freshness of computational tasks by leveraging the age of information (AoI) as a performance metric. Although prior studies have primarily focused on reducing data transmission delays and energy consumption in UAV-assisted computing networks, overlooking the cooperative trajectory design of UAVs, environment-specific channel variations between MDs-UAVs and MDs-ground BSs and the limited battery capacities of MDs and UAVs. Hence, this proposal aims to maximize computational and energy efficiency by systematically offloading tasks to UAVs' computing servers while maintaining low AoI requirements. Achieving this involves optimizing transmission power, computing resource allocations, and UAVs’ cooperative trajectory. Due to the non-convex and combinatorial nature of this problem, conventional analytical methods are impractical. Therefore, the problem is divided into two sub-problems using a Markov decision process. First, the global optimal instantaneous transmission power for each MD and UAV and the computing clock frequencies of computing servers will be determined. Then, deep reinforcement learning (DRL) frameworks will be employed to optimize the cooperative trajectories of UAVs, allowing UAVs to adjust their speed and headings dynamically while adhering to practical constraints by efficiently shaping rewards. Finally, experimental setups with appropriate hardware configurations will validate the proposed DRL algorithms. This validation will provide key insights into system parameters and demonstrate significant performance improvements compared to benchmark methods. The outcomes of this research are expected to contribute theoretically and practically toward advancements in implementing cellular-connected UAV communication systems.