Abstract
This paper focuses on leveraging coordinated multi-point (CoMP) to improve the sum downlink rate in air-terrestrial integrated networks. Considering the CoMP transmission, the problem of maximizing the sum downlink rate possesses two-timescale characteristics, i.e., trajectory planning varies of aerial base station in a long-timescale manner whereas CoMP resource allocation varies in a short-timescale manner. To solve it, a two-timescale parallel framework is proposed. Specifically, the initial two-timescale problem is decomposed into multiple single-timescale subproblems via the alternating direction method of multipliers and then all subproblems are parallelly solved. Furthermore, to resolve the high complexity arising from continuous-discrete hybrid variables in each single-timescale subproblem, we propose an online algorithm that embeds optimization programming (OP) into deep reinforcement learning (DRL). Particularly, discrete CoMP cluster variables in sequential time slots are optimized with DRL to eliminate the need for large-scale combinatorial optimization. Then, the continuous resource allocation variables in each time slot are solved by OP, which is embedded between the output and reward of DRL, to speed up the convergence. Simulation results show that the proposed algorithm achieves less than a 7% total downlink data gap while decreasing the computation time by more than an order of magnitude compared with numerical optimization.