Abstract
conventional integrated sensing and communication (ISAC) systems, the multiplicative fading of radar signals and space division multiple access (SDMA) restrict system performance. This letter proposes a novel rate splitting multiple access (RSMA)-reconfigurable intelligent surface (RIS)-assisted ISAC system to improve the sensing performance, where RIS is deployed to simultaneously assist target detection and RSMAaided multi-user communication for a dual-functional base station (BS). The objective of this letter is to maximize the SNR of the target detection by optimizing the rate splitting, active beam forming at BS, and passive beamforming at the RIS. To solve this non-convex problem, an iterative algorithm is proposed exploiting majorization-minimization (MM), the approximation of the difference of two convex functions (DC), and the penalty method. Finally, simulation results verify the effectiveness of the proposed algorithm and demonstrate a clear sensing gain of the proposed RSMA-RIS-assisted ISAC scheme, approaching the upper bound, due to the special transmission mechanism of rate splitting in RSMA, and the cascaded passive beamforming of order O(N-4) from RIS, respectively.