Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

Abstract

We consider a cognitive heterogeneous network (HetNet), in which multiple pairs of secondary users adopt sensing-based approaches to coexist with a pair of primary users on a certain spectrum band. Due to imperfect spectrum sensing, secondary transmitters (STs) may cause interference to the primary receiver (PR) and make it difficult for the PR to select a proper modulation and/or coding scheme (MCS). To deal with this issue, we exploit deep reinforcement learning (DRL) and propose an intelligent MCS selection algorithm for the primary transmission. To reduce the system overhead caused by the MCS switching’s, we further introduce a switching cost factor in the proposed algorithm.

The simulation results show that the primary transmission rate of the proposed algorithm without the switching cost factor is 90% ~ 100%  of  the  optimal  MCS  selection  scheme, which assumes that the interference from the STs is perfectly known at the PR as prior information, is 30% higher than that of the upper confidence bandit (UCB) algorithm, and is 100% higher than that of the signal-to-noise ratio (SNR)-based algorithm. Mean while, the proposed algorithm with the switching cost factor can achieve a higher primary transmission rate than those of the benchmark algorithms without increasing system overheads.. Here we added an extension to current model like considering a number of frames in Energy Efficiency (EE) system where the users have different subcarrier spacing (SCS). Unlike in single numerology EE systems, mixed frames EE systems suffer from inter-numerology interference (INI). We first derive the interference pattern and find that the variance of interference energy increases due to the difference in SCS. This increase in variance negatively affects decoding performance, since the interference energy is unbalanced between subcarriers