Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms

Table 12

RL model for the DCS scheme [27].

Action	𝐚^𝐢𝐭=𝑎^𝑖1,𝑡,𝑎^𝑖2,𝑡,…,𝑎^𝑖𝐾,𝑡∈𝐴1×𝐴2×⋯×𝐴𝐾; each subaction 𝑎^𝑖𝑘,𝑡∈𝐴𝑘={0,1} represents the presence of PU activities. Specifically, 𝑎^𝑖𝑘,𝑡=0 if a PU agent cannot transmit in channel 𝑘 and so it becomes white space, and 𝑎^𝑖𝑘,𝑡=1 if the PU agent can transmit in channel 𝑘.
Reward	𝑟^𝑖𝑡+1𝐚^𝐢𝐭=0,if𝑇𝐻𝐚^𝐢𝐭<𝑇th𝜆⋅𝜂𝐚^𝐢𝐭+𝜇⋅𝑊𝐚^𝐢𝐭,otherwise.