Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms

Table 13

RL model for the channel sensing scheme [20].

State	𝐬^𝐢𝐭=𝑏𝑠^𝑖1,𝑡,𝑏𝑠^𝑖2,𝑡,…,𝑏𝑠^𝑖𝐾,𝑡∈𝑆1×𝑆2×⋯×𝑆𝐾; each substate 𝑏𝑠^𝑖𝑘,𝑡∈𝑆𝑘={0,1} indicates SU 𝑖’s belief about channel 𝑘, and it has a value of 0 (busy) or 1 (idle)
Action	𝑎^𝑖𝑡∈𝐴={1,2,…,𝐾}; each action represents a single channel chosen for channel sensing purpose
Reward	𝑟^𝑖𝑡+1𝐬^𝐢𝐭+𝟏,𝑎^𝑖𝑡 represents the number of channels identified as being idle by SU node 𝑖