Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms

Table 4

RL model for joint dynamic channel selection and channel sensing [11].

State	𝑠^𝑖𝑡∈𝑆={1,2,…,𝐾}; each state represents an available channel
Action	𝑎^𝑖𝑡∈𝐴={𝑎se,𝑎tx,𝑎sw}, where action 𝑎se senses a channel for the duration of 𝑇^𝑖se, 𝑎tx transmits a data packet, and 𝑎sw switches the current operating channel to another one which has the lowest best-known average transmission delay for a single-hop
Reward	𝑟^𝑖𝑡+1𝑠^𝑖𝑡+1 represents the difference between a successful single-hop transmission delay and the maximum allowable single-hop transmission delay