Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms

Table 6

RL model for joint DCS and channel sensing [34].

State	𝐬^𝐢𝐭=𝑠^𝑖1,𝑡,𝑠^𝑖2,𝑡,…,𝑠^𝑖𝐾,𝑡∈𝑆1×𝑆2×⋯×𝑆𝐾; each substate 𝑠^𝑖𝑘,𝑡∈𝑆𝑘={0,1} indicates an idle or busy channel; specifically, 𝑠^𝑖𝑘,𝑡=0 if PU activity does not exist in channel 𝑘, and 𝑠^𝑖𝑘,𝑡=1 if PU activity exists in channel 𝑘
Action	𝑎^𝑖𝑡∈𝐴={1,2,…,𝐾}; each action represents a single channel available for data transmission
Reward	𝑟^𝑖𝑡+1𝐬^𝐢𝐭+𝟏=1,ifsuccessfultransmission0,ifunsuccessfultransmissionbecausethesensedchannelisbusy−0.5,ifunsuccessfultransmissionandbackoffbecausethereiscollisionwithotherSUs