Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms

Algorithm 1

RL algorithm for joint DCS and channel sensing [10].

Repeat

(a) Choose action 𝑎^𝑖_𝑡∈𝐴_𝑝

(b) Update 𝑄-value:

           𝑄^𝑖_𝑡+1𝑎^𝑖_𝑡=𝑄^𝑖_𝑡𝑎^𝑖_𝑡+𝑟^𝑖_𝑡+1𝑎^𝑖_𝑡

(c) Update preferred channel set 𝐴_𝑝∈𝐴

          𝑎^𝑖_𝑡∈𝐴_𝑝=𝑎^𝑖_𝑡∈𝐴∣𝑄^𝑖_𝑡+1𝑎^𝑖_𝑡>𝑄_th