Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms

Algorithm 6

RL algorithm for the channel sensing scheme [20].

Repeat

(a) Take action 𝑎^𝑖_𝑡∈𝐴

(b) Exchange collaboration message 𝐷^𝑖_𝑡,1 with SU neighbor agents // First round of collaboration

(c) Determine delayed reward 𝑟^𝑖_𝑡+1𝐬^𝐢_𝐭+𝟏,𝑎^𝑖_𝑡

(d) Exchange collaboration message 𝐷^𝑖_𝑡,2 with SU neighbor agents // Second round of collaboration

(e) Choose action 𝑎^𝑖_𝑡+1∈𝐴

(f) Update 𝜃_𝑡+1 (see (9))

(g) Update 𝑄-value, 𝑄^𝑖_𝑡+1𝐬^𝐢_𝐭,𝑎^𝑖_𝑡