Reinforcement Learning for Routing in Cognitive Radio Ad Hoc Networks

Algorithm 1

Dynamic softmax algorithm at SU node 𝑖.

initialize  𝑀=[𝑀_min,𝑀_max]

while  (updating a new 𝑄-value 𝑄^𝑖_𝑡+1)

   if  (𝑄^𝑖_𝑡+1>(𝑄^𝑖_𝑡+𝜗)&&𝑀<𝑀_max)  then  𝑀+=𝑓

   else if  (𝑄^𝑖_𝑡+1<(𝑄^𝑖_𝑡−𝜗))  then  𝑀=𝑀

   else  if  ((𝑄^𝑖_𝑡−𝜗)≤𝑄^𝑖_𝑡+1≤(𝑄^𝑖_𝑡+𝜗)&&𝑀>𝑀_min)  then  𝑀−=𝑓

   end  if

end