Reinforcement Learning for Routing in Cognitive Radio Ad Hoc Networks

Table 1

CRQ-routing model embedded at SU node 𝑖.

State	𝑠^𝑖𝑡∈𝑆={1,2,…,𝑁−1}, each state 𝑠^𝑖𝑡 representing a SU destination node 𝑛. 𝑁 represents the number of SUs in the entire network.
Action	𝑎^𝑖𝑡∈𝐴^𝑖={1,2,…,𝐽}, each action 𝑎^𝑖𝑡 representing the selection of a SU next-hop node 𝑗 along with its operating channel. 𝐽 represents the number of SU 𝑖’s neighboring SU nodes.
Cost	𝑟^𝑖𝑡(𝑎^𝑖𝑡) represents the link-layer delay incurred to successfully deliver a packet from SU node 𝑖 to SU neighbor node 𝑎^𝑖𝑡=𝑗, including retransmission delays as a result of PU-SU packet collision and packet loss.