Reinforcement Learning for Routing in Cognitive Radio Ad Hoc Networks

<table>SU end-to-end delay for varying PU mean arrival rate <svg height="12.9375" id="M470" style="vertical-align:-3.25792pt" version="1.1" viewbox="0 0 32.037498 12.9375" width="32.037498" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,7.675)"><path d="M538 96q-38 -45 -84 -76.5t-71 -31.5q-44 0 -15 117q12 49 18 89h-2q-99 -128 -163 -178q-36 -28 -70 -28q-26 0 -44 38h-2q-16 -69 -16 -129q0 -43 11 -75.5t24 -42.5l1 -6q-7 -12 -24.5 -23t-37.5 -11q-40 0 -40 76q0 52 35 202l94 407l78 24l5 -5q-32 -122 -72 -310
q-7 -33 0 -55t25 -22q36 0 105.5 75t107.5 145l32 146l75 26h10q-50 -197 -78 -347q-7 -38 6 -38q7 0 31 17t47 39z" id="x1D707"></path></g>
<g transform="matrix(.012,-0,0,-.012,9.5,12.712)"><path d="M46 650h247q112 0 169 -41q64 -49 64 -141q0 -91 -56.5 -143t-138.5 -62q-23 -3 -45 -1l-74 19v-157q0 -63 15.5 -77t83.5 -19v-28h-271v28q60 5 74 20t14 77v400q0 65 -12.5 79t-69.5 18v28zM212 559v-245q31 -12 73 -12q59 0 102 38t43 124q0 77 -42.5 115.5
t-108.5 38.5q-44 0 -56 -12q-11 -8 -11 -47z" id="x50"></path></g><g transform="matrix(.012,-0,0,-.012,16.199,12.712)"><path d="M716 650v-28q-46 -3 -65 -15.5t-22 -43.5q-6 -54 -6 -162v-82q0 -184 -75 -264q-68 -69 -180 -69q-98 0 -160 48q-88 67 -88 238v249q0 66 -13.5 81t-73.5 20v28h258v-28q-59 -4 -72.5 -19t-13.5 -82v-227q0 -122 48 -188t140 -66q189 0 189 286v75q0 111 -7 165
q-4 29 -24 40.5t-73 15.5v28h238z" id="x55"></path></g><g transform="matrix(.012,-0,0,-.012,24.956,12.712)"><path d="M495 163l30 -6q-19 -99 -36 -157h-453v28q63 5 77.5 20t14.5 78v400q0 63 -13 77t-75 19v28h261v-28q-61 -5 -74.5 -19t-13.5 -77v-395q0 -41 5 -59t18 -25q24 -13 96 -13q85 0 112 31q30 33 51 98z" id="x4C"></path></g>
</svg> for different levels of standard deviation of PUL <svg height="12.3375" id="M471" style="vertical-align:-3.25792pt" version="1.1" viewbox="0 0 32.212502 12.3375" width="32.212502" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,8.038)"><path d="M548 455q-32 -77 -74 -77q-37 0 -103 10l-3 -2q36 -30 48.5 -62t12.5 -80q0 -103 -75 -179.5t-175 -76.5q-70 0 -113 45t-43 128q0 115 83.5 200.5t209.5 85.5q31 0 77.5 -4t67.5 -4q36 0 62 30zM350 274q0 54 -17 86q-19 35 -57 35q-50 0 -90 -37.5t-59 -91.5t-19 -109
q0 -61 24.5 -96.5t65.5 -35.5q51 0 87.5 46.5t50.5 100.5t14 102z" id="x1D70E"></path></g>
<g transform="matrix(.012,-0,0,-.012,9.663,12.112)"><path d="M46 650h247q112 0 169 -41q64 -49 64 -141q0 -91 -56.5 -143t-138.5 -62q-23 -3 -45 -1l-74 19v-157q0 -63 15.5 -77t83.5 -19v-28h-271v28q60 5 74 20t14 77v400q0 65 -12.5 79t-69.5 18v28zM212 559v-245q31 -12 73 -12q59 0 102 38t43 124q0 77 -42.5 115.5
t-108.5 38.5q-44 0 -56 -12q-11 -8 -11 -47z" id="x50"></path></g><g transform="matrix(.012,-0,0,-.012,16.361,12.112)"><path d="M716 650v-28q-46 -3 -65 -15.5t-22 -43.5q-6 -54 -6 -162v-82q0 -184 -75 -264q-68 -69 -180 -69q-98 0 -160 48q-88 67 -88 238v249q0 66 -13.5 81t-73.5 20v28h258v-28q-59 -4 -72.5 -19t-13.5 -82v-227q0 -122 48 -188t140 -66q189 0 189 286v75q0 111 -7 165
q-4 29 -24 40.5t-73 15.5v28h238z" id="x55"></path></g><g transform="matrix(.012,-0,0,-.012,25.119,12.112)"><path d="M495 163l30 -6q-19 -99 -36 -157h-453v28q63 5 77.5 20t14.5 78v400q0 63 -13 77t-75 19v28h261v-28q-61 -5 -74.5 -19t-13.5 -77v-395q0 -41 5 -59t18 -25q24 -13 96 -13q85 0 112 31q30 33 51 98z" id="x4C"></path></g>
</svg>.</table>

The Scientific World Journal

fig17

Figure 17

Figure 17: Reinforcement Learning for Routing in Cognitive Radio Ad Hoc Networks 