Dynamic Request Routing for Online Video-on-Demand Service: A Markov Decision Process Approach

<table>An example to find departure probability with a small <svg height="11.375" id="M30" style="vertical-align:-0.17555pt" version="1.1" viewbox="0 0 49.0625 11.375" width="49.0625" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,11.112)"><path d="M764 650l-7 -26q-61 -10 -91.5 -22.5t-76.5 -48.5l-235 -184q104 -152 208 -276q31 -37 54.5 -48.5t68.5 -16.5l-7 -28h-156q-138 170 -213 284q-19 29 -33.5 35t-33.5 -7l-29 -173q-15 -75 -5.5 -90.5t74.5 -20.5l-5 -28h-260l7 28q58 5 74 21.5t30 89.5l70 378
q12 68 2 83t-72 22l6 28h257l-7 -28q-62 -7 -76 -21.5t-27 -83.5l-32 -172q29 11 78 46q112 84 217 179q18 16 23.5 25.5t-1 14.5t-26.5 8l-30 4l5 28h249z" id="x1D43E"></path></g><g transform="matrix(.017,-0,0,-.017,17.962,11.112)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,32.666,11.112)"><path d="M384 0h-275v27q67 5 81.5 18.5t14.5 68.5v385q0 38 -7.5 47.5t-40.5 10.5l-48 2v24q85 15 178 52v-521q0 -55 14.5 -68.5t82.5 -18.5v-27z" id="x31"></path></g><g transform="matrix(.017,-0,0,-.017,40.826,11.112)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g>
</svg>. <svg height="15.8375" id="M31" style="vertical-align:-3.2316pt" version="1.1" viewbox="0 0 60.125 15.8375" width="60.125" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,11.55)"><path d="M649 676l-22 -187l-33 -2q3 56 -12 94q-8 20 -31.5 26.5t-86.5 6.5h-74l-90 -491q-4 -23 -6 -36.5t1 -25t7 -16.5t18 -9t27 -5t41 -3l-6 -28h-286l4 28q68 5 84 18.5t28 76.5l94 491h-55q-74 0 -100.5 -6t-41.5 -23q-24 -29 -54 -98l-32 1q32 98 53 188h22q7 -18 15 -22
t37 -4h417q23 0 33.5 5t25.5 21h23z" id="x1D447"></path></g>
<g transform="matrix(.012,-0,0,-.012,11.388,15.637)"><path d="M244 607q0 -25 -15.5 -43t-37.5 -18q-19 0 -32 13t-13 35q0 21 15 41t39 20q20 0 32 -14t12 -34zM222 91q-29 -33 -79 -68t-75 -35q-13 0 -19 7.5t-6 31t10 65.5l62 253q5 26 -1 26q-21 0 -72 -43l-13 24q43 40 91 68t71 28q30 0 10 -78l-71 -274q-8 -30 3 -30
q16 0 76 48z" id="x1D456"></path></g>
<g transform="matrix(.017,-0,0,-.017,19.975,11.55)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,34.662,11.55)"><path d="M153 550l-26 -186q79 31 111 31q90 0 141.5 -51t51.5 -119q0 -93 -89 -166q-85 -69 -173 -71q-32 0 -61.5 11.5t-41.5 23.5q-18 17 -17 34q2 16 22 33q14 9 26 -1q61 -50 124 -50q60 0 93 43.5t33 104.5q0 69 -41.5 110t-121.5 41q-53 0 -102 -20l38 305h286l6 -8
l-26 -65h-233z" id="x35"></path></g><g transform="matrix(.017,-0,0,-.017,42.821,11.55)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g><g transform="matrix(.017,-0,0,-.017,53.82,11.55)"><path d="M319 325l-25 -7q-33 99 -103 99q-29 0 -47 -19.5t-18 -49.5t22 -49.5t62 -36.5q63 -26 95 -57t32 -79q0 -64 -50 -101t-115 -37q-35 0 -67.5 10.5t-46.5 23.5q-5 11 -11 51t-6 67l27 5q14 -53 46.5 -88.5t75.5 -35.5q29 0 50 19.5t21 50.5t-19.5 51.5t-59.5 39.5
q-28 12 -46 22.5t-38.5 27t-30.5 38.5t-10 49q0 54 42.5 92t109.5 38q48 0 88 -18q6 -15 13 -50.5t9 -55.5z" id="x73"></path></g>
</svg>, <svg height="11.375" id="M32" style="vertical-align:-0.17555pt" version="1.1" viewbox="0 0 53.549999 11.375" width="53.549999" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,11.112)"><path d="M485 501l-30 -4q-15 64 -35 90q-12 16 -33.5 22t-68.5 6h-74q-21 0 -26.5 -5t-5.5 -26v-458q0 -62 15.5 -78t83.5 -20v-28h-270v28q58 4 72 20t14 78v399q0 61 -13.5 77t-71.5 20v28h433q0 -51 10 -149z" id="x393"></path></g><g transform="matrix(.017,-0,0,-.017,13.39,11.112)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,28.093,11.112)"><path d="M384 0h-275v27q67 5 81.5 18.5t14.5 68.5v385q0 38 -7.5 47.5t-40.5 10.5l-48 2v24q85 15 178 52v-521q0 -55 14.5 -68.5t82.5 -18.5v-27z" id="x31"></path></g><g transform="matrix(.017,-0,0,-.017,36.253,11.112)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g><g transform="matrix(.017,-0,0,-.017,47.251,11.112)"><path d="M319 325l-25 -7q-33 99 -103 99q-29 0 -47 -19.5t-18 -49.5t22 -49.5t62 -36.5q63 -26 95 -57t32 -79q0 -64 -50 -101t-115 -37q-35 0 -67.5 10.5t-46.5 23.5q-5 11 -11 51t-6 67l27 5q14 -53 46.5 -88.5t75.5 -35.5q29 0 50 19.5t21 50.5t-19.5 51.5t-59.5 39.5
q-28 12 -46 22.5t-38.5 27t-30.5 38.5t-10 49q0 54 42.5 92t109.5 38q48 0 88 -18q6 -15 13 -50.5t9 -55.5z" id="x73"></path></g>
</svg>. The upper bound of sojourn time is <svg height="11.125" id="M33" style="vertical-align:-0.17555pt" version="1.1" viewbox="0 0 33.674999 11.125" width="33.674999" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,10.862)"><path d="M384 0h-275v27q67 5 81.5 18.5t14.5 68.5v385q0 38 -7.5 47.5t-40.5 10.5l-48 2v24q85 15 178 52v-521q0 -55 14.5 -68.5t82.5 -18.5v-27z" id="x31"></path></g><g transform="matrix(.017,-0,0,-.017,8.222,10.862)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g><g transform="matrix(.017,-0,0,-.017,16.381,10.862)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g><g transform="matrix(.017,-0,0,-.017,27.362,10.862)"><path d="M319 325l-25 -7q-33 99 -103 99q-29 0 -47 -19.5t-18 -49.5t22 -49.5t62 -36.5q63 -26 95 -57t32 -79q0 -64 -50 -101t-115 -37q-35 0 -67.5 10.5t-46.5 23.5q-5 11 -11 51t-6 67l27 5q14 -53 46.5 -88.5t75.5 -35.5q29 0 50 19.5t21 50.5t-19.5 51.5t-59.5 39.5
q-28 12 -46 22.5t-38.5 27t-30.5 38.5t-10 49q0 54 42.5 92t109.5 38q48 0 88 -18q6 -15 13 -50.5t9 -55.5z" id="x73"></path></g>
</svg>. We obtain that <svg height="16.025" id="M34" style="vertical-align:-3.2316pt" version="1.1" viewbox="0 0 61.849998 16.025" width="61.849998" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,10.862)"><path d="M570 304q0 -108 -87 -199q-40 -42 -94.5 -74t-105.5 -43q-41 0 -65 11l-29 -141q-9 -45 -1.5 -58t45.5 -16l26 -2l-5 -29l-241 -10l4 26q51 10 67.5 24t26.5 60l113 520q-54 -20 -89 -41l-7 26q38 28 105 53l11 49q20 25 77 58l8 -7l-17 -77q39 14 102 14q82 0 119 -36
t37 -108zM482 289q0 114 -113 114q-26 0 -66 -7l-70 -327q12 -14 32 -25t39 -11q59 0 118.5 81.5t59.5 174.5z" id="x1D45D"></path></g>
<g transform="matrix(.012,-0,0,-.012,10.138,15.825)"><path d="M244 607q0 -25 -15.5 -43t-37.5 -18q-19 0 -32 13t-13 35q0 21 15 41t39 20q20 0 32 -14t12 -34zM222 91q-29 -33 -79 -68t-75 -35q-13 0 -19 7.5t-6 31t10 65.5l62 253q5 26 -1 26q-21 0 -72 -43l-13 24q43 40 91 68t71 28q30 0 10 -78l-71 -274q-8 -30 3 -30
q16 0 76 48z" id="x1D456"></path></g>
<g transform="matrix(.017,-0,0,-.017,18.738,10.862)"><path d="M537 353q-54 -70 -120 -70q-47 0 -137 43q-78 38 -108 38q-55 0 -91 -57l-31 33q51 74 122 74q45 0 137 -44q76 -37 108 -37q55 0 91 57zM537 168q-54 -70 -120 -70q-47 0 -137 43q-78 38 -108 38q-55 0 -91 -57l-31 33q51 74 122 74q45 0 137 -44q76 -37 108 -37
q55 0 91 57z" id="x2248"></path></g><g transform="matrix(.017,-0,0,-.017,33.424,10.862)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g><g transform="matrix(.017,-0,0,-.017,41.584,10.862)"><path d="M113 -12q-24 0 -39.5 16t-15.5 42q0 24 16 40.5t40 16.5t40 -16.5t16 -40.5q0 -26 -16 -42t-41 -16z" id="x2E"></path></g><g transform="matrix(.017,-0,0,-.017,45.46,10.862)"><path d="M412 140l28 -9q0 -2 -35 -131h-373v23q112 112 161 170q59 70 92 127t33 115q0 63 -31 98t-86 35q-75 0 -137 -93l-22 20l57 81q55 59 135 59q69 0 118.5 -46.5t49.5 -122.5q0 -62 -29.5 -114t-102.5 -130l-141 -149h186q42 0 58.5 10.5t38.5 56.5z" id="x32"></path></g><g transform="matrix(.017,-0,0,-.017,53.619,10.862)"><path d="M153 550l-26 -186q79 31 111 31q90 0 141.5 -51t51.5 -119q0 -93 -89 -166q-85 -69 -173 -71q-32 0 -61.5 11.5t-41.5 23.5q-18 17 -17 34q2 16 22 33q14 9 26 -1q61 -50 124 -50q60 0 93 43.5t33 104.5q0 69 -41.5 110t-121.5 41q-53 0 -102 -20l38 305h286l6 -8
l-26 -65h-233z" id="x35"></path></g>
</svg> in this scenario.</table>

Mathematical Problems in Engineering

fig3

Figure 3

Figure 3: Dynamic Request Routing for Online Video-on-Demand Service: A Markov Decision Process Approach 