Dynamic Request Routing for Online Video-on-Demand Service: A Markov Decision Process Approach

Algorithm 1

Solutions to the BMDP formulation.

Input: a BMDP 𝑀_↕={𝑆,𝐴,𝑃_↕,𝑅_↕}

Output:  𝑉_↕ and 𝜋

(1) Create 𝜋; {𝜋 holds the strategy in each iteration.}

(2) Create 𝑉_↕; {𝑉_↕ is an initial interval value function.}

(3) loop

(4)  𝑉_↕=IVI_↕opt(𝑀_↕,𝑉_↕,&𝜋);

(5) end loop