Composition of Web Services Using Markov Decision Processes and Dynamic Programming

Algorithm 3

Value iteration algorithm.

(1) foreach state do

(2)  

(3) end

(4) repeat

(5)  foreach state do

(6)   

(7)  end

(8) until convergence