Decomposition Methods for Solving Finite-Horizon Large MDPs

Table 1

Characteristics of the three racetracks.

	MDP’s horizon	Number of possible states	Number of SCCs
Race-1	31	132750	5120
Race-2	18	106200	5103
Race-3	48	123300	3554

Note. The value iteration (VI) algorithm under the infinite-horizon discounted MDP is used in [30] in order to solve racetrack problems. presents the comparison between VI, BI, and HBI algorithms. As it can be seen, the BI algorithm outperforms the VI algorithm, but the proposed HBI algorithm is more efficient than the BI algorithm.