Decomposition Methods for Solving Finite-Horizon Large MDPs

Algorithm 3

Hierarchical backward induction.

(1)	HBI (In MDP: , Out ()
(2)	Find SCCs and their belonging levels using Tarjan’s algorithm
(3)	For each level Lp, ,…, Lp Do
(4)	For each class Clpk, in level Lp
(5)	Construct the restricted finite-horizon MDPpk
(6)	End For
(7)	End For
(8)	For each level Lp,,…, Lp Do
(9)	For each class Cpk, in level Lp over planning horizon T, Do
(10)	ABI (MDPpk)
(11)	End For
(12)	End For
(13)	Return