Kernel Recursive Least-Squares Temporal Difference Algorithms with Sparsification and Regularization

Algorithm 1

OSKRLSTD-.

)  Input: to be evaluated, , , ,

()  for    do

()    if    then

()     Initialize ,

()     Take given by , and observe ,

()     Initialize

()     Initialize

()     Initialize

()    else

()    Take given by , and observe ,

()     Update ,

()     Update , by (21) and (20)

()      if   satisfies the sparsification condition then

()        , ,

()       Compute , , and by (23)

()       Update , by (25) and (24)

()      end if

()    end if

()   

() end for