Dynamic Energy Storage Control for Reducing Electricity Cost in Data Centers

Algorithm 1

Dynamic energy storage control strategy based on -Learning and Speedy -Learning algorithms.

(1) Initialize:

for each ,   do

 Initialize arbitrarily

end for

Initialize learning counter

Initialize starting state

(2) Learning:

repeat

 Decide to explore/exploit action with probability

 if exploration then

  Choose action at random

 else if exploitation then

  choose action

 end if

 Take action

 Observe the next state

 Receive an immediate cost

 Calculate according to (20)

 Calculate according to (21)

 Update the estimate as follows:

      +

       +

 Update the current state

 Update learning counter

until