A Reinforcement Learning Based Auto-Scaling Approach for SaaS Providers in Dynamic Cloud Environment

Algorithm 1

Q-learning based self-adaptive VM instances renting algorithm.

Input: Initialize parameters , and

StateNum←1, s← an initial state, S[StateNum]←s, add A(s) to A

For , initialize Q-values

Output: action a for each renting decision period

(loop (for each renting decision period)

( choose an action from A(s) using -Greedy policy

(if Random(0,1) < ε  then   exploration

(a←random A(s)

(else //exploitation

(a←

(end if

(submit renting plan(i.e. action a) to IaaS provider

(observe customer workload w over this period

(move to new state

(if    then  // add new state into state space

(StateNum++

(S[StateNum]←

(For , initialize Q-values

(end if

( calculate reward r← using (5) - (12)

(update

(s ←

(end loop