Abstract
In recent years, the energy consumption of cloud data centers has continued to increase. A large number of servers run at a low utilization rate, which results in a great waste of power. To save more energy in a cloud data center, we propose an energy-efficient task-scheduling mechanism with switching on/sleep mode of servers in the virtualized cloud data center. The key idea is that when the number of idle VMs reaches a specified threshold, the server with the most idle VMs will be switched to sleep mode after migrating all the running tasks to other servers. From the perspective of the total number of tasks and the number of servers in sleep mode in the system, we establish a two-dimensional Markov chain to analyse the proposed energy-efficient mechanism. By using the method of the matrix-geometric solution, we mathematically estimate the energy consumption and the response performance. Both numerical and simulated experiments show that our proposed energy-efficient mechanism can effectively reduce the energy consumption and guarantee the response performance. Finally, by constructing a cost function, the number of VMs hosted on each server is optimized.
1. Introduction
Cloud computing is a paradigm that provides computing, communication and storage resources as a service on demand [1]. The cloud computing paradigm has experienced rapid growth, which has led to increases in the size and number of cloud data centers [2]. Data centers have experienced rapid growth in the number of hosted servers. Some large international business companies have more than 1 million servers in their data centers, including Google and Microsoft [3]. In 2017, the total number of cloud data centers deployed worldwide was estimated to have reached 8.6 million by the IDC (International Data Corporation) [4].
Consequently, the worldwide energy consumption of these massive data centers has increased dramatically. In 2010, electricity usage in global data centers accounted for approximately 1.3% of the total electricity usage worldwide [5]. The total energy usage of data centers has increased dramatically each year. Recently, according to Forbes Technology Council’s report in 2017, electricity usage in global data centers accounted for approximately 3% of the total electricity usage worldwide [6]. In addition to the enormous electricity costs, the energy usage of cloud data centers creates a number of environmental problems. Therefore, energy efficiency is now considered a chief concern for data center operators, even more so than the traditional considerations of availability and security [7]. Green cloud computing has become a trend in the development of current data centers. It is very important to solve the energy-efficient resource scheduling problems in cloud data centers [8].
One of the major reasons for energy inefficiency is the waste of idle power when data center servers are providing computing and storage capabilities at low utilization rates [9]. The task scheduling algorithms have a significant impact on the energy consumption of large data centers [3]. Therefore, how to improve server utilization to save energy has become a difficult problem in cloud data centers that urgently needs to be resolved. Moreover, the rapid development of high-performance computing techniques requires increased response performance in cloud data centers. The trade-off between energy consumption and response performance has also become a hot research topic [10].
Currently, various approaches have been proposed to optimize and manage energy consumption in cloud data centers. Some approaches have investigated techniques such as processor speed controls [11], voltage adjustments [12], service migration [13], switching on servers, and hibernation or sleep modes [14]. Among these techniques, the popular scheme of switching idle servers to sleep mode or powering them down was studied by many researchers and helped provide good improvements in the energy efficiency and power consumption within data centers.
According to the previous work [15], most studies only consider switching the state of the physical server to improve energy efficiency. Few researchers have taken into account the server virtualization of the cloud data center. Meanwhile, cloud data center virtualization has become more popular. Other studies consider switching servers to a sleep mode after VMs migration. However, few of them model the energy-efficient scheme as a queueing system and the overhead of VM migration is not negligible. The goal of this paper is to study the energy-efficient mechanism in virtualized data centers based on multiserver vacation queueing theory. In this paper, we model an energy-efficient task scheduling mechanism with switching on/sleep mode of servers as a multiserver vacation queueing system and then analyse the system’s performance. First, we introduce an energy-efficient task scheduling mechanism with the switching on/sleep mode of servers in a virtualized data center. Then, we establish a two-dimensional Markov chain and estimate the energy consumption and the response performance. We also present numerical results with analyses to show the validity of the novel energy-efficient task scheduling mechanism. Finally, by constructing a cost function, the number of VMs that can be hosted on each server is optimized.
We make three contributions to solve the research problem of the energy-efficient task scheduling of virtualized cloud data centers.(1)We propose an energy-efficient task scheduling mechanism with switching servers on/sleep mode for virtualized cloud data centers. When the number of idle VMs reaches a specified threshold, the server with the maximum idle VMs will be switched to sleep mode after migrating all the running tasks to other servers. In this way, the idle energy consumption in the virtualized cloud data center can be reduced.(2)We present a method to model the proposed mechanism and evaluate the system performance mathematically. We model the virtualized cloud data center with the proposed mechanism as an M/M/n vacation queueing system. By constructing a two-dimensional Markov chain, we evaluate the system performance in terms of the response performance and the energy consumption.(3)We construct a cost function by considering the system performance metrics for virtualized cloud data centers. The optimal number of VMs hosted on each server is obtained using the cost function.
The remainder of this paper is organized as follows. In Section 2, we summarize the related work for the technique of switching idle servers off or to sleep in cloud data centers. In Section 3, we propose an energy-efficient task scheduling mechanism with switching on/sleep mode of servers and introduce the features of the mechanism and then construct the system model and assess the stability of the system. In Section 4, we analyse the queueing model by the matrix-geometric solution method. In Section 5, we derive the formulas for the performance measures and then demonstrate the influences of the system parameters on the system performance with numerical results. In Section 6, by constructing a cost function, we optimize the number of VMs that are hosted on each server. Finally, we draw the conclusions in Section 7.
2. Related Work
2.1. Energy-Efficient Strategies of Physical Servers
Some researchers employ the technique of switching idle servers off or to sleep mode to conserve energy.
In [16], Cheng et al. applied vacation queueing theory to model a cloud computing system and analysed the expectations of the task sojourn time and the energy consumption of a cloud computing system. Based on their analysis, they proposed a task scheduling algorithm based on similar tasks to optimize the energy consumption and evaluate the performance through simulations.
In [17], Phung-Duc and Kawanishi considered an M/M/c/c retrial queue where a server in the queueing system corresponds to one physical server in data centers. They analysed the multiserver retrial queue with setup time by a level dependent QBD process and established a sufficient condition of the stability for the level dependent QBD process. Finally, they proposed a heuristic technique to determine the truncation point, which has been shown to be practically useful.
In [18], Meisner and Wenisch use both DPM (Dynamic Power Management) with the sleep state and dynamic voltage scaling to reduce the power consumption and meet response time constraints. This work has been done considering a single physical server. One server is modelled as an M/G/1/PS queue. The goal of this work is to minimize the mean power consumption of a server under a given response time.
In [19], Vakilinia et al. proposed an energy-efficient strategy to turn off a server pool to save power if the servers are not currently serving any jobs. They developed a mathematical model using queueing theory to analyse the performance of the proposed strategy for a data center with N server pools. Moreover, they considered four schemes, i.e., always on, reactive, proactive, and optimal prediction, and compared their performances.
In [20], Sakata and Kasahara first modelled a data center as a multiserver queueing system. The system alternates independently between two modes, i.e., the normal-operating mode and the power-saving mode, according to the number of jobs in the system. When the system works in the power-saving mode, part of the servers in each cluster are powered off after completing the existing job. Second, the authors constructed a continuous-time Markov chain for the system and derived the steady-state probability vector by the matrix-geometric method. Finally, they investigated how the performance measures are affected by the system parameters, including the number of clusters and the energy efficiency grade.
In [21], Nguyen et al. proposed a novel three-state model for cloud servers. The model was deployed in both single and multiple finite capacity queues. They also proposed several strategies to control multiple queues simultaneously. This approach reduces the service waiting time for jobs and elastically manages the service capability for the system. The power consumption was also evaluated to provide insights into the system performance with respect to the energy-performance trade-off.
In [22], Wang et al. modelled a data center as a heavy traffic approximate large-scale queueing system, and then they designed a load routing mechanism between multiple data centers to reduce the energy costs. Based on multiserver vacation queueing theory, the authors presented a dual-threshold to dynamically adjust the various states (On/Off/Idle) of a single server to reduce the energy costs in the cloud data center.
In [23], Liao et al. minimized the power consumption by switching on/off a certain group of servers according to whether the current queue size reached the threshold. They introduced two activation thresholds and modelled the data center as an M/M/n + m1 + m2 queueing system. Finally, they developed a mathematical model using queueing theory to determine the activation threshold.
In [24], Cheng et al. introduced an M/M/n queueing model with impatient tasks to model the task scheduling in the cloud data center. Based on the M/M/n queueing system model, they analysed the average number of task betrayals and the power consumption costs. To save energy, they proposed a strategy of dynamically adjusting the threshold of the task number. A server is chosen to be awakened according to the cold area where the server located and the length of its shutdown time.
2.2. Energy-Efficient Strategies of Virtual Systems
The above studies are focused on switching the state of the physical servers to improve the energy efficiency; however, the server virtualization is neglected. Fewer studies propose the strategy of shutting down a subset of servers or switching some servers to sleep mode in virtual systems.
In [25], Shen et al. investigated the problem of “right-sizing” a cloud data center for energy efficiency through virtualization, which allows for the consolidation of workloads into smaller number of servers while dramatically powering off the idle ones.
In [26], Deng et al. proposed a VM consolidation based size decision approach, which migrates VMs from underutilized servers to more active ones according to the monitored resource utilization in real time and then turns off the underutilized servers.
In cloud data centers, the arrival of tasks is batch and random. The capability of the servers decreases as the number of processing tasks increases. Large-scale task is generally coordinated by a cluster of multiple servers. Therefore, the task scheduling process of the cloud data center is very consistent with the classic M/M/n multiserver queueing model.
3. Energy-Efficient Mechanism and Model Description
In this section, we propose an energy-efficient task scheduling mechanism with switching servers on/sleep for virtualized cloud data centers and introduce the features of the mechanism. Then, we model the data center with the energy-efficient mechanism as an M/M/n queueing system with a threshold vacation policy.
3.1. An Energy-Efficient Mechanism
To ensure responsiveness, many servers in conventional data centers are always on, even though there are not any tasks to be processed. Many idle VMs lead to a waste of energy. To reduce the extra energy consumption of idle VMs, we propose an energy-efficient mechanism with switching servers on/sleep in a virtualized cloud data center with some homogeneous physical servers. In the cloud data center, each physical server hosts a fixed number of VMs with a fixed number of CPU cores and random access memory. The fixed number of VMs hosted on each server is a threshold, say d. VMs are created, managed, and assigned by the cloud management system (CMS). When users submit tasks to the cloud center, the CMS allocates VMs to process the tasks. Each task is assigned to one VM.
In our proposed energy-efficient mechanism, the server will be switched between two modes, namely, on mode and sleep mode. When a server is in on mode, all the VMs hosted on this server are activated and can run incoming tasks one by one. When a server is in sleep mode, it means that all VMs hosted on the server are closed and the server is deactivated. The proposed energy-efficient mechanism is detailed as follows:(1)The proposed energy-efficient task scheduling mechanism assumes that all the tasks can be scheduled by the CMS to continue running on other VMs during the running process.(2)The CMS monitors the total number of idle VMs in real time. The idle VMs will be distributed among different servers. If the number of idle VMs reaches d, the CMS will schedule all the running tasks on the server with the most idle VMs, migrating these tasks from the busy VMs to other idle VMs on other servers.(3)The scheduling process of a running task mainly includes the following two steps. First step: the CMS sends messages to the VMs on the selected server to terminate the execution of the running task, and then the VMs stop executing the running task. Second step: the CMS sends messages to the idle VMs on the other server to perform the running task with the relevant parameters, and then these idle VMs start to continue the running task.(4)The CMS switches the physical server without busy VMs to sleep mode.(5)Once a physical server switches to sleep mode, a sleep timer will be started, and the sleep period is set by this sleep timer. If there are no tasks waiting to run in the system, the server will start another sleep period once the sleep timer is over. Conversely, if there are tasks waiting to run in the system, the server will be switched to on mode and d hosted VMs will run the waiting tasks immediately.
We note that the energy consumption in sleep mode is lower than that in on mode. The energy consumption will be reduced in the data center with the proposed energy-efficient mechanism.
3.2. Features of the Energy-Efficient Task Scheduling Mechanism
In current cloud data centers, Torque [27], Slurm [28], Google Cloud Scheduler [29], Cloud Foundry [30], and other resource management systems of the server clusters provide many functions, and related APIs, including system status monitoring, task scheduling, server operating mode switching, and so on. These functions and APIs provide support for the efficient operation and energy saving of server cluster systems in cloud data centers.
In the proposed energy-efficient mechanism, VMs switch the operating mode in units of VM groups, avoiding thrashing of the VM clusters, which is caused by frequent switching on/sleep mode of a small number of VMs in a short period of time. In this paper, the proposed energy-efficient mechanism is based on the sleep mode (low power mode) of the physical servers. In the sleep mode, the physical server is not shut down, and the runtime resources of the VMs hosted on the physical server are still stored in the memory of the physical server. It takes a short time for physical servers to switch operating modes between on and sleep. Once the operating mode of a physical server is switched from sleep to on, all of the VMs hosted on the physical server can immediately perform tasks. The proposed energy-efficient mechanism does not involve the process of switching operating mode of the VMs from sleep to on. Therefore, the time for the VMs to switch from nonserviceable to serviceable in the proposed energy-efficient mechanism is very short.
In virtualized cloud data centers, the proposed energy-efficient mechanism avoids thrashing of the VM clusters in the process of implementing energy-efficient task scheduling. And in the proposed energy-efficient mechanism, the method of switching operating modes of physical servers is applied to improve system response performance.
3.3. Model Description
To analyse the performance metrics of the cloud data center, many studies assume that the service times of tasks are exponentially distributed and model the cloud data centers as M/M/n queueing systems [21, 22, 24]. In our paper, we also assume that the service times of tasks are exponentially distributed. The assumptions are as follows.
We assume that the cloud data center consists of M (), homogeneous physical servers, and each server hosts d () VMs. The total number of VMs in the cloud center can be denoted as . We assume users’ tasks arrive at the data center according to a Poisson process with the parameter λ (λ > 0), and the service rate of each VM is exponentially distributed with parameter μ (μ > 0). The sleep period time of the sleep timer for each sleeping server follows an exponential distribution with parameter θ (θ > 0). Tasks are independent and processed on a First-come First-served (FCFS) basis. Random scheduling algorithm is selected as the VM scheduling strategy. If there are idle VMs, the tasks will be processed immediately, and otherwise they wait in the queue. We assume that the migration time of the task is so fast that we can ignore it. Based on these assumptions above, we model the cloud center with the proposed energy-efficient mechanism as an M/M/n queueing system with a threshold vacation policy, as shown in Figure 1.
From the perspective of the number of tasks and the number of servers in sleep mode in the system, we construct a two-dimensional Markov chain to estimate the system’s performance.
Let be the number of tasks to be sent to the data center at time t and be the number of physical servers in sleep mode at time t. Because all random variables are exponentially distributed, constitutes a two-dimensional continuous-time stochastic process with the state space as
We define πi,j as the steady-state probability when the number of tasks is i and the number of physical servers in sleep mode is j in the system. πi,j is then as
We define πi as the steady-state probability vector where the number of tasks is i. Given that i = 0, π0 is given as
Given that kd + 1 ≤ i ≤ (k + 1)d (k = 0, 1, … , M − 2), πi is given as
Given that i > (M − 1)d, πi is given as
We define Π as the steady-state probability distribution of the two-dimensional continuous-time Markov chain, and Π is composed of πi. Π is given as
4. Model Analysis
4.1. Transition Rate Matrix
We define Q as the one-step transition rate matrix of the two-dimensional continuous-time Markov chain. Considering the number of tasks in the system, we give the transition rate matrix Q in a block structure. The matrix Q is composed by several submatrices. The submatrix Bi represents the one-step transition rate for the number of tasks changing from i to i − 1. The submatrix Ci represents the one-step transition rate for the number of tasks changing from i to i + 1. The submatrix Ai represents the one-step transition rate for the number of tasks fixed at i. According to different numbers of tasks in the system, we address Bi, Ai, and Ci with the following conditions.(1)Given that , it means that there are no tasks in the system. In this case, all the servers in the system are in sleep mode and the total number of sleeping physical servers is M. The submatrices A0 and C0 are given as follows:(2)Given that , the possible values for the j sleeping physical servers are . The transition matrices Bi, Ai, and Ci are given as follows:where . where is an identity matrix of order (k + 2).(3)Given that and , the possible values for the j sleeping physical servers are . The transition matrices Bi,Ai, and Ci are given as follows: where . where is an identity matrix of order (k + 2).(4)Given that , the possible values for the j sleeping physical servers are . When the number of tasks in the system reaches N, the submatrices in Q are repeated forever. The repeated submatrices are B, A, and C, which are given as follows.where .where is an identity matrix of order (N + 1).
Based on the discussions above, all the submatrices in the Q have been addressed. The Q is a partitioned matrix
4.2. Matrix-Geometric Solution
To analyse the two-dimensional continuous-time Markov chain with the one-step state transition rate matrix Q, we need to solve the minimal nonnegative solution of the matrix quadratic equation . The solution R is called the rate matrix. The rate matrix R plays a crucial role in the analysis of the proposed mathematical queueing model. Under the condition that , the matrix quadratic equation has a minimal nonnegative solution R and the spectral radius [31].
It is difficult to give the expression of the rate matrix R in a closed form with the higher-order matrix equation. Here, we give an iterative algorithm to obtain the rate matrix R numerically. The main steps of the algorithm are shown in Table 1.
To ensure the convergence of the algorithm, the accuracy ε is given in the algorithm. The algorithm will repeatedly determine whether the value of ||R∗ − R||∞ is greater than ε. When the value of R∗ − R||∞ is not greater than ε, the R is the approximate solution to meet the accuracy ε. With the rate matrix R obtained by the iterative algorithm in Table 1, we construct a square matrix B[R] as follows:
By using the balance equation and the normalized condition, we give the system of equations as follows, where e is a column vector with ones.
We conclude that Q is the one-step transition matrix of a Quasi Birth-and-Death Process. By using the geometric-matrix method solution [32], we can give the system of equations as follows:
By applying the Gauss–Seidel method [33] to solve equation (17), can be obtained. Then, the steady-state probability distribution of the system can be given numerically.
In equation (18), is composed of the vectors of , and is the steady-state probability vector where the number of tasks is i. is composed of the values of , and is the steady-state probability when the number of tasks is i and the number of physical servers in sleep mode is j in the system. When i goes to infinity, the sum of is 1.
5. Performance Analysis and Numerical Results
5.1. Performance Analysis
(1)Let Ls be the average number of tasks in the system. According to Little’s law, we can obtain the mean sojourn time of tasks as Ws. Therefore, we have(2)Let Lbusy be the average number of busy VMs in the system, Lidle be the average number of idle VMs in the system, and Lsleep be the average number of physical servers in sleep mode. The average number of physical servers in on mode is M-Lsleep. The power consumed by a server in on mode is determined by the power of the physical server in on mode and the power of the VMs that are hosted on the physical server. We define the power consumed by a physical server in on mode as Pon and the power consumed by a physical server in sleep mode as Psleep. We define the power consumed by a busy VM as Pbusy and the power of consumed by an idle VM as Pidle. Let Ptotal be the total energy consumption. We have
5.2. Numerical Results
To evaluate the system performance with the proposed energy-efficient mechanism, we provide numerical experiments and simulation experiments with analyses. The results are calculated using Matlab 2011a. The simulation program is written in Java language. The simulation results are obtained by averaging the results from 100 independent runs.
We assume there are 10 physical servers hosted in the data center, and each server can host d VMs. Each VM is configured with a fixed number of CPU cores and random access memory. Each physical server can host at least 5 and at most 20 VMs. The number of VMs that can be hosted on each server can be dynamically adjusted according to the workload. The parameters of the numerical experiments are set in Table 2.
Figure 2 examines how the mean sojourn time of tasks changes in relation to the number of VMs hosted on each server with different task arrival rates. From Figures 2(a) and 2(b), we find that, for the three different arrival rates, when the number of VMs hosted on each server increases, the mean sojourn time decreases. The reason is that as the number of VMs hosted on each physical server increases, the total number of VMs will increase. This means that there are more servers to execute the tasks in the queueing system, and consequently the mean sojourn time of tasks will decrease.
(a)
(b)
By comparing the results of Figures 2(a) and 2(b), we see that the mean sojourn times of tasks increase as the sleeping parameter θ decreases. The reason is that 1/θ is the average length of the sleep period time of the sleep timer. As the sleeping parameter θ decreases, the sleep time of the server in sleep mode will be longer. Then, the tasks will wait for a longer time in the buffer and the mean sojourn time of tasks will increase.
According to the classic multiserver vacation queueing theory, the sleeping parameter θ reflects the idleness of the VMs, and the mean sojourn time reflects the task execution performance. The smaller the value of θ, the longer the sleep time of the VMs, and the greater the probability that the VMs are busy. The longer the mean sojourn time, the worse the QoS.
As shown in Figure 2, the value of θ of Figure 2(b) is smaller than the value of θ of Figure 2(a), indicating that the VMs in Figure 2(b) are more busy. Under the condition that the number of VMs hosted on each server is the same and the value of θ is the same, the value of the mean sojourn time in Figure 2(b) is smaller than the value of the mean sojourn time in Figure 2(a). This shows that when the number of busy VMs hosted on each server is large, the QoS of the system deteriorates, and the service level is reduced.
Figure 3 examines how the total energy costs change in relation to the number of VMs hosted on each server with different task arrival rates.
(a)
(b)
From Figures 3(a) and 3(b), we find that, for the three different arrival rates, when the number of VMs hosted on each server increases, the total energy costs increase. The reason is that as the number of VMs hosted on each physical server increases, the total number of VMs will increase. This means that there are more servers to execute the tasks in the queueing systems, and consequently the total energy costs will increase. By comparing the results of Figures 3(a) and 3(b), we see that the total energy costs of tasks decrease as the sleeping parameter θ decreases. The reason is that 1/θ is the average length of the sleep period time of the sleep timer. As the sleeping parameter θ decreases, the sleep time of the server in sleep mode will be longer. A server in sleep mode consumes less energy, and so the total energy consumption of systems will decrease.
6. System Optimization
To ensure the response performance, one factor affecting the system costs is the mean sojourn time of tasks. As the mean sojourn time of tasks increases, the total numbers of tasks processed in the system per unit of time will decrease. From the perspective of reducing traffic in the throughput-sensitive system, this factor should be set higher. Another factor affecting the system costs is the total energy consumption. As the total energy consumption increases, the electricity costs for the data center increase too. From the perspective of improving energy efficiency in the energy-efficient system, this factor should be set higher.
To obtain the optimal amount of VMs hosted on one server, we construct a system cost function as follows:where f1 is the impact factor of the system costs introduced by the mean sojourn time of tasks and f2 is the impact factor of the system costs introduced by the energy consumption. The values of f1 and f2 are set as needed in practice. In this paper, as an example, we set f1 set to 25 and f2 to 0.4. The change trends for the system costs are shown in Figure 4.
(a)
(b)
Figure 4 examines how the system costs change in relation to the number of VMs hosted on each server with different task arrival rates. From Figures 4(a) and 4(b), we can see that when the number of VMs hosted on each server increases, the system costs decrease. The reason is that the main factor affecting the system costs is the mean sojourn time of tasks, and when the number of VMs hosted on each server increases, the mean sojourn time of tasks decreases. However, when the number of VMs hosted on each server is greater, the system costs do not decrease but rather increase gradually. The reason is that the energy consumption also affects the system costs to a certain degree. Consequently, there is minimum system cost when the parameters d are set optimistically. According to the results shown in Figures 4(a) and 4(b), the optimal parameters d can be given as shown in Table 3.
7. Conclusions
In this paper, we propose an energy-efficient sleep mechanism in a virtualized cloud data center with homogeneous servers. Each server hosts d VMs, and when the number of idle VMs reaches the threshold d, the server with the maximum number of idle VMs will be switched to sleep mode after migrating all the running tasks to other servers. The advantage of the proposed mechanism is that it can dynamically switch servers to sleep mode according to the workload. To analyse the proposed mechanism, we construct a two-dimensional continuous-time Markov chain to estimate the system’s performance and energy consumption. The experimental results are valuable for studying the power-performance trade-off in cloud data centers. By considering the influence of different performance metrics, a cost function for the cloud data center is constructed. We obtain the optimal number of VMs hosted on one server by the cost function.
VM migration technology is very important in virtualized cloud data centers. In the future, we plan to study the energy-efficient task scheduling mechanism with VM migration in virtualized cloud data centers.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported in part by National Natural Science Foundation (no. 61472342), China, and the Strategic Priority Research Program of the Chinese Academy of Sciences (no. XDA06010401).