Abstract
We propose a Lyapunov drift-plus-penalty- (LDPP-) based algorithm to optimize the average power cost for a data center network. In particular, we develop an algorithm to minimize the operational cost using real-time electricity pricing with the integration of green energy resources from the smart grid. The LDPP technique can achieve significant energy cost savings under quality of service (QoS) constraints. Numerical results are presented to evaluate and validate our solution. These results illustrate significant operational/energy cost reductions for a data center network over the conventional approach which optimizes the predicted values of stochastic parameters under a fixed QoS constraint.
1. Introduction
Data centers are primary information and communications technology (ICT) energy consumers. For example, data centers in the USA consume 100 billion kWh per year which costs approximately 7.4 billion dollars [1]. Recently, the next-generation power grid, known as the smart grid, has been introduced to facilitate the participation of power consumers and the integration of renewable generation to balance power supply and demand in real-time [2]. This provides consumers with the ability to dynamically track electricity price variations and efficiently manage their power consumption. In this case, we can improve system efficiency by shaping and balancing the power loads (e.g., servers). The smart grid also promotes the use of sustainable and green energy resources such as wind and solar panels (SPs).
The number of demand applications, particularly video streaming traffic, e.g., video on demand (VoD), web browsing, online gaming, and IPTV, has substantial impact on the energy consumption of data centers and can dynamically change according to the number of user demands. Thus, as emerging services grow as well as the number of connected smart devices rises, the energy consumption of data centers is anticipated to increase significantly.
In the current work, we propose and investigate the performance of a Lyapunov optimization method considering actual real-time prices. Experimental data and computer simulation are used to assess the performance of the LDPP-based energy cost reduction strategy. The results obtained show that this approach can significantly outperform the expected value algorithm (e.g., linear programming or LP (the LP method is a well-known benchmark method for solving linear constrained optimization problems)) in terms of energy cost savings. In this case, we also derived the probability of violating the QoS (connection delay) and processing (connection handling capacity of the data center) constraints. It is also shown that the proposed LDPP algorithm has a lower probability of violating the QoS and processing constraints compared to the conventional linear programming (LP) approach.
In summary, the main contributions of the present paper are as follows.(i)We consider energy and cost savings for a data center network in the context of the smart grid. A real-time energy management algorithm is proposed to reduce the cost of server cluster operation according to real-time pricing, energy demand, and power supply estimation. We also incorporate green power generators such as energy storage devices and solar panels as a power supply for the server clusters.(ii)We develop an algorithm based on optimizing the total energy consumption cost in data center networks. This allows us to intelligently route requests among data centers according to the type of user demand application (web/VoD) and energy efficiency considerations.(iii)LP and LDPP methods are investigated to solve the optimization problem. This leads to a tradeoff between operational cost saving and computational complexity.(iv)We examine the number of batteries required to support the average daily streaming traffic for a data center network.(v)Finally, a probabilistic model is given to model the complementary cumulative distribution function (CCDF) of violating the QoS and processing constraints for data center network users, and the performance with LP and LDPP is compared.
The remainder of this paper is organized as follows. Section 2 provides a review of the related work including system models and some assumptions used in the paper. Section 3 presents the formulation of a optimization problem for real-time cost reduction for a green data center. Section 4 describes the proposed LDPP-based algorithm. The performance evaluation results are presented in Section 5, and finally some conclusions are given in Section 6.
2. Background and System Models
2.1. Related Work
There has been significant research on reducing data center power costs and emissions in the smart grid context [3–12]. Previous approaches have focused on the design of real-time energy management systems which optimize the data center network power costs according to system uncertainties while considering a quality of service (QoS) constraint such as connection delay. These uncertainties are related to stochastic parameters including real-time pricing, power supplies from renewable sources, and data center workloads. To characterize these uncertainties, the expected values of the parameters during specific time periods have been employed under QoS constraints. Thus, the problem formulations have been deterministic and solved using techniques such as linear programming (LP) to minimize the power costs. However, this does not necessarily maximize the power cost savings.
Previous methods for saving data center power costs can be categorized as minimizing electricity costs [3–7], decreasing energy consumption [8, 9], or using renewable energy resources [10, 12]. Methods in the first category typically use the temporal and spacial variations in electricity prices from the smart grid to exchange connections among data centers to reduce operational costs. The second class of techniques aims at lowering the energy consumption of a data center and/or a data center network by considering cloud computing to reduce the power consumption of end users. The impact of real-time pricing on the operational cost and energy efficiency of the cloud transport network infrastructure was examined in [10]. The third class of techniques employs renewable energy from local sources to reduce emissions and operational costs. For example, the green star network (GSN) testbed [10] was used to examine the practicality of powering a network of data centers with solar and wind power.
In [13], a stochastic optimization problem was formulated to tackle the stochastic renewable generation and workload arrival processes. Then, an online control algorithm based on Lyapunov optimization was proposed to solve it. In [14], Mao et al. proposed a low-complexity online algorithm to minimize the long-term average network service cost, namely, the Lyapunov optimization-based base station assignment and power control (LBAPC) algorithm. The main advantage of this algorithm is that the decisions depend only on the instantaneous side information without requiring distribution information of channels and energy harvesting processes. In [15], a gamebased traffic exchange mechanism for green data center networks was proposed. Unlike in [15], the current work, a fast online optimization method based on the Lyapunov drift is employed which can reduce the system complexity more efficiently.
Different from the approaches in [13], we integrate smart grid concepts into the optimization formulation and use a QoS metric for user satisfaction in the proposed constraints. In [14], a similar Lyapunov optimization approach was used in a different context which is base station assignment and power control.
Current results in the literature such as in [3–12] focus on reducing the power cost of data center networks according to dynamic electricity pricing, integration of renewable energy resources, and considering fixed QoS constraints. Rather than predicting the values of stochastic parameters as in [3–12], we consider an algorithm based on Lyapunov optimization [16]. This method minimizes the long-term average power cost under QoS and processing constraints using the Lyapunov drift-plus-penalty (LDPP) algorithm. This approach can reduce the optimization to a greedy minimization problem which provides a significant cost reduction.
In this paper, an optimization model is developed for a data center network based on stochastic parameters extracted from green data centers, and a drift-plus-penalty framework is used to optimize the time-averaged power cost reduction. This model includes the dynamic and static power consumption of the servers, electricity prices, the availability of renewable generation, QoS and processing constraints, and service payments as well as penalties.
2.2. Green Data Centers
The data center model is composed of servers, users, power management control units, the electrical grid, renewable power sources, power storage, and batch schedulers, as illustrated in Figure 1. The total power consumption within a data center typically consists of the consumption by the facility infrastructure and the ICT infrastructure, as shown in Figure 1. In the facility infrastructure, a cooling system is the main source of energy consumption, while in the ICT infrastructure, computing resources (servers) and switches are the major consumers. The facility infrastructure also includes additional power loads such as uninterrupted power supplies (UPSs), batteries, lighting, and switchgear. The sensor network in Figure 1 is one of the smart grid components for measuring energy usage.
The power consumption of the computing servers has two major components: static and dynamic . The latter is associated with the CPUs and is directly related to user traffic demand. Static power consumption does not depend on this demand and can be expressed as [17]where is the average idle power of a server, U is the power usage effectiveness [17], and is the average server peak power when it processes application requests (i.e., user demand). The dynamic power consumption can be expressed aswhere the superscripts and denote real-time and non real-time (elastic) applications, respectively. and are two positive weighting constants where , (it is assumed that the real-time applications are broadband so they consume more resources than their narrowband counterparts). represents the number of real-time connections (such as video on demand, gaming, and IPTV), and is the number of elastic connections which are served from the data center. Since static power consumption does not depend on user demand and the focus of this paper is on dynamic power consumption, only the latter is considered in the remainder of this paper.
Electrical energy resources including renewable generators, local power generators (e.g., diesel generators), and batteries for storing energy are illustrated in Figure 1. Renewable energy sources are solar panels and wind turbine, while local generators include diesel generators and batteries. A renewable source generates a random amounts of power due to the weather conditions at different time slots. For example, wind energy depends on wind power, and solar energy depends on the level of sunlight. The data center has power storage (i.e., batteries), to store the power generated from renewable sources as well as power purchased from the electrical grid. An intelligent sensor network which is composed of sensors, actuators, and a control unit as shown in Figure 1 is essential for stable and reliable data center operation. Sensors/actuators are attached on rectifiers (in the case of wind generator), converters, power generators, and loads to sense and measure parameters such as active power and passive power for processing in the control unit. The control unit facilitates real-time energy management via energy consumption scheduling. An energy consumption scheduling module can work in disconnected or connected modes of operations according to the shortage or surplus of electrical power. In the former mode, it locally balances supply and demand and if there is a surplus of energy, there is the possibility to save this energy in the UPS batteries. In the latter mode, the power grid and local energy resources are employed to balance supply and demand. In this case, there is a capability for energy consumption scheduling to facilitate real-time energy management and consequently reduce operation costs. It also obtains real-time pricing via a gateway and bidirectional communications with the utility operations center.
3. Optimization Model for Minimizing Power Cost
Assume that the composite random at time slot t is defined as , . , , and denote the set of powers generated from renewable sources, the set of grid spot power prices, and the set of user demands due to real-time and nonreal-time applications, respectively, within the data center at time step t. The random variables , , , and take values from these sets. The time step t is from the set , with duration τ. The variable m from set indicates the number of applications.
The minimization of the power cost can be formulated as an optimization problem with uncertainty . The objective is to minimize the expected cumulative cost over all time steps, i.e., over random variables . To minimize the cost at time t, the objective function is defined with random variables and . The cost minimization problem can then be expressed aswheresubject to
We have considered the composite random λ at (T-1) because, in the real-time energy management methodology, the energy costs in the previous time slot will be used to estimate current costs. For simplicity, we assume that the set of purchase prices of power from the grid is equal to the set of prices of power sold to the electrical grid . We also assume that the power generated is saved in batteries so it incurs the set of costs associated with battery losses due to charging and discharging. Equation (5) ensures that the number of available servers is sufficient to handle all user demands. β is a constant based on the user demand according to the number of available servers. Equations (4)–(6) provide the optimal solutions for and .
4. Proposed LDPP-Based Algorithm for Cost Reduction Optimization
To solve the uncertainty model defined in (4)–(6), the deterministic equivalent model [18] given is employed:subject towhere expectation in the stochastic model is replaced by the corresponding probability denoted by in (8). In fact, the objective function defined in (8) is to minimize the expected total cost. Further, the time-dependent decision variables are defined based on specific values, i.e., and . In particular, if occurs at time t, then the values of and are used. For example, is the amount of power bought from the grid with in time step t. The constraint in (11) ensures that the power input is equal to the power output in time step t.
Instead of solving the stochastic constrained optimization model defined in (8)–(11), it can be rewritten with the random variables replaced by their expected values. This formulation has much lower complexity since the number of variables and constraints are reduced. The corresponding deterministic formulation issubject towhere the expected value of is , and the other expected values are obtained similarly, and represents the maximum storage capacity of the batteries.
Linear programming (LP) can be used as a solution to the expected value in (5). However, LDPP can provide a solution with large power cost savings as explained in the following section.
4.1. LDPP-Based Algorithm for a Data Center Network
In order to employ the Lyapunov-drift algorithm for the above stochastic optimization problem, let represent the ith data center within a data center network where the number of data centers is I. It is also assumed that real-time connection arrivals within each data center have a Poisson distribution. The service rate is exponential with parameter for application m at data center i, where is the average service time [19]. Further, the QoS and processing requirements must be satisfied, so the transmission delay from a front end server to data center i denoted as should be less than a delay threshold for real-time application m at data center i. For a data center network, the problem formulation (8)–(11) is then given bysubject towhere is the maximum number of applications in data center i.
Table 1 summarizes the parameters and variables used in the paper.
Let , , denote the stable mean rates for the queues associated with the network routers. We also assume that and denote the packet arrival and packet departure processes, respectively, for the th queue.
We now propose a Lyapunov stochastic optimization for the problem formulated in (16)–(20). To achieve this, we define virtual queues for values of m and i so that
The penalty function incurred by the virtual queues at time step t is defined as
Finally, the stochastic optimization model based on the drift-plus-penalty framework defining vectors and is given in the following steps [16].(1)Observe and greedily choose the power consumption vector such that the following expression is minimized:(2)Update the queues according to (21):
Algorithm 1 describes the proposed LDDP algorithm for a green data center network.
|
As mentioned above, an advantage of the LDPP-based optimization algorithm is that the decision variables depend only on the instantaneous stochastic information without requiring their distribution information. To determine the power cost reduction, we only need to solve a deterministic per-time step problem which validates the effectiveness of the proposed algorithm.
In the proposed constrained optimization problem (16)–(20), we have a nonlinear constraint (20) which is due to end user QoS requirements. So, in the proposed optimization, we have nonlinear constraints, and this is the main reason that the conventional LP method may be suboptimal with respect to its LDPP counterpart. Next, we derive the constraint violation probabilities for the processing capability constraint (14) and the QoS constraint (20).
4.2. Probability of Constraint Violation
In this section, the probabilities of violating the processing and QoS constraints in (14) and (20) are calculated. The probability that the delay (QoS) constraints are satisfied in the proposed multiapplication/multidata center system, i.e., , is denoted by and can be expressed for time step t aswhere is the probability distribution function (PDF) of the real-time connection arrivals. The QoS violation probability (or ) is then
The probability that the processing constraints are satisfied in the proposed multiapplication/multidata center system, i.e., , is denoted by and is expressed for each time step t aswhere denotes convolution and and are the PDFs of the real-time and nonreal-time connection arrivals, respectively. The processing violation probability (or ) is then
5. Numerical Results and Discussion
In this section, numerical results are presented to demonstrate the performance of the proposed real-time management algorithm for both the expected value (LP) and Lyapunov optimization solutions. The power cost savings are determined with regards to the optimized amount of grid energy which can be bought and/or sold. To obtain realistic results, we use the actual hourly real-time prices for 24 hour periods from [19] for various dates. Therefore, data centers at different geographic locations within the data center network employ different real-time pricing. The price of electricity is updated every two hours, while the real-time energy management algorithm is executed every one hour for both the LP and Lyapunov optimization solutions. All simulation results are averaged over 1000 iterations with real-time prices from different 24 hour periods. and are set to 0.98 and 0.02, respectively. All the thresholds in Algorithm 1 are set to 0.01. The simulations were done using MATLAB on a Quad-Core Intel processor with a 3 GHz clock speed and 4 GB of RAM.
It is assumed that the packet generation process follows a Poisson distribution [20, 21] for voice (real-time) and elastic (nonreal-time) applications. It is also assumed that the video packet generation process follows a heavy-tail distribution, namely, a Markov-modulated Poisson process (MMPP) [20, 21]. An MMPP is a doubly stochastic Poisson process whose average number of events in an interval, i.e., the event rate, varies according to a Markov process. There are data centers, and each employs different prices from [22]. Practical 5 KWh batteries are used for storage of the energy from solar panels according to [23]. We also assume that the data centers have different storage capacities, i.e., different numbers of solar panels.
The proposed algorithm is evaluated for both LP and LDPP optimization. We compare a regular data center network which only uses the grid for power with a green data center network which uses both the grid and solar panels and balances the power supply and demand using real-time energy management. For simplicity, we only consider real-time traffic for streaming connections. Other types of traffic can be similarly examined.
The number of solar panels is allocated based on the variation in traffic within different data centers. In this case, the number of allocated batteries versus the average daily streaming traffic for a data center network is shown in Figure 2. This indicates that when the traffic within a data center network is increased, the number of batteries is also increased in order to use more green energy as well as to reduce the power costs. It is important that the marginal cost increases due to increasing battery costs is much smaller than the increasing costs related to a comparable fossil-based (nongreen) energy consumption scenario because the nonrenewable energy costs are typically much higher.
Figures 3(a) and 3(b) present the cost savings over a day (24 hours) versus the average number of streaming connections for the LP and Lyapunov optimization methods, respectively, when the total number of batteries in the data center network is 800. Figures 3(a) and 3(b) show that while considering a maximum amount of tolerable delay ( s) for video streaming applications according to [24], algorithms with real-time energy management (LDPP) can achieve significant power cost savings over a conventional system without real-time energy management (LP) for different average real-time connection densities . Further, the LDPP-based optimization performs much better than the LP-based optimization while meeting the QoS or maximum tolerable delay constraint, i.e., . These cost savings are due to the real-time stochastic nature of the Lyapunov-based optimization method which can improve the performance in environments with uncertainty (random user arrival processes) and nonlinear characteristics (nonlinear QoS constraints).
(a)
(b)
The complementary cumulative distribution function (CCDF) of violating the QoS constraint, , is shown in Figure 4 for both the LP and LDPP algorithms. Similarly, the CCDF of the probability of violating the processing constraint, , for both the LP and LDPP algorithms is illustrated in Figure 5. As expected, LDPP has a lower violation probability for both QoS and processing constraints compared to the LP algorithm. Figures 4 and 5 show that the difference between the two algorithms increases as the maximum tolerable delay and average numbers of connections increase.
Figures 4 and 5 indicate that, as the simultaneous average real time connection density increases, the CCDF (QoS) and CCDF (process) increase because of the added uncertainty in the system due to larger user traffic behavior variations. On the other hand, by increasing the maximum tolerable delay , for real-time connections, the CCDF (QoS) and CCDF (process) decrease because of lower QoS sensitivity of real-time applications.
Table 2 presents the time complexity of the conventional LP method and the proposed LDPP algorithm. This shows that the LDPP algorithm performs better than the LP method in terms of operational cost reduction. We define the operational cost reduction ratio for the proposed optimization (using LP and LDPP) as . Here, we mean by Conventional an ordinary data center network without employing optimized real-time energy management. The LDPP algorithm can save between 55% and 60% of the operational costs over the conventional system for different numbers of solar panels (SPs). This saving is decreased between 22% and 41% if we consider LP over the conventional system.
The computation time of the LDPP algorithm and LP approximation for different values of is shown in Table 2. We use the ratio of the LP optimization execution time for the proposed technique to that of the LDPP algorithm in order to compare the computational complexity, i.e., . The optimization time for each algorithm is averaged over 1000 optimizations. Table 2 shows this ratio versus the total number of solar panels, . This indicates that, in the best scenario, the computation time of the LP approach can be as small as 88.5% (100–11.5) that of the LDPP algorithm. Thus, the proposed approach can provide an operational cost saving versus complexity tradeoff for various total numbers of solar panels (). However, mean execution time for the LDPP algorithm is not much longer than the mean execution time for the LP algorithm. Thus, LDPP is superior to LP in terms of operational cost savings and has reasonable computational complexity.
6. Conclusion
In this paper, a Lyapunov drift-plus-penalty- (LDPP-) based algorithm was with green energy resources to improve the operational cost of data center networks. The LDPP technique can achieve significant power cost savings via minimizing the long-term average power cost under quality of service (QoS) constraints according to the drift-plus-penalty algorithm. Numerical results were presented which demonstrate the reduction in power cost and the robustness of the proposed LDPP-based technique. Moreover, the proposed method has reasonable computational complexity compared to the conventional LP approach. Further, the proposed LDPP method can deliver better quality of service compared to conventional techniques based on the CCDF metric. For future research, machine learning approaches (specifically deep learning) can be used to enhance the system performance in terms of convergence speed and overall cost savings.
Data Availability
The real-time price data used to support the findings of this study are available from [1], and other data are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The authors thank ITRC for its financial support during this research.