Energy Efficiency Strategy for Big Data in Cloud Environment Using Deep Reinforcement Learning

Pandey, Neeraj Kumar; Diwakar, Manoj; Shankar, Achyut; Singh, Prabhishek; Khosravi, Mohammad R.; Kumar, Vivek

doi:https://doi.org/10.1155/2022/8716132

Mobile Information Systems

On this page

Abstract Introduction Related Work Results and Discussion Conclusion Data Availability Conflicts of Interest References Copyright Related Articles

Research Article | Open Access

Volume 2022 | Article ID 8716132 | https://doi.org/10.1155/2022/8716132

Energy Efficiency Strategy for Big Data in Cloud Environment Using Deep Reinforcement Learning

Neeraj Kumar Pandey,¹Manoj Diwakar,²Achyut Shankar,³Prabhishek Singh,⁴Mohammad R. Khosravi,⁵and Vivek Kumar⁶

Academic Editor: Robin Singh Bhadoria

Received29 Dec 2021

Revised24 Feb 2022

Accepted22 Jun 2022

Published11 Aug 2022

Abstract

Big data entails massive cloud resources for data processing and analysis, which consumes more energy to run. The resources and tasks are increasing exponentially in the cloud environment for the processing of big data, which results in an increment in power consumption to run the cloud data center. So, there is always a scope for optimizing the energy utilization in cloud data centers. This paper presents a visionary architecture in a cloud environment for big data with a proposed energy-efficient strategy based on LSTM-DQN (long-short-term memory-deep Q network) using reinforcement learning (RL). The traditional techniques are not so efficient when the tasks are allocated dynamically, and the generic RL strategies are not able to store the data iterated in the last cycles of processing, so the LSTM is considered for this purpose. In the proposed model, integration of DPSO and DQN is used for better estimation and rectification of the curse of dimensionality. The proposed strategy is compared with different variants of PSO (particle swarm optimization) such as DPSO and QoS-PSO. The improvement in results through proposed model is recoded over the algorithm such as load aware (8.01%), DQN (13.36%), EA-DQN (34.16%), L-No-DEAF (15.62%), DPSO (62.68%), QoS-PSO (72.69%), FFO-EVSM (75.42%), and MIMT (76.39%) on the parameter of energy efficiency, tasks completion time, and energy consumption over the timeline. So, the proposed model is encouraging in the energy-efficient cloud environment for big data with the challenges that the technological world is facing and the emergence of deep learning as one propitious field.

1. Introduction

Big data is the most advanced data analysis field used to process massive data in a high-speed dynamic environment. In this scenario, the most advanced real-time resources are required to cope-up with the industrial requirement. Cloud computing has proven the most reliable computing paradigm of sustainable computing in big data. Big data requires massive storage, high bandwidth for real-time data streams, high-performance data analysis applications, and high-end visualization of data, so these needs cannot be fulfilled with ordinary computing infrastructure available on-premises.

Cloud computing technology is known for aggregating the physical resources in data centers and provides a single system view to each user in a fully virtualized environment. The vicinity of the cloud has increased exponentially in the last five years due to ad-hoc mobile [1] cloud services where a user is accessing cloud utilities through mobile devices. This mechanism has increased the challenges of the cloud (like resource allocation and scheduling, load balancing, and power management) and it requires new dynamic strategies to handle traditional optimization problems. Assignment of every single task consumes some resource over the cloud and that too consumes some power to run in the cloud server. So power management is always a hot spot for the researchers of cloud computing.

In last few years, swam intelligence has gained popularity in solving complex problem in diversified research areas and has given very efficient solutions for dynamic problems. There are many strategies, which come in the meta-heuristic category, which helps as individual or in integration with other nature-inspired approaches to solving complex problems of scheduling, load balancing, and resource migration. The integrated solutions are more focused as it may address multiple parameters of a single problem.

1.1. LSTM

LSTM is a special strategy used in combination with RNN to deal with vanishing gradient issues. The RNN can improve the results in every recurrence, but it cannot store the previous results required for future predictions. RNN works on the large sequence and it cannot carry the data from starting of the sequence to the later stages so LSTM is used to resolve this issue, which gives efficient results in RL with RNN. It is used in different optimization problems in cloud environment, specifically in energy efficiency [2], resource allocation [3, 4], VM migration [5], and load balancing [6]. The CIES (Cloud-based intelligent evaluation service) framework has been used for video homework using LSTN-CNN [7]. It is the tool used for the assessment of video assignments and classifies the data using different features captured from the videos. This is based on the classification; the qualified/unqualified category has been allotted in a cloud environment using LSTM.

1.2. DPSO

PSO is the most commonly used nature-inspired resource allocation strategy in the cloud environment for resource allocation. There are various formats and modifications of PSO used by different researchers with integration and with different algorithms. Integer-PSO, MPSO (modified-PSO), GA-PSO (genetic algorithm-PSO), DPSO (discrete PSO), EE-PSO (energy efficient PSO), RND-PSO (rounding–off PSO), SPV-PSO (smallest position value-PSO), EEDPSO (energy efficient dynamic-PSO), BPSO (binary-PSO), and MOPSO (multiobjective-PSO) are the most common PSO used in the management of resources of the cloud data center. This paper focuses on the DPSO for improving energy efficiency in resource allocation and scheduling. The major strategies used are PSO and its different combinations, but it does not look sufficient in the dynamic environment of cloud, so the RL is applied along with scheduling strategies so that more optimization can be achieved.

DPSO is used with the discrete and binary form of PSO in cloud environment for workflow scheduling. In DPSO every particle parameter is used to calculate the fitness of the particles. The major steps involved in the particle are as follows [8]:(1)Initialize the data center and dimension of all particles where the best previous position of each and all particles are Pbest and Gbest, respectively.(2)The objective fitness of all particle are calculated between the node points considering the distance and cost factor.(3)Updating the position of each particle is done depending on the number of generations.(4)Updating position of all particles and new values are assigned to Gbest.(5)Position and velocity of every particle are calculated and updated values that are assigned to corresponding variables.(6)The velocity and position values of particles are measured, which exceed the corresponding range.(7)If the termination criteria are not met, the process will be repeated from step 2. It depends on the number of generations is taken for consideration or if not achieved.

1.3. DQN

In deep Q network (DQN), RL plays a big role in capturing the features of dynamic resource allocation strategies in cloud environments. RL is used to integrate the state with the corresponding action so that immediate decisions can be triggered in a dynamic environment. This mapping function is framed to evaluate action value based on the reward expected. So the primary focus is on the process of resource scheduling with different parameters like throughput, makespan, energy efficiency, and VM migration. This process creates an abstracted layer between the CSP and the client. The primary interface of cloud does not disclose the strategy used for VM management and resource orchestration. In the proposed methodology, RL is combined with LSTM along with DQN. DQN is used to store transition data generated in training so that it can make the method more efficient due to the reusability of data. So, DQN makes the sample data selected in more wide way, which provides speed to the convergence and stability. In the paper [9], the job scheduling model is used using double DQN so that the overestimation can be reduced produced by DQN.

The above-mentioned strategies are used by different researchers in various optimization techniques, which motivates them to work in this area, where there is a lot of scope of optimizing the energy efficiency in the cloud environment for big data, specifically the paper [10–12].(i)The amalgamation of different static and nature inspired technique is used for reducing energy consumption in cloud with conservative energy reduction models.(ii)The RL based DQN and LSTM with DPSO is used in this paper for addressing the energy efficiency issue in cloud where LSTM helps to process the time stamp data with larger size and gap. It helps DPSO in handling such issues and making it more efficient with DQN.

The paper is organized in the following way as Section 2 describes the architecture of big data services in cloud environment. In Section 3, related work of different researchers is compared in table by considering technique used, parameter covered, goal, and simulating environment. In Section 4, proposed model for resource scheduling of cloud data center has been discussed, which includes LSTM, RL-based model, DQN structure, and DQN-DPSO algorithm. In Section 5, implementation and results have been discussed along with the comparison with different strategies. The Section 6 discusses the conclusion and future scope.

2. Architecture of Big Data Services in Cloud

The integration of two major computing paradigm (cloud and big data) can be a game-changer for all big players in industrial computing. The architecture of big data services in the cloud is shown in Figure 1. Big data provides a user ability platform for the distributed query processing for large datasets, while in cloud, an engine for the data processing in a distributed platform like Hadoop [13]. The architecture has three major parts i.e., data sources, cloud integration with big data platforms like Hadoop, data visualization, and decision-making. Data sources are the combination of data generation sources through cloud like real-time data streaming sources, web sources, etc. These sources work like raw material for the big data cloud platform. The big data platform started from the HDFS, which is a distributed file system and storage technology for Hadoop and used for storing unstructured data received from random data sources. Then, there is creation of databases for the unstructured data, which is an input for the distributed programming model along with algorithms used for the processing of large datasets. These data are then available for the query engine for filtering, analysis, and report generation. Finally, data analysis is used for data visualization for better understanding and decision-making. Different tools, API, and libraries are used to visualize extensive data generated on the big data platform over the cloud. Cloud computing provides not only the platform and resources for massive big data processing but also a dedicated service model for big data services and applications. There are specific big data services provided by all major cloud computing players like IBM, Google, Microsoft, Cloudera, and Salesforce. The major big data service on cloud platforms includes availability, data imports, NoSQL, machine learning, relational DBMS, MapReduce, and Hadoop.

In the paper [14], the author has state-of-the-art review of the energy efficiency strategy of big data in cloud environment. The contribution of high-performance computing (HPC) for big data analysis has been underlined with the help of supercomputer, cluster, and grid computing. In the paper [15], an energy efficiency technique in the cloud data center for big data has been proposed. Hadoop is the most famous platform for big data in cloud environment. It is highly available and known for services over a highly distributed environment for the high-speed data stream.

2.1. Challenge in Handling Big Data in Cloud Environment

Handling such a big volume of data in a real-time environment is itself a big challenge, and it gets multifold when handled in a cloud environment. The cloud challenges are also added with big data in this scenario. So the major challenges increase, like scalability, which is generally handled by cloud in every situation, but in this case, the actual test of this basic property of cloud has started. The volume of data in big data is too high that CSP can easily reach on the threshold, so big data services provided by cloud have to address this issue in particular. Different researchers [16–19] have projected many problems and challenges for big data in cloud environment, which respect to the energy efficiency using traditional and nature-inspired strategies. The major concerns include availability, security, storage, heterogeneity, privacy, data quality, energy efficiency, and data staging and scaling.

Storing and processing high volume data streams require high availability, robustness, and scalability in the system. So the cloud computing plays a significant role to provide all aforementioned services through virtualization for big data. Cloud is capable to provide all network based services in all its generic service models to big data in highly scalable environment. So, due to high demand of big data services in cloud environment, many cloud service providers are providing big data as a dedicated service in their service models. The major big data issues handled by cloud are security, storage, data handling policy, heterogeneity, network services, and disaster recovery policy.

Energy efficiency is a rich area of exploration and full of the possibility of optimization in cloud environment for big data services. Cloud datacenter is always in need of huge power because of high consumption of resources, scalability, and availability in a dynamic business environment. Every other computing paradigm is an option for researchers in integration with cloud computing for optimization of energy consumption in cloud datacenter. In the paper [20], an efficient energy-aware strategy for mobile edge computing (MEC) is used for the prediction of offloads. LSTM is used for the task prediction and connecting the MEC server. A three-layer architecture is proposed for the data transmission, energy efficiency of resource allocation, and communication. In the paper [21], an integration model of CNN-LSTM is proposed for resource utilization. The input data of energy efficiency are analyzed by regression and residual data are passed with CNN for analysis. The feature extraction of each VM is achieved after passing on the LSTM model.

This paper claims to achieve the accuracy from 3.8% to 10.9% and the error in the implementation is reduced by 7% to 8.5%. In Table 1, various energy-efficient strategies have been discussed using different simulating environment. They are satisfying different goals and covering many parameters. If the major contributions are analyzed, it can be concluded from Table 1 that most strategies are heuristic, optimization-based, and multiobjective. The focus of covering parameters includes SLA violation, energy efficiency, makespan, and task scheduling. Clousim, Python, and different public cloud environment are used for simulation and performance analysis for big data in cloud. In the paper [29], dynamic PSO is used for resource allocation with higher efficiency. The energy-efficient DPSO (EEDPSO) is used to optimize the energy utilization used in the resource allocation in dynamic environment for big data applications. The strategy is implemented through Google workload trace to minimize resource wastage and achieve better resource utilization. In the paper [30], the two-stage multitype PSO (TMPSO) is used for container consolidation in an energy-aware environment. The major focus of the paper is on energy-efficient, VM selection, and placement. The strategy was implemented on WS-dreams for validating a set of applications with different resources requirement. In the paper [31], DPSO and its variants (MOPSO, SBDMPSO, and APDPSO) are used for the load balancing, and time taken in communication is based on number of task executed. The CloudSim and MATLab were used for this. In the paper [32], a swarm optimization-based workload optimization (SOWO) strategy is proposed to improve the efficiency and performance of resources in cloud data centers. In the implementation part, the performance of the strategy is compared with the OpenStack scheduler and 50% less consumption of resources is recorded. In terms of handling big data, including cloud security, there are other researchers that show its major impact on other applications related to medical, satellite, and optical imaging field, IoT based cloud security, decision support system, etc.

The above methods and strategies have used the heuristic methods for energy efficiency. PSO work in iterative manner and it can process time series data, but if the time intervals are long so it may not store the previous data. This motivates us to propose LSTM-DQN model for energy efficiency, including reinforcement learning (DQN) with LSTM. In addition to this, it addresses the more energy consumption, no memory to store previous values, and not performing well for EE limitations mentioned in the Table 1. This models plays vital role to increase the efficiency using DPSO in cloud environment.

4. DQN-DPSO Model

The PSO is most widely used nature inspired strategy for different optimization technique. It follows very different way of sharing information, unlike ACO, GA, PIO, EHO, RCO, etc. These strategies do not have crossover and mutation operator. PSO and its variation algorithms update the operator values with the help of memory elements and internal velocity, which makes it more advantageous. The inbuilt guidance strategy of PSO helps to find more accurate and useful solution of its nearby solution and update itself accordingly in every iteration.

So, in this section, energy-efficient solutions have been provided to minimize the energy consumption by cloud data centers in resource scheduling. The proposed model will cover the LSTM (long short-term memory) based model for prediction and RL (reinforcement learning) based model. The major reason for using RL is because energy efficiency is a sequential decision problem due to time varying energy states. It entails precise model with temporal characteristic between sequential decisions where supervised/unsupervised learning strategies do not give predictable results because of dynamic work state and high computational cost. The supervised/unsupervised learning methodology is used in the problems where class type and underlying patterns are predictable, while EE does not come under this category, so the RL is used, in which learning agent works as a reward and action system. This plays a major role in the proposed model to optimize the existing methods mentioned in the result section (Algorithms 1 and 2).

4.1. LSTM-Based Model for Load Prediction

Resource scheduling in cloud data center requires calculating load of the node in the upcoming time slot. The summation of the load of every node can help to calculate the load of the data center. LSTM is a combination of LSTM units and RNN (recurring neural network). In RNN, hidden layers of the network is connected to the hidden layers of RNN. The most important properties lie in RNN that it can learn time series data efficiently. It has the ability to process the prior data during learning in the existing hidden layer. So, it can easily handle the dynamic nature of the time sequence. This property helps RNN to predict future directions using the past data values. If the time intervals between data are large, RNN cannot store the past data in an efficient way so to overcome this issue, LSTM is combined with RNN.

The proposed model is used to predict the load of the resource node for which a fixed time slot is along with using RNN. LSTM is used for load prediction when abnormal characteristics of data are found. At the time t₁ LSTM prediction model is given as

In the above equation, i_t is input at time t, o_t is an output unit at time t, and the status of RNN is determined by s_t (current status), while end time status is represented by h_t−1 at input i_t. X_h is a weight from input to hidden layer,Y_h is weight of self-circulation, d_h is used for representing deviation, and Δ_h and Δ_y are the activation functions. The time sequence represents data of RNN and network output layer data shows the prediction of time sequence in the next time interval.

4.2. RL-Based Model for Decision

Discrete particle swarm optimization (DPSO) generic algorithm is used for optimizing resource scheduling. There are P resource nodes with R tasks to be assigned. The distribution matrix and velocity matrix is given below to minimize the energy consumption while allocating the resource to the task, so the green computing can be achieved in the cloud data center.

In equation (2), A_ij represents that i^th task is assigned to j^th server, and A_ij ∈ {0, 1} and B_ij have the probability lies between 0 and 1 of the A_ij position. The particle speed b_ij the sig(b_ij) is mapped with b_ij between 0 and 1. Equation (3) is used to calculate the power consumed, which is in the focus of optimization. pow_ij shows that power require to i^th task executed on j^th server. The following fitness function (equation (4)) is used for the positions of the particles. The p_ij is the optimal matrix of the position of the swarm where k decides the number of iterations to be taken for positioning with α (weight factor). Here, c1 and c2 are the factors used in learning with ω.

DPSO has great ability for optimization of scheduling the resources in cloud environment, but when the resource and tasks are dynamic in nature, it will be very hard by DPSO to handle because it takes a long time to execute. It reduces the efficiency of the model of scheduling. So, RL is used to learn from every iteration of scheduling and accumulate experience. Q-learning is used to learn from some reward after some action on specific position. In each iteration, the reward value is updated and approached towards maximization, which results the output state. A table is used to record the rewards and actions on a specific state known as state-action (in combination). So, this Qtable is updated in each iteration using the following equation:where s is known as current state and a is the action, r is reward (noted in Qtable), is loss factor, ś is next state of action, and á is action taken in next position. In new state, LSTM is used for prediction and the task is used as input to RL-based decision model for nonlinear and distributed relationship. So, Q value from Qtable is used to select initial state of particles. Many other particles correspond to optimized solution in many other positions. This position can be formed by searching other particles. The following figure represents the stable structure of DQN (deep Q network). It is divided into convolutional layer and fully connected later to reach out to Qtable with updated Q value. This Q value is received on every iteration and maintained in Qtable so that the optimized (max.) values can be achieved. The Q value is achieved by equation (6), where Q function is measured in terms of status at time t and the action over it. So, it is Q(s, a), where a can be specific policy at status s i.e., a = π(s) while if there any change in the action, it will lead to the next state ś, so the updated action function will be like á = π(ś) and so on. And finally, Q value can be represented mathematically like

Here, (t) shows the total rewards achieved on the given time slot t and μ is the discount factor that lies between 0 and 1. μ = 0 shows the agents consider current rewards only, while μ ≤ 1 shows that the rewards are considered on later time slots also. As per Bellman’s equation, the Q learning is defined aswhere β is represented as learning rate. The Qtable contains all possible values generated by equation (7) but when the state-action increases, the convergence of Qtable is like a challenge. It generates new problem known as the curse of dimensionality [33]. So here, DQN structure (Figure 2) is used as a solution to this problem, where DQN is used as an alternative for better estimation, in which Conv 1 and Conv 2 are the convolutions, and FC denotes the fully connected.

The power consumption and CPU utilization can be represented using a linear relationship equations (8) and (9), in which Ƥ is used to calculate the total power consumed, is the total power used by server in idle position, is the used for the total power used in fully utilized position, and x is the CPU utilization. As per the study, total power consumption by idle servers are 60% more than the total power consumption by the server on threshold utilization.

The total energy consumption can be calculated using equation (10), which defines that the total energy is the integration of power consumption with time (from starting to till the CPU is in awake state).

In order to calculate the complexity of proposed model, n is number of particles, d is the number of dimension, dn is total number of complex multiplication for update the velocity, and Nn is total number of complex additions, which are required in every iteration in DPSO. So, the complexity of the model is represented as dNn + 4dn + 4 + (N − 1)n, where N is a size of block. The DQN behavior is like ensemble model for the evaluation of complexity, so the DQN-DPSO is complexity, which is totally based on the number of complex addition and multiplication per iteration and the size of blocks into the consideration. With this complexity RL agents deals with environment in discrete way and DPSO is used to support RL, whereas supervised/unsupervised learning cannot work in such scenario. RL have a capability to learn from the feedback in previous iteration, which makes it more efficient for energy efficiency problems.

	DPSO algorithm
(1)	t = 0
(2)	for (i = 1 to N)
(3)	()
(4)	Ē()
(5)
(6)
(7)	do{
(8)	for (i = 1 to N)
(9)
(10)
(11)	If ()
(12)
(13)	If ()
(14)
(15)
(16)	} while ()

	DQN algorithm
(1)	for (i = 1 to N)
(2)	Memory initialization
(3)	for (i = 1 to t)
(4)	for (i = 1 : N)
(5)	Prob. T and random action a_t
(6)	Set a_t = Q(s_t, a, θ)
(7)	Observe reward and state S_t+1
(8)	Add
(9)
(10)	Set θⁿ = θ in every iteration
(11)	End for
(12)	End for
(13)	Agent replay update
(14)	End for
(15)	End procedure

5. Results and Discussion

In this paper, energy-efficient LSTM-RL based cloud computing model is proposed. The proposed model is implemented and verified using ClodSim. So the RL model of prediction is implemented with DPSO using CloudSim stimulating environment. CloudSim provides features of resource scheduling and management. The large scale cloud cluster can display and test on CloudSim using single machine and processing elements. The simulating environment parameter is given in Table 2 contains 50 VM’s. Every VM’s has same configuration and MIPS speed (i.e., 2500 MIPS).

The weight factor value (i.e., 0.5) is based on the capacity of local and global search. In the entire process, the value of inertia is decreased gradually with some constant difference (0.05), which results slow down the particle velocity and make it converge in an easier and linear way. There are various impractical studies [34] available for optimal particle size for getting most useful results, which is in the range of 10 to 30. So the particle size is 25 and α = 0.5 in the proposed model of DPSO is used.

Based on the configuration stated in Table 2 the simulation is performed with all parameters. A similar dataset is used to train the prediction model using RL. Table 2 has all parameter list used for simulating DPSO in cloud environment using CloudSim. The load aware resource allocation algorithm is used to compare the proposed model of RL-based resource allocation algorithm in cloud computing for big data. The following couple of algorithms are used for the proposed model, in which the DQN-DPSO is used to improve energy utilization during the execution of big data tasks in cloud environment. The DQN improves the utilization in each iteration, which can store the data for the improvement in every cycle. Figures 3(a) and 3(b) represent the total energy consumed using CloudSim for three different resource scheduling algorithm. In Figure 3, the x-axis shows the size of requested tasks and the y axis shows the power consumption for the task in execution. For observation, the size of the task is arranged in increasing order. The proposed algorithms are compared with traditional DQN and load aware algorithms. Through the above figure, the performance of DQN-DPSO strategy can be easy analyzed. The two set of task is implemented for better understanding, comparison, and visualization. In Figure 3(b), the task set and the number of algorithms have been increased for better view of comparison and understanding. It shows that as the number of tasks increases the DQN-DPSO performs better in comparison to the DQN and load aware algorithm in big data.

(a)

(b)

In Figure 3(a), energy consumption of DQN-DPSO is shown (round-1) with the comparison with load aware, EA-DQN, and DQN algorithms [35]. As the task size increases, the power consumption also get increases (per task), but DQN-DPSO has minimum average energy consumption per unit task i.e., 0.3746 while load aware, EA-DQN, DQN, FFO-EVMM (firefly optimization-energy aware virtual machine migration), and MIMT (minimization of migration based on Tesa) have 0.4072, 0.5690, 0.4325, 1.489, and 1.7238, respectively. In the next cycle (round-2), number of tasks has been increased to check the performance of the proposed model (as shown in Figure 3(b). These two rounds of evaluation are considered for Case-1. The comparison of the proposed model is also being made with the load aware strategy and DQN. It has optimized performance as the average energy consumption per unit time of DQN-DPSO is 0.4596 and Load- Aware strategy, DQN, FFO-EVMM, and MIMT have 0.4739, 0.4850, 1.489, and 1.7238, respectively.

Big data applications have a great impact on the resources due to high volume and unstructured content, so it is necessary to store the previous data for the cleaning process. So, DQN supports in this way and DPSO has more impact on the energy-aware resource selection. DQN-DPSO makes a unique integration to meet the big data requirements so that the cloud resources may be used in such a way. This method optimizes the energy efficiency of strategies and is able to allocate resources to efficient processes. Every public cloud environment is providing dedicated big data services so this methodology may help them to optimize the energy efficiency so that it can deal with the hindrance of process allocation and save energy in the entire mechanism.

Another comparative study has been done among the proposed strategy and DQN, Worn-Dear [36], and L-No-Deaf on the parameter of energy efficiency with DQN-DPSO proposed model. In Figure 4, it is shown that DQN-DPSO has less energy consumption as compared to DQN, Worn-Dear, L-No-Deaf, FFO-EVSM, and MIMT. These strategies and considered in literature survey for the comparison and motivation for proposed model (Case-2). Similarly, in Case-3, the completion of execution time as a parameter is also taken for the proposed model, which is compared with different variations of PSO, DBC, and EDF (Figure 5) [37]. The DQN-DPSO performs better than other compared strategies as the time of completion increases when the number of tasks increases because of big data applications specifically like EA-PSO, MIMT, and FFO-EVMM. So the performance of the proposed strategy is performing better than other integrated variants of PSO for big data cloud applications.

The improvement is shown in Figures 3–5 can be analyzed with the help of following Tables 3 and 4. The improvements are calculated with respect to the DQN-DPSO strategy over other algorithms. The proposed model has value 41.21 and 74 for round-1 and round-2, respectively, calculated in Case-1. In the next case (Case-2), a significant improvement has been recorded as compared to other proposed model i.e., 108 and in Case-3 the completion time is 2810 per second. The detailed improvement against all compared algorithms is shown in Tables 3 and 4.

The proposed model with PSO is extended with FIS for not getting premature convergence and local minimums, so that it may not lead to unexpected level of optimization but in proposed system, PSO is getting more optimal value in efficiency, which is not significantly increased in extended strategy with called FISDPSO. The improvement recorded is not so significant so it is not discussed in the results section but shown in the Tables 3 and 4. While two more strategy FFO-EVSM and MIMT is also compared with proposed model and it shows the significant improvement, i.e., 75.42% and 76.39%.

6. Conclusion and Future Scope

As the green cloud is today’s computing requirement, there is a need for an intelligent self-learning strategy used with traditional optimization problems for big data in cloud environment. This paper focuses on the usage of energy while executing the task in cloud perspective for big data. The efficient integration of intelligent strategies (DQN and DPSO) gives the solution where less power is required for resource allocation issues. CloudSim provides such features for simulation so that cluster systems can run in a simulated environment and DQN strategy can be integrated with DPSO. The results can be compared with traditional load aware algorithms for big data. The performance of DQN-DPSO is found more efficient for two different set of task in execution. It shows that the proposed model provides an energy-efficient solution to the resource allocation problem in the green cloud environment. As the future strategy, more reinforcement learning can be explored more for the development of integrated strategies for energy efficiency for big data in the cloud environment. The traditional and streamlined algorithms may not perform well as the challenges in big data is increasing every day in terms of velocity and volume, so there is a need of the integration of multiple optimized algorithms to meet the challenges of present big data applications in cloud data centers. The energy efficiency can be used in the industrial strategy of energy optimization in cloud based industrial control system (especially during scheduling and load balancing of resources). This helps the industrial cloud and private cloud deployments to save energy in VM migration during handling of big data (e.g., AWS, Eucalyptus, and OpenNebula).

Data Availability

The conclusion and comparison data of this article are included within the article. For inquiries regarding raw data and codes, please contact the first author (email: [email protected]).

Conflicts of Interest

The authors declared that they have no conflicts of interest.

References

D. L. Van and C. K. Tham, “A deep reinforcement learning based offloading scheme in ad-hoc mobile clouds,” in Proceedings of the IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 760–765, IEEE, Honolulu, HI, USA, April 2018.
View at: Publisher Site | Google Scholar
R. Yazdani, O. Ruwase, M. Zhang, Y. He, J. M. Arnau, and A. Gonzalez, “LSTM-sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory,” 2019, https://arxiv.org/abs/1911.01258.
View at: Google Scholar
Y. Liu, L. L. Njilla, J. Wang, and H. Song, “An LSTM enabled dynamic stackelberg game theoretic method for resource allocation in the cloud,” in Proceedings of the 2019 International Conference on Computing, Networking and Communications (ICNC), pp. 797–801, IEEE, Honolulu, HI, USA, February 2019.
View at: Publisher Site | Google Scholar
S. Vakilinia, “Energy efficient temporal load aware resource allocation in cloud computing datacenters,” Journal of Cloud Computing, vol. 7, no. 1, p. 2, 2018.
View at: Publisher Site | Google Scholar
H. Ren, Y. Wang, C. Xu, and X. Chen, “SMig-RL,” ACM Transactions on Internet Technology, vol. 20, no. 4, pp. 1–18, 2020.
View at: Publisher Site | Google Scholar
R. K. Ramesh, H. Wang, H. Shen, and Z. Fan, “Machine learning for load balancing in cloud datacenters,” in Proceedings of the 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pp. 186–195, IEEE, Melbourne, Australia, May 2021.
View at: Publisher Site | Google Scholar
R. Song, Z. Xiao, J. Lin, and M. Liu, “CIES: cloud-based Intelligent Evaluation Service for video homework using CNN-LSTM network,” Journal of Cloud Computing, vol. 9, no. 1, p. 7, 2020.
View at: Publisher Site | Google Scholar
J. Liu, T. Shi, and P. Li, “Optimal cloud storage problem in the distributed cloud data centers by the discrete PSO algorithm,” in Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC), pp. 156–163, IEEE, Sendai, Japan, May 2015.
View at: Publisher Site | Google Scholar
B. Han and J. Yang, “Research on adaptive job shop scheduling problems based on dueling double DQN,” IEEE Journals & Magazine | IEEE Xplore, IEEE, vol. 8, 2020.
View at: Google Scholar
A. Iqbal, M.-L. Tham, and Y. C. Chang, “Double deep Q-network-based energy-efficient resource allocation in cloud radio access network,” IEEE Access, vol. 9, Article ID 20449, 2021.
View at: Publisher Site | Google Scholar
M. Siddaraju and Dr. K. Siddaraju, “Energy efficient dynamic particle swarm optimization (EEDPSO) resource allocation in cloud computing,” Webology, vol. 17, no. 2, pp. 128–149, 2020.
View at: Publisher Site | Google Scholar
K. Oukfif, L. Bouali, S. Bouzefrane, and F. Boumghar, “Energy-aware DPSO algorithm for workflow scheduling on computational grids,” in Proceedings of the 2015 3rd International Conference on Future Internet of Things and Cloud, pp. 651–656, IEEE, Rome, Italy, August 2015.
View at: Publisher Site | Google Scholar
I. A. T. Hashem, I. Yaqoob, N. B. Anuar, S. Mokhtar, A. Gani, and S. Ullah Khan, “The rise of “big data” on cloud computing: review and open research issues,” Information Systems, vol. 47, pp. 98–115, 2015.
View at: Publisher Site | Google Scholar
EC. Inacio and M. Dantas, “A survey into performance and energy efficiency in HPC, cloud and big data environments,” International Journal of Networking and Virtual Organisations, vol. 14, no. 4, pp. 299–318, 2014.
View at: Publisher Site | Google Scholar
F. Mehdipour, H. Noori, and B. Javadi, “Chapter two—Energy-efficient big data analytics in datacenters,” Advances in Computers, vol. 100, pp. 59–101, 2016.
View at: Publisher Site | Google Scholar
A. Kobusińska, C. Leung, C.-H. Hsu, S. Raghavendra, and V. Chang, Emerging Trends, Issues and Challenges in Internet of Things, Big Data and Cloud Computing, Elsevier, Amsterdam, Netherland, 2018.
A. Dhabhai and Y. K. Gupta, “A study of big data in cloud environment with their related challenges,” International Journal of Engineering Sciences, vol. 25, p. 11, 2016.
View at: Google Scholar
S. Subbalakshmi and D. K. Madhavi, “Security challenges of big data storage in cloud environment,” Surveyor, vol. 13, no. 17, p. 8, 2018.
View at: Google Scholar
F. Mehdipour, H. Noori, and B. Javadi, “Energy-efficient big data analytics in datacenters,” Advances in Computers, vol. 100, pp. 59–101, 2016.
View at: Publisher Site | Google Scholar
G. Wu, Y. Miao, Y. Zhang, and A. Barnawi, “Energy efficient for UAV-enabled mobile edge computing networks: intelligent task prediction and offloading,” Computer Communications, vol. 150, pp. 556–562, 2020.
View at: Publisher Site | Google Scholar
S. Ouhame, Y. Hadi, and A. Ullah, “An efficient forecasting approach for resource utilization in cloud data center using CNN-LSTM model,” Neural Computing & Applications, vol. 33, no. 16, Article ID 10055, 2021.
View at: Publisher Site | Google Scholar
N. Somu, G. R. M R, and K. Ramamritham, “A hybrid model for building energy consumption forecasting using long short term memory networks,” Applied Energy, vol. 261, Article ID 114131, 2020.
View at: Publisher Site | Google Scholar
M. Yakhchi, S. M. Ghafari, S. Yakhchi, M. Fazeli, and A. Patooghi, “Proposing a load balancing method based on Cuckoo Optimization Algorithm for energy management in cloud computing infrastructures,” in Proceedings of the 2015 6th International Conference on Modeling, Simulation, and Applied Optimization (ICMSAO), pp. 1–5, IEEE, Istanbul, Turkey, May 2015.
View at: Publisher Site | Google Scholar
B. Patel and H. Patel, “Multi-objective optimization oriented policy for performance and energy efficient resource allocation in Cloud environment,” Journal of King Saud University - Computer and Information Sciences, vol. 32, no. 7, pp. 860–869, 2020.
View at: Publisher Site | Google Scholar
S. Murali and M. Murali, “RETRACTED ARTICLE: Deep learning based energy efficient novel scheduling algorithms for body-fog-cloud in smart hospital,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 7, pp. 7441–7460, 2021.
View at: Publisher Site | Google Scholar
M. H. Askarizade, M. Maeen, and M. Haghparast, “An energy-efficient dynamic resource management approach based on clustering and meta-heuristic algorithms in cloud computing IaaS platforms,” Wireless Personal Communications, vol. 104, no. 4, pp. 1367–1391, 2019.
View at: Publisher Site | Google Scholar
S. Y. Zahedi Fard, M. R. Ahmadi, and S. Adabi, “A dynamic VM consolidation technique for QoS and energy consumption in cloud environment,” The Journal of Supercomputing, vol. 73, no. 10, pp. 4347–4368, 2017.
View at: Publisher Site | Google Scholar
M. Khorsand and R. Khorsand, “Energy-aware scheduling algorithm for time-constrained workflow tasks in DVFS-enabled cloud environment,” Simulation Modelling Practice and Theory, vol. 87, pp. 311–326, 2018.
View at: Publisher Site | Google Scholar
M. Bala Krishna, “Hybrid evolutionary algorithm based task scheduling mechanism for resource allocation in cloud environment,” Revista Gestão Inovação e Tecnologias, vol. 11, no. 4, pp. 194–209, 2021.
View at: Publisher Site | Google Scholar
T. Shi, H. Ma, and G. Chen, “Energy-aware container consolidation based on PSO in cloud data centers,” in Proceedings of the 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8, IEEE, Rio de Janeiro, Brazil, July 2018.
View at: Publisher Site | Google Scholar
Z. Miao, P. Yong, Y. Mei, Y. Quanjun, and X. Xu, “A discrete PSO-based static load balancing algorithm for distributed simulations in a cloud environment,” Future Generation Computer Systems, vol. 115, pp. 497–516, 2021.
View at: Publisher Site | Google Scholar
J. Yan, H. Zhang, H. Xu, and Z. Zhang, “Discrete PSO-based workload optimization in virtual machine placement,” Personal and Ubiquitous Computing, vol. 22, no. 3, pp. 589–596, 2018.
View at: Publisher Site | Google Scholar
A. Koppel, G. Warnell, E. Stump, P. Stone, and A. Ribeiro, “Policy evaluation in continuous MDPs with efficient kernelized gradient temporal difference,” IEEE Transactions on Automatic Control, vol. 66, no. 4, pp. 1856–1863, 2021.
View at: Publisher Site | Google Scholar
A. P. Engelbrecht, Computational Intelligence: An Introduction, John Wiley & Sons, NJ, USA, 2007.
A. Oroojlooyjadid, M. Nazari, L. Snyder, and M. Takáč, “A Deep Q-Network for the Beer Game: A Deep Reinforcement Learning Algorithm to Solve Inventory Optimization Problems,” 2020, https://arxiv.org/abs/1708.05924.
View at: Google Scholar
P. Prabukumar and M. Prabukumar, “Design and implementation of energy efficient reconfigurable networks (WORN-DEAR) for BAN in IOT environment (BIOT),” International Journal of Reasoning-Based Intelligent Systems, vol. 10, no. 3/4, p. 258, 2018.
View at: Publisher Site | Google Scholar
W. Jing, C. Zhao, Q. Miao, H. Song, and G. Chen, “QoS-DPSO: QoS-aware task scheduling for cloud computing system,” Journal of Network and Systems Management, vol. 29, no. 1, p. 5, 2020.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Neeraj Kumar Pandey et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

452

Downloads

451

Citations

Mobile Information Systems

Energy Efficiency Strategy for Big Data in Cloud Environment Using Deep Reinforcement Learning

Abstract

1. Introduction

1.1. LSTM

1.2. DPSO

1.3. DQN

2. Architecture of Big Data Services in Cloud

2.1. Challenge in Handling Big Data in Cloud Environment

3. Related Work

4. DQN-DPSO Model

4.1. LSTM-Based Model for Load Prediction

4.2. RL-Based Model for Decision

5. Results and Discussion

6. Conclusion and Future Scope

Data Availability

Conflicts of Interest

References

Copyright