Abstract
Facing more and more severe global warming problems, renewable energy, as an alternative to traditional fossil fuels, is attracting more and more attentions due to its capability of reducing carbon emission. This paper considers twotier HetNets with orthogonal frequency division multiple access (OFDMA), where the macro base station (MBS) is powered by power grid and small base stations (SBSs) have hybrid energy supplies. Through smart grid, SBSs can share their renewable energy with each other. We consider the problem of crosslayer interference caused by spectrum reuse, the burst of user data, and the randomness of renewable energy arrivals. Through energy cooperation, this paper investigates maximizing the timeaverage energy efficiency of SBSs. Based on user data queue and SBS energy queue, the optimal problem is decoupled into two subproblems by Lyapunov optimization: resource allocation subproblem and energy scheduling and energy cooperation subproblem. By solving two subproblems, the online solution to the optimization problem is obtained. Through theoretical analysis and simulations, both user data queues and energy queues have an upper bound, the network is stable, and the proposed algorithm performs better than the basic algorithm without energy cooperation.
1. Introduction
With the dramatic increase of smart devices, a high burden will be posed on the fifthgeneration (5G) wireless networks, which are expected to provide user equipment (UE) with ubiquitous service. To achieve higher capacity and better coverage, more base stations (BSs) are deployed, which means higher energy consumption. As claimed in [1], energy consumption of information and communication technology (ICT) occupies 10% of world’s total energy consumption. Concerned about the environment and energy saving, operators are exploring energy harvesting technology swhich is able to harvest renewable energy source (RES) by solar panels and wind turbines.
It is widely known that, in order to meet the requirements of 5G wireless networks, the heterogeneous network (HetNet), where various small cells are densely underlaid in a macro cellular network, is a promising technique to achieve more spectrumefficient and energyefficient communications. HetNets significantly boost the overall spectral efficiency and energy efficiency through a full spatial resource reuse. However, the severe intralayer interference among dense macro base stations (MBSs) and crosslayer interference between small cell base stations (SBSs) and MBSs are challenging the successful commercial operations of HetNets. In recent years, a great emphasis has been given on advanced techniques that take interference control, resource allocation, and selforganization into account to enhance both the spectral efficiency and energy efficiency [2].
On the other hand, the rapid developments in energy harvesting technologies have made the dream of selfsustaining devices and BSs potentially possible. As such, energy harvesting is highly desirable for prolonging the battery life and improving the energy efficiency of networks, especially for HetNets with lowpower SBSs [3, 4]. Reference [5] summarized recent contributions in energy harvesting wireless communications and, reference [6] presented an overview of the past and recent developments in energy harvesting wireless communications and highlighted a number of possible future research directions. Particularly, for energy supply by RES, some works considered the solely RES powered communication networks [7–9]. However, owing to the dynamics of renewable energy generation and limited capacity of energy storage, sole RES may not guarantee enough power supplies for all kinds of BSs. Hence, HetNets are more likely to adopt hybrid energy supplies with both traditional power grid and RES [3–5], and the works in [10–13] focused on resource allocation in hybrid energy powered cellular networks, where the power grid is needed to provide reliable energy to BSs.
Then it is noted that, with its ongoing paradigm shift from traditional grid to smart grid, the power grid also provides new opportunities for BSs’ cost saving. Since smart grid enables both twoway information and energy flows between the grid and end users, the energy cooperation in cellular networks which allows the BSs to trade and share their unevenly harvested energy to support nonuniform wireless traffic in a costeffective way is emerging [14]. Reference [15] studied energy group buying with load sharing, where two network operators are aggregated as a single group to make the dayahead and realtime energy purchase and their BSs share the wireless traffic at the same time. Reference [16] proposed a hybrid energy sharing framework for cellular network, where a combination of physical power lines and energy trading with other BSs using smart grid is used. In [17], to minimize the ongrid power consumption, the BS operation and the power distribution were jointly optimized. Reference [18] presented a contemporary review of recent breakthroughs on the utilization, redistribution, trading, and planning of energy harvested in future wireless networks interoperating with smart grids. In a word, the above works in [14–17] on energy cooperation and resource allocation emphasized on energy planning and/or energy trading through smart grid, while the traffic coordination or resource allocation was performed on a large timescale.
In this work, we focus on the smalltimescale resource scheduling and energy cooperation in a HetNet powered by hybrid energy. Particularly, the random renewable energy arrivals, the inherent crosslayer interference constraint, and the burst of user data in the twotier HetNet are considered. With the objective of maximizing the timeaverage energy efficiency of SBSs, the transmission power allocation, the subcarrier assignment, the energy scheduling, and energy cooperation among SBSs are jointly optimized. The main contributions of this paper are summarized as follows:(1)Firstly, considering a twotier downlink HetNet with hybrid energy supplies, we jointly optimize the resource allocation, energy scheduling, and energy cooperation with the burst of user data, the crosslayer interference constraints, and the random renewable energy arrivals.(2)Secondly, a stochastic optimization problem that maximizes the timeaverage energy efficiency is modelled for tradeoff between user throughput improvement and ongrid energy consumption reduction. Leveraging Lyapunov optimization technique, an online algorithm that decides transmission power allocation, subcarrier assignment, discharging and charging of the storage devices, energy cooperation among SBSs, and ongrid energy consumption is developed after several decompositions.(3)Thirdly, both theoretical analysis and simulation results are presented to show the stability and superiority of the proposed online resource scheduling and energy cooperation algorithm.
The rest of this paper is organized as follows. In Section 2, we illustrate the system model that contains network model, traffic model, data queue model, energy supply and demand model, and energy queue model. Section 3 shows the formulation of the stochastic optimization problem and its transformation based on Lyapunov optimization method. An online algorithm for transmission power allocation, subcarrier assignment, energy scheduling, and energy cooperation is developed in Section 4. Performance analysis is given in Section 5. Section 6 presents simulation results and discussions. Finally, Section 7 concludes the whole work.
2. System Model
2.1. Network Model
We consider the downlink transmission of an OFDMA twotier HetNet system as shown in Figure 1. MBS of the first tier provides basic coverage and SBSs of the second tier enhance the capacity of the whole network. In this system, MBS is powered by power grid, while SBSs are powered by hybrid energy supply containing both power grid and RES. SBSs are connected through smart grid and they can exchange renewable energy through smart grid, so that the unbalanced renewable energy sources can be effectively utilized to match with the uneven user data.
In the system, one MBS with M active MUEs corresponds to SBSs, and each SBS has active SUEs. Here, MUEs, SBSs, and SUEs are denoted as indexed by , indexed by , and indexed by , respectively. All SBSs only provide service to the preregistered SUEs as SBSs have a closed access. We assume that all SBSs are connected to the MBS via high speed backhaul with negligible delay. To match with the realworld scenario of suburban areas, we consider sparse deployed small cells. So we could ignore the cotier interference between SBSs.
As wireless channels and renewable energy arrivals change dynamically, the transmission between BSs and UEs could be regarded as slotbased transmission. Meanwhile, during each time slot, the wireless channels and renewable energy arrivals remain stable. We consider the problem of a fixed scheduling time containing time slot, denoted as .
2.2. Traffic Model
This paper considers the downlink transmission of a twotier HetNet with orthogonal frequency division multiple access (OFDMA). In the network, the total bandwidth is equally divided into subcarriers, denoted as . To avoid intracell interference, at time slot , subcarrier can be allocated to zero or one SUE in each small cell. The allocation policy of subcarrier at time slot is denoted as , whose value represents that subcarrier is allocated to SUE in small cell if it equals 1 and otherwise if it equals 0. Then, we have
is used to denote the power allocated by SBS to SUE on subcarrier . Limited by hardware, each SBS has its energy consumption upper bound, denoted as . Depending on subcarrier allocation and power allocation, the total energy consumption of SBS at time slot cannot exceed the aforementioned upper bound. This restriction is denoted aswhere is the static energy consumption of SBS related to cooling and baseband processing.
For wireless channel, denotes the channel gain of SUE from SBS on subcarrier , denotes the interference channel gain of MUE served by MBS from small cell on subcarrier , and denotes the interference channel gain of SUE in small cell from MBS on subcarrier . Generally, the channel gain contains two parts: one is the largescale fading component with path loss and shadowing, and the other is the smallscale fading component with multipath Rayleigh fading. is used to denote the power allocated by MBS to MUE on subcarrier at time slot . So the received SINR of SUE served by SBS and occupying subcarrier at time slot is denoted aswhere is the background noise.
Meanwhile, the achievable rate on subcarrier of SUE served by SBS at time slot is defined as
To reuse subcarrier of MBS to transmit information, the subcarrier has a tolerable interference level , meaning that the interference on subcarrier cannot exceed the interference threshold. Then, we have
2.3. Data Queue Model
Each SBS has an independent buffer for every one of its preregistered SUEs. The arriving data is first stored in the buffer and then transmitted to the corresponding SUE. We use to denote the arriving data for SUE at time slot , where SUE is severed by SBS . The arriving data of SUE in SBS can be modelled as an independently and identically distributed (i.i.d) process with a mean of . At each time slot, the arriving data of SUE cannot exceed the maximum arriving data .
Let denote the data backlog of SUE in SBS at time slot . Based on the admitted data rate and service rate of each SUE, the data backlog of SUE in SBS at time slot can be updated aswhere . We first give the definition of queue stability.
Definition 1. (see [19]). When a discretetime process , it is strongly stable if it satisfiesDepending on rate stability theorem [20], the queue is ratestable when holds, where and . Moreover, when all individual queues of the network are strongly stable, a network is strongly stable.
According to the stability theorem, the queue is stable if holds, where and .
2.4. Energy Supply and Demand Model
We use to represent the renewable energy collected by SBS at time slot , which is based on the i.i.d stochastic process. It should be noted that there is no correlation between renewable energy generations among SBSs. In this paper, the collected renewable energy is firstly used by the SBS and then the remaining energy is stored in the storage device. At time slot , there are two cases considering the relationship between the energy consumption of SBS and the renewable energy arrival of SBS .(a): At time slot , the renewable energy collected by SBS cannot satisfy its energy consumption. For the insufficient part, it is denoted as , which can be satisfied by three ways:(1)Obtained from its storage device. At time slot , SBS gets from its storage device for the insufficient part of renewable energy.(2)Energy cooperation among SBSs. Due to the energy cooperation between SBSs, SBS can use the surplus renewable energy of SBS , which is denoted by . The energy taken from other SBSs has upper bounds, and the maximum energy available from SBS is .(3)Getting from the power grid. When the renewable energy getting from its storage devices and other SBSs cannot meet the energy demand, the SBS can get energy from the power grid. The amount of energy getting from power gird is expressed as . By the above three ways, the insufficient energy of SBS can be satisfied, which means(b): At time slot , the renewable energy collected by SBS can satisfy or exceed its energy consumption. The renewable energy collected is first used to meet its energy consumption. For the remaining renewable energy, which is denoted as , there are two applications:(1)Sharing with other SBSs. The renewable energy sharing from SBS to SBS at time slot is denoted by .(2)Stored in its storage device. At time slot , the stored energy of SBS is denoted as , where is the maximum storage renewable energy and . Thus, at time slot , we have
2.5. Energy Queue Model
At each time slot , the remaining renewable energy of the storage device of SBS is denoted by . Since the energy changes of the storage device at each time slot are mainly the charging process and the discharging process, we can describe this as an energy queue whose updating process is
At each time slot , the discharging of storage device can neither be more than its remaining energy nor exceed the maximum discharging limit . Therefore, we get the following constraint:
Similarly, at each time slot , the charging of storage device can neither exceed the remaining battery capacity nor exceed the maximum charging limit . Thus, we have
According to (10), by taking iterated expectation and using telescoping sums over , we get
We further divide on both sides of (13), where . According to the stability theory, the energy queue is stable due to and , where is the timeaverage discharging of SBS and is the timeaverage charging of SBS .
For clarity, the important notations of this paper are listed in Table 1.
3. Problem Formulation and Transformation
3.1. Problem Formulation
Taking into account the random arrivals of each SUE’s traffic data and each SBS’s renewable energy, our optimization goal is to minimize timeaverage ongrid energy consumption while maximizing the timeaverage throughput of all SUEs. At the same time, it is also needed to meet the network stability constraints, the subcarrier allocation constraints, and the crosslayer interference constraints. Then, we formulate the following optimization problem:whereand it is SUE ’s timeaverage service rate, andand it is SBS ’s timeaverage ongrid energy consumption.
We use to denote the optimal objective value of P1. In P1, the control variables , , , and denote the subcarrier assignment, transmission power control on each subcarrier, SBS’s discharge control, and energy sharing, respectively. and are nonnegative constants, which are used to balance the throughput and ongrid energy consumption. C1 indicates that one subcarrier can be allocated to at most one SUE in one SBS. C2 indicates an index of subcarrier allocation. C3 is the transmission power consumption constraint of SBSs. Constraint C4 is the tolerable crosslayer interference for each subcarrier occupied by SUEs. C5 is the stability constraint of data and energy queues. C6 is the compensation component of the insufficient renewable energy. C7 is the allocation of extra renewable energy for each SBS. C8 and C9 are discharging and charging constraints of each SBS, respectively. C10 is the renewable energy sharing constraint.
3.2. Problem Transformation
It is observed that, in P1, the decision variables , , , and are coupled together. We decompose the optimization problem P1 into joint resource allocation and energy scheduling by the Lyapunov optimization method in each time slot. The design principle of Lyapunov optimization method is to minimize the upper bound of Lyapunov drift minus objective function greedily without knowing the profile of stochastic events.
and are used to represent the energy queue and the data queue, respectively. Then, we use to denote all queues in the network system. At each slot , we define the Lyapunov function aswhere is a perturbation factor, which ensures that there is enough energy in the energy queue. Without loss of generality, we assume that all queues are empty when such that .
The Lyapunov function is a scalar measure of network congestion. Intuitively, if is small, then all queues are small; and if is large, then at least one queue is large. Thus, by minimizing the Lyapunov drift (i.e., by minimizing a difference in the Lyapunov function from one time slot to the next), the data queues and energy queues can be stabilized.
By using expression in (17), the Lyapunov drift can be written asand, to solve optimization problem P1, we now use the driftpluspenalty minimization method [21], where a control policy that solves P1 is obtained by minimizing the upper bound of the following driftpluspenalty term:where is a nonnegative control variable that represents the relative importance of minimizing the queue length and maximizing the objective function. According to [21], when is sufficiently large, the optimization algorithm approaches the optimal policy.
The upper bound of the driftpluspenalty can be derived as follows.
Lemma 1. The upper bound of driftpluspenalty in (19) iswhere is a positive constant that satisfies the following inequality constraint for all time slots:he proof of Lemma 1 is shown in Appendix A.
By Lemma 1, we can transform the optimization problem P1 into minimizing the rightside term of (20) at each time slot . Thus, we will decompose the optimization problem P1 and present an online dynamic control algorithm for the resource allocation and the energy scheduling and energy cooperation, respectively.
4. Online Solution
The proposed problem is solved online based on two subproblems. The resource allocation subproblem determines the subcarrier assignment and the transmission power allocation on each subcarrier. The energy scheduling and energy cooperation subproblem determines the discharging and charging, the energy cooperation, and ongrid power consumption of each SBS. Solving the above two subproblems in each time slot can finally determine the online resource allocation scheme and the energy scheduling and the energy cooperation scheme, so as to maximize the timeaverage energy efficiency of SBSs.
4.1. Resource Allocation Subproblem
The resource allocation subproblem jointly considers transmission power control and subcarrier allocation. At the beginning of each time slot, we can solve the resource allocation subproblem if we get the subcarrier status and queue information. The resource allocation subproblem is given below:
Problem P2 is an NPhard mixedinteger programming problem. To solve this problem, we extend the binary variable to the continuous variable . At the same time, we introduce a new variable . Ignoring the static power consumption, problem P2 can be rewritten as
Since the optimization problem P3 is a convex optimization problem, it can be solved in polynomial time. Here, we consider using the Lagrange dual method to solve problem P3. The Lagrange function iswhere are Lagrange multipliers for constraints , respectively. Further, we have the dual function as follows:
The dual problem is given as
The dual problem (26) can be further decomposed into independent subproblems, making each SBS independently decide on each subcarrier. Therefore, problem (26) is equivalent towhere
For the solution of , we first introduce the Lagrange multiplier for the constraint . Then, we find the partial differential function of with respect to . Then, we havewhere .
In order to obtain the optimal transmission power control for the resource allocation subproblem, the above problem (27) must satisfy its KarushKuhnTucker (KKT) conditions, which are denoted as
Therefore, the optimal can be obtained by setting the partial differential function to be 0. Then, the specific optimal transmission power allocation is as follows:
We use the subgradient method to update the Lagrange multipliers and , which are updated aswhere and are the step sizes of iteration .
Based on the above optimal transmission power allocation results, we can find the optimal subcarrier assignment. Decomposing the dual problem (27) into subproblems, for each subproblem, we havewhere
Subcarriers will be assigned to SUEs with the largest , which meanswhere .
Therefore, the subcarrier assignment decision is as follows:
Besides, we also use the subgradient method to update the Lagrange multiplier , which is updated aswhere is the step size of iteration .
4.2. Energy Scheduling and Energy Cooperation Subproblem
The energy scheduling and energy cooperation subproblem jointly considers the control of SBS’s discharging, charging, energy cooperation, and ongrid energy consumption. At each time slot , SBS solves the following problem based on local information for energy scheduling and energy cooperation:
According to constraint (8) and , P4 can be further written as
The optimal solution to the above problem is , , and . The energy of SBS getting from the grid can be expressed as
Since P5 is a linear programming problem, it can be solved directly by linear programming tools.
4.3. Online Algorithm
As presented in the above subsections, the formulated timeaverage delayaware and interferenceaware energy efficiency problem is solved distributively through several decompositions. Firstly, we use Lyapunov optimization method to decompose the optimal problem into two subproblems at each time slot, that is, resource allocation subproblem and energy scheduling and energy cooperation subproblem. Secondly, the resource allocation subproblem is solved through Lagrange dual decomposition. Then, the dual problem is decomposed into independent subproblems for optimal power allocation and independent subproblems for optimal subcarrier assignment. Meanwhile the energy scheduling and energy cooperation subproblem is decomposed into independent subproblems for the optimal discharging, charging, energy cooperation, and ongrid energy consumption.
Based on the above solution, we further present the distributed algorithm for joint resource scheduling and energy cooperation in interferenceconstrained HetNets powered by hybrid energy, which is shown in Algorithm 1.

5. Performance Analysis
Theorem 1. When andhold, the following relationships have been established:(i)If , .(ii)If , .
Proof. See Appendix B.
Theorem 2. When andare satisfied, the renewable energy collected by the storage device satisfies at any time slot.
Proof. See Appendix C.
Lemma 2. Assume that the coverage range of the network is . As the data arriving process , the channel gain , and the renewable energy generation all follow i.i.d processes, a random control strategy is implemented. Then,where is a finite positive number. When the constraint of is satisfied, it can be proved by Carathéodory’s theorem. Please refer to the specific proof in [22, 23].
Theorem 3. (a) The upper bound of the timeaverage data queue is(b) The timeaverage energy efficiency of the proposed algorithm satisfies
Proof. See Appendix D.
6. Simulation Results
In this section, the simulation results of the proposed algorithm are given. We consider a hybrid network that consists of one MBS and SBSs, where the coverage radius of MBS is 500m and the coverage radius of SBS is 30m. The MBS has 10 MUEs and each SBS has 2 SUEs. In this paper, we consider wireless channels with smallscale Rayleigh fading component and a largescale path loss. Every SUE has a buffer to its related SBS, where the buffer size . At each time slot, SBS receives burst data of its corresponding SUE which follows a Poisson distribution with average rate . SBS receives renewable energy which follows a Poisson distribution with average rate [9, 24]. The other parameters are listed in Table 2.
Firstly, we demonstrate the stability of network. We choose ; the trend of each queue is shown in Figure 2. As the same types of queues follow a similar trend, we only show the trends of SUE1 of SBS1, SUE2 of SBS2, and SUE3 of SBS3 in the data queue and show the trends of SBS1, SBS2, and SBS3 in the energy queue . From Figure 2, all data queues are strictly less than the maximum data buffer size, which is in line with Theorem 3 (a); and all the energy queues do not exceed the capacity of the storage devices, which is in line with Theorem 2.
(a)
(b)
When , we give the relationship between the longterm admitted rate and the service rate for all SUEs. As shown in Figure 3, each SUE’s admitted rate is less than the service rate, which means that the actual queues are stable.
Figure 4 shows the trend of timeaverage objective value with parameter when and are fixed. For each , we take . With the increase of , which means the emphasis on timeaverage ongrid energy consumption, the timeaverage objective values decrease. On the other hand, Figure 5 shows the trend of timeaverage objective value with parameter when and are fixed. For each , we take . As increases, which means that the timeaverage service rate weighs more, the timeaverage objective value increases. The results of both Figures 4 and 5 agree with Theorem 3 (b).
Figure 6 compares the proposed algorithm with a basic algorithm. To highlight the proposed joint resource allocation and energy cooperation algorithm, in the basic algorithm, all SBSs are supplied by hybrid energy and perform energy scheduling noncooperatively. Besides, the resource allocation in the basic algorithm is optimized with the proposed algorithm in this work. It is shown that, at , the value of the objective function of the proposed algorithm is about 7% higher than that of the basic algorithm without energy cooperation, which demonstrates that the proposed algorithm has a better timeaverage energy efficiency than that of the basic algorithm. Moreover, as shown in Figures 4 and 5, the tradeoff factors and also greatly affect the value of the objective function.
7. Conclusion
In this paper, we consider the resource scheduling and energy cooperation for a twotier downlink HetNet, which is powered by hybrid energy supplies. By jointly adjusting the transmission power allocation, the subcarrier assignment, the discharging and charging, the energy cooperation, and the ongrid energy consumption, we can maximize the timeaverage energy efficiency of SBSs in an online manner. Compared with the other works, this paper considers the burst of user data, the crosslayer interference from SBSs to MBS, and the scheduling of random renewable energy sources. Through theoretical analysis and simulations, we prove the stability and superiority of the proposed algorithm.
Appendix
A. Proof of Lemma 1
Since , according to the update process of queue , we have
Depending on the update process of energy queue , we have
By employing above inequalities, we can prove Lemma 1.
B. Proof of Theorem 1
In order to minimize the target value of the energy scheduling problem (36), we can infer that when ; then . Thus, when , we have
Putting the constraint of into (38), we have
For the second case, in order to minimize the objective function, we have . Bringing to (B.1), we get
Therefore, holds when .
C. Proof of Theorem 2
At the first time slot 0, the constraint holds, since SBS has no operation. We consider that there are four cases for the energy in storage devices.(i)We have when . So, according to the energy update process of storage device, holds.(ii)We have when . Thus, if is introduced into the inequality, holds.(iii)When , we get . Then there is .(iv)When , we have and .
Based on the above proofs, is still satisfied when is satisfied.
D. Proof of Theorem 3
Using the random control strategy , we have