Abstract

Device-to-device (D2D) communications and femtocell systems can bring significant benefits to users’ throughput. However, the complicated three-tier interference among macrocell, femtocell, and D2D systems is a challenging issue in heterogeneous networks. As D2D user equipment (UE) can cause interference to cellular UE, scheduling and allocation of channel resources and power of D2D communication need elaborate coordination. In this paper, we propose a joint scheduling and resource allocation scheme to improve the performance of D2D communication. We take UE rate and UE fairness into account by performing interference management. First, we construct a Stackelberg game framework in which we group a macrocellular UE, a femtocellular UE, and a D2D UE to form a two-leader one-follower pair. The cellular UE are leaders, and D2D UE is the follower who buys channel resources from the leaders. We analyze the equilibrium of the game and obtain solutions to the equilibrium. Second, we propose an algorithm for joint scheduling of D2D pairs based on their utility. Finally, we perform computer simulations to study the performance of the proposed scheme.

1. Introduction

With the increasing demand for larger system capacity and ubiquitous service quality in wireless communication, device-to-device (D2D) communication is a promising technology which has been considered as an important feature to be integrated into the long term evolution-advanced (LTE-A) system. As a type of proximity communication, D2D communication enables user equipment (UE) to communicate with each other directly without traversing the evolved NodeB (eNB) when UE is in close distance [1, 2]. There are multiple transmission modes for D2D communication coexisting with cellular networks. Inband mode indicates the method of D2D reusing cellular spectrum, while outband mode indicates that D2D occurs on unlicensed band [3]. In inband mode, the interference among participators can be properly controlled, due to the controlling mechanism of cellular spectrum based networks [4, 5]. Therefore, D2D communication makes a great contribution in improving system throughput and extending UE’s battery lifetime.

On the other hand, network topology has been considered as one of the key issues to make a leap on the performance of networks [6]. Using a mix of several cellular systems, heterogeneous networks enables flexible and low-cost deployments with higher spectral efficiency and better user experience [7]. As a type of such technology, femtocells have gathered considerable interest recently, since they can efficiently offload traffic burden of macrocell base station (MBS), which will consequently improve network coverage and capacity.

In the literature, studies on resource management of D2D communication focus on power optimization, resource allocation, and mode selection when they perform as an underlay in traditional cellular networks reusing uplink resource [810]. In [11], the optimal selection of possible resource sharing modes of D2D pairs with the cellular network in a single cell is studied, and a mode selection procedure for a multicell environment is proposed. A greedy heuristic algorithm that can lessen interference from D2D to cellular network utilizing channel gain information was proposed in [12]. In [13], the optimum power allocation method is derived for the scenario with prioritized cellular communication and an upper limit on the maximum transmission rate of all links. In [14], an open-access algorithm for femtocell base stations in D2D LTE-advanced networks is proposed to optimize network connectivity. Distributed power allocation strategies are discussed in [15] using Stackelberg game, where a central macrocell is underlaid with several femtocells.

Generally, as a secondary underlay which reuses the spectrum resources of the primary system, resource allocation of D2D pairs is an important but challenging task to improve the performance of heterogeneous macrocell-femtocell networks [16, 17], since the receivers in this system suffer from three-tier interference, that is, macro-to-device, femto-to-device, and device-to-device interference [18]. In previous SINR-threshold-based power control method [19, 20], only one system, either macrocell or femtocell, has been considered to determine transmit power of D2D pairs, which can be inefficient and impede sufficient deployment of D2D communications. Other related research on multilayer interference management in heterogenous networks is limited in number.

In this paper, we study resource management for D2D communication in heterogeneous networks utilizing game theory approach. Given D2D’s underlay status in the system, Stackelberg game framework is well suited for the situation. The rest of this paper is organized as follows. In Section 2, system model of heterogeneous networks with D2D communication as an underlay is illustrated. In Section 3, we formulate the problem as a two-level Stackelberg game. In Section 4, solutions for the proposed game are derived by attaining Stackelberg game equilibrium. In Section 5, simulation results are presented to validate the proposed scheme. Finally, Section 6 concludes the paper.

2. System Model

We consider the uplink of a macro/femto/D2D system in a single cell with a macrocell base station (MBS) in the center. One femtocell is randomly located in the same cell. The femtocell is assumed to be round-shaped with a femtocell base station (FBS) deployed at the center of the house. The femtocell serves several indoor users, which are randomly located within the house. Several macrocell UE are distributed out of the femtocell. There are multiple outdoor D2D UE around the considered femtocell. The D2D UE are in pairs, each consisting of one transmitter and one receiver between which the communication distance is . If D2D pairs are far enough away from the femtocell, the interference they cause to the considered femtocell can be neglected. Therefore, we locate D2D pairs around the central house in the shadowed area with width of , as shown in Figure 1.

We assume that the number of macrocell UE is , and hence there are orthogonal channels which are occupied by the corresponding macrocell UE. The channels allocated to the macrocell UE are fixed. The femtocell system and D2D pairs share the channels with macrocell UE. The number of the femtocell UE is . One channel is only allowed to be used by one macrocell UE, one femtocell UE, and one D2D pair. We consider a dense D2D environment, where the number of macrocell UE is smaller than the number of D2D pairs . The set of macrocell UE, femtocell UE, and D2D pairs are , , and , respectively. In LTE, scheduling takes place in every transmission time interval (TTI) [21]. Channels are allocated among D2D pairs according to their priority. During each TTI, D2D pairs are selected to share the channels with macrocell UE and the femtocell UE while other D2D pairs wait for transmission.

We use a set of , to denote the current D2D pair in communication. if the th D2D pair is selected to use channel , and otherwise. Define , , and as transmit power of the th macrocell UE, th femtocell UE, and th D2D transmitter, respectively. Therefore, the received SINR at MBS corresponding to th macrocell UE is where denotes channel gain between th macrocell UE and MBS. denotes channel gain between th D2D transmitter and MBS. The received SINR at FBS corresponding to th femtoocell UE is where denotes channel gain between th femtocell UE and FBS. denotes channel gain between the th D2D transmitter and FBS. The SINR at th D2D receiver can be expressed as where denotes channel gain between th D2D transmitter and th D2D receiver, which are in a pair. denotes channel gain between th macrocell UE and th D2D receiver. denotes channel gain between th femtocell UE and th D2D receiver. The channel rate of a UE can be obtained by

3. Stackelberg Game Formulation

As D2D communication takes place underlaying the heterogeneous networks, we focus on power control and scheduling of D2D UE, while transmit power of macrocell UE and femtocell UE are assumed to be fixed. D2D communication can utilize the proximity between UE to improve the throughput performance of the system. In the meanwhile, interference from D2D pairs to cellular network should be limited. Thus, transmit power of D2D UE should be properly controlled. Another goal is to guarantee the fairness among D2D UE when scheduling. In this section, we first formulate this problem as a resource allocation method using Stackelberg game based scheme; then we first obtain solutions to the outcomes of the proposed game.

3.1. Two-Level Game Framework

Interactions among selfish cellular UE and D2D UE sharing a channel can be modeled as a noncooperative game using game theory framework. When players choose their strategies independently without any coordination, it usually leads to an inefficient outcome. If we simply model this scenario as a noncooperative game, D2D transmitters will choose to use the maximum transmit power to maximize their own payoffs regardless of other players, whereas cellular UE will choose not to share channel resources with D2D UE. This is an inefficient outcome, as either the interference is too serious or D2D cannot get access into the network.

Therefore, we employ the Stackelberg game to coordinate the scheduling, in which macrocell UE and femtocell UE are leaders and D2D UE are followers. We focus on the behavior of a two-leader one-follower pair, of which a macrocell UE and a femtocell UE are the leaders; a D2D UE is the follower. They share the same channel resource. The leaders own the channel resource and they can charge D2D UE some fees for using the channels. The fees are fictitious money to coordinate the system. Thus, the cellular UE have an incentive to share the channel with D2D UE if it is profitable, and the leaders have the right to decide the price. For D2D UE, under the charging price, they can choose the optimal power to maximize their payoffs. In this way, an equilibrium can be reached.

3.1.1. D2D UE/Follower-Level

The D2D pair can be modeled as a buyer and aims to obtain the most benefits, at least possible payments. The utility of the follower can be defined as its own throughput performance minus the cost it pays for using the channel. The fees should be decided according to the leaders’ consideration. Thus, the fee is charged proportionally to the amount of interference the leaders observe, which can be expressed as where and are the charging prices (, ) of MBS and FBS, respectively. We denote the set of prices for th D2D as , where . is a scale factor to denote the ratio of the D2D’s gain and its per unit of rate. Since the two terms in the utility function are uniform in measurement units, the scale factor is introduced to better integrate them. is a key parameter to influence the outcome of the game, which we will discuss later.

The optimization problem of follower-level game can be formulated as

3.1.2. Macrocell UE and Femtocell UE/Leader-Level

The macrocell UE and femtocell UE can be seen as two seller and aim to not only earn the payment but also gain as many extra profits as possible. The utility of the leaders can be defined as their gain from the follower minus the interference they observe from the D2D pair. The utility function of the leaders can be, respectively, described as where denotes utility of MBS and denotes utility of FBS. is a factor to denote the ratio of the leader’s gain and the follower’s payment. It mainly influences the speed of convergence of the proposed game.

The optimization problem of leader-level is to set a set of charging prices that maximize their utility, that is, The choice of the optimal prices and is affected not only by distance and channel conditions between D2D transmitter and base stations, but also by each other.

The outcome of the proposed game will be shown in detail in the following section.

3.2. Analysis of the Proposed Game

In the Stackelberg game, the leaders move first and the follower moves sequentially; that is, the leaders set the prices first, and the follower selects its best transmit power based on the price. The leaders know ex ante that the follower observes their action. Therefore, the game can be solved by backward induction.

3.2.1. Analysis of the Follower-Level Game

In Stackelberg game, the leader has the preferential right of pricing on the follower cost, while the follower has no direct influence on the prices. Therefore, in every step of pricing updating process, since prices have been set by the leaders, they remain constant at this very step from the perspective of the follower. As illustrated in (5), given and decided by the leaders, when approaches 0, the utility of D2D approaches 0 as well. As increases, also increases. If grows too large, will begin to decrease since the logarithmic function grows slower than the cost. The follower wants to maximize its utility by choosing proper transmit power. The best response is derived by solving where

The solution of (9) is

From (11), we know the power is monotonically decreasing with and , which means when the price is higher, the amount of power bought is smaller.

3.2.2. Analysis of the Leader-Level Game

Substituting (11) into (8), we have

We can note that (12) and (13) are a noncooperative game by MBS and FBS, and there exists a trade-off between prices and the base stations’ utility. We take MBS, for example. If MBS asks for a relatively low price at first, the D2D pair sharing the same channel will buy more power from MBS, and will increase as grows according to (12). When keeps growing and exceeds a certain value, it is no longer beneficial for D2D to buy so much power from MBS. In this way, will shrink and hence results in decrement of . Therefore, there is an optimal price for MBS to ask for. Besides, the optimal price is also affected by FBS’s price.

From the analysis above, by taking the derivative of to and to , and equating it to zero, we have where , . We take short for when , and for when .

Solving the above equations of , we denote the optimal prices as where denotes the set of channel gains among disparate participators.

4. Stackelberg Equilibrium

In this section, we prove that the solutions in (11) and in (15) are the Stackelberg equilibrium (SE) for the proposed game and show the conditions for the SE to be optimal by the following properties, propositions, and theorems. Furthermore, we show that the set of the solutions is a unique fixed point and the proposed game converges to that point.

4.1. Existence of the Equilibrium for the Proposed Game

We first define the SE of the proposed game as follows.

Definition 1. and are the SE of the proposed game if when is fixed and when is fixed

Then, we show that the optimizer of (11) can be solved by equating to zero by the following property.

Property 1. The utility function of the follower is a concave function of , with and fixed ().

Proof. The second-order derivative of (5) is
Thus, the solution in (11) is a maximum point.

Due to Property 1, in (11) is the global optimum that maximizes the D2D UE’s utility . Therefore, satisfies (16) and is the SE .

In the following two properties, we show that the base stations cannot infinitely increase by asking arbitrarily high prices.

Property 2. The optimal power of D2D transmitter is decreasing with a base station’s price or when the other base station’s price () is fixed.

Proof. Taking the first-order derivative of the optimal power by , we have Similarly, taking the first-order derivative of the optimal power by , we have
Therefore, is decreasing with and . This is because when a base station individually increases its price while the other keeps the same price as before, the D2D transmitter will adopt a lower power to avoid loss of its own utility.

Consequently, there is a trade-off for base stations to ask for proper prices, and we can solve the optimal prices by equating , the reason of which is shown as follows.

Property 3. The utility function of each base station is concave in its own price when the D2D transmitter in the same channel adopts the optimized power as calculated in (11) and the other base station’s price is fixed.

Proof. We can prove Property 3 by taking the derivatives of a base station’s utility . We take the proof of , for example, while the proof of is similar. Taking the second-order of results in the following: where ; then . Therefore, is concave with respect to .

Based on the above properties, we conclude the following theorem.

Theorem 2. The in (11) and in (15) are the SE for the proposed game, where the SE is defined in (16) and (17).

In the next section, we will show that the SE is unique, and the proposed game converges to the unique SE when each base station updates its price according to a simple function.

4.2. Convergence of the Price Updating Function

From the previous section, one base station needs to modify its own price after the other base station changes its price. Consequently, each base station updates so that its utility satisfies the following equality: with the equality holding if and only if reaches the optimum.

After rearranging (22) we have

In order to calculate in (23), each base station listens to the instantaneous feedback information about and from the D2D transmitter. Then the updating of the base stations’ prices can be described by a vector equality of the form where and , with representing the price competition constraint to one base station from the other. Therefore, with the competition constraints in (24), the iterations of the price updating can be expressed as follows:

We show next the convergence of the iterations in (25) by proving that the price updating function is a standard function.

Definition 3. A function is standard if for all , the following properties are satisfied. (i)Positivity. .(ii)Monotonicity. If , then .(iii)Scalability. For all , .

Proposition 4. The price updating function is standard.

Proof. Consider the following.
Positivity. By Property 2, . Moreover, if and , then by the definition of (23), . Therefore, in a real price updating progress, each base station starts increasing its price from .
Scalability. Comparing and , we have Since , . Then, the problem reduces to proving that the second term in the RHS of (27) is positive. We take , for example. If we define as follows: where and . Then, from (11), we can get
Therefore, to prove the positivity of the second term of RHS of (27) is equivalent to proving , where . Since is continuous and differentiable in , we only need to prove that . Consider
After extensive numerical tests for a wide range of parameters when the nodes are randomly located, we observe that the numerator of (29) is positive. Then, . We can claim that .
Monotonicity. Suppose and are different price vectors, and the vector inequality means that , for all . Therefore, the problem reduces to proving and , for all . Consider
After order-of-magnitude estimation, we can note that . Similarly, we can also prove that , so monotonicity holds for the price updating function.
Finally, from the above three parts, we prove that the price updating function is standard.

In [22], a proof has been given that starting from any feasible initial power vector , the power vector produced after iterations of the standard power control algorithm gradually converges to a unique fixed point.

From (22) we know that for one base station, its utility satisfies every time after the base station updates its price given the feedback of from the source. After the vector converges to , no base station can gain a higher utility by further varying its price, meaning that . From (15), we know that is exactly the optimal price vector. As Property 1 shows, is concave in , so the D2D transmitter can gradually increase the power from 0 and find the optimal . Thus, if the prices of both MBS and FBS converge to their optima, then the D2D transmitter will correspondingly buy the optimal power. Therefore, once converges to , and converge to the SE.

5. Joint Scheduling and Resource Allocation

The scheduling process is conducted at each TTI. The D2D UE forms a priority queue for each channel. During each TTI, the MBS selects D2D UE with the highest priority for each channel sequentially and other D2D UE has to wait.

In our Stackelberg game framework, the priority is based on the utilities of the followers, which express the satisfaction of the followers. In the design of scheduling scheme, fairness is considered as an important goal. The scheme should take the outcome in the previous TTIs into account. This can be achieved by adjusting prices for using the channel. The follower has to pay an additional fee for using the channel at TTI if it has been selected in previous TTIs, which will lead to a decrease in the priority. The additional fee is decided by the cumulative utility of follower. The priority for follower at TTI can be defined as the following: where and are the optimal strategy under the Stackelberg equilibrium at TTI . is the additional cost and can be defined as where is the fairness coefficient. For a larger , the cumulative utility has a larger influence on the priority. If , the scheduling scheme does not take fairness into account.

Based on the above discussion, during each TTI, every macrocell UE, femtocell UE, and D2D UE form a leader-follower pair and play the Stackelberg game. The optimal price and power can be decided for each pair. The priority for each pair can be calculated and they form a priority queue. Then, the eNB schedules the D2D pairs sequentially according to their order in the queue. If there is a tie, in which one channel has been allocated to another D2D pair, or the D2D pair has been scheduled to another channel, the pair is skipped. When each channel is allocated to one D2D pair, the eNB records the outcome and the scheduling is over. The algorithm is summarized in Algorithm 1.

(1)  Given CSI, TTI , the factors , , the fairness coefficient , and the additional cost , .
(2)  Initialize
(3)  Update prices according to the iterative function
           
(4)  Calculate the optimal power
        
(5)  Calculate priorities
            .
(6)  Sort in descending order to form a priority queue.
(7)  while do
(8)    Select the head of the queue. The pair is .
(9)    if and then
(10)     Schedule the pair .
(11)      Set and
(12)    end if
(13)    Delete the head of the queue.
(14)  end while

The algorithm has a low complexity, as the optimal strategy for each leader-follower pair is searched in a set with a constant number of elements. To form the priority queue with length , the complexity is .

6. Simulation Results

To evaluate the performance of the proposed algorithm, we perform several simulations. We consider a single circular cell environment. The macrocell/femtocell UE and D2D pairs are uniformly distributed in the cell. The two D2D UE in a D2D pair are close enough to satisfy the maximum distance constraint of D2D communication. The received signal power is , where and are received power and transmit power, respectively. is the distance between the transmitter and the receiver. represents the complex Gaussian channel coefficient that satisfies ~. The scheduling takes place every TTI. Simulation parameters are summarized in Table 1.

6.1. Convergence of the Proposed Game

As described in previous section, the base stations start increasing their price from . Denote the price vector at time as . From (11) the optimal power purchased by the source node at time can be denoted as In order to obtain and update their prices by (23), the base stations will simultaneously increase each by a small amount . The D2D pairs receive this price updating and calculate using the following approximation: Substituting the above approximation signaled from the D2D pairs into (23), the base stations can obtain .

We conducted simulations to observe the convergence behavior of the proposed game. In Figure 2, we plot the convergence of the prices of MBS and FBS under different (), where denotes the gain per unit of rate as defined in (5). It is seen that the proposed scheme has fast convergence to the SE . It takes less than 15 iterations until the price vector converges to the optimum. For one , the speed of convergence for MBS and FBS is almost the same. When gets smaller, the prices asked for D2D transmission from MBS and FBS get smaller consequently. From (11), we denote that when gets smaller, D2D transmitter will choose a smaller transmit power, which is illustrated in Figure 3 (). In this case, the interference D2D causes to base stations gets relatively weaker, and there is no necessity for base stations to ask for high prices. Thus the prices are relatively lower.

6.2. Analysis of the UE Rate

In Figure 4, we plot CDF of D2D rate under different . With larger , D2D transmit power rises as has been shown in Figure 3. Therefore, D2D rate goes up correspondingly, causing more interference to cellular UE and a decrease in the rate of cellular UE.

In Figures 5 and 6, we plot CDF of MBS and FBS rate with/without power optimization method proposed in this paper. It is clearly shown that both MBS and FBS rate is improved by approximately 2-3 bps/Hz.

6.3. Analysis of Scheduling Process

In Figure 7, we study the effect of the fairness coefficient . We plot CDF of UE rate under different . For a small , D2D UE rate is distributed in a large range and has a tendency to converge with a larger . Thus, scheduling with a larger achieves better fairness. If we set too large, the previous utility is the deciding factor and the utility of the current TTI has little influence. If D2D scheduling is not considered, there will be only a few D2D pairs that can get access to the network, resulting in other D2D UE that cannot achieve any data transmission.

7. Conclusion

In this paper, we constructed a Stackelberg game framework for joint power control and channel allocation and scheduling of D2D communication in heterogeneous macrocell/femtocell network system. Based on properly designed utility functions, prices for reusing the channel resource and appropriate transmit power of D2D transmitters are adjusted to maximize the utility obtained by base stations and D2D pairs, respectively. Based on the proposed scheme, we analyzed the optimal strategy for every participator (D2D pairs/femtocell users/macrocell users), worked out the solutions for equilibrium state, and proposed an algorithm to allocate resources to schedule D2D UE, where interference management and fairness of the system were considered. Simulation results show that the proposed algorithm can achieve an increase in transmit rate performance for both the cellular and the D2D UE. The D2D UE can be fairly served. The scale factor and fairness coefficient have an important effect on the performance of the algorithm.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is partly supported by the National Science Foundation of China (Grant nos. NFSC no. 61071083 and no. 61371073) and the National High-Tech Research and Development Program of China (863 Program) no. 2012AA01A506.