Abstract
The unmanned aerial vehicle (UAV) base station plays a significant role in enhancing the terrestrial network, when the ground base station (GBS) is destroyed in emergent cases or its load exceeds the capacity of the terrestrial network. Presently, many papers focus on optimizing the UAV position deployment and user access, while ignoring the optimization about the spectrum resource management. To solve this problem, we formulate a joint optimization problem of the spectrum resource management and the position placement for UAVs with the constraint of the limited backhaul capacity. Later, the joint optimization problem is modeled as a hierarchical game decision architecture comprised of a UAV position placement game and a spectrum resource management game. Further, we analyze the equilibrium property of the two games and propose two best response- (BR) based optimization algorithms to reach the Nash Equilibriums (NEs) of the two games, respectively. Specifically, the proposed algorithm about the UAV deployment considers the variable granularity local exploration and global random exploration. Simulation results show that the proposed UAV deployment algorithm can improve the total throughput by 7% and 20% at least in comparison with the K-means deployment algorithm and the fixed granularity exploration algorithm, respectively.
1. Introduction
When the ground base station (GBS) is destroyed in emergent cases or when the data demand of users increases sharply, the excessive load will cause network congestion and reduce the quality of service for users. Due to the advantages such as high flexibility, reliable communication, and low cost, unmanned aerial vehicles (UAVs) are popularly used to help the terrestrial base station and are expected to be deployed in the next generation Wireless Communications Networks (WCNs) to enhance the communication and expand the coverage [1].
Presently, many papers investigate the UAV-assisted network, and the problem of UAV deployment and user access receives much attention [2–6]. The farther the UAV is from the base station, the lower the backhaul link capacity is, and the farther the UAV is from the user, the lower the UAV-user link capacity. The network throughput is the minimum value of the backhaul link capacity and the UAV-user link capacity, so the reasonable position placement helps the network achieve higher throughput. Because UAVs have different backhaul link capacities, reasonable user association can make the UAV with higher backhaul link capacity serve more users so as to make full use of UAV resources.
However, the management of spectrum resources is equally important [7] but has been neglected in related researches. Due to the limited spectrum resources and the increasing communication users, spectrum resources become scarce, and it becomes unavoidable for different communication users to use the same segment of spectrum resources at the same time. Although different communication users will interfere with each other when using the same spectrum resources, the intensity of interference depends on the transmission power and the distance between communication users. Therefore, the reasonable management of spectrum resources, that is to say allow users far from each other to use the same spectrum resource, can reduce the interference among users in the network and improve the total throughput of the network. If the communication is maliciously interfered, the identification technology about interference signal is also essential [8, 9].
We investigate the joint optimization problem of the spectrum resource management and the position placement for UAV base stations. To simplify the optimization problem, the part about the UAV-user association is solved by a matching-based mechanism. While there are some challenges needing to overcome. Firstly, considering the decision optimization of multiple UAVs and multiple users, it is difficult to describe and analyze the influence between UAVs and users. Then, there is a coupling relationship between the optimization variable of the spectrum resource allocation and the UAV position. UAVs that use the same spectrum tend to be spaced far from each other, and UAVs that spaced far from each other tend to use the same spectrum. At last, the strategy space of the optimization problem is huge. If UAVs and users exist in the network, channels can be used and the space size of discrete location of UAVs is , then the space size of the joint strategy is . So the optimal solution is hardly to obtain by a search method.
To solve the challenge of multiuser optimization, we model the joint optimization problem as a game model. Because the game theory is a theoretical tool often used in the multiagent decision making, and through a clever design, all agents can reach a Nash Equilibrium through the distributed optimization of agents, and the best solution to the optimization problem is often a Nash Equilibrium (NE) [10–17]. To solve the challenge of the coupling relationship between optimization variables, we adopt a hierarchical optimization framework. We use the inner and outer layer structure instead of the upper and lower layer structure to improve the network performance. To solve the challenge of huge strategy space, we propose two best response- (BR- [18]) based optimization algorithms to optimize the spectrum resource management and the UAV position, respectively.
Our paper’s main contributions are the following: (i)In the network assisted by the UAV base station, the problem of spectrum resource management in nonorthogonal channel is considered, and the spectrum resource management and UAV position placement are jointly optimized(ii)The above joint optimization problem is modeled as a hierarchical game decision architecture comprised of a UAV position placement game and a spectrum resource management game, and both the two games are proved to be exact potential games (EPGs) [19](iii)Two BR-based optimization algorithms are proposed to optimize the spectrum resource management and the UAV position, respectively. Simulation are conducted to show that the proposed UAV deployment algorithm can improve the total throughput by 7% and 20% at least in comparison with the K-means [20] deployment algorithm and the fixed granularity exploration algorithm, respectively
2. Related Work
The UAV deployment has been extensively investigated because UAVs have faster deployment speed, lower cost, and larger coverage in comparison with terrestrial base station UAVs [21–23]. In [21], the authors proposed a multi-UAV coverage model and investigated the multi-UAV deployment considering energy efficiency. The authors in [22] considered the minimum average UAV-user distance as the quality of coverage, and the UAVs were deployed in a distributed way without global information. A new framework to predict the traffic in hot spots for the UAV deployment in wireless networks was proposed in [23]. The problem of UAV-user association in UAV-assisted networks is addressed in [24–26]. In [24], the authors maximized users’ QoE jointly optimizing the UAV position placement, caching deployment, and user access. In [25], the authors considered a UAV communicating with the sensors along the way in wireless sensor networks (WSNs) and considered that ground sensors converged data on several head nodes and got the head nodes to communicate with the UAV to improve the data transmission efficiency. The authors in [26] jointly optimized the UAV 3D deployment and user access to improve users’ satisfaction.
However, the papers mentioned did not consider the impact of UAV backhaul links in the study of UAV deployment. Therefore, some authors made a further study considering the constraint of backhaul links [27–33]. [27] investigated the 3D deployment of a single UAV in two different networks to serve as many users as possible and maximize the sum-rates of the network. [28] investigated the UAV deployment in a post-disaster scenario and proposed an algorithm based on artificial bee colony to deploy the UAV. The authors in [29] proposed an efficient heuristic algorithm with lower computational complexity to address the resource management problem and used a search algorithm to determine the UAV location. The authors in [30] investigated a scenario using a UAV-network to replace the terrestrial backhaul network and proposed a heuristic approach to address the access problem and a genetic algorithm to deploy the UAVs. The authors in [31] considered the in-band wireless backhaul and proposed an novel method to optimize the user access and UAV deployment. Then the authors made a further study proposing a novel framework to optimize the network throughput with consideration of fairness among the users in [32]. In [33], the authors proposed a decentralized deployment algorithm to decrease the average distance between the UAVs and users, and this algorithm could be applicable to large-scale. While none of the above papers [27–33] investigated the spectrum resource management about users’ channel selection and almost all papers fail to consider users sharing the same channel. Thus, the above researches cannot meet the increasingly access requirement of users.
3. System Model and Problem Formulation
3.1. Network Model
As shown in Figure 1, there is a GBS, multiple UAVs, and multiple ground users (GUs) in the network. It is assumed that GUs cannot be served by the GBS directly due to a large path loss caused by the blockage. The UAVs are deployed to provide wireless communication service for GUs and have a backhaul link with the GBS. The sets of the UAVs and GUs are denoted as and , respectively. The ground locations of the GBS, UAVs, and GUs are denoted by , , and , respectively. The UAVs are assumed to be deployed at a fixed altitude . The altitude of the GBS and GUs is negligible. The mobility of the GUs is assumed to be low and the GUs’ locations are seemed as unchanged during the placement update process of the UAVs.
3.2. Channel Model
According to [34], UAVs communicate with ground users in a line-of-sight (LoS) transmission link and a non-line-of-sight (NLoS) transmission link. For convenience, we denote the GBS and UAVs set as , and 0 is used to denote the GBS. Then, the probability of LoS between the UAVs and the GBS or between the UAVs and the GUs is denoted by where and are environment impact factors which are related to the density, height of buildings, and street width, etc., and denotes the 2-norm. Furthermore, the probability of NLoS is denoted by .
Hence, the average pathloss [32] is expressed as where , , , , and are the carrier frequency, speed of light, and average additional losses for LoS and NLoS links, respectively.
The UAVs use different channels to communicate with the GBS, and the channel bandwidth is . Therefore, the backhaul rate of UAV is expressed as where is the transmission power of the GBS and is the variance of the additive white Gaussian noise.
The channel set used by the UAVs to communicate with the GUs is denoted as . The channel bandwidth is . Every GU can access a UAV at most, and should be assigned only one channel after access. Each channel should only be assigned once at most by every UAV. Limited by hardware conditions, each UAV can serve up to GUs at the same time. The GUs served by different UAVs can use the same channel, but there will be interference. Hence, the rate of GU received from UAV on channel is written as where is the transmission power of the UAV, and is a binary association indicator variable for UAV , GU , and channel and 1 indicates that UAV communicates with GU in channel .
3.3. Problem Formulation
The backhaul rate of the UAVs is closely related to the UAVs’ locations, and choosing which UAV and channel to access also affects the GUs’ communication rate. Hence, to improve the throughput of the entire network, we optimize the locations of the UAVs and the channel selections of the GUs. The problem is modeled in the following:
In this optimization problem, constraint (5b) indicates that each GU should access one UAV and one channel at most. Constraint (5c) represents that each channel should only be assigned once at most by every UAV. Constraint (5d) indicates that each UAV can serve up to GUs at the same time. At last, constraint (5e) requires that each UAV’s backhaul rate is greater than the total communication rate of the users they served, respectively.
4. Hierarchical Game Decision Architecture
Game theory has been widely used in multiagent decision making, and when there are multiple optimization variables, the problem is usually modeled as a hierarchical game decision architecture [10, 12, 26]. Similarly, a hierarchical game decision architecture is proposed to optimize the spectrum resource management and UAV position placement jointly. The schematic diagram of the hierarchical game decision architecture is shown in Figure 2. The UAV position placement game is in the outer layer, while the spectrum management game is in the inner layer.
When the UAVs choose a new position strategy in the outer game, the UAV-user association strategy corresponding to the new position strategy will be obtained through a mechanism based on matching theory. Then according to the position strategy and the UAV-user association strategy, UAVs constantly update their channel selection strategy for users they serve until they reach an NE of the inner game. Further, the channel selection strategy is outputted to the outer game, and the UAVs calculate their payoffs in the outer game. At last, the UAVs decide whether to update the position strategy based on the payoffs and start the next new position update process until they reach an NE of the outer game.
The mechanism used to decide the UAV-user association is described as follows: (1)Users prefer to be served by the nearest UAV. So each user applies to access to its nearest UAV(2)Adhering to the principle of first application, first service, UAVs also tend to serve the closer user. Each UAV agrees to the user’s application under the limit of the number of access and rejects unwanted applications(3)The remaining rejected users continue to apply to access to the nearest UAVs that have not rejected them until all the users have accessed UAVs, or all UAVs have accessed the maximum number of UAVs
4.1. Inner Game: Spectrum Resource Management Game
The inner game is expressed as . is the set of GUs served by the UAVs. is the channel set. and are the channel selection strategy and payoff of GU , respectively. Because users who use the same channel will interfere with each other, and if the interference decreases, the throughput of the network will increase. The inner game is modeled as a bilateral symmetric interaction game [35], and the payoff of GU is defined as the negative sum of the interference it received from other UAVs and the interference it caused to other users [36].
Therefore, the payoff of GU is expressed as where is the channel access selection of the GUs in set except GU , and denotes the interference received by GU from GU . is defined as where is the UAV access selection of GU , and is boolean variable which denotes the interference situation between GU and . is defined as
Then, the game is expressed as
The important concept about Nash equilibrium is defined as follows:
Definition 1 (Nash Equilibrium [37]). A strategy profile is a pure strategy NE if and only if no player can improve its payoff by changing its strategy unilaterally:
where and are the strategy and payoff of player , respectively, and is the number of players in set .
Further, we define Exact Potential Game (EPG) in the following, which has several nice properties.
Definition 2 (Exact Potential Game [19]). A game is EPG if there is a potential function satisfying For an EPG, the most important properties are as follows: (i)Every potential game has at least one pure strategy NE(ii)Any global or local maxima of the potential function constitutes a pure strategy NE
Theorem 3. The game is an EPG and has at least one NE.
Proof. The potential function of the outer game is constructed as
Firstly, we define a set as follows:
which means the set of the GUs who use channel except GU . If an arbitrary GU , , changes its channel from to , only the interference between GU and the GU in set will change, where .
Then, if GU changes its channel selection from to , the change of its payoff function is expressed as
The change of potential function is expressed as
Because
then the formula (15) is transformed as
Therefore,
Then, according to the definition 2, the inner game is an EPG and has at least one NE [38]. This completes the proof.
4.2. Outer Game: Position Deploy Game
The outer game is expressed as . is the UAV set. is the position of the UAV . is the payoff of the UAV .
Because the marginal contribution [21] can well reflect the influence of the UAV’s position strategy on the global optimization objective, is defined as UAV ’s marginal contribution to the overall network throughput and is given as where is the communication rate without deploying the UAV and .
Then, game is expressed as follows:
Theorem 4. The game is an EPG and has at least one NE.
Proof. The potential function of the outer game is constructed as
If UAV , , changes its position from to , then the change of its payoff function is expressed as
where is equal to .
The change of potential function is expressed as
Therefore,
Then, according to the definition 2, the inner game is an EPG and has at least one NE [38]. This completes the proof.
| ||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||
Remark 5. Both game and game are potential games with at least one NE. The physics significance of potential function is the negative sum of the total interference in network and the physics significance of potential function is the throughput of the entire network. Due to the important properties of an EPG, the best NE of is the channel strategy with the minimum interference of the entire network and the best NE of is the position strategy of the UAVs with maximum throughput of the entire network based on the game theory.
Remark 6. When the UAV position strategy is inputted to game from game , UAVs start to update their channel selection strategies until they reach the corresponding NE of game . Then, the channel selection strategy is outputted to game and UAVs update their positions. After multiple iterations, UAVs reach an NE of game .
4.3. Solution Approach
We propose two BR-based algorithms to optimize the spectrum resource management and the UAV position placement, respectively. If the finial channel selection strategy and the finial UAV position strategy do not satisfy constraint (5e), there are many heuristic methods to reduce the UAV transmission power to solve the problem, such as dichotomy.
4.3.1. BR-Based Channel Selection Algorithm
The details of channel selection algorithm are shown in Algorithm 1. Every time the UAVs choose a new position strategy in game , Algorithm 1 will be executed to produce the corresponding UAV-user association and channel selection strategy. Assume that there are UAVs, channels and every UAV serves up to GUs. Because Algorithm 1 randomly picks between 1 and users to update their channel selections in each iteration, then the computational complexity of Algorithm 1 is expressed as , where is the computational complexity required for a user to update its channel selection in each iteration and is a constant.
4.3.2. BR-Based Position Placement Algorithm
The details of position placement algorithm are shown in Algorithm 2. In each iteration of Algorithm 2, the UAV firstly explores the surrounding position of the current position. If there is no better position around, the UAV explores a random position in the whole space. Assume that the max iteration times of Algorithm 2 is , then the computational complexity of Algorithm 2 is at least .
where is the computational complexity required for a UAV to update its position and is a constant.
Remark 7. In each iteration of the proposed algorithm, the optimization of the strategy will improve the potential function. Because the strategy space of the problem is limited, the potential function has a maximum value, and the potential function will not increase all the time in the proposed algorithm, and the strategy will converge to the Nash Equilibrium at last. In the proposed algorithm, the strategy selected by the player always moves towards a Nash Equilibrium in the strategy space and players do not search the whole strategy space. Therefore, the proposed algorithm greatly saves the computation and solves the challenge of the huge strategy space to a certain extent.
5. Simulation Results and Analysis
In this section, simulations are made to verify the convergence performance and effectiveness of the proposed algorithm. The corresponding simulation parameters and analysis of results are also presented.
5.1. Simulation Parameter Setting
The mission area of the UAVs deployment is a square area of 2 km. The GBS is deployed in the lower left corner of the mission area and its coordinate is denoted as . The GUs are evenly distributed within a circle whose center is and radius is 750 m. The specific simulation parameters are shown in Table 1, where the UAV height m, GBS transmission power dBW, UAV transmission power dBW, number of UAVs , number of GUs , maximum number of GUs which the UAV can serve , number of channels , maximum carrier frequency GHz, channel bandwidth between the UAV and the GBS 20 MHz, channel bandwidth between the UAV and the GU 1 MHz, propagation environment parameters and attenuation factors . Specifically, we referred to , , , , and in reference [32].
5.2. Deployment Effect Description
The deployment effect is shown in Figures 3 and 4. The GBS is indicated by the pink pentacle, the UAV is indicated by the black triangles, and the GU is indicated by the dot. The red dots represent the unserved GUs and the green dots represent the GUs served by the UAVs. As shown in Figure 3, in the initial state, the UAVs are evenly deployed along the boundary of the GU’s distribution. Figure 4 shows the result of deployment solution. According to the matching theory, the UAV tends to serve GUs who are closer to it. Affected by the constraint (5e), the UAV tends to be closer to the GBS.
5.3. Result of Convergence Performance Simulation
To avoid the contingency, the convergence performances of Algorithm 1 and Algorithm 2 are obtained by running each algorithm 1500 times and 100 times, respectively.
Figure 5 shows the convergence time of Algorithm 2 in networks with different numbers of UAVs. The network which has more UAVs needs more time to converge. It is seen that 1500 iterations are enough for the network who has no more than 5 UAVs and 20 GUs. Figure 6 shows the cumulative distribution function (CDF) of the convergence time of Algorithm 1. It is seen that no more than 300 iterations are needed by the network with 4 UAVs, 20 GUs, and 10 channels to converge, and there is an 80% chance that Algorithm 1 will converge within 150 times.
5.4. Results of Different Deployment Methods’ Performance
In this subsection, we analyze the impact of different factors on the throughput of the network: (the bandwidth of the backhaul link), (the number of channels), (the maximum number of GUs which the UAV can serve), (the number of UAVs), and (the number of GUs). In addition, the proposed UAV deployment algorithm is compared with three other deployment methods, which are listed as follows: (i)K-Means Deployment. Use K-means algorithm [20] to deploy the UAVs, and then use Algorithm 1 to decide the channel selection of the GUs(ii)Fixed Granularity Exploration. Different from Algorithm 2, the exploration step size of the UAV is fixed and the UAV only explores positions near the current position(iii)K-Means with Fixed Granularity Exploration. Use K-means algorithm [20] to decide the UAVs’ initial positions, then use fixed granularity exploration approach to further optimize the UAVs’ positions
5.4.1. Impact of the Backhaul Bandwidth
Figure 7 compares the network throughput of different deployment methods when varying the backhaul bandwidth. As shown in Figure 7, the higher backhaul bandwidth improves the network throughput, while the improvement is limited. The performance of the proposed algorithm outperforms the other methods, the main reason is that the proposed algorithm considers the global random exploration, it makes the result less trapped in local optimality. Compared the performance of K-means and K-means with fixed granularity exploration, the fixed granularity exploration method can further improve the performance of K-means method. Compared the performance of fixed granularity exploration and K-means with fixed granularity exploration, the initial position of UAVs is closely related to the performance of fixed granularity exploration method.
5.4.2. Impact of the Number of Channels
The network throughputs of different deployment methods with different numbers of channels are shown in Figure 8. The performance of the proposed deployment solution outperforms the other methods in networks with different numbers of channels. When the number of channels exceeds 13, the performance of the proposed algorithm keeps increasing, while the performance of K-means deployment algorithm no longer increases because the deployment result is independent of the number of channels, and 13 channels are enough for the GUs to communicate without interference.
5.4.3. Impact of the Number of UAVs
Figure 9 compares the network throughput of different deployment methods when varying the number of UAVs. Similarly, the performance of the proposed algorithm is the best and the network throughput increases with the increase of the number of the UAVs. With the increase of the number of the UAVs, the performance difference between the proposed algorithm and other deployment methods decreases.
5.4.4. Impact of the Number of GUs
Figure 10 compares the network throughput of different deployment methods when varying the number of GUs. With the increase of the number of the GUs, the performance of the proposed algorithm increases and is always better than the other deployment methods. The reason is that the GUs are evenly distributed, the greater the number of GUs, the more GUs are close to the GBS, then the UAVs are deployed closer to the GBS, and the throughput of the network is improved at last.
5.4.5. Impact of the Radius of GU Distribution
The influence of the radius of GU distribution on the network throughput in different deployment algorithms is shown in Figure 11. The distribution radius varies from 500 to 1000 meters and the performance of the proposed algorithm always outperforms other algorithms. Furthermore, with the increase of the distribution radius, the performance of the proposed algorithm decreases. The reason may be that the GUs are assumed to obey uniform distribution, and when the distance between GUs is farther, the average distance between UAV and served GUs is farther, causing the throughput of GUs decreased. In addition, the network throughput trend of the purple line is different from the other three. The difference between the algorithm of the purple line and the algorithm of the orange line is that the initial position of UAVs is different. We can see that the network throughput trend of the orange line is the same as the proposed algorithm and the deployment method based on K-means algorithm. So the initial position of UAVs is the key factor that needs to be considered.
6. Conclusion
Different from the previous researches that ignored spectrum resource management, we investigated the joint optimization of the position placement and the spectrum resource management for UAV base station networks with the constraint of the limited backhaul capacity in this study. To resolve the challenge of multiuser optimization and the challenge of the coupling relationship between optimization variables, we modeled the joint optimization problem as a hierarchical game decision architecture. The UAV placement was modeled as an outer game while the spectrum resource management was modeled as an inner game, and both of the two games were proved to be EPGs. Furthermore, we proposed two BR-based algorithms to optimize the spectrum resource management and the UAV position, respectively. Compared with the K-means deployment algorithm and the fixed granularity exploration algorithm, the proposed UAV deployment algorithm can improve the total throughput by 7% and 20% at least, respectively.
Data Availability
The simulation data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant No. 61901517, No. 62101595, No. 62001514, and No. 61931011, in part by the Jiangsu Province Natural Science Foundation under Grant BK20200580, and in part by the National Postdoctoral Program for Innovative Talents under Grant BX2021371.