Abstract
Nowadays, the popularity of the unmanned aerial vehicles (UAVs) is high, and it is expected that, in the next years, the implementation of UAVs in day-to-day service will be even greater. These new implementations make use of novel technologies encompassed under the term Internet of Things (IoT). One example of these technologies is Long-Range (LoRa), classified as a Low-Power Wide-Area Network (LPWAN) with low-cost, low-power consumption, large coverage area, and the possibility of a high number of connected devices. One fundamental part of a proper UAV-based IoT service deployment is performance evaluation. However, there is no standardized methodology for assessing the performance in these scenarios. This article presents a case study of an integrated UAV-LoRa system employed for air-quality monitoring. Each UAV is equipped with a set of sensors to measure several indicators of air pollution. In addition, each UAV also incorporates an embedded LoRa node for communication purposes. Given that mobility is key when evaluating the performance of these types of systems, we study eight different mobility models, focusing on the effect that the number of UAVs and their flying speed have on system performance. Through extensive simulations, performance is evaluated via multiple quality dimensions, encompassing the whole process from data acquisition to user experience. Results show that our performance evaluation methodology allows a complete understanding of the operation, and for this specific case study, the mobility model with the best performance is Pathway because the LoRa nodes are distributed and move orderly throughout the coverage area.
1. Introduction
The Internet of Things (IoT) is gaining momentum. IoT represents a heterogeneous network scenario with virtually unlimited uses [1, 2]: Smart-Homes, Smart-Cities, Industry 4.0, Smart-Grids, etc. At the same time, unmanned aerial vehicles (UAV), also known as drones, are becoming a very interesting tool for traffic surveillance, crop monitoring, border patrolling, disaster management, remote areas control, or wildfire monitoring, and among others [3]. Examining the characteristics of both, IoT and UAVs, it can be presumed that UAVs could become a natural symbiotic element of the Internet of Things (IoT) [1]. Let us delve into this idea.
On the one hand, UAVs can be classified in terms of several features such as size, communication capacity, flight mode, and wing types. UAVs can work isolated or in groups, giving rise to a new type of communication network called Flying Ad hoc Networks (FANETs) [4]. FANET can be seen as an extension of Mobile Ad hoc Networks (MANETs) with singular features in terms of mobility, topology, wave propagation, and energy constraints. In contrast to other communication networks as MANET or Vehicular Ad hoc Networks (VANETs), the UAVs move freely in the air, including a third axis (z) to the mobility of the devices (x, y) considered so far. Mobility models used for UAVs are usually classified according to its nature and are either created for other networks and then adapted to this new environment or specifically introduced as mobility models [5]. Mobility models are key for an optimal UAV deployment [6]. Numerous factors have a notable effect on the trajectory of UAVs such as energy constraints, collision avoidance, flight time, ground users’ demands, and the specific service in use. For instance, UAVs employed for extension coverage of wireless telecommunications network (better connectivity to terrestrial networks) will need to consider the Quality of Service (QoS) as a key element to optimize their performance [7].
On the other hand, it is a common practice in IoT to use small (and sometimes low cost) sensor devices to capture data from multiples sources. Then, these data are usually sent by means of wireless technology to a gateway that provides Internet connectivity to the cloud, where network servers are located. Network servers are responsible for collecting and processing the data and also making decisions or defining specific actions to be carried out. Although there are several technologies being used for the wireless part of this general IoT communication architecture, Low-Power Wide-Area Network (LPWAN) solutions are standing out [8]. Among their benefits, we can highlight the following: energy efficiency, low cost, possibility of dense deployments, and high performance in a wide coverage area. However, it is important to note that the low data rate (DR) and the duty cycle constraint (1%) make this technology not appropriate for time-sensitive traffic. One of the most popular LPWAN technologies is Long-Range (LoRa) [9]. LoRa uses a proprietary modulation algorithm patented by Semtech [10] as a derivative of chirp spread spectrum (CSS), operating in the industrial, scientific, and medical (ISM) band and spreading a narrowband signal over a wider channel bandwidth. The communication architecture of LoRa (layers and protocols) is defined by LoRaWAN [11, 12].
Consequently, these small sensors used in IoT together with their communications capabilities could be easily embedded in UAVs (e.g., Figure 1). By doing so, the UAVs provide a new framework to deploy IoT-based services. Although the maturity level of UAVs and LoRa/LoRaWAN for IoT is higher and higher, there are some important unsolved issues. From the UAV perspective, mobility is a challenge. Specific mobility models have been proposed in the scientific literature for UAVs and FANETs. However, how to select the best mobility pattern for a given service is still an open issue. From the IoT perspective, there is not yet a standardized methodology for performance evaluation in terms of quality [13].
In this paper, we address these two questions using a case study. Particularly, we evaluate the performance of an IoT air-quality monitoring system that integrates LoRa/LoRaWAN and UAVs. Each UAV incorporates an IoT device that has sensors, which measure the quality of the air, and a LoRa node for sending these data to a gateway. Eight different UAV mobility models are tested, namely, Random Walk (RW), Random Waypoint (RWP), Random Direction (RD), Gauss–Markov (GM), Reference Point Group Mobility (RPGM), Pathway, Semi-Random Circular Movement (SRCM), and Smooth Turn (ST). Performance is measured using four quality components, namely, Quality of Data (QoD), Quality of Information (QoI), Quality of user Experience (QoE), and Quality Cost (QC). These quality components encompass the whole process from data acquisition to user experience and were introduced in previous works [14, 15]. Observe that, for each mobility model, we will measure the effect on performance of both the number of UAVs, equivalent to the number of IoT end-nodes, and the UAVs’ flying speed. Through intensive computer simulations, we find out the best mobility model to be used for this IoT system. Results show that the mobility models that orderly cover all the areas obtain the best performance for the service under study.
The rest of the paper is organized as follows. A review of the state-of-the-art literature is included in Section 2. Section 3 describes the materials and methods used in this study. Simulation results are shown and discussed in Section 4. The paper ends with a conclusion in Section 5.
2. Related Work
There are many proposals in the scientific literature addressing the trade-off between resource optimization and performance in UAV deployments. It is well known that the mobility model is one of the key factors with a high impact of performance. Therefore, in this section, we first review previous works related to optimal location, distribution, and trajectories of UAVs. Then, we focus on studies that analyzed the impact of UAV mobility in the performance of the system in terms of quality metrics. Finally, we explore those works that have proposed the combined used of IoT wireless communication technologies, e.g., Wireless Sensor Networks (WSN) or LoRa, and UAVs.
In [3, 16], authors studied mobility models in UAVs. The goal was to identify the best mobility model in order to achieve higher WLAN coverage without decreasing performance. They took into account the qualitative and quantitative communications needs and used the number of drones as the main investigated parameter. Similarly, Chen et al. [17] suggested maximizing QoE at the expense of minimizing the total transmitted power by each UAV considering the channel communication constraints between the deployed devices and the base station (or base stations). Gao et al. [18] proposed a high energy-efficient resource allocation scheme considering the mobility of the UAVs and measuring the performance of devices in terms of QoE in dynamic aerial channel conditions and different transmission DR. Vashisht et al. [19] analyzed the impact of the increase of peripherals into UAVs; the idea was that only needed peripherals should be fixed in drones with limited resources to perform all the tasks in an energy-efficient way and increasing as much as possible the flight time.
Cheng et al. [20] proposed a security alternative regarding UAV trajectories and time scheduling with an iterative algorithm solving a convex optimization problem. Similarly, Zhao et al. [21] solved an optimization problem for UAV trajectories and Non-Orthogonal Multiple Access (NOMA) precoding by erasing the interference from the base station to UAVs or minimizing to a given threshold. Also coupling with QoE, contributions [17, 22, 23] resolved different optimization problems to guarantee the QoE requirement using the minimum transmission power in the UAV. In this case, the goal was to provide wider coverage for all nodes in a particular area maintaining QoS requirements in the 3D deployment or maximizing the number of nodes in the coverage area but always being aware of energy efficiency. Other studies [18, 19, 24] addressed different challenges such as high node mobility, fluid topology, or low node density. As an example, Bouachir et al. [24] studied the Random Waypoint (RWP) mobility pattern, having a great impact on QoS metrics, mainly due to interference and packet collisions [25].
Regarding the combination of UAVs and IoT, several studies are addressing the use of WSN in these scenarios. For instance, in [3], the authors studied the features and conditions in a UAV network using different metrics to guarantee connectivity, security, and scalability. Other authors [26–29] evaluated the use of WSN to acquire raw data from the environment and employed LoRa/LoRaWAN to transmit the data to the server, assuming that LoRa covers a range up to 30 km. In [30, 31], the authors integrated WSN with UAVs and proposed an optimal trajectory design, minimizing the total path length, passing close to the main interesting points, and ensuring a minimal delay in the communication. Sharma et al. [32] tested a LoRaWAN network using UAVs for urban surveillance focusing on stress areas, being able to preserve 40% of the network energy consumption.
From a more complete perspective, Yuan et al. [1] assessed and compared the performance in rural and urban environments of UAVs equipped with LoRa, WiFi, and Long-Term Evolution (LTE) networks; results showed that LoRa achieved the best performance with greater swarm density and longer coverage range when LTE was not supported. Kirichek and Kulik [33] addressed the use of different elements in the networks as WSN acquiring data from terrestrial segments. In their proposal, LoRa/LoRaWAN devices transmitted the data acquired from the WSN to a UAV gateway. Then, the UAV acting as a gateway and/or as a repeater forwarded the data to reach the LoRa base station, which, in turn, forwarded the data to a network server. Finally, Trasviña-Moreno et al. [34] developed a complete system with buoys equipped with WSN for marine-coastal environment monitorization and transmitted the acquired data using LoRa technology. This provided an easy-to-use and low-power solution with a large coverage area, one of the main features of LoRa. In parallel, a UAV with LoRa technology acted as a gateway to collect the data up to forwarding to the server that processes it. Finally, security is also one important concern because these devices could be an easy target. Some contributions addressed this issue using an efficient resource management and planning the strategy in accordance with QoS [19, 35, 36].
Despite existing many contributions that evaluated the performance of different mobility models in UAV or the incorporation of LoRa to UAV deployments, they have based the performance analysis only on two quality components, namely, QoS and QoE. For QoS, the examined metrics are the classic ones: delay, jitter, throughput, and packet losses. However, the QoE component does not have a standardized model for IoT services. Consequently, the metrics used to assess QoE in the state of the art are extremely unalike. With this paper, we verify a proposal that harmonizes the performance evaluation of IoT services using a complete UAV/LoRa/LoRaWAN deployment as a case study. The novelty resides on the use of multiple quality dimensions (QoD, QoI, QoE, and QC).
3. Materials and Methods
In this section, we describe the main features of the mobility models used in UAV deployments. Then, we describe the characteristics of the simulation framework that we have used in this study as well as the performance evaluation methodology.
3.1. Mobility Models
Dynamic topology, high mobility, etc., are significant challenges in the design of UAV networks and services. Taking into account that testing with real devices is costly and depends on region restrictions, Camp et al. [25] suggested the use of mobility models under simulation to assess the performance of UAV networks and FANETs. The mobility of UAVs is largely different to the mobility of vehicles placed in the ground, so in most cases the MANET models are not directly applied. According to [5], two primary groups can be found for mobility models in this scenario: adapted mobility models and specific models defined for FANETs.
On the one hand, Xie et al. [5] adapted and extended traditional MANET mobility models from 2D to 3D classifying them into five categories: random, temporal dependency, spatial dependency, geographic dependency, and hybrid mobilities.
The first category encompasses Random Mobility, which, in turn, includes three models, namely, Random Walk (RW), Random Waypoint (RWP), and Random Direction (RD). In RW, the node randomly chooses the orientation and speed during a time interval, and before it ends, it chooses a new random orientation and speed for the next period reflecting or wrapping from the boundaries. In RWP, each node randomly selects a target in the coverage area and the travelling speed. When the node achieves the target, it waits for a random time; then, it chooses a new target and a new speed to reach the new target. Finally, in RD, the node chooses a course and speed moving to the border, where it rests and after that chooses a new direction to go. The difference among RD and RW is the travelling duration, being constant or random, respectively. In contrast, the aim of RWP model is to measure the influence of the range, speed, the number of hops, and the density of nodes in a FANET. Observe that despite the random mobility, the destination position is always in the constrained area range.
The second category is the temporally dependent mobility models that claim to avoid sudden changes in direction and speed, such as the Gauss–Markov (GM) [37] for tracking trajectories of targets. The GM equations are found in [5] and depend on heading speed, direction, and pitch, avoiding abrupt changes close to the borders. The GM model has been largely used for network performance evaluation. Smooth Random Mobility correlates the behavior of the vehicles in the ground by the Stop-Turn-Go model.
The third group is space-dependent, where the mobility of a node depends on the available space and the behavior of the closest nodes. One example is Reference Point Group Mobility (RPGM) that follows the master-slave model, where the members of a group follow the group leader [38]. Another example is Spatially Correlated Mobility, where the behavior of a node depends on the actions of the other nodes.
The fourth category is the geographical dependence classification, including the models with trajectory restrictions due to pathways or obstacles in the way [39]. Lastly, the hybrid models share at least two features of different categories as the Free-way Mobility model that relies on the present context (temporally dependent) and the position of surrounding nodes restricted to lanes on highways (space-dependent). Another case is the Disaster-Area model where many mobility models are included [5].
On the other hand, many mobility models have been specifically created for FANETs. First, in the Semi-Random Circular Movement (SRCM) model, all UAVs are placed around a fixed center and turn around it with a radius, speed, and initial angle. This model is not suitable in MANETs because these conditions are not feasible on ground due to geographical or item restrictions. But it is possible in FANET because of the available free space in the air. Second, the Three-Way Random Mobility assumes the heading speed and three possible states defined on a Markov chain: going straight, turning right, and turning left. Third, the Pheromone Repel model splits the area into small grids trying to cover all the available networks. The behavior of Three-Way Random and Pheromone Repel models close to the border is similar to GM model, choosing to turn completely the direction of the UAV pointing to the interior of the area. Fourth, the Smooth Turn (ST) captures the free-space mobility of the nodes making smooth trajectories with a large radius as aircrafts in 2D and 3D. In the second case, the 3D model has two versions, z-dependent and z-independent, which vary in the correlation along z-dimension and the plane (x, y). Fifth, the Flight-Plan (FP) mobility defines flights where the initial and final point are known beforehand (e.g., regular and commercial), not suitable for autonomous UAVs. Finally, the Multi-Tier Mobility model use different aircraft types that flight at diverse heights. Figure 2 contains a brief summary of the mobility models mentioned in [5].
3.2. Performance Evaluation in LoRa
The simulation is carried out using OMNeT++ [40], INET framework [41], FLoRa framework [42], and Crypto++ [43]. The simulated scenario is composed of one server (called for this purpose Network Server, NS), four LoRa gateways (LoRaGWs), and a variable number of LoRa nodes ranging from five to twenty in the LoRa network. The scenario is depicted in Figure 1. Each UAV incorporates a set of sensors, so that the UAV can act as a mobile air-quality station. Specifically, each UAV includes nine pollution metrics.
During simulations, the UAVs will move around the coverage area at different speeds (10, 25, and 50 km/h). In addition, given the favored characteristic of LoRa nodes, these are also embedded into the UAVs to send the collected data to the LoRaGW. In our scenario, we employ four LoRaGWs to cover the complete area (40 km × 40 km), thus maximizing the coverage area, but considering that duplicated packets might be received. Behind the LoRaGWs, four GWRouters (one for each LoRaGW), one InternetCloud, one NSRouter, and one Network Server compose the backhaul network (Figure 1). When a LoRaGW receives a frame, it forwards it through the cloud to the Network Server. This element processes the packet and obtains valuable information for decision-making. In summary, each UAV in the topology represents an air-quality station with sensors that acquire data from the environment where the UAV is flying, and the embedded LoRa node transmits the raw data to the Network Server through a LoRaGW. Each LoRa node generates packets randomly using an exponential distribution with mean 100 seconds. For a better understanding, sensor measurements are obtained from a real dataset [44] that corresponds to a suburban area. Table 1 and Figure 3 show the configuration parameters used in the simulations.
Our goal is to evaluate the performance of eight different mobility models: RW, RWP, RD, GM, RPGM, Pathway, SRCM, and ST. For each model, we analyze the effect of the number of LoRa nodes (i.e., the number of UAVs) and also their motion speed. The evaluation is carried out using four quality components introduced in [14, 15], each one covering a different dimension of the performance, and thus avoiding overlapping. In this sense, these four quality components are divided into two magnitudes: profit (QoD, QoI, and QoE) and cost (QC). Each component assesses one dimension:(1)QoD: it measures the quality of raw data as acquired by sensors. It is calculated as shown in (1) and it refers to the precision of the sensor reading, the truthfulness of the measurement (it is within range), and the completeness of the measured data (if all sensors work properly).(2)QoI: it measures the quality of the obtained information after raw data have been processed in the server. It includes seven metrics dealing with the quality of the processed data, from the amount of data received to the accuracy of these data. It is obtained as shown in (2).(3)QoE: it measures network performance (i.e., classical QoS values) and the user experience. Initially, only the use of the network interface of the LoRa gateway is considered as a metric different from the well-known QoS metrics used for performance evaluation (delay, jitter, packet loss, and throughput). It is calculated as shown in (3).(4)QC: it measures the cost in terms of resources, e.g., energy consumption, computation capacity, and duty cycle limitation (1%). Its expression is depicted in (4).
It is important to observe that each quality component is the product of several metrics, previously normalized, as shown in (1)–(4). Therefore, QoD, QoI, QoE, and QC are also normalized values [0-1], being 1 the best possible performance and 0 the worst one. Quality components will be calculated at time intervals, called Teval, at the network server. This evaluation period (Teval) is customized according to the monitoring needs. In this paper, we use Teval = 500 s.
4. Results and Discussion
In this section, we show the results obtained after extensive simulations. The eight mobility models have been tested with a variable number of UAVs/LoRa nodes {5, 10, 15, and 20} and different speeds {10, 25, and 50 km/h}. Results are obtained for QoD, QoI, QoE, and QC. The term QoX will be used in this section to refer to any of these quality components.
Generally, the performance in terms of quality components QoX improves as the number of LoRa nodes/UAVs increases. This result applies to all the evaluated mobility models. The only exception is for RPGM because all flying nodes are usually located into a small part of the studied area; therefore, at each Teval, the system only receives packets from one or two grids. This entails the worst value for QoI when the number of LoRa nodes/UAVs is higher in RPGM. In the situation where the number of flying devices is constant and we only change the speed of the devices, the different mobility models obtain similar results with a slight performance enhancement as the speed increases. As we commented previously, the QoI component of the RPGM model is highly affected by mobility, due to the coverage; therefore, the faster the movement, the best the results for this model.
Now, we discuss the particular results for each QoX component. In relation to the data acquisition process, we have measured the quality of the raw data obtained by the UAV-incorporated sensors. Because we are focusing on collecting data, the QoD component penalizes pollution measurements that are not taken. If we observe how QoD varies as a function of the number of nodes, we can see that, in general, the more the number of UAVs, the better the performance (Figure 4). In the scenario with 5 nodes, the raw data obtained by the first LoRa node (node0) is weaker because it has not all the sensors to obtain all the air-quality values. This fact penalizes the QoD in this scenario. As the number of UAVs increases, this “poor” UAV goes unnoticed because the rest of UAVs collect a high number of measurements, thus increasing the value of QoD. The QoD value is practically the same with a constant number of UAVs, though with a certain variance due to the random message generation by each UAV. On this component, the speed at which the UAV moves within the coverage area does not have any influence on the results. Lastly, the scenario with 20 UAVS is slightly better than others in terms of QoD for all mobility models because the effect of that “poor” UAV is lower. The performance for all mobility models are quite similar because the QoD metrics only depend on the raw data, which is obtained from the same dataset that is shared by all the tested mobility models. The best performance is acquired by RPGM and RW models (Figure 4) and the worst by RWP and RD (Figure 5).
(a)
(b)
(a)
(b)
After the data acquisition process, the QoI component measures the quality of the obtained information. QoI depends on seven metrics, as shown in (2), which make it very volatile. In other words, mobility has a higher impact on QoI than in QoD. To this particular case, the better the performance, the faster the nodes because the Network Server receives packets from all grids (i.e., more data are available to monitor all grids in the studied area). The recall metric is responsible for measuring this effect on QoI. The mobility model with the best performance for this quality component is Pathway Mobility due to the orderly arrangement of the UAVs in the coverage area (Figure 6). This means that once processed the data, the Network Server assesses information from all the monitored areas, which is very important for decision-making.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
The use of four LoRaGWs guarantees that there are no hidden or “blank” areas without LoRa coverage. Therefore, the performance in terms of QoE is very similar for all the mobility models. The only metric that varies a little bit more is gatewayavailability, see (3). The reason is that this metric evaluates how busy is the gateway because of the need for sending data, which could create a bottleneck in specific congested scenarios (which is not the case). All mobility models obtain similar results with RD as the best one (Figure 7); this is because this model looks for the edge of the area to change the direction, coinciding with the location of LoRaGWs, so the ratio of delivery packets is higher than in other mobility models. The speed of the flying LoRa nodes/UAVs does not affect performance in the QoE quality component.
(a)
(b)
Finally, we discuss the QC quality component. The energy consumption depends on the state of the transceiver (higher consumption in transmitting state). Due to a random message generation when the number of UAVs is lower, the number of times that transceiver is in transmitting mode is different for each Teval. This fact makes this component unstable because the consumption is highly fluctuating between two time intervals Teval. The best result is obtained for the GM model and the worst for RPGM model (Figure 8), although the difference is small.
(a)
(b)
5. Conclusion
The combination of technologies to design a multifaceted system is a current trend with greater impact on future smart services. UAVs, as well as LoRa, is a very flexible technology with an increasing number of applications. However, today, there is still not a standard to evaluate the performance of these technologies (or combination of them). Therefore, we used a model based on different quality components that address different dimensions of the operation of the service. We compared the results obtained for eight mobility models (Random Walk, Random Waypoint, Random Direction, Gauss–Markov, Reference Point Group Mobility, Pathway Mobility, Semi-Random Circular Movement, and Smooth Turn). From the results, we found out that whereas QoD and QC components hardly vary among models because these quality components do not depend on communication factors, some variation is perceived in terms of QoI and QoE. These two quality components, QoI and QoE, depend on wireless and wired network conditions, and thus, the provision and the movement of the flying nodes across the network have a direct impact on performance. On the one hand, the QoI component achieves a better performance for those mobility models whose UAVs move orderly through the network. On the other hand, QoE relies on network conditions and the best results were achieved for the RD mobility model which minimizes network metrics such as delay, jitter, and packet delivery rate. As future work, we plan to improve the quality model to find out the best relationship among quality components.
Data Availability
The data used to support the findings of this study is open access and cited at relevant places within the text as references [44].
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by the AEI/FEDER-UE Project grant (TEC2016-76465.C2-1-R) (AIM).