Abstract
Internet of Things (IoT) devices frequently utilize wireless networks operating in the Industrial, Scientific, and Medical (ISM) Sub-1 GHz spectrum bands. Compared with higher frequency bands, the Sub-1 GHz band provides broader coverage and lower power consumption, which are desirable properties for low-cost IoT applications. However, low-power and low-cost IoT modules cause high variability in network performance. The varying influence from real-world environments additionally undermines wireless propagation and aggravates this variability. We explore these influences and provide a checklist of potential factors affecting wireless network performance in real-world environments. Using multiple low-cost IoT modules, we conduct multiple experiments in five real-world scenarios: indoor, street, open field, ground-to-drone (G2D), and drone-to-drone (D2D). Specifically, the tests are conducted inside a building, on a straight street with wooded sidewalks and aligned houses, on an open field golf course, and high up in the air between drones. To understand the difficulty of reproducibility in IoT deployments, we studied the effect of factors in four categories. This includes the effect of path (line of sight, distance, and obstruction), configuration (transmit power level), weather (precipitation, temperature, and humidity), and installation (IoT module mobility and position). We find that some of the factors in the path and weather categories have the most influence among all the factors, while the rest have moderate to low impacts. In the end, we provide a complete checklist of all the tested factors, which we believe would be constructive not only to academics but also to industrial practitioners working on wireless IoT systems.
1. Introduction
The wireless signal continuously varies and attenuates as it propagates from the transmitter to the receiver [1]. Apart from the attenuation along the line of sight, many other factors would affect the wireless signal propagation in a real-world deployment. The combined impacts are exceptionally intricate and undetermined, especially for the low-cost Internet of Things (IoT) modules. These modules are produced in large quantities and have high variability in quality and may also be highly sensitive to minor changes in deployment scenarios. As is indicated in [2], the production of low-cost wireless modules is significantly different from the regular networking environments. The wide range of the frequency bands, 169 MHz to 2.4 GHz, used by these modules makes the testing more complicated and expensive [2]. It is also incredibly arduous to reproduce a specific experiment since uncontrollable factors such as weather and surroundings are almost impossible to replicate.
Reducing the influence of different real-world environments, such as indoor, urban, outdoor, and line of sight, becomes increasingly noteworthy. We believe that a checklist to keep in mind when conducting experiments and analyzing raw data would significantly help academics and practitioners working with IoT technologies. Hence, we have recorded and studied the observed behaviors of several IoT modules under selected environments and conditions, aiming to provide an initial checklist of potential factors affecting wireless network performance in realistic situations. This paper is the first to present a full checklist of realistic factors and each factor’s influence based on empirical pieces of evidence to the best of our knowledge.
Moreover, Sub-1 GHz wireless networks are developing into one of the critical elements of low-cost IoT applications. These are low-frequency networks that consume less energy, cover a broader area, and are less prone to interference than higher frequency networks [3]. However, the need for low-cost IoT modules also brings variability and instability to the low- and high-frequency networks. Thus, to understand the performance disparity of different frequency bands, we selected two Sub-1 GHz modules Digi XBee-PRO 900HP (900 MHz) [4] and two 2.4 GHz modules Digi XBee Zigbee (2.4 GHz) [5], for initial analyses. Technical details of the modules and protocols will be introduced in Section 3. More information about the 900 MHz frequency band can be found in [6, 7].
Higher frequency bands (e.g., the 2.4 GHz band used by the Digi XBee Zigbee modules) carry more information than lower frequency bands. However, they have higher attenuation and higher interference. Also, the 2.4 GHz band is now congested with signals from Wi-Fi and Bluetooth. Lower frequency bands, such as Sub-1 GHz Industrial, Scientific, and Medical (ISM) band, which in our case is the 900HP module, have longer ranges and lower interference. Sub-1 GHz ISM band is used in primarily proprietary links with fewer competing applications using the same spectrum [3]. Because of their long wavelengths, Sub-1 GHz signals can pass through walls and turn corners better (bend farther around obstacles and reduce the blocking effects), thus making them propagate among buildings in urban environments better. However, they have a lower data rate. Nonetheless, the extended range and lower energy consumption generally make them preferable to IoT applications [3].
Therefore, our experiments are targeted in the context of the typical lower frequency band networks, Sub-1 GHz wireless networks. Specifically, we use the Digi XBee 900HP radio frequency (RF) modules to analyze and determine the common factors affecting network behaviors in real-world scenarios. We have conducted controlled indoor and uncontrolled outdoor experiments in different real-world scenarios and conditions to record the experimental environments’ potential influence. Five experimental setups were arranged for the field tests: indoor, street, open field, ground-to-drone (G2D), and drone-to-drone (D2D). In other words, the experiments were conducted inside a building, on a straight street with wooded sidewalks and aligned houses, on an open field golf course, and high up in the air between drones.
Our study’s most important contribution is to show the impact of factors affecting Sub-1 GHz wireless network performance in real-world conditions by recording and analyzing field tests’ experimental results. We classify these factors into the following four categories: (1)Path: the physical medium between the modules, such as the distance, the line of sight, and the environment (obstacles and surroundings)(2)Configuration: the configurable parameters of the modules, such as the transmit power level, number of packet delivery attempts, and channel mask(3)Weather: precipitation (such as rain, hail, snow, sleet, or thunderstorm), along with temperature and humidity(4)Installation: IoT module mobility, position, antenna direction, and hardware components
Some of these parameters are not reported when IoT devices’ performance is specified (by manufacturers or scholars), but they affect the performance often in a significant way. At the end of this paper, a complete checklist of all the tested factors is provided.
The organization of the paper is as follows. In Section 2, we present the related work and the motivation. In Sections 3, we discuss the experimental setup and the potential factors. In Section 4, we analyze and present the experimental results in addition to the complete checklist. Finally, we give overall conclusions in Section 5.
2. Related Work
Wireless communication technologies have been developed and standardized for years. Recently, the growth of IoT applications has resulted in increasing interest in the performance of these technologies. Therefore, performance analyses become indispensable and crucial for the comparison and selection of wireless modules.
For example, Ferrari et al. evaluate wireless sensor networks’ indoor performance in realistic scenarios by comparing the Zigbee and Z-Wave protocols in different topologies [8]. Rathod et al. test Sub-1 GHz modules in real-world environments to select a network deployment inside their campus and differentiate the wireless signal propagation performances in different indoor and outdoor ambiances [9]. Vondrouš et al. evaluate mesh networks’ performance in the ISM frequency band to find the leading cause of the degradation in network stability and the decrease of Quality of Service (QoS) [10]. Aust et al. evaluate transmission characteristics’ performance and discuss the transmission boundaries and the modulation schemes of Sub-1 GHz modules [11]. Robinson and Knightly investigate the deployment factors in the aspects of topological and structural characteristics [12]. However, these experimental results are mostly not reproducible because the wireless signal is susceptible to even minor changes in the environments. Hence, it is essential to record as many factors as possible to see their extent of influence in the measured settings since they potentially affect wireless signal propagation.
With the increasing investigation on performance evaluation and technology comparison, leveraging realistic environments and network emulation are drawing more attention and are being continuously discussed. For example, Khan et al. indicate the time-consuming, expensive elements of extensive hardware and human resources in real-world experiments [13]. Jardosh et al. use simulations to produce real-world scenarios for mobile network evaluation using a mobility and signal propagation model [14]. Suranata et al. record and analyze their field experiment results on the practical efficiency of Sub-1 GHz networks for low-power systems [15]. Researchers also try to ease the tension between the realistic environment and simulation by adding realistic simulation features. For example, Judd and Steenkiste develop a wireless emulator for practical and repeatable experimentation in the physical layer to leverage the natural environment and the repeatability of simulations [16]. They use the emulator to understand real-world experiments better and improve wireless network applications [17]. Moreover, performance evaluations on wireless networks often use specific protocols, standards, and metrics, for example, received signal strength indicator (RSSI), throughput, round-trip time (RTT), and packet loss rate (PLR) [10, 12, 16, 18, 19]. However, few research papers have worked on balancing theory and practice and capturing the target configuration and the environment’s realistic behavior.
Moreover, Sub-1 GHz wireless links have gained more attention due to their practicability for low-cost IoT applications. However, due to the complexity of real-world situations, the real-world robustness of Sub-1 GHz wireless networks has become a critical issue. It cannot solely rely on abstraction and assumption when developing new protocols or new hardware. Srinivasan et al. present their observations of low-power wireless links and summarize the standard assumptions for network protocols based on the observations [20]. Their experimental observations indicate that these assumptions are not always valid. For instance, “link quality is the same on all channels” is not always valid. Thus, it is crucial to involve the realistic behaviors of wireless hardware modules. Sha et al. propose a protocol for data-intensive sensing applications based on empirical power control and interference models [21]. They study the correlations between transmit power and the RSSI by implementing real-world experiments. Liando et al. conduct real-world experiments with practical conditions to verify the performance of Long Range (LoRa) networks. The results show that LoRa’s performance is severely affected in real-world situations with obstructions such as buildings and vegetation. Kim et al. investigate the Sub-1 GHz frequency-hopping-based 6LoWPAN and verify the impact of network size and other factors on the network performance in natural environments [22]. Chandu et al. analyze the performance of a proposed Sub-1 GHz IoT system integrated with temperature and humidity sensors in different settings [23].
Even though prior research on wireless performance has been intensively investigated, there is no complete guideline on what should be examined and how they should be examined for real-world deployments of wireless devices, specifically, Sub-1 GHz wireless networks. Our goal is to provide a helpful checklist of potential factors affecting wireless performance in realistic environments. We believe it will facilitate and speed up future experiments and serve as a primary reference for analyzing raw data. Specifically, we resolve to initiate this checklist of realistic parameters in the context of Sub-1 GHz wireless networks, crucial for low-cost and long-range IoT systems.
3. Experimental Setup
To explore the realistic factors for signal propagation and wireless network performance, we set up controlled and uncontrolled experiments in indoor and outdoor environments, respectively. As indicated earlier, five experimental setups and multiple low-frequency IoT modules were selected for the tests. The RSSI values and PLR were recorded during the tests. The five setups are listed below: (1)Indoor. This setup is inside a building. The transmitter is stationary, while the receiver moves away from it and stops every 10 m. We recorded the RSSI values at every stop(2)Street. This setup is on a straight road with wooded sidewalks, aligned apartments, and houses. We recorded the RSSI values every 50 m(3)Open Field. This setup is on a golf course that has very few obstacles around. We recorded the RSSI values every 50 m(4)Ground-to-Drone (G2D). In this setup, the receiver and its battery are attached to a drone. Meanwhile, the transmitter is installed to a table on the ground (G2D), as shown in Figure 1(a). We recorded the RSSI values every 50 m(5)Drone-to-Drone (D2D). In this setup, the transmitter and receiver with batteries are attached to two different drones. Additionally, a third module is set up on the ground to monitor and initiate the experiments. This module controls the operations by sending messages to the transmitter. The transmitter then starts the test and continuously sends RSSI values between the transmitter and the receiver to the ground. We recorded the RSSI values every 50 m
(a)
(b)
The modules on the ground in all the experiments are set up on a 70 cm high, round table, as shown in Figure 1(b). Based on the above setups, we aimed to study the factors’ influence in the four categories described in Section 1.
Our aim here is to experiment with commonly used Sub-1 GHz protocols (i.e., Zigbee and its variations). We did not include Low-Power Wide-Area Networks (LPWANs) in this study. We have experimented with LPWANs in the past [24]. Also, while LPWANs are designed for IoT, they have been mostly ignored by IoT devices and designers. Popular LPWANs Sigfox and LoRa both require a service provider. Therefore, the deployment has been extremely slow. While they may have been used by some utility companies, there are hardly any cities in the United States where these services are available for the general public. Instead, 2G (GSM), 3G, and 4G are commonly used for all long-distance IoT applications. 5G has numerous features exclusively designed to support IoT. It is expected to be commonly available everywhere in the near future further diminishing the prospects for other LPWANs. Even Wi-Fi 802.11ah, which has been designed by IEEE 802.11 WG specifically for IoT, is not in common use. To the best of our knowledge, Zigbee and Wi-Fi (not including 802.11ah) are used in most IoT devices used in homes currently.
Therefore, out of simplicity and their popularity, we selected the Digi XBee-PRO 900HP modules, as discussed in Section 1, to construct an end-to-end wireless network in our experiments. Each module works with a 2.1 dBi, half-wave dipole and omnidirectional antenna on a specific frequency in the 902-928 MHz band using pulse-width modulation (PWM). According to the specifications, the RF data rate and the transmit power can support up to 200 kbps and 250 mW, respectively.
The networking protocol of the 900HP module is the DigiMesh protocol developed by Digi International Incorporation [25]. This protocol is similar to the Zigbee protocol utilized by the Zigbee module, with fewer complexities and more flexibility. Zigbee and DigiMesh are based on the IEEE 802.15.4 standard [26]. The physical layer and Medium Access Control (MAC) sublayer are built as defined in the standard. The differences between the Zigbee and DigiMesh protocols can be found in [25]. The DigiMesh protocol is suitable for low-power IoT devices since it targets power-sensitive applications relying on low-power batteries or power-harvesting technologies.
Notably, we set up an extra modular coordinator connecting to the transmitter for test management and monitoring, as shown in Figure 2. With this coordinator and the configuration software, XBee Configuration & Test Utility (XCTU), we could remotely change the transmitter’s settings during the tests [27].
RSSI and PLR are used as performance metrics to determine the transmission quality of the wireless modules initially. The RSSI value can differentiate the channel status (i.e., crowded or not) and show how the broadcast signal strength can be optimized. Still, it only indicates the energy of the signal detected at the antenna port, which means that the value may carry undesired amounts of background noise and other interference. Using it in a combination of different metrics such as SINR (Signal-to-Interference-plus-Noise Ratio), SNR (Signal-to-Noise Ratio), PDR (Packet-Delivery Ratio), or PLR makes it a good indication of link quality [28, 29].
A low baud rate may affect the throughput or data rate of the wireless transmission if serial data is lost or delayed due to operational conditions. Depending on the specific implementation, a wireless module might strategically adjust the data rate according to RSSI or SNR, such as [30]. Theoretically, the baud rate and payload do not influence the signal strength, but they are constant throughout our investigation to reduce the possible bias of implementation and practical experience. On the other hand, the transmitted power value does affect signal strength, so it is vital to keep it constant throughout the experiments as a control variable. An independent investigation on the relationship between transmit power and RSSI is presented later in Subsection 4.3.
After conducting several test runs with the number of samples ranging from 100 to 1000 samples to determine the number of samples required in each experimental run, we found out that the 95% confidence intervals of the results stabilized and stayed constant between 200 and 1000 samples. Hence, for convenience, we decided to use 200 samples per experimental run. Each experiment took about six minutes per run because a packet error or loss can take a fixed five-second timeout, which occurs more frequently when the distance between the transmitter and receiver is considerable. The other settings have been chosen as the default values recommended by the manufacturer. Thus, the critical settings of the wireless modules are as follows: (i)Baud rate: 9600 b/s(ii)Packet payload size: 50 bytes(iii)Transmit power: 24 dBm (250 mW)(iv)Number of packets (samples): 200 packets
As indicated earlier, all the ground modules are set up on a 70 cm high, round table, as shown in Figure 1(b). In addition, each module is connected to Arduino Uno development boards with the Xbee Shield for Arduino for power, control, and data collection. With these general settings and metrics, we describe the details of specific experimental setups in this section.
3.1. Indoor
We configure two XBee-PRO 900HP modules with XCTU and set them up inside a building. The transmitter module is plugged into a Windows laptop. A power bank battery powers up the receiver. The transmitter outputs the raw data to the computer using two scripts, one of which is written in processing [31]. The first script is an Arduino script that sends requests and synchronously reads RSSI values from the receiver until it collects the desired number of observations, which we set at 200. The second script is a processing script that reads the raw data from the transmitter and stores it as a comma-separated value (CSV) file to the computer. We use the stored data to draw graphs and analyze the experimental results afterward.
The transmitter is set up stationary on the first floor of the building while the receiver is mobile. The receiver is moved from a distance of 0 m to 50 m, in steps of 10 m, away from the transmitter. The maximum distance is 50 m because PLR exceeds 10% and significantly increases after 50 m (starting 60 m) for our modules. This is a typical range limit for most current IoT wireless modules. Since the modules are in common areas in the building, the ground truth direct distance between the modules is calculated approximately by measuring the floor’s dimensions. The direct distance means the straight-line distance from the transmitter to the receiver along the building walls. Since the indoor test is for understanding the effect of reflection, scattering, diffraction, or the multiobstacle environment, we consider the approximate distance measurements acceptable. The place is a long hallway, 3-meter wide with two walls alongside, with no significant obstacles in the way. There are objects along the walls but not directly blocking the transmitter and the receiver. We avoided direct obstacles between the transmitter and the receiver for all experiment setups. The modules were both placed on a round table.
Existing indoor wireless networks have been studied to identify potential interference. To the best of our knowledge, only conventional Wi-Fi networks (no Wi-Fi HaLow/IEEE 802.11ah networks that use the 900 MHz band) were present inside the buildings besides cellular networks. Our aim is for the Sub-1 GHz wireless network, so the conventional Wi-Fi networks, whose frequency bands are at 2.5 GHz and 5 GHz, would not interfere much with our experiments. Also, to avoid cellular interference from cell phones, the tests were performed during weekends when only a few people were present in the building. Therefore, we consider that the interference from other wireless networks is minimal and acceptable for the experiments.
3.2. Street and Open Field
We set up the range tests on a straight street with moderately wooded sidewalks and an open field of a golf course. The street is generally flat and straight at the segment of the tests. There were no obstacles, vehicles, or pedestrians directly between the transmitter and the receiver during the tests. The golf course has fewer obstacles and surrounding objects than the straight street. The receiver was moving straight from a distance of 0 m to 300 m, 50 m at a time. The transmitter sent 200 packets at each stop.
3.3. Ground-to-Drone (G2D)
In this experiment, with the transmitter staying on the same table, the receiver is in the air, attached to a drone (see Figure 1(a)). As shown in Figure 3, 200 m was the height limit for the drone. The drone with the receiver is flying vertically at the beginning until it reached 200 m. When it hits the height limit, it begins to fly horizontally until the slant distance reached 600 m. We collect the metric data at every 50 m of the direct distance between the ground transmitter and the drone receiver. The transmitter collects the results on a computer during the tests.
3.4. Drone-to-Drone (D2D)
In the D2D setup, we attach the transmitter to one drone and a receiver to another drone. As shown in Figure 4, the ground module initiates the experiment and then collects the results. The transmitter starts working according to the command from the ground module. The Arduino script of the transmitter synchronously sends repeated requests to the receiver and collects RSSI values. The transmitter then sends the RSSI values to the ground module for storage after the experiment run is complete to avoid any possible interference effect on the experiments. Finally, the ground module saves the collected data on a computer. The ground module and two drone modules are configured to be on the same network. The communication between the transmitter and the ground module only happens at the beginning and the end of the experiment to avoid possible interference with the testing requests and responses. The distance between the two drones starts from 50 m to 700 m, in increments of 50 m.
4. Experimental Result
This section presents the experimental results and discusses the potential factors affecting Sub-1 GHz wireless performance in real-world environments. As discussed earlier, Sub-1 GHz networks provide a more extended range and broader coverage and consume lower power than the higher frequency networks, such as 2.4 GHz. However, the long range requires a clear line of sight [32] or a clear Fresnel zone [33, 34]. Otherwise, the range would be severely reduced by a non-line-of-sight transmission path with obstructions, shown in our experiments and the observations conducted in [35]. Moreover, Sub-1 GHz networks are also known for their capability of large-scale deployments with many connected devices. The performance of such deployments usually correlates with the network protocols and the hardware configuration. The impact of network size is out of this paper’s scope and can be found in [22].
Generally, the experiments show that wireless signal propagation is affected by the characteristics of the medium of the transmission path (distance, path, and surroundings) [36], configuration, and qualitative conditions, such as weather and installation. Qualitative factors are problematic to be quantified but cannot be neglected in real-world experiments. As mentioned earlier, we categorize them into four categories: path, configuration, weather, and installation. In the following sections, we examine and present the factor effects in the context of the line of sight (path), Fresnel zone (path), configurable parameter (configuration), weather, and installation, along with the overall extent of influence.
4.1. Line of Sight
The line of sight represents the medium through which the wireless signal travels from the transmitter to the receiver antennas. The wireless signals change characteristics as they propagate through the line of sight [32]. These changes come along with the distance between the transmitter and receiver. They also depend on the surroundings’ variations (buildings, trees, vehicles, and people). Simultaneously, the reflection and absorption along the path accelerate the changes. It is hard to generalize a realistic channel model due to the continuous variation of the real-world environments. Hence, many studies use empirical models layered on measurements in different real-world situations [37, 38].
As stated in [36], the leading influencers of the line of sight are path loss, multipath, and shadowing, which were demonstrably reflected in the experimental results. As shown in Figure 5, RSSI values decrease as the distance increases and increase as the environments become more spacious (from indoor to D2D). When the signal travels further, it becomes weaker. On the other hand, the space correlation reflects multipath and shadowing effects, such as reflection, absorption, diffraction, and scattering. These effects attenuate and interfere with the signal further in the path. When the environment becomes more complicated (from D2D to indoor), the signal strength becomes weaker, even if the modules’ distance is the same. Similarly, the furthest distance with no packet loss increases in the same way, from 30 m (or indoor) to 600 m (or D2D), as shown in Figure 6.
From Figures 5 and 6, we see that the signal travels the worst in the indoor environment. Street and the open field are slightly better than indoor. The G2D scenario is slightly worse, and D2D (high in the air) is the best. For the indoor scenario, 50 m is the furthest distance. In the street and open field scenarios, the operating distance of wireless modules is also limited. They can barely communicate with each other after the distance is more than 350 m. In the G2D test, the signal reaches 600 m with 8.6% PLR, while in the D2D experiment, communications can be maintained up to 700 m with 10% PLR.
To determine the PLR threshold for our experiments, we conducted the experiments until the successful transmission rate reached 90%, after which we found that the PLR increases rapidly to about 90% from below 10%. It means it has almost reached the limit of the communication range for the devices. Also, based on our experience and conclusions in some literature, such as [39], we believe that PLR above 10% is undesired. In most such cases, additional measures are needed to ensure the minimum performance (e.g., data retransmission to ensure data reception). Therefore, we set 10% PLR as the threshold for our experiments. As shown in Figure 6, the indoor experiment begins to have packet losses and failures after 30 m. In the street and open field scenarios, the furthest distances with no packet loss are 250 m and 150 m, respectively, much less than those in the air. No packet loss is detected until 450 m and 600 m in the G2D and D2D scenarios, respectively, much better than the indoor, street, and open field scenarios. As expected, the fewer objects around, the better the performance. In addition to that, the closer the receiver to the transmitter, the less chance of losing a packet or encountering a transmission failure. In other words, a simpler environment with fewer obstacles provides more stable conditions for the wireless signal. Finally, according to the 95% confidence interval bars shown in Figure 5, wireless performance is generally more stable in the G2D and G2D scenarios because their results have lower variance (narrower confidence intervals) at all distances.
4.2. Fresnel Zone
A Fresnel zone is an ellipsoid region of space between and around a transmitter and a receiver [33, 34]. Transmitted radio, sound, or light waves can follow slightly different paths before reaching a receiver, especially if there are obstructions or reflecting objects between the two ends.
As shown in Figure 5, the signal propagates the best in the D2D scenario. This conforms to how the wireless signal travels in the Fresnel ellipsoid region. The fewer obstacles in the ellipsoid zone result in better signal propagation. Even though there is a direct line between the modules and not many objects around in the open field experiment, the RSSI value is still not significantly higher than the street experiment, from which we speculate that the signal is also reflected and shadowed by the ground. As shown in Figure 7, the Fresnel zone intersects with the ground in the street and open field scenarios. This also happens in the G2D scenario, although its effect is not as much as in the street and open field scenarios. We compare the specific midpoint radii of the first Fresnel zones of various testing distances with the height of the wireless modules in the street and open field tests in the following paragraphs.
The midpoint radius of the th Fresnel zone radius can be calculated by Equation (1) [33]: where is the distance between the midpoint and the transmitter, is the distance between the midpoint and the receiver, and is the wavelength of the wireless signal. We used meter (m) as the distance unit for both and .
We know that the frequency band range of the experimental modules is 902-928 MHz; thus, the wavelength should be approximately from 0.32 m to 0.33 m according to Equation (2), the wavelength formula, where is the signal velocity or phase speed of the signal wave and is the frequency. We assume that the signal velocity equals the speed of light in free space, which is m/s.
Combining Equations (1) and (2), we calculate the midpoint radius of the first Fresnel zone for each experimental distance, as shown in Table 1.
Table 1 shows that the experimental Fresnel zones’ midpoint radii exceed the table height (0.7 m) in all experiments. Thus, in the experiments set up on the table, the first Fresnel zones of the transmitter and the receiver all intersected with the ground. It explains the experimental result that the performance in the air was better than those on the ground, no matter what the surroundings were.
4.3. Configurable Parameter
Manufacturers of wireless modules progressively release their products with software-defined parameters, giving customized functionalities to users who want more flexibility. With different settings to particular parameters, the deployed modules may have significant differences in performance. In the open field experiment, we set different transmit power levels from 7 dBm to 24 dBm for both modules at each distance. As shown in Figure 8, settings of higher power levels provide better performance than the lower ones. However, a higher power level causes a higher power drain. There is a trade-off between performance and power consumption for this setting. In other words, setting a higher power level gives the module better immunity to the interference, but it costs more energy than a lower power level.
Furthermore, for lower transmit power levels at the same distance, it is more likely to experience packet errors or losses with moving objects such as people or pets coming across [40]. We want to note that the other scenarios on different power levels have similar results, but we show only one scenario for simplicity. As indicated earlier, this paper’s experiments were conducted without moving objects, people, or animals between the transmitter and receiver.
Configurable parameters of the wireless modules peculiar to certain manufacturers also affect the performance at different levels. For example, the channel mask parameter of Digi modules allows the user to select different frequencies from a set of supported rates. This reduces the chance of using interfering frequencies by averting interference among modules using the same frequency band. Proper settings of these kinds of configurable parameters may provide more sensible performance. Parameters such as preamble ID, number of packets for insurance of reception, times of packet delivery attempts, sleep mode, and wake time have similar effects. Mainly, preamble IDs help avoid interference among networks operating in the same radio frequency band. Lastly, the interference between neighboring nodes can also affect wireless performance, which can be mitigated by employing an interference-aware routing protocol, as stated in [19].
4.4. Weather and Installation Parameter
In addition to the propagation path and configuration effects, wireless performance is also affected by the weather and installation parameters. Some of these parameters may not directly change the wireless module or the line of sight but may change other equipment pieces working with the modules. For example, the temperature may influence the battery discharge and consequently affect the emitted wireless signal [41]. We did not find any suspicious influence of battery in the experiments reported here, but it happened in our previous experiments with LoRaWAN [24]. Similar effects may exist in other pieces of hardware devices. In this subsection, we discuss such factors based on empirical evidence.
As stated in [42], weather conditions affect wireless performance at different levels. Different forms of precipitation, such as rain and snow, have the most significant influence because the wireless signal could be absorbed and interfered with by raindrops and snow [43]. The signal will be reflected, scattered, and obstructed during the propagation. We conducted experiments on a rainy and sunny day, respectively, with the D2D setup. As shown in Table 2, the mean RSSI value is much lower on a rainy day than a sunny day. However, it is worth noting that while the drone was flying on a rainy day, it was hard to keep the drone stationary in the air. The drone’s slight movements could be a confounding parameter to cause the wireless modules’ worse behavior on a rainy day. We speculate that other forms of precipitation (hail, snow, sleet, or thunderstorm) have similar effects on the wireless modules from the rainy day experiments.
In addition to different forms of precipitation, which directly interfere with the electromagnetic waves, temperature and humidity are also potential factors that indirectly affect wireless performance. As indicated earlier, the temperature may not directly change the propagation path or the signal. Instead, it may affect other hardware pieces of equipment, which could indirectly cause performance differences. By conducting experiments under different outdoor temperatures, we find that low temperature is more likely to cause unstable wireless performance. We used a lithium-ion power bank in the experiments. Its permissible discharge temperature is -20°C to 65°C, and the operating temperature of the XBee-PRO 900HP module is -40°C to 85°C. Tests at the temperature of -3°C and 15°C should have similar results. However, as shown in Table 2, experiment results at -3°C (clear sky at night) have a higher standard deviation than results at 15°C at the same distance, 100 m.
We also find that humidity affects wireless performance more in long-range than in short-range communication from the experiments. As shown in Table 2, the standard deviation of the RSSI values on a foggy day with a 500 m distance is much higher than that at a distance of 100 m in the D2D setup. The weather-related factors are confounded with each other, and thus, the experimental results of one factor may also be affected by other factors. We tried our best to keep the variables as consistent as possible, but this difference is inevitable in real-world experiments. Thus, weather-related factors are considered qualitative parameters in this research.
Additionally, the wireless performance is also affected by the transmitter or receiver’s mobility [18, 24, 43]. We conducted a new experiment with a receiver on a constantly flying drone and compared it with the original stationary case. As shown in Table 2, the moving receiver has a lower RSSI value than a stationary receiver. Due to Doppler shift [44], when either end or both ends of the end-to-end network are moving, the frequency of the received signal changes. The speed in the moving receiver experiment is about five m/s. The receiver was attached to a drone, and the constant speed was set along with the drone waypoints. We want to point out that there was no appreciable wind on the ground during the experiment, but the wind speed could be different high in the air.
The hardware installation also subtly affects signal propagation via positions of specific devices [24, 45, 46]. For example, wireless modules are installed on top or below the drone carrier. Individually, we did experiments with the wireless module installed in different positions, top or bottom, of the drone in the G2D experiment. The two tests were done consecutively to avoid weather influences as much as possible. In both cases, the antenna on the drone module pointed downward. As shown in Table 2, modules installed on top of the drone perform slightly worse than those installed below it.
Theoretically, the antenna direction should not affect the signal strength in our experiments because we use omnidirectional antennas. Still, practically, this is not always the case, for example, in the G2D experiments. As shown in Table 2, the air modules’ antenna pointing upwards gets weaker signal strength than when it is pointing downwards. We speculate that this is caused by the metal objects in the vicinity of the antennas, such as the battery and the drone body.
Overall, from the above experiments, we see that the qualitative conditions impacting the outdoor wireless performance include different forms of precipitation (rain, hail, snow, sleet, or thunderstorm), temperature, humidity, and installation, along with module mobility.
4.5. Extent of Influence
In this subsection, we explore how (to what extent) each of the factors discussed above influences wireless performance. We quantify the degree of influence by comparing the RSSI differences based on two conditions: condition 1 (C1) and condition 2 (C2), which are recorded in Table 3. This table’s data is from the experiments where the distance between the transmitter and receiver is 100 m.
In the last column of Table 3, the extent of influence is calculated by dividing the difference (Diff) between RSSI values of condition 1 () and condition 2 () by the RSSI value of condition 2 (). As shown in the table, rainy weather has a significant influence on performance. These effects are much higher than those of temperature, humidity, and module mobility, from the same setup of the D2D experiment. The street scenario’s path, which has a complex line of sight and various surroundings, has a similar influence due to the weather. The path of G2D, on the other hand, has a much smaller effect than street but is similar to the open field. The more complex the environment becomes, the more significant influence it brings. The extent of influence depends on how complex the environment is. Finally, temperature, humidity, and module mobility have similar and moderate impacts, while the modules’ installed position has a minor influence in all experiments.
4.6. The Checklist
All the tested factors considered in this work and their effects are summarized in the four categories as shown in Table 4.
5. Conclusion
Real-world environments can be highly dynamic and complex for deployments of Sub-1 GHz IoT wireless networks. We conducted many different types of experiments, which comprise wireless modules on the ground and in the air, to study real-world factors affecting wireless performance. These experiments include five different scenarios: indoor, street, open field, G2D, and D2D. We recorded factors that potentially cause the nonreproducibility of real-world deployments. In particular, factors affecting wireless performance in Sub-1 GHz networks are classified into four categories: path (distance, obstruction), configuration (transmit power level), weather (precipitation, temperature, and humidity), and installation (IoT module mobility status and position).
We found that line of sight and precipitation have much higher degrees of influence than other factors. Temperature, humidity, and module mobility have moderate impacts. The hardware components and their installed position have a relatively slight effect. Finally, we made a checklist out of them to help potential future experimenters. We believe that this checklist would be constructive not only to academics but also to industrial practitioners whose work would involve low-cost IoT wireless modules. As future work, open Sub-1 GHz standards such as IEEE 802.15.4g and open-source hardware platforms such as Zolertia REmote and OpenMoteB [47] could be included to further validate the checklist for completeness and comparison among different Sub-1 GHz hardware and protocols.
Data Availability
The module configurations and experimental data used to support the findings of this study can be accessed on http://www.cse.wustl.edu/~jain/sub1ghz.
Disclosure
The findings achieved herein are solely the responsibility of the authors.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This publication was made possible by the NPRP grant (NPRP10-0125-170250) from the Qatar National Research Fund (a member of Qatar Foundation) and NSF grant CNS-1718929 from the National Science Foundation and in part by Prince Sattam bin Abdulaziz University, Al-Kharj, Saudi Arabia.