Abstract

Canopy temperature is an important variable directly linked to a plant’s water status. Recent advances in Unmanned Aerial Vehicle (UAV) and sensor technology provides a great opportunity to obtain high-quality imagery for crop monitoring and high-throughput phenotyping (HTP) applications. In this study, a UAV-based thermal system was developed to directly measure canopy temperature, skipping the traditional radiometric calibration process which is time-consuming and complicates data processing. Raw thermal imagery collected over a cotton field was converted to surface temperature using the Software Development Kit (SDK) provided by the sensor company. Canopy temperature map was generated using Structure from Motion (SfM), and Thermal Stress Index (TSI) was calculated for the test site. UAV temperature measurements were compared to ground measurements acquired by net radiometers and thermocouples. Temperature differences between UAV and ground measurements were less than 5%, and UAV measurements proved to be more stable. The proposed UAV system was successful in showing temperature differences between the cotton genotype. In conclusion, the system described in this study could possibly be used to monitor crop water status in a field setting, which should prove helpful for precision agriculture and crop research.

1. Introduction

Canopy temperature is an important indicator of water availability, water stress, and irrigation status in agriculture [1]. Measuring crop canopy temperature can help establishing relationships with harvest yield, as well as support water management decisions [2]. Remotely sensed data, which are acquired by sensors on space-borne, air-borne, or ground-based platforms, have been widely used in agriculture to estimate crop parameters such as vegetation indices and Leaf Area Index (LAI) [3, 4]. However, limitations of traditional remote sensing technologies include low spatial and temporal resolutions for time-series analysis, as well as high cost and low efficiency [5, 6].

Unmanned Aerial Vehicle (UAV) and sensor technology are quickly evolving and offering a great opportunity for the development of precision agriculture and high-throughput phenotyping (HTP) systems for a variety of applications. Most studies using UAV for agriculture have focused on red-green-blue (RGB) and/or multispectral sensors to calculate vegetation indices and monitor crop development for yield forecasting. Advanced UAV systems can provide fine spatial and high temporal resolution data at relatively low cost so that crop traits such as height, canopy morphology, and greenness can be estimated [2, 68]. Vegetation indices were calculated by remotely sensed data from a multispectral sensor mounted on a UAV to estimate LAI, which is one of the key parameters determining photosynthesis, respiration, and transpiration of vegetation [2, 9]. Chang et al. [6] proposed a method to monitor the growth of sorghum using a commercially-available UAV, while Anthony et al. [7] used a Micro-UAV system with a laser scanner to measure crop height. Furthermore, Patrick and Li [8] generated 3D models of blueberry bushes from UAV data to extract morphological traits for genotype selection and found a strong relationship between traditional growth indices and image-derived bush volume. The UAV platforms with hyperspectral sensors have used to extract phenotypes and to predict biomass for sorghum [1012]. Ramanurthy et al. [10] extracted features from hyperspectral and RGB images for predictive modeling of sorghum plants. Zhang et al. [11] developed nonlinear regression models to predict sorghum biomass from multitemporal UAV-based hyperspectral and RGB data, while Ali et al. [12] adopted hyperspectral, LiDAR, and RGB data for sorghum biomass prediction.

Although canopy temperature may be used to detect crop stress [13], there are challenges associated with the ability to accurately measure canopy temperature using thermal cameras, whether mounted on a ground-based platform [13], manned and unmanned aerial platforms [1417] or satellite [18]. Data from each platform could be useful for different purposes, however, as long as their different spatial resolution output is accounted for. Postharvest and quality evaluation operations of fruits and vegetables have been conducted using sensors and ground-based platforms [1921]. Bulanon et al. [22] studied fruit recognition using thermal imaging to enhance the robotic harvesting of citrus. Berni et al. [14], Zarco-Tejada et al. [15], and Gonzalez-Dugo et al. [16] used a UAV platform to collect multispectral, hyperspectral, and thermal data to calculate vegetation indices, water stress, and canopy temperature of fruit tree species. Remotely acquired thermal images from aerial platform have also been used to derive the Crop Water Stress Index (CWSI) and map canopy conductance in olive orchards [17]. Thermal imagery collected from fixed-wing platforms at altitudes of 150 m or higher to cover large agricultural areas complicates image preprocessing methods including radiometric calibration and atmospheric correction [14, 15]. Ribeiro-Gomes et al. [23] proposed an uncooled calibration algorithm for thermal camera used in UAV applications for agriculture, while Berni et al. [14] performed laboratory calibration using a blackbody source to estimate stabilization procedure and absolute temperature shifts for the radiometric sensor calibration. Atmospheric correction methods have been applied to thermal imagery based on the MODTRAN radiative transfer model to calculate surface temperatures [15, 16]. Single-layer atmosphere (uniform conditions including air temperature, relative humidity, and barometric pressure) corrections of thermal imagery for fixed-wing UAV have been proposed [14]. Additionally, thermal sensors can be affected by camera tilt angles caused by the platform’s orientation, viewing angle, and directional effects [17]. Rotary-wing UAV systems, on the other hand, could be a good alternative to overcome limitations of fixed-wing platforms. Although rotary-wing UAVs cover smaller areas at lower altitudes (<100 m) usually, these platforms tend to be more stable and provide reliable data with higher spatial resolution (<10 cm) [6]. In recent days, a multirotor UAV with a thermal camera used to estimate the adaptive CWSI for precision agriculture [24]. UAV-based thermal system provided relevant instantaneous and seasonal variations of water status [25].

In this study, we developed a UAV-based thermal system using a quad-copter platform and a radiometric thermal sensor. The framework of data collection and processing is proposed to directly measure crop canopy temperatures without the traditional radiometric calibration and/or atmospheric correction as well as field-based measurement. Thermal images can be collected over a cotton field to generate a canopy temperature and the crop stress index map. Canopy temperature measured by the UAV-mounted sensor was compared with ground-based measurements to evaluate the performance of the proposed system.

2. Materials and Methods

2.1. Study Area and Data Acquisition

The test site was located at the Texas A&M AgriLife Research and Extension Center in Corpus Christi, Texas, USA (-97°33E, 27°47N) (see Figure 1). Two genotypes of cotton (Gossypium hirsutum L.) contrasting in leaf pigmentation were used. The line TAMU4920 (Red leaves) has a marked presence of anthocyanins, whereas DP1044 (Green leaves) do not. Plot size for each line was limited to 20 rows that were 12.2 m long, spaced 0.96 m apart, and followed East-West orientation. The materials were planted on June 20, 2016. Irrigation tapes were installed on July 3, 2016, and plots were irrigated as needed throughout the season, to promote adequate vegetative growth. The crop height between red and green cotton was similar at UAV data collection.

Ground-based leaf temperature measurements were collected in each plot using ten type-T thermocouples (OMEGA Engineering, Bridgeport, NJ), placed on row 11 (Figure 1). The average of ten measurements was used to estimate the leaf temperature of the plots. The sensors were installed at the centre of the abaxial surface of sunlit main-stem leaves using clear surgical tape with the sensor wires secured around the leaf petioles by zip ties. Two 4-channel net radiometers (model CNR1, Kipp & Zonen, Delft, Netherlands) were used to measure net radiation (Rn) over the plots. The instruments were installed at the centre of each field at a height of 2 m above the soil surface. The radiometer measured four components of the surface radiation balance separately: direct incoming shortwave radiation (), reflected shortwave radiation (), longwave radiation from sky (), and longwave radiation emitted by the surface (). was converted to surface temperature [26, 27] as where is the emissivity of the crop, assumed to be 0.96 [28], and is the Stefan-Boltzmann constant ().

The sensors were controlled by dedicated data-loggers (model CR1000, Campbell Scientific, Logan, UT). The thermocouples were multiplexed. For TAMU4920, an AM25T (Campbell Scientific, Logan, UT) multiplexer was used, while for DP1044, an AM16/32B (Campbell Scientific, Logan, UT) multiplexer was used. The data-loggers were programmed to scan the sensors every 30 seconds and compute 10-minute averages.

The UAV system and thermal sensor used in this study were a 3DR X8 octocopter system, (3D Robotics, Berkeley, USA), and FLIR Vue Pro R 640 radiometric thermal camera (FLIR, Wilsonville, USA), respectively (Figure 2). A PixHawk flight controller (3DR, Berkeley, USA) was used for platform and sensor integration; the sensor was mounted to the platform using a custom 3D-printed mount. The thermal sensor was equipped with a 9 mm lens, producing images with pixels and a spectral response in the range of 7.5-13.5 μm. The detectable temperature range of the sensor is -20-50°C. The raw images were saved as uncompressed 14-bit radiometric images with telemetry in standard metadata fields. The camera was factory-calibrated to calculate temperatures with 5% measurement accuracy. The sensor manufacturer, FLIR, provides a Matlab library for conversion of pixel values in the raw image output to surface temperatures using radiometric metadata information and parameters such as emissivity, atmosphere temperature, relative humidity, reflective temperature, and distance of the sensor to target. Raw images were converted to canopy temperature before applying the Structure from Motion (SfM) algorithm to generate the orthomosaic image and canopy temperature map.

The UAV flight parameters were determined based on the field size and sensor specification (see Table 1). We adopted a grid-style flight pattern and nadir view for data collection, as described by Chang et al. [6]. A total of 89 geo-tagged thermal images were collected for the study site.

2.2. Conversion of Thermal Imagery and Image Mosaicking

A Matlab Software Development Kit (SDK) is currently being offered by FLIR as an “Add on Tool” to support the use of FLIR thermal sensors. Although it is not open-source, the user can install it for free and use the tools to view, analyze, and capture data from FLIR thermal sensors directly in Matlab. Five parameters, including emissivity, atmosphere temperature, relative humidity, reflective temperature, and sensor distance to target, have to be entered in order to convert raw imagery to surface temperatures. Emissivity is defined as the ratio of infrared energy emitted by the object, compared to that emitted by an ideal blackbody. We used an emissivity value of 0.96 for cotton [29]. Reflective temperature is any thermal radiation originating from other objects that reflects off the target [30]. We used the temperature measured by a weather station in the field as a reflective temperature. Atmosphere temperature and relative humidity were selected in the National Centers for Environmental Information (NCEI) provided by NOAA [31]. Flight altitude was used as the sensor’s distance to target. The digital number (DN) of each pixel in raw thermal images was input to the function of the SDK with the parameters to calculate surface temperatures. Figures 3(a) and 3(b) show individual raw (digital number) and converted surface temperature images, respectively.

After image conversion, surface temperature images were processed using the Agisoft Photoscan Pro software to generate an orthomosaic image of the study area. Photoscan Pro adopts the Structure from Motion (SfM) algorithm, which is a photogrammetric imaging technique to estimate 3D structures from 2D images [2, 6, 32, 33]. Generally, UAV images can be stitched using tie-points between the images for alignment and geo-referenced with GPS coordinate of image location determined by GPS module equipped in the UAV platform. In this study, the flight controller, PixHaw, recorded GPS coordinates (X, Y, Z) when a trigger signal was sent to the thermal sensors to capture images. Although the single GPS module was used, all images included a GPS coordinate in its metadata. Geo-tagged information was input into Photoscan Pro to generate a geo-referenced orthomosaic canopy temperature map. Image alignment to generate a sparse point cloud, the point optimization, dense point cloud generation, and DSM/Orthomosaic generation were conducted sequentially.

2.3. Thermal Stress Index (TSI)

Canopy temperature has been considered as a proxy for monitoring crop water status [16]. Although CWSI can be calculated from thermal images, upper and lower baseline temperature of air and canopy should be measured over the whole growing season. However, Thermal Stress Index, which was optimized for cotton, was defined as Equation (2) as below. where is foliage temperature and is the biochemically determined baseline temperature. The crop-specific biochemical temperature optimum was suggested as baseline temperature. In the literature, The 27.5°C midpoint temperature of the Thermal Kinetic Window, which means the temperature range for which the value of the apparent Michaelis constant remained within 200% of the minimum observed value, for cotton was examined as a baseline temperature () for the TSI [34]. The values of TSI range from zero to some positive limit. The biochemical-based TSI and the physically based CWSI were highly correlated for cotton across a range of environmental conditions. In this study, the temperature value in the orthomosaic temperature map was considered as the foliage temperature () to calculate TSI.

3. Results and Discussion

3.1. Canopy Temperature & TSI Map

A canopy temperature map was created from thermal images collected with the UAV-based system using the SfM algorithm and geo-tagged temperature images (see Figure 4(a)). The final canopy temperature map had a 7 cm spatial resolution. In this study, an orthomosaic image with finer spatial resolution could be generated since rotary-wing UAVs can fly at lower altitude with fairly stable orientation, when compared to a fixed-wing system. The temperature of bare soil and road was higher than those of vegetation. Temperature differences were also visible between green and red cotton, with the green genotype exhibiting lower canopy temperatures.

A TSI map was computed using Equation (2) and thermal data (see Figure 4(b)). As expected, similar to what was seen for the surface temperatures, TSI values for the green cotton genotype were lower and ranged from 0.2 to 0.3, while the red cotton genotype exhibited TSI values approximately 0.1 higher. In the TSI map, it was found crops in the northern area had more pressure of water. It could be supposed that water was not enough due to the irrigation system, terrain slope, or vegetation cover.

3.2. Evaluation of Canopy Temperature

The canopy temperature from UAV and ground-based sensors was compared (see Figure 5). The average of pixels on 11 rows in orthomosaic temperature map was adopted as UAV measurement. As indicated by ground and UAV measurements, the red plants were warmer than the green ones. The temperature of the red cotton measured by the net radiometer, thermocouples, and UAV was 33.69, 33.77, and 34.65°C, respectively. For the green cotton, the net radiometer, thermocouples, and UAV measured 31.96, 33.60, and 33.22°C, respectively. For both plots, the UAV measurements slightly overestimated the temperature of the plants. For the red cotton, the UAV overestimated the temperature with respect to the net radiometer and thermocouples by 2.85% and 2.60%, respectively, while for the green cotton, it overestimated by 3.94% and underestimated by 1.1%. The trend can be explained by the measuring location of temperature and atmosphere effect. The temperature at the top of the plant canopy should be higher than at the middle layer of the plant. Since UAV measures of the canopy surface, especially the top area, UAV overestimated the temperature. Although UAV thermal imagery was collected at the lower altitude (40 m) than the conventional platform, the higher atmosphere temperature (34.11°C) than canopy affected the thermal camera capturing the energy in the wavelength range. The leaf level measurements showed a greater variability than the canopy temperature derived by the UAV as shown by the standard deviations in Table 1. It is well known that leaf temperature measurements tend to be rather variable, since they are strongly influenced by the angle of incidence of [35]. Although the temperature measured by the thermocouple was closer to it of the net radiometer, the UAV measurements were in agreement with the ground sensors and were successful in showing temperature differences between the plots (see Table 2).

4. Conclusions

In this study, a UAV-based thermal sensor system was developed to measure canopy temperature using a radiometric calibrated thermal camera. Geo-tagged thermal imagery was collected over a cotton field including two different cotton genotypes exhibiting red and green leaves. Raw thermal data was converted to surface temperature using Matlab SDK provided by the sensor manufacturer. Radiometric calibration was performed using environmental parameters such as emissivity and weather conditions. Canopy temperature from UAV measurements was compared with that of net radiometer and thermocouples. The results show that the UAV system slightly overestimated canopy temperature when compared to the ground sensors. However, the errors did not exceed 5%, thus showing that the deviations were small and not significant for practical purposes. Additionally, the UAV system was successful in showing temperature differences between the plots. The proposed method showed the advantages of measuring canopy temperature and generating crop stress index map without field-based measurement. In the future, UAV and ground-based multiple thermal datasets will be collected for a newly designed plot to verify the proposed UAV-based thermal system for precision agriculture.

The proposed methodology could be applied to various agriculture fields to monitor crops for water stress and possibly the development of precision irrigation management applications in the future. However, there are a number of challenges that need to be addressed in order to make UAV technology practical for large commercial operations. One of the current limitations is the difficulty to generate precise geo-referenced orthomosaic thermal image without ground control points. Timely processing of a large volume of UAV data is also an issue that needs attention. Especially for crop precision management applications, the time between data collection and the output of actionable information needs to be drastically reduced.

Data Availability

The data used to support the findings of this study have not been made available because it is only available for researcher and collaborators of Texas A&M AgriLife Research and Extension.

Conflicts of Interest

The authors declare no conflict of interest.

Acknowledgments

This research was supported by Texas A&M AgriLife Research.