Applied Mathematics for Engineering Problems in Biomechanics and RoboticsView this Special Issue
EO Sensor Planning for UAV Engineering Reconnaissance Based on NIIRS and GIQE
When unmanned aerial vehicles (UAVs) support the Corps of Engineers in reconnaissance operations, in order to gather visible image information that should meet the mission’s need, we grouped the engineering reconnaissance information interpretation tasks into 10 levels by using the National Imagery Interpretability Rating Scale (NIIRS). The quantitative relationship between the engineering targets, sensor performance, and flight altitude was established through the general image quality equation (GIQE) and the geometrical property of the ground sampled distance (GSD). Through some simulations, the influence of variable factors of the EO sensor imaging quality was analyzed, and the imaging height of the sensor for an engineering reconnaissance scenario was calculated. The results showed that this study could solve the problem of poor image quality caused by the flight altitude not meeting the mission requirements.
The main task of engineering reconnaissance is to detect or identify the terrain, geology, hydrology, traffic conditions, the enemy’s engineering facilities, the resources available locally on the battlefield, etc. When engineering corps reconnaissance operations are supported by unmanned aerial vehicles (UAVs), the mission planners of engineering corps may have limited knowledge about the use of UAV sensors, and because of the temporary assignment of engineering reconnaissance, the UAV operators of other units may not be familiar with the engineering targets. This brings uncertainty to the effectiveness of UAV engineering reconnaissance. In hostile and dangerous environments, UAV operators are generally willing to make the UAVs fly as high as possible; however, if the UAVs fly only at high altitude, some smaller targets will exceed the sensors’ capabilities and might not be detected. In this case, the UAV operators will have to detect certain targets repeatedly, which is inefficient and will increase the risk of loss of the UAVs. If the UAV reconnaissance altitude corresponding to different types of engineering targets can be calculated in advance, the abovementioned problems could be avoided to some extent.
At present, there have been many research studies on the planning of the flight altitude of UAVs [1–4], but the main task of these studies was to avoid antiaircraft fire or missiles, radar detection, obstacles, and other threats by adjusting flight altitude. This approach does not focus on the relationship between the sensor imaging height and quality. Most studies on imaging height and quality are about sensors of satellites [5, 6], and only small parts are about UAV sensors in order to provide theoretical methods for sensor design and performance evaluations [7, 8]. Qiao et al.  discussed a mission-oriented UAV path planning algorithm, and they pointed out that the quality of image information should be considered in mission planning. However, how to set the UAV to meet the image quality requirements was not discussed in their study.
This paper focuses solely on the imaging quality and height of EO sensors in UAVs supporting engineering reconnaissance. The problems of threat avoidance, flight paths, and resource consumption will not be discussed here. The main study is structured as follows. Section 2 is a general presentation of NIIRS and GIQE, and we group a series of engineering information interpretation tasks into 10 NIIRS levels according to military and civil visible NIIRS criteria. Section 3 describes a method to build a quantitative relationship between the engineering targets, sensor performance, and flight altitude and to provide a solution for how high the UAV should fly in engineering reconnaissance operations. In Section 4, some simulations are carried out, and the results are discussed. Then, an engineering reconnaissance scenario is given to illustrate how to implement sensor planning. In Section 5, conclusions are given.
2. NIIRS and GIQE
2.1. National Imagery Interpretability Rating Scale
The NIIRS is a set of subjective image quality assessment criteria: a 10-level scale of 0 to 9 for image interpretability [10, 11]. NIIRS was developed under the auspices of the United States Government’s Imagery Resolution Assessment and Reporting Standards (IRARS) committee. Each NIIRS level from 1 through 9 is defined by a series of interpretation tasks that range from very easy (requiring low image quality) to very difficult (requiring high levels of image quality). The tasks that define the NIIRS are related to an empirically derived perceptual image quality scale. Similar scales have been developed for use with radar, IR, and multispectral imagery. There are a large number of descriptive tasks in each scale that could not be listed here; refer to [12–14] if needed. NIIRS is probably the best measure of assessing the quality of images. It has been used extensively by the intelligence community. The performance of intelligence-surveillance-reconnaissance (ISR) sensors of UAVs was specified in NIIRS form, including “Global Hawk,” “Dark Star,” “Predator,” and a large number of other platforms.
The NIIRS is predictable and is a subjective measure of information extraction. For nonprofessional users of remote sensing images, it is technically not dependent on a large number of data, and the subjective score of a target according to the NIIRS criteria guide is available [5, 15]. The NIIRS value and the spatial resolution (the ground sampled distance and relative edge response are measures of the system spatial resolution) have an ideal linear relationship , and the spatial resolution is defined as the minimum size that sensors can distinguish between targets whose length and width are at the same magnitude in the case of good contrast and similar background . Therefore, for criteria not listed on the scale, NIIRS levels can be roughly estimated according to the shape, size, contrast, and other information of the targets.
2.2. Visible NIIRS of Engineering Reconnaissance Operations
For sensor planning of UAV, it is necessary to know the NIIRS levels of the engineering targets. Our solution is to extract the criteria that are relevant to the tasks of engineering reconnaissance from the current version of military and civil visible NIIRS and list a set of information interpretation tasks that are related to common engineering facilities, engineering equipment, personnel, the environment of the battlefield, and other targets according to the mission of UAV engineering reconnaissance. Next, we studied the background, state, shape, size, and other information of the engineering targets through a detailed comparison of the criteria of current military and civil visible NIIRS. We grouped the engineering information interpretation tasks into corresponding levels according to the scales and merged them with the previous extracted criteria. Finally, we listed a rough estimated visible NIIRS of common engineering reconnaissance tasks in Table 1. As the focus of this study is sensor planning rather than image intelligence interpretation, some engineering targets were selected as similar features as the targets of the original criteria of visible NIIRS in order to avoid significant errors.
2.3. General Image Quality Equation
The NIIRS can express the requirements of reconnaissance mission well. It is meaningful to predict the NIIRS value when the sensor parameters of a UAV and information of reconnaissance targets are known. The general image quality equation (GIQE) is capable of completing this prediction. GIQE is an empirical model that is developed through a statistical analysis of the judgment of the image analyst. It originally predicted the interpretability of visible sampled imagery .
Although GIQE is subjective, it is impossible to predict the NIIRS by other methods. In the verification and comparison of image statistical models and estimation models, it is found that the two are correlated . For example, an automobile salesman’s ability is related to the number of automobiles that he sells. Without assessing his professional knowledge, we can verify the salesperson’s ability through his sales performance. Image analysts are good predictors of image quality, and GIQE meets their needs well. Until a better method is developed, people will have to rely on this empirical model. The GIQE provides NIIRS predictions as a function of perceptual-quality attributes of scale, resolution, and sharpness, and of contrast and noise.
GIQE has undergone several revisions. The current version is 4.0:where is the geometric mean of the ground sampled distance in inches, is the geometric mean of the normalized relative edge response, is the geometric mean height owing to edge overshoot resulting from modulation transfer function compensation (MTFC), G is the noise gain resulting from MTFC, and SNR is the signal-to-noise ratio. and contribute as much as 92% of the NIIRS value. Other factors take up only 8% .
The complete calculation of the parameters , , , G, and SNR in GIQE involves complex physical processes and is closely related to the specific physical parameters of the sensors. Therefore, we will not discuss the calculation here. The impact of the target (orientation, size, and contrast) is reflected in and implied in the SNR. The effects of the atmosphere are reflected in the SNR, and a standard target contrast is assumed for most applications. The impact of the sensor is included in and MTFC-related items (RER and lower-impact G and H). The effects of image processing include MTFC and grayscale transformations (dynamic range adjustment and gray-level transformation compensation), and the GIQE model assumes that the grayscale transformations are optimal .
3. Sensor Planning Method
According to the GIQE, factors that affect the value of NIIRS can be divided into two categories: one determined by the intrinsic properties of sensors, the environment, or engineering targets, and the other is related to the specific use of the sensors. For sensor planning, the intrinsic properties part cannot be changed, so only using the sensor properly in engineering reconnaissance operations can meet the needs of NIIRS.
The parameters related to sensor planning are mainly reflected in . is determined by the sensor focal length, flight altitude of the UAV, imaging distance, and other factors. These are the operational parameters of the UAV in the course of an engineering reconnaissance mission, so they are very important to sensor planning. From the mathematical expression of GIQE, the influence of on NIIRS is significant. The influencing factors of are decomposed and discussed below.
is the geometric mean of the horizontal and vertical ground sample distances based on a projection of the pixel pitch distance to the ground. is computed in inches in both the X and Y dimensions :For systems in which the along-scan and cross-scan directions are not orthogonal, is modified by the angle between these directions:For a CCD-array EO imaging sensor, the imaging scale depends on the focal length of the sensor and the flight altitude of the UAV. Assumption, Figure 1: a UAV flies from left to right, the EO sensor payload of the UAV has a focal length f, and the pixel pitchs of the vertical and horizontal are DP and DP’, respectively. The pixel pitch is center-to-center distance of a pixel, relative to the pixel shape, usually the same as pixel edge length. The projection of the pixel is a trapezoidal area on the ground, the short edge of the trapezoid is x, the long edge is , and the hypotenuse is ’. The imaging height is , the slant distance is , and the look angle that is between the sensor to the target line and the ground horizontal line is .
In Figure 1, the size of the pixel projection changes with the ground undulation, and for some military systems, it is not meaningful to compute the value on the ground. Thus, the usual practice is to compute the value on a plane perpendicular to the sensor sight, on which the sensor projection changes from a trapezoid on the ground to a rectangle or a square, and y’ becomes y. In addition, the slant distance is kilometer-level, and GSD is centimeter-level, and the projection effect from ground to plane that vertical of sight on slant distance can be ignored. Thus, the distance from the sensor to the plane of the vertical sight can still be calculated by r here. If the pixel of an EO sensor is rectangular, this is known by the geometrical relationThe rectangular area isAccording to (3), the value of is the square root of the rectangular area, and it is more direct and convenient to use the imaging height in calculations. Here, we use to replace r, and because the unit of is the inch, it needs to be converted to meters for calculation:Thus,Further, is brought into (1) to establish an association with the sensor:Make, and bring this into (9):Then,To sum up, when the NIIRS level is known before a reconnaissance operation, the model of senor planning for UAVs based on NIIRS and GIQE is as shown in Figure 2.
4. Simulation and Results Discussion
The EO equipment of “Global Hawk” and “Predator” was chosen as an example to carry out some simulations. A partial list of performance parameters [7, 13] for the EO camera of “Global Hawk” and “Predator” is shown in Table 3. When sensor planning for other types of UAVs, simply replace these with the parameters of the EO sensor payload of the other UAVs.
For the EO camera, the values of , , and have the following typical data [8, 22]: = 0.75, = 1.4, and = 10, making SNR = 66. Parameters such as , HGM, G, and SNR are generally fixed values, which are usually considered in the design of new EO equipment. Because = 0.75, then a = 3.16 and b = 2.817 according to (1).
Assume that the along-scan and cross-scan directions are orthogonal. The pixel of the sensor array selected for the test is square, so that = = GSD. In the following, the relationship between the flight altitude, focal length of the sensor, angle of view, and the NIIRS will be analyzed, and an example of sensor planning of engineering reconnaissance supported by UAVs will be given to explain how to use NIIRS and GIQE for sensor planning.
4.1. Relationship between Flight Altitude and NIIRS Level
The EO sensor of “Global Hawk” is designed to provide a minimum NIIRS level of 6.5  for visible light images (angle of view 45° and sensor-to-target distance of 28 km). The imaging height of 19,802 m can be calculated through a trigonometric relationship, that is, the maximum flight altitude of “Global Hawk,” rounded to 19,800 m for calculation. The EO sensor of “Predator” is designed to be at a 45° angle of view and at a height of 15,000 ft (4570 m), providing a minimum NIIRS level of 6 for visible light images .
In order to study the relationship between the flight altitude and the NIIRS level of the EO sensor, to verify (9) by the NIIRS requirements for the sensor design, and to verify the reliability of calculated by DP, f, θ, and h, a range of 11,000 m to 19,800 m of the flying altitude of “Global Hawk” was selected for the simulation. Because the sensor design requirement that stipulates the imaging quality should reach a certain level at a certain altitude, the focal length parameter value was set to the maximum focal length of 1.75 m. A range of cruising altitude of 4570 m to the maximum height of 7620 m for “Predator” was selected, and a focal length of 0.16 m was set. The relationship between the imaging height and the imaging quality was calculated by a simulation, as shown in Figure 3.
According to the results, the imaging height of “Global Hawk” increased from 11,000 m to 19,800 m, and the NIIRS level changed from 7.4 to 6.6. The imaging height of “Predator” increased from 4570 m to 7620 m, and the NIIRS level changed from 6.1 to 5.4. It can be seen directly from Figure 3 that the NIIRS value decreased as the imaging height increased, and the trend of value decreasing was slowing down.
When the imaging height of “Global Hawk” was 19,800 m, the NIIRS value was 6.6 (keeping two digits after the decimal point, the value was 6.55, and the result is marked with a red circle in Figure 3), and it met the design requirement of NIIRS > 6.5. When the imaging height of “Predator” was 4570 m, the NIIRS value was 6.1 (keeping two digits after the decimal point, the value was 6.08 and is also marked with a red circle in Figure 3). This also met the design requirement of NIIRS > 6. The calculation results can be further explained in that (9) was correct and reliable for calculating the NIIRS by using DP, f, θ, and h to establish the relation with .
4.2. Relationship between Focal Length of Sensor and NIIRS Level
The instantaneous field of view (IFOV) can be adjusted by changing the focal length of an EO sensor. When the focal length is short, the IFOV is wide, and a large area can be detected, but the resolution is usually low. When the focal length is long, the IFOV is narrow, and the detector covers a small area, so the resolution is improved. However, this sacrifices the ground coverage, and very much like a “glimpse,” the target detection is more difficult.
Therefore, it is necessary to set the focal length parameters reasonably in order to get the image to meet the task requirement and to improve the coverage of IFOV before engineering reconnaissance operations begin. For simulation parameters, the imaging height of the UAVs was set to their cruise altitude: “Global Hawk” was 18,000 m, “Predator” was 4570 m, and the angle of view was set to 45°. The results are shown in Figure 4.
The results showed that the focal length of “Global Hawk” was 1–1.75 m, the range of the NIIRS value was 5.9–6.7, and when the focal length of “Predator” was 0.016–0.16 m, the range of the NIIRS value increased from 2.9 to 6.1. The NIIRS value increased with the focal length of the sensor, and the trend of increasing speed of NIIRS slowed down with an increase in focal length. Because the EO sensor of “Global Hawk” has a zoom of only 1.75, the overall increase in the NIIRS value is small, but because of its long focal length, it can obtain high-quality images in case of a wide IFOV. The EO sensor of “Predator” has a zoom of 10; therefore, the NIIRS value fluctuates greatly when the focal length changes. Because of the short focal length of the sensor, the image resolution of a wide IFOV can be lower.
4.3. Relationship between Angle of View and NIIRS Level at Different IFOVs
Here, the imaging height was the cruising altitude. According to the IFOV, the focal length was calculated by the minimum and maximum values, and the range of the angle of view was set to 45°–90°. The results are shown in Figure 5.
In case of a wide IFOV, when the angle of view increased from 45° to 90°, the range of the NIIRS value of “Global Hawk” was 5.9–6.4, and the range of the NIIRS value of “Predator” was 2.9–3.4. In case of a narrow IFOV, the range of the NIIRS value of “Global Hawk” was 6.7–7.2, and the range of the NIIRS value of “Predator” was 6.1–6.6. From the curve in Figure 5, we can see that the NIIRS value increased with an increase in the angle of view, the growth slowed down gradually, and it finally tended to be horizontal.
4.4. Solution of Sensor Planning of a Scenario
Taking engineering reconnaissance of landing attack supported by UAVs as an example, this paper shows how to plan the imaging height of the sensors in engineering reconnaissance operations. According to the operation methods of engineering reconnaissance supported by UAVs in landing attack and the estimated visible NIIRS of common engineering reconnaissance tasks (Table 1), the main reconnaissance tasks and the required NIIRS level are sorted as shown in Table 4. Among them, if multiple details need to be detected in a task, the image quality needs to be planned according to the highest NIIRS level.
For the parameter setting of the EO sensor of the UAV, the angle of view continues to be 45°, which was specified by the sensor design standards of NIIRS. Because the engineering reconnaissance task needs to detect more targets, a wide IFOV should be chosen as far as possible. The focal length of the EO sensor of “Global Hawk” was set to 1 m. The range of the IFOV of “Predator” is 2.3° 1.7°–23° × 17°. Because of its short focal length, the flight altitude of the UAV will descend to hundreds of meters if the IFOV is too wide, and this does not conform to practical use. Therefore, a 5 zoom was selected for which the IFOV was 16.5° 8.5° and the focal length was 0.08 m. The values of , , , , , and were consistent with the previous text. The results of the EO sensor planning are shown in Table 5, and an imaging height requirement comparison of the two types of UAV is shown in Figure 6.
For tasks that demand a high NIIRS level, some other parameters of the sensor are fixed, so the flight altitude of the UAVs must be lowered to meet the imaging quality requirements. For a task with a lower demand of NIIRS level, the cruise altitude of “Global Hawk” is close to its ceiling, and the increasing part of the height has little effect on the image quality and detection range. Thus, the UAV should continue reconnaissance at the cruising altitude. By contrast, “Predator” has a different cruising altitude and maximum flight altitude, although it can easily obtain low-level NIIRS engineering target images without changing altitude. However, a large increase in the imaging height can increase the sensor’s detection range, and thus more targets will be detected and the efficiency of reconnaissance will be improved.
Aiming at the problem of how to obtain visible-light image intelligence in engineering reconnaissance operations supported by UAVs, the NIIRS and GIQE were studied in this paper. According to the NIIRS criteria and the properties of the engineering targets, the visible NIIRS level was specified for the engineering reconnaissance tasks, and the relationship between the NIIRS level of engineering reconnaissance tasks, the EO sensor performance, and the ground sampled distance was established through GIQE. Then, the ground sampled distance in the GIQE was further decomposed into sensor parameters such as pixel pitch, focal length, angle of view, and imaging height. A model for sensor planning was established by using a geometrical method. Finally, some simulations were carried out, and a scenario of an engineering reconnaissance operation was examined.
The results showed that the NIIRS level decreased with an increase in the imaging height and increased with an increase in the angle of view and the focal length. The value of the height in the results was the highest that a UAV could fly during an engineering reconnaissance task, and it was difficult to meet the imaging quality requirement if the flight altitude was exceeded. Exceeding the flight altitude could lead to re-reconnaissance, increasing the time. In addition, in the model for sensor planning, several variables interacted with each other. The flight altitude is different when the angle of view and focal length are different for the same task. Thus, reasonable sensor planning should be combined with the requirements of specific engineering reconnaissance operations.
It is complicated to determine the flight altitude of UAVs in military operations. Threat avoidance, flight paths, and resource consumption should be considered in mission planning. The abovementioned problems will be studied in the future.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This study was supported by the Military Science Project of the National Social Science Foundation of China (no. 15GJ003-141) and the Military Postgraduate Funding Project of the PLA (no. 2016JY370).
A. Tsourdos, B. White, and M. Shanmugavel, Cooperative Path Planning of Unmanned Aerial Vehicles, John Wiley & Sons Ltd, 2010.
W. Naifeng, Research on the small uav online path planning algorithms in complex and low-altitude environments, Harbin Institute of Technology, Harbin, China, 2016, Research on the small uav online path planning algorithms in complex and low-altitude environments”. Harbin Institute of Technology, 2016.
B. Honggang, A study on image quality evaluation of remote sensing systems based on NIIRS, Xi’an Xidian University, Xi'an, China, 2010.
H. Xiao-juan, K. Sheng, and L. Kan, “Analysis of Image Quality Influencing Factors of Aero Visible Camera,” Optics & Optoelectronic Techenology, vol. 12, no. 4, pp. 65–68, 2014.View at: Google Scholar
Q. Ming, Z. Xiao-lin, X. Wen-jun et al., “A Mission Oriented Path Planning Algorithm for Unmanned Reconnaissance Air Vehicles,” Journal of Air Force Engineering University: Natural Science Edition, vol. 16, no. 2, pp. 38–42, 2015.View at: Google Scholar
R. G. Driggers, M. Kelley, P. G. Cox, and G. C. Holst, “National imagery interpretation rating system (NIIRS) and the probabilities of detection, recognition, and identification,” in Proceedings of the Joint Precision Strike Demonstration Project Office SETA Member EOIR Measurements, Inc., pp. 349–360, Orlando, FL, 2007.View at: Publisher Site | Google Scholar
J. M. Irvine and J. C. Leachtenauer, “A Methodology for Developing Image Interpretability Scales,” ASPRS/ASCM Annual Convention Exhibition, pp. 273–279, 1996.View at: Google Scholar
R. G. Driggers, J. A. Ratches, J. C. Leachtenauer, and R. W. Kistner, “Synthetic aperture radar target acquisition model based on a national imagery interpretability rating scale to probability of discrimination conversion,” Optical Engineering, vol. 42, no. 7, pp. 2104–2112, 2003.View at: Publisher Site | Google Scholar
J. C. Leachtenauer and R. G. Driggers, Surveillance and Reconnaissance Imaging Systems-Modeling and Performance Prediction, Artech House Publishers, 2000.
Imagery Resolution Assessments, and Reporting Standards (IRARS) Committee (1996), “Civil NIIRS Reference Guide”, http://www.fas.org/irp/imint/niirs_c/guide.htm.
Editorial Office of Equipment Reference, Ground resolution, Equipment Reference, no. 41, pp. 21-21, 2004.
Q. Ye, X. Li, and Y. Chen, “Evaluating aerophoto image quality based on character statistics,” Remote Sensing, vol. 5, pp. 20–23, 2006.View at: Google Scholar
H. Mao, S. Tian, and A. Chao, UAV Mission Planning, National Defense Industry Press, 2015.
Y. Chang-feng, Y. Wen-Xian, and S. Yi, “he Situation Awareness Evaluation of S&R System,” Journal of National University of Defense Technology, vol. 27, no. 3, pp. 49–53, 2005.View at: Google Scholar
USAF, “Air combat command concept of operations for endurance unmanned aerial vehicles”, https://fas.org/irp/doddir/usaf/conops_uav/part02.htm.