Abstract
Insects can cause a major loss in stored grain, and early identification and monitoring of insects become necessary for applying corrective action. Considering the effectiveness and practicability, an image processing approach and the corresponding application embedded in smartphones are proposed to identify and count the insects. For the insect images acquired by mobile phones, one sliding window-based binarization is adopted to release their nonuniform brightness, and then connected domain-based histogram statistics is presented to identify and count the insects in stored grain. Finally, the experiments are performed on the corresponding application based on Android, and it has shown that the proposed approach can be applicable for different random insect images from mobile phones with the counting accuracy of 95%, which is superior to the traditional approach.
1. Introduction
Detecting the insects in stored grain quickly and accurately is the basis of preventing and controlling storage grain insects and avoiding severe grain losses and unnecessary excessive treatment. Current standards of international trade are used as insect-free detection by standard method (ISO 6639-3,2001). Recently, several detection techniques have been developed for insect detection in stored grain such as electromagnetic spectrum-based methods [1–6], acoustic-based methods [7], conductance-based methods [8], and probes [9]. However, some of these techniques are time-consuming, expensive and less efficient to detect the dead insects. The image processing-based method is nondestructive and convenient in application design, which has been one of the hot topics in recent years [10–12]. There have been some methods that use image processing to count insects, which mainly include counting the morphological features of the grain insects [13, 14], the binary sketch of the grayscale image [15, 16], and counting the insect pixels [17, 18]. However, these methods have lower accuracy in counting smaller grain pests, the hardware of image acquisition and monitoring system are expensive, and the higher cost in computation cannot ensure the real-time detection. Therefore, considering the accuracy and practicability of insect detection, the image processing approach and application embedded in smartphones are researched. The focus is to preprocess the acquired insect image with nonuniform brightness and dark background and to separate the different insect pixels for higher accuracy, especially for tiny storage grain insects. The main work and innovation is as follows:(i)Sliding window with changeable optimal threshold is used to binary processing the acquired original insect image with uneven brightness, which affects the insect identification accuracy based on pixels difference.(ii)The probable occupied connected-domain pixels area of one insect is analyzed, thereby identifying and counting the insects, which is applicable for the booklice and other tiny insects.(iii)The application based on Android is portable to install in cell phones, and less memory space is occupied.
To verify the effectiveness, the traditional insect detection based on image processing [15] is compared.
2. Algorithm Descriptions
The overall structure of the automatic identification and counting system of storage grain insects based on the smartphone image is shown in Figure 1. Modular design is adopted, including original photo collection module, image capture module, image preprocessing module, data statistics module, and so on. In the first two modules, the original photos are collected in mobile phones with the high-resolution camera which can be clipped into different size to be detected to obtain the detected area.
The test environment is shown in Table 1.
While the system is used in the barn, mobile phone photos are easily affected by external environmental factors, including insufficient illumination and noise interference; therefore, the quality of images cannot be guaranteed. In order to solve the aforementioned problem and increase the difference between the insects and the background, it is necessary to preprocess the acquired insect image.
2.1. Preprocessing Algorithm Based on Sliding Window
For the acquired insect image, the main preprocessing units include the color space transformation, binary processing, boundary tracing, and so on, which are adopted to extract the contour of grain insects in the white wall of barn, as shown in Figure 2.
Obtaining the optimal threshold for the following binarization is the key preprocessing unit. In existed references, the fixed threshold with standard weighted average was often used [11], which is helpless for distinguishing the insect pixels area with different brightness, further maximizing the difference between the insect areas and the background. Therefore, the optimal threshold should be changeable with the brightness of different insect areas. Thereby, the key difficulty is to select a series of segmentation thresholds corresponding to different insects’ areas.
Assume that the insect image is , where and are its row and column separately. One square window with a size of 3∗3 pixels is adopted as the basic template operation. In each area with a size of 3∗3 pixels, the optimal threshold is equal to the weighted average of the nine pixels; for the first 3∗3 pixel area, the optimal threshold is
Here, is the weighted coefficients, typically take 1. In a similar operation, the second optimal threshold is
Similarly, can be obtained, where is the number of 3∗3 pixel areas in the whole insect image.
The pixels whose gray value is bigger than the optimal threshold will be assigned to “255” and the converse will be assigned to “0.” Generally, the lower gray pixels area represents the image background.
After image binarization, the gray difference between the insect pixel and the background is maximized to an extreme, which is helpful to increase the identifying and counting accuracy.
2.2. Histogram Statistics Counting Insects by Connected Domains
The commonly used grain pest counting algorithms mainly include template matching method [12, 13] and connected domain method [15]. The template matching method is used to identify insect body posture and body characteristics, and a matching benchmark must be established to complete the process of registration. The template matching method can obtain higher matching accuracy, but is less efficient for more similar insect body features and the complexity will increase with the insect body samples. In the connected domain method, one center pixel and its adjacent several pixels with the same gray value of “255” will be called as one connected domain, also called as black connected domain. One black connected domain represents one insect probably. The key difficulty is to decide the number of pixels in one black connected domain occupied by one kind of grain insect. For a given grain insect image, the number of black connected domains with different pixels numbers is shown in Figure 3. Here, N is the number of pixels in one black connected domain, and M is the number of homogeneous connected domains over the whole image. M is 6 when N is less than 18, M is 110 when N is from 18 to 24, and M is 2 when N is more than 24, as shown in Table 2.
From Table 2, it has shown that one grain insect occupies 18 to 24 pixels in one connected domain with most possibility. Therefore, the black connected domain with 18 to 24 pixels will be regarded as one insect.
3. Evaluations
3.1. Android-Based Mobile Application Development
Using the Windows system, Android Studio and SDK (software development kit) [19] are used to develop the application (APP), and the completed APK (Android package) is installed on the Android smartphone. The main functions are listed as follows:(i)Choosing one photo. One way to obtain the photo is from the camera of the cell phone, and the other way is from the albums.(ii)Clipping the photo. The photo is a square image with one side 10 cm to 15 cm, in which hundreds of grain insects are usually included. At this time, the sampled photo area can be chosen by clipping.(iii)Resizing the sampled photo. In order to increase the detection resolution, the detected image should keep certain size; usually, the clipped image is enlarged.(iv)Identifying and counting the insects. After binarization and connected domain statistics, the number of insects in stored grain is displayed.
3.2. Results
For the experiment, three original image samples with different brightness and insect density are selected as shown in Figures 4(a)–4(c). To verify the effectiveness, the traditional insect detection based on image processing [10] is compared, in which the binarization under the fixed threshold is adopted, called as the fixed threshold method. The proposed binarization in this paper is called as the sliding threshold method. The preprocessed images under the fixed threshold and under the sliding threshold are shown in Figures 5(a)–5(c) and Figures 6(a)–6(c), respectively.
(a)
(b)
(c)
(a)
(b)
(c)
(a)
(b)
(c)
From Figures 4–6, it can be seen that the gray difference between the grain insect and the background is more obvious by the sliding window preprocess and that the side identification is enhanced. According to the histogram statistical algorithm proposed in this paper, the accuracy of identifying grain insects is shown in Table 3. Obviously, the sliding threshold method can provide the higher identification accuracy.
4. Conclusion
In this paper, one insect identifying and counting approach and Android-based APP are presented, in which the sliding window is used in the preprocess unit that can maximize the difference between the detected targets and the background and remove the uneven background brightness. By adapting the sliding window size, it is possible for tiny insects to be distinguished. Compared with the existing technology, the proposed scheme can provide the higher counting accuracy and is portable with online identifying and counting the insects. Besides, in the proposed connected domain method, the insects with one same shape and size will correspond to one same black connected domain, which can be applied to identify and count the other kinds of insects with different shapes and sizes. Thereby, the separation of the connected domain will be one of the key issues to improve identification accuracy in the future.
Data Availability
The [Android-Based Mobile Application] data used to support the findings of this study were supplied by [Chunhua Zhu] under license and so cannot be made freely available. Requests for access to these data should be made to [Chunhua Zhu, [email protected]].
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This research was financially supported by the National Science Foundation of China (61741107 and 61871176), Henan Provincial Department of Science and Technology Project (172102210230), and Key Scientific Research Projects in Henan Colleges and Universities (19A510011).