Cognitive Modeling of Multimodal Data Intensive Systems for Applications in Nature and Society (COMDICS)View this Special Issue
Plant Disease Identification Based on Deep Learning Algorithm in Smart Farming
The identification of plant disease is the premise of the prevention of plant disease efficiently and precisely in the complex environment. With the rapid development of the smart farming, the identification of plant disease becomes digitalized and data-driven, enabling advanced decision support, smart analyses, and planning. This paper proposes a mathematical model of plant disease detection and recognition based on deep learning, which improves accuracy, generality, and training efficiency. Firstly, the region proposal network (RPN) is utilized to recognize and localize the leaves in complex surroundings. Then, images segmented based on the results of RPN algorithm contain the feature of symptoms through Chan–Vese (CV) algorithm. Finally, the segmented leaves are input into the transfer learning model and trained by the dataset of diseased leaves under simple background. Furthermore, the model is examined with black rot, bacterial plaque, and rust diseases. The results show that the accuracy of the method is 83.57%, which is better than the traditional method, thus reducing the influence of disease on agricultural production and being favorable to sustainable development of agriculture. Therefore, the deep learning algorithm proposed in the paper is of great significance in intelligent agriculture, ecological protection, and agricultural production.
Plant disease can directly lead to stunted growth causing bad effects on yields [1–3]. An economic loss of up to $20 billion per year is estimated all over the world [4–6]. Diverse conditions are the most difficult challenge for researchers due to the geographic differences that may hinder the accurate identification [7, 8]. In addition, traditional methods mainly rely on specialists, experience, and manuals , but the majority of them are expensive, time-consuming, and labor-intensive with difficulty detecting precisely . Therefore, a rapid and accurate approach to identify plant diseases seems so urgent for the benefit of business and ecology to agriculture.
Internet technologies, in particular the availability of multimodality data from various sensors including the Internet of things and sensor networks, have developed rapidly . Herein, a novel plant leaf identification model based on deep learning algorithm is designed to solve the above issues. The function contains leaf retrieval, image segmentation, and identification with the utilization of integrated deep learning algorithm throughout the whole process. The first task is leaf retrieval, but many factors pose the challenge of identification accuracy such as soil and illumination in the complex environment . Hence, the model is investigated RPN algorithm for manipulating retrieval and represents the good adaption in practice. Image segmentation is the second step that is considered to be the most crucial because diagnostic precision plays an important role in detection results. The Chan–Vese algorithm based on region shows promising results for segmenting images free of noise and weak edge. The last step is to identify the disease of leaves based on the migration learning algorithm. Based on the pretrained model, the migration learning model uses the dataset of disease leaves in a simple background to train the model. The rest of the paper is constructed as follows: Section 2 previews other scholars’ researches thoroughly. The detailed information about the model is shown in Section 3. Section 4 demonstrates the procedure of experiment and study. Conclusions and discussions are in Section 5.
2. Literature Review
At present, the research of plant disease recognition in the complex environment mainly focuses on three aspects: disease leaves image segmentation, feature extraction, and disease identification.
2.1. Image Segmentation
In the complex environment, the most crucial task is how to segment the images while localizing and detecting diseased plant leaves, since the major aim of image segmentation is to set the symptom information apart from the background. There are many researchers making a deep investigation on it. In 2017, Ali et al. applied the Delta E color difference algorithm to separate the disease-infected area . In general, four major methods are used to perform the image segmentation which are discussed the detail in the following paragraph .
Some researchers integrate the region of interest (ROI) and other methods to segment images. For example, Kao et al. claimed that the convolutional autoencoder served as the background filter to determine the ROI in an image . The second method only concerns region segmentation. In 2013, Pujari et al. claimed that images were divided into various regions which had a special meaning and extracted the images’ feature . Akram and other colleagues provided an image processing model with real-time synchronous processing. By dividing the image into three color spaces, it can carry out contrast stretching, feature vector, and salient region recognition . In addition, other researchers chose deep learning techniques to segment and detect images. Marko et al. recommended a depth-based target detection algorithm and used the two-stage algorithm to optimize plant disease images detection . At last, the thresholding is common in segmentation. In 2018, Li et al. applied multilevel thresholding techniques based on gray histogram for image segmentation . Mohamed and Diego presented a new multiobjective metaheuristic on the basis of a multiverse optimization algorithm to segment grayscale images via multilevel thresholding .
However, there is a fact that cannot be ignored. Because of the complexity of color information in the complicated environment, the machine vision algorithm based on color, ROI, and threshold performs poorly in practice.
2.2. Feature Extraction
The feature extraction of plant disease faces many problems in identifying plant disease. The distinct image features include textures, shape, color, and motion-related attributes, which are the essential conditions for disease feature extraction [21, 22]. Raza and his colleagues described a method that uses color and texture features to extract disease spots . Hu et al. proposed the Dempster–Shafer (D-S) evidence theory and multifeature fusion for extracting features as well as the results were processed by introducing variance to improve decision rules of D-S evidence theory . In addition, Turkoglu depicted improved versions of the Local Binary Patterns (LBP) methodology, which uses the original LBP local quadratic value to transform the image into grayscale and processes the R and G channels of the image by considering overall and region . Li et al. researched an IoT feature extraction for the intelligent city based on the deep migration learning model . There was an application in music, which can extract meaningful audio features in order to enable the visualizations to be responsive to the music . And in recent studies, lots of novel approaches have been put forward to implementing feature extraction. For example, concerning the challenging task that the extraction of relevant and distinct features from electroencephalogram (EEG), Meziani et al. proposed two new spectral estimators that were robust against non-Gaussian, nonlinear, and nonstationary signals . What is more, as Liu et al. reported, the high-dimensional time–frequency spectrum features were extracted by using the residual neural network and the improved signal-to-clutter ratio (SCR) . Xu et al. introduced a feature extraction method based on the Hilbert marginal spectrum to perform the wear of milling tools .
2.3. Disease Identification
As for the precise identification, so many techniques are developed and researched to get accurate results. The identification model focused on using class labels for training images and built a fine-grained image classification system . Zhang et al. reported a recognition method for plant disease leaf images based on a hybrid clustering . In 2017, Patil et al. described a content-based image retrieval (CBIR) system to extract texture features and means value to compute color features, and support vector machine (SVM) classifier was used for classification . Through above researches, the major goal was to design the classification schemes and image analysis for feature extraction and identification. Recently, other new approaches have been introduced to identify the disease more accurately and precisely. A novel system based on the selection of pictures and short text descriptions helped nonexperts in identifying plant diseases that can be used remotely from a desktop as well as in a smart phone or personal digital assistants . Pertot et al. presented a scheme that used mobile phones for real-time on-field imaging of diseased plants and used mobile devices for leaf image segmentation and spotting of disease patch with improved k-means clustering . Yang et al. presented a microscopy image detection methodology based on the synergistic judgment of texture and shape features and the decision tree-confusion matrix .
Additionally, the convolutional neural network is numerously utilized in identifying diseases. Chad et al. established a system capable of automatically identifying plant disease in field-acquired images of maize plants . Ni et al. used the deep convolution neural network to train 1632 images of corn kernels and designed an automatic corn detector . Lu et al. proposed a rice diseases identification method based on deep convolutional neural networks (CNNs) techniques . Zhang et al. designed an agricultural machinery image recognition network using the deep learning algorithm . Zhang et al. improved deep convolution neural network to improve the accuracy of maize leaf disease identification . Images were input into two deep learning-based architectures, namely, AlexNet and VGG-16 net, to perform detection . Coulibaly et al. suggested a method using transfer learning for feature extraction to build an identification system . However, due to the requirement for high hardware resources and traditional neural network models of high quality and quantity of data sets in the training process, the training wastes much time that is not conducive to the promotion and use of the model. In this paper, we recommend a transfer learning model for identification combined with the pretrained model, using the dataset of disease leaves to train the model.
From the above research findings, some achievements have been achieved in three aspects: leaf image segmentation, leaf lesion feature extraction, and leaf disease recognition. However, there are still many problems to be solved to realize plant disease identification in the complex environment.
3.1. The Solution Framework
The full plant disease identification model framework based on deep learning is shown in Figure 1, including three steps, the localization of plant leaves, the segmentation of images, the extraction of plant disease, and the identification of disease. The model used in this paper mainly consists of the following three steps. The first step is to locate the diseased leaves. The RPN algorithm is used to train the leaf dataset in the complex environment, and the frame regression neural network and classification neural network are used to locate and retrieve the diseased leaves in the complex environment. The second step is the segmentation of diseased leaves. The Chan–Vese algorithm is used to segment the image of diseased leaves. Based on the set zero level set and the minimum energy function as the goal, the leaf contour is obtained by iterative calculation, so as to realize the image segmentation of diseased leaves in the complex environment. The third step is the identification of leaf disease species. The pretrained transfer learning model is trained to realize plant disease recognition in the simple background.
3.2. The Leaf Localization
Aiming at the localization of disease-plant leaves, the paper manipulates the leaf dataset under complex background to train the RPN algorithm and integrates boundary regression neural network and classification neural network to perform localization and retrieval.
As for the classification neural network, the core task is to distinguish whether the image in the boundary box is an object or a background. During the process of training, making Intersection over Union (IoU) as a criterion of classification, the boundary box with IoU greater than 0.5 is annotated as an object and the boundary box with IoU less than 0.1 is labeled as a background. IoU is applied in calculating the relevance between predicting boundary box and artificial marked boundary box. The formula of IoU is shown as follows:where represents the overlap area of predicting boundary box and artificially marked boundary box, and represents the total area of it. Due to the fact that the classification neural network is only used for binary classification problem, sigmoid function is employed as loss function.
With regard to the adjustment parameters of boundary regression neural network and output boundary box, one boundary box can be represented by four-dimensional variable . represents the given boundary box, represents the target boundary box, and represents the predicting boundary box. In order to find a mapping relationship of boundary regression neural network, and are defined.
The movement of boundary consists of pan and zoom.
The parameter of pan is , given that and . The formula is shown as
The parameter of zoom is , given that and . The formula is shown as
According to the above formula, the real learning objectives of boundary regression neural network are represented with , and the real transform parameters between predicting boundary box and artificially marked boundary box are shown as .
The objective function of boundary regression neural network is , where represents the learning parameter of boundary regression neural network. The loss function is shown as follows:
3.3. The Leaf Image Segmentation
As for the segmentation of images in complex environment, based on the results of previous step, the model performs the segmentation by Chan–Vese algorithm. Laying the foundation of the set zero level set, aiming at minimizing the energy function and obtaining blade profiles by iterative computing, the model may perform the segmentation of diseased plant leaves images. The Chan–Vese algorithm uses the level set to constructing an energy function to constrain the whole region rather than to control surface evolution by the explicit control speed . The energy function is defined as the minimum sum of variances between the gray values of the image inside and outside the contour, and the contour length is increased to make it converge. Given a closed curve in the image, the energy function is expressed as follows:
represents inside the contour, represents outside the contour, represents the gray values of the image, represents the average gray values in contour, and represents the average gay values out contour. Then, the given formulas are as follows:
When and , the computing ends.
Level set method is used to solve (11) and zero level set is used to express contour lines. Heaviside’s function and Dirac’s function are introduced:
The level set equation of energy function is as follows:
By minimizing (16) with variational method and combining Euler–Lagrange equation, the following partial differential equations are obtained, where and .
3.4. The Diseased Leaf Identification
In identifying disease types, the paper utilizes the disease leaf dataset training model under simple background to train the pretrained transfer learning model. This method finishes training in a short period of time and performs the disease identification in the simple environment, reducing the requirement of deep learning algorithms for the hardware equipment. Due to the fact that shallow network has similar characteristics for various learning objects, the shallow neural network for source task can be transferred to the neural network for a target task by using the transfer learning algorithm. Transfer learning has better performance in convergences and ultimate results than new learning in practice.
Setting the domain as , it includes two contents, where represents feature space and contains all possible characteristic values. represents a specific feature sampling instance in the feature space:
Setting the task as , it includes two parts, where represents label space, that is, all vector space consisting of all tags. As prediction function, is obtained by learning from the features and labels of input data:
4. Experimental Study
4.1. The Acquisition of Data
First of all, this study needs to obtain the leaf dataset in the complex environment. The paper employs Crawler technology and obtains 1000 leaf photos from the Plant Photo Bank of China (PPBC), including the leaves of various plants at each growth stage. The shapes of these leaves are different, and the health of the leaves is also different. Aiming at watermark less shelter, obvious leaves, and easy labeling, 189 images are screened out as leaf photos in the complex environment. This dataset is used to train the RPN algorithm to detect and locate the leaf in the complex environment.
Then, using LabelImg, images are quickly annotated and XML files are generated in PASCAL VOC format. It can directly input the target detection neural network as training data. Finally, this study needs to obtain the dataset of diseased leaves in the simple environment. This paper downloads four kinds of images of black rot, bacterial plaque, rust, and healthy leaves from PlantVillage Agricultural Question-and-Answer Forum as training data of transfer learning model, including 537 black rot, 1032 bacterial plaque disease, 293 rust, and 2852 healthy leaves. This dataset is used to train the transfer learning model.
4.2. The Parameter Setup
4.2.1. The Parameter Setup of Leaf Localization
4.2.2. The Parameter Setup of Leaf Segmentation
The main parameter setup of Chan–Vese algorithm is initial zero level set and iteration number setting. In this paper, we set the initial zero level set as a circle with the center of the picture and one-third of the diagonal length of the picture as the radius and set up the Chan–Vese algorithm to calculate 500 iterations.
The image obtained by RPN algorithm is input into the Chan–Vese algorithm and the image in the zero level set is preserved. The image outside the zero level set is set to black to get the image segmentation result.
4.2.3. The Parameter Setup of Leaf Retrieval
Resnet-101 is selected as the pretraining model, and the network is trained by using the dataset of disease leaves under a simple background in this paper. Its network parameters are shown in Table 3.
In this paper, all its parameters are modified and initialized in the last output layer of Resnet-101, and the classification number is changed from 1000 to 4, which corresponds to the identification results of four kinds of leaf diseases.
5.1. The Result of Leaf Localization
The test image is input into VGG-16 model and RPN algorithm, and the results are shown in Figure 2.
With regard to the above images, there is inaccuracy of the frame selection range in Figure 2(b) and the blades in Figure 2(d) are missing. But RPN algorithm can basically frame the main blade structure, which has better performance than the original model.
5.2. The Result of Leaf Segmentation
According to the above results, it can be found that after 500 iterations, Chan–Vese algorithm can get better leaf image segmentation results. Although Chan–Vese algorithm cannot effectively extract the edge contour of the blade compared with the watershed algorithm, it retains the complete structure of the central blade including leaf venation, spot color, and spot shape. The complete central structure of the blade obtained by Chan–Vese algorithm can be used for disease identification of the next step.
5.3. The Result of Disease Identification
The test image is input into VGG-16 model and RPN algorithm, and the results are shown in Figure 8.
According to the above pictures, although it can be found that the frame selection range in Figure 8(b) is not accurate enough and there are omissions in Figure 8(d), RPN algorithm can basically frame the main blade structure and can be used for the next operation.
5.4. Comparative Test
In this paper, the parameters of the transfer learning model include gradient descent optimization parameters and training parameters. The specific parameters are set as shown in Table 4.
After 4000 iterations of training, the loss value and training set accuracy of the transfer learning model and the traditional model are shown in Figure 9.
In Figure 9(a), ResNet-101 represents traditional model. According to Figure 9(a), after the same 4000 iterations training, the transfer learning model has faster convergence speed and lower model loss value than the traditional model. According to Figure 9(b), it can be found that in the process of model training, transfer learning has higher accuracy, lower variance, and better recognition effect than new learning.
Therefore, compared with the new learning, this paper uses the transfer learning to converge faster and achieve better model identification effect. It can meet the requirements of smart agriculture for low hardware resources, fast training time, and high training efficiency.
Then, the image is input into transfer learning model based on the segmentation of Chan–Vese algorithm. As a contrast, the image that has not been processed in this paper is input into the traditional ResNet-101 model for identification, and the results are shown in Table 5.
According to the comparison results in the above table, the average correct rate of the proposed method is 83.75%, which is significantly better than that of the traditional ResNet-101 model (42.5%). Comparing the performances of this method in four samples, we can find that rust and healthy leaves can get better results than black rot and bacterial plaque.
6. Discussion and Conclusion
This paper shows that the plant disease recognition model based on deep learning has the characteristics of unsupervised, high accuracy, good universality, and high training efficiency. However, there are many challenges in accuracy practicability of plant disease detection in the complex environment. In order to solve these problems and optimize the identification method, this paper proposes a recognition model integrating RPN algorithm, CV algorithm, and TL algorithm, which can effectively solve the problem of plant disease identification in the complex environment. The model not only adapts to complex environments, but also increases the accuracy of identification. Compared with the traditional model, the model proposed in this paper not only guarantees the robustness of the convolutional neural network, but also reduces the number and quality requirements of the convolutional neural network on the data set and obtains better results. Therefore, the model could help agricultural production personnel to prevent and cure the plant disease quickly. The model which overcomes the problem of environment complexity can get an accurate identification result in practical application. Furthermore, this study enriches the existing theory and helps to improve the accuracy. At the same time, it is of great significance for the study of plant disease identification in the field of environmental complexity and helps researchers pay attention to the important role of environmental complexity in plant disease identification. Therefore, the model applies information technology to agricultural production and is favorable to sustainable development of smart agriculture.
Although the plant disease identification model based on deep learning proposed in this paper can overcome the complexity of the environment and improve the accuracy of identification, there are still some problems to be pointed out. For example, the Chan–Vese algorithm needs repetitive iterative calculation and runs for a long time, which is not conducive to the fast identification results of this method. In future research, we will use the neural network to generate zero initial set corresponding to different leaves, which will increase the end of calculation limit for the iterative process of Chan–Vese algorithm, speed up the training speed, and end the iteration ahead of time.
Data were made available with the help of the Key Laboratory of Agricultural Information Engineering of Sichuan Province. The data used to support the findings of this study were provided by the laboratory under license. Access to these data will be considered by the corresponding author upon request, with permission of the laboratory.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
This study was supported by the Key Laboratory of Agricultural Information Engineering of Sichuan Province and Social Science Foundation of Sichuan Province in 2019 (19GL030).
A. Breukers, D. L. Kettenis, M. Mourits, W. V. D. Werf, and A. O. Lansink, “Individual-based models in the analysis of disease transmission in plant production chains: an application to potato brown rot,” Academy of Sciences, vol. 90, no. 1–3, pp. 112–131, 2006.View at: Publisher Site | Google Scholar
P. F. Konstantinos, “Deep learning models for plant disease detection and diagnosis,” Computers & Electrical Engineering, vol. 145, pp. 311–318, 2018.View at: Google Scholar
Y. Guo, X. Hu, Y. Zou et al., “Maximizing E-tailers’ sales volume through the shipping-fee discount and product recommendation system,” Discrete Dynamics in Nature and Society, vol. 2020, pp. 1–14, 2020.View at: Google Scholar
R. Amanda, B. Kelsee, M. C. Peter, A. Babuali, L. James, and D. P. Hughes, “Deep learning for image-based cassava disease detection,” Frontiers in Plant Science, vol. 8, p. 1852, 2017.View at: Google Scholar
D. Pujari, R. Yakkundimath, and A. S. Byadgi, “Grading and classification of anthracnose fungal disease of fruits based on statistical texture features,” International Journal of Advanced Science and Technology, vol. 52, pp. 121–132, 2013.View at: Google Scholar
A. Marko, K. Mirjana, S. Srdjan, A. Andras, and S. Darko, “Solving current limitations of deep learning based approaches for plant disease detection,” Symmetry-Baseline, vol. 11, no. 7, p. 939, 2019.View at: Google Scholar
Y. Xu, H. Ding, Y. Xue, and J. Guan, “High-dimensional feature extraction of sea clutter and target signal for intelligent maritime monitoring network,” Comput. Commun., vol. 147, pp. 76–84, 2019.View at: Google Scholar
C. Xu, Y. Chai, H. Li, Z. Shi, L. Zhang, and Z. Liang, “A feature extraction method for the wear of milling tools based on the Hilbert marginal spectrum,” Machining Science and Technology, vol. 23, pp. 847–868, 2019.View at: Google Scholar
J. K. Patil and R. Kumar, “Analysis of content based image retrieval for plant leaf diseases using color, shape and texture features,” Engineering in Agriculture, Environment and Food, vol. 10, pp. 69–78, 2016.View at: Google Scholar
N. Yang, Y. Qian, H. S. EL‐Mesery, R. Zhang, A. Wang, and J. Tang, “Rapid detection of rice disease using microscopy image identification based on the synergistic judgment of texture and shape features and decision tree-confusion matrix method,” Journal of the Science of Food and Agriculture, vol. 99, no. 14, pp. 6589–6600, 2019.View at: Publisher Site | Google Scholar
D. Chad, W. H. Tyr, S. Chen et al., “Automated identification of northern leaf blight-infected maize plants from field imagery using deep learning,” Phytopathology, vol. 107, pp. 1426–1432, 2017.View at: Google Scholar
R. A. Krishnaswamy, P. Raja, and R. Aniirudh, “Tomato crop disease classification using pre-trained deep learning algorithm,” Procedia Computer Science, vol. 133, pp. 1040–1047, 2018.View at: Google Scholar