Abstract
Special symptoms could be observed on rice leaves when exposed to potassium deficiency, and these symptoms usually display differently under different potassium levels, which offer a foundation for rapid nutrition diagnosis. In this research study, two years of hydroponic experiments on rice (providing 5 levels of potassium nutrition from extremely short to normal) were carried out and the leaf images were acquired by optical scanning at four growth periods. To diagnose the potassium nutrition content, the special symptoms including the yellowish brown leaf margin and the necrotic spots were segmented and quantized by the object-oriented method from leaf images, and the 6 further spectral characteristics of leaf were extracted by the image color analyzing function of MATLAB software. Based on the relationship between potassium content and leaf characteristics, the G value (average value of G channel in the RGB color model) calculated from the entire leaf and leaf tip, the area of yellowish leaf margin, and the number of necrotic spots were applied in the establishment of the identification model of potassium stress by using the support vector machine (SVM). The results indicated that the overall identification accuracies of rice potassium nutrition contents were 90%, 94%, 94%, and 96% at four different growth periods (productive tillering stage, invalid tillering stage, jointing stage, and booting stage), respectively. The data obtained from another year were used to validate the model, and the identification accuracies were 94%, 78%, 80%, and 84%, respectively. Generally speaking, the extraction of the specific symptoms by using object-oriented segmentation is an extension of machine vision technology in diagnosing potassium deficiency, and its application in diagnosing plant nutrition is valuable for the quantization of effective characteristics and improvement of identification accuracy.
1. Introduction
The hyperspectral characteristics of rice (the reflectance at rice leaves and canopies) have been widely demonstrated for identification of the rice nutrition status [1–6]. However, it has limitations to decide its nutrition status merely by the hyperspectral information. For instance, when rice grows under different nutrition conditions, the spectral information exhibits the similar waveform, which makes it difficult to identify the nutrition status by discriminating the critical value. In addition, external environmental factors like diseases, pests, drought, and waterlogging could have interference on the reflectance of leaves and canopies. Consequently, it is hard to diagnose nutrition status accurately only by using hyperspectral characteristics of rice leaves.
When it comes to the special symptoms, potassium (K) deficiency commonly results in dark green plants with yellowish brown leaf margins, and sometimes, dark brown necrotic spots initially appear on the tips of older leaves [7]. As shown in Figure 1, due to K deficiency, the leaf tips show yellowish brown and the symptom always changes with rice growth. The symptom appears on older leaves first, then along its leaf edge, and finally on the leaf base. If the deficiency is not treated, the discoloration will gradually appear on younger leaves.
Recently, some research studies have applied the hyperspectral sensor for identification of crop nutrition, which could synchronously acquire the digital and hyperspectral image of rice leaves, but with low image resolution. In addition, due to the interference of the external environment during the image acquisition process, only some obvious deficiency symptoms could be captured. Therefore, it is difficult to capture and quantify these microsymptoms such as the yellowish brown leaf margins and necrotic spots. Since scanning is conducted in a closed environment, the external disturbance could be eliminated during the imaging process, and thus, the color and size reproduction could be enhanced. Compared with the common digital camera, the scanning image has no complex background, multiredundant information, and image noise, which could reduce errors in the image analysis process [8, 9]. Thus, in this research study, the static scanning was used to obtain the digital image of rice leaves to capture the special microsymptoms.
In order to explore the most sensitive leaf characteristics to K nutrition in this research, the scanning images of rice leaves were analyzed and compared to find the difference of characteristics under a different K nutrition status. The analysis of plant leaf color has been proved to be useful in assessing plant nutrition status [10–14]. In addition, when rice suffers from K deficiency, the leaves will also display specific symptoms: yellowish brown leaf margins and dark brown necrotic spots [9].
Recently, image processing technology has been widely used to analyze the digital image by using the information of single pixel rather than pixel groups [11, 12]. This whole process not only focuses on the per-pixel spectral information but also ignores the spatial relationships between the pixels, which are crucial information to identify K deficiency by analyzing the rice leaf image.
The information to be extracted is not a single pixel but spatial pixel clusters defined through an aggregation algorithm known as “multisolution segmentation.” However, due to user bias in setting scale and homogeneity parameters, we propose the object-oriented classification method in which the segmentation algorithm integrates homogeneity criteria by considering size uniformity, pixel values, and local contrast for the elements contained in a given segment [13]. Object-oriented classification is proposed to be different from the per-pixel classification method, whose basic unit is the object that is composed by a group of similar pixels, and this object is more consistent with the reality [13, 14]. Furthermore, additional properties like shape and spatial information could be extracted in conjunction with spectral information in this method.
In this study, we conducted the object-oriented classification to efficiently extract the area of yellowish brown leaf margins and dark brown necrotic spots according to its leaf information such as spectral, shape, and topology properties. Then, combined with leaf color characteristics, we propose to build the identification rule and model of rice K nutrition status, and the support vector machine (SVM) method would be used here. Given the above analysis, this study could provide a new idea to diagnose the rice K nutrition by using the machine vision technology.
2. Materials and Methods
2.1. Experimental Design
For the purpose of research, the experiment was conducted in a greenhouse on Zijingang Campus, Zhejiang University, Hangzhou, China, during 2012 and 2013. Rice seeds (cultivar ZheYou-NO.1) were pregerminated for 3 days. 7 days later, the seedlings were individually transplanted to the similar hydroponic conditions as in the previous research study [10] for precise nutrient control. As shown in Figure 2, to understand the response of rice growth to different K nutrition conditions, 5 K treatment levels (K2SO4: 0 mg/L (K1), 22.3 mg/L (K2), 44.7 mg/L (K3), 67 mg/L (K4, normal nutrition), and 114.3 mg/L (K5)) were provided to seedlings via hydroponic solutions. The solutions were replaced every 14 days, and the pH value was always maintained at 5.
2.2. Acquisition Images and Feature Extraction
The top 3 fully expanded leaves from the productive tillering stage, invalid tillering stage, jointing stage, and booting stage were scanned using EPSON GT20000 (the scanner could create an output image of 16 bits/pixel/internal color and 1 to 8 bits/pixel/external color) with a resolution of 300 dpi (dots per inch). To build the diagnosis rule and identification model, 600 rice leaf samples including 480 (under 4 degrees of K deficiencies) and 120 (with normal nutrition) were collected from 4 growth stages. To validate the identification model, 300 samples (60 samples per K level) were collected on July 29th and August 13th, 20th, and 31st (2012). As shown in Figure 3, the rice leaves appeared significantly different under 5 levels of K nutrients. Similar to the nutrition mechanism of plant growth, the yellowish brown leaf margins and dark brown necrotic spots first appear on the tips of older leaves. Under severe K deficiency, leaf tips show the serious yellowish brown. Under slight K deficiency, the necrotic spots show a decrease [7].
2.3. Extraction of Leaf Characteristics Based on Color Mean Value Function
According to the nutrition mechanism of rice growth, when suffering in the different degrees of K deficiencies, the difference mainly reflects in the color of leaf and leaf tip [7]. Therefore, we mainly selected the color characteristics of leaf and leaf tip to identify the K nutrition status. Here, the RGB mean value function in MATLAB (MathWorks Inc., USA) was applied for the quantification of color information. Consequently, a total of six color characteristics were extracted and quantified in this study (Table 1).
As depicted in Figure 3, with the aggravation of K stress, the symptoms become more obvious, and the area of yellowish brown leaf margins and dark brown necrotic spots is also more distinctive under different K nutrition conditions. Thus, to emphasize these differences of special symptoms, a suitable method to quantify the characteristic information is necessary since the yellowish brown leaf margins and dark brown necrotic spots cannot be expressed with single pixel color information. Hence, an object-oriented classification method was applied to extract and quantify the symptoms in this research.
2.4. Extraction of Leaf Characteristics Based on the Object-Oriented Method
The preliminary process of object-oriented classification is the segmentation which divides the whole image into numbers of heterogeneous objects, and here, the scale is considered the most important parameter to decide the size of the object [13, 14]. When the value of scale is too large, there would be mixed errors in the object. But when the value of scale is too small, the segmented object would be fragmentary and consequently makes the sequent process much more difficult.
In this study, we aimed to separate the dark brown necrotic spots area (Figure 4) and the yellowish brown leaf margins (Figure 5), causing K deficiency on the leaves. So the multisegmentation was executed in order to achieve two scales that arise from these two situations. This resulted in the choice of optimal scale value of segmentation.
When the value of scale was set to 20, the leaf was segmented with high degree of fragmentation; even some spots was oversegmented, which was contrary to our original intention that the spot was treated as the primary object. Nevertheless, when we set the scale value to 40, many spots were segmented into the objects those also contained the healthy area, decreasing the whole accuracy of the classification. When we set the scale value to 30, the spots could be distinguished from the background well, and through the later image fusion, we could also extract the healthy region properly. As a result, 30 was selected as the final value of the scale by trial and errors. In this way, the whole leaf image was segmented to get the satisfactory spots as the aim object for classification.
After extracting the spots, we began to segment the yellowish brown leaf margins from leaves. As shown in Figure 5, when the scale value was set to 500, the yellowish brown leaf margin area was oversegmented as the complete etiolated area was separated, which increased the difficulty in characteristic extraction. The results of the segmentation also demonstrated that there was no obvious difference between the results when the scale value was 600 and 700.
Multiscale segmentation is processed through the following steps, which first identifies a single image object of one-pixel size and then merges it with adjacent objects which have the same homogeneity criteria. The homogeneity of image regions could be expressed by the computation of heterogeneity (f), which is performed in the color heterogeneity and shape heterogeneity, as shown in Figure 6. Here ω is the artificial weighting [15, 16].
Therefore, apart from the scales, the color and shape are also two important parameters to decide the homogeny of the objects in the segmentation process. And shape is decided mostly by other two components: compactness and smoothness. However, in view of the aims of this study, we considered the shape parameter being less important compared to color. Therefore, the shape was set to 0, which means we only took the spectral information into consideration.
Apart from segmentation, parameter selection is an important process when applying object-based classification. The most important fact influencing the abstraction of yellowish brown leaf margins and dark brown necrotic spots is to select the most suitable typical parameters to build the distinction and quantization rules. In order to optimize the parameter combination, different parameters of object were selected to build the parameter sets. Table 2 shows the object spectral and textual parameters utilized in the nearest neighbor classification.
After all the parameters are obtained, the flowchart involved in the segmentation process is shown in Figure 7.
According to the parameters utilized in the nearest neighbor classification, the image was segmented to three parts: yellow (yellowish brown leaf margin), green (background), and red (dark brown necrotic spots). Compared with the original image, the yellowish brown leaf margin, the dark brown necrotic spots, and the background had been successfully discriminated, laying a foundation for further quantization and analyzing (Figure 8).
After the extraction of the yellowish brown leaf margin area and dark brown necrotic spots of leaf, analysis of the relationship between the two symptoms and K nutrition status would allow for further achieving of the K deficiency diagnosis.
Figure 9 displays the 1st, 2nd, and 3rd leaf of rice under K deficiency. The image shows that the area of yellowish brown leaf margin and quantity of necrotic spots at different leaf positions is marked differently under extreme K deficiency. The symptoms of K deficiency in the 3rd fully expanded leaf is extremely visible, compared with the 1st leaf which shows no specific symptoms, which results from the mobility of the rice potassium. Therefore, only the 3rd fully expanded leaf was selected for diagnosis of K nutrition status.
In order to further validate the relevance of the specific symptoms of rice leaves and the K nutrition contents, the digital images of the 3rd leaf with five K nutrition levels were further compared and analyzed.
As shown in Figure 10, with the decrease in K nutrition content, the area of yellowish brown leaf margin and quantity of necrotic spots showed increasing trend in the 3rd leaf.
After analyzing the symptoms of rice leaves, in total 8 parameters were extracted from the scanned images of the leaf (Table 3), including 6 color parameters and additional 2 shape parameters (area of yellowish brown leaf margin (AYLM) and quantity of necrotic spots (QNSs)).
2.5. Support Vector Feature Selection Method
The support vector feature selection (SVFS) method was applied to select the optimal characteristics for diagnosis of K nutrition status. SVFS provides an effective and efficient mechanism to select the optimal characteristics for the small training samples [17, 18]. Furthermore, the core of SVFS is the employment of SVM (support vector machine) which selects the optimal characteristic set by identifying the high correlations between these characteristics to remove the redundant characteristics and achieve the maximum classification prediction ability [17, 18]. Therefore, the optimal characteristics subset could represent the set of sensitive characteristics under K deficiency. In this research study, SVFS was used to screen the optimal characteristic set from the 8 characteristics in LibSVM-3.12. The SVM classification has two steps: training and classification [19]. In LibSVM platform, RBF kernel function (equation (1)) was adopted in this research study:
The classification function iswhere σ and c are both kernel parameters which are calculated by grib.py function of LibSVM.
3. Results and Discussion
The most sensitive characteristics of each growth stage for K deficiency are highlighted in Table 4. Further analysis was undertaken by combining all the growth stages of the leaf samples to generate 4 universal characteristics (LG, LTG, AYLM, and QNS) in order to understand the identification of K nutrition status at all growth stages. The 4 characteristics could sensitively and overall express the differences between the 5 levels of K stress, which conform to the nutrition mechanism of rice growth when suffering from K deficiency.
After screening the optimal characteristics, SVM in the LibSVM platform was used to build the diagnostic model of rice K nutrition status. A total of 600 leaf samples from all growth stages were used to build the identification model. As shown in Table 5, the identification accuracy of K nutrition status in rice are different among different leaf characteristics of different leaf positions. The identification of the characteristics of the 3rd leaf has the highest accuracy for all growth stages, which are 90%, 94%, 94%, and 96%, respectively.
Rice tends to tiller effectively at the early growth stage. At this time, the field has sufficient nutrition to supply the growth of these tillers, which has little effect on the growth of new leaves. Meanwhile, because the third leaf has experienced deficiency for a relatively long time, the main symptoms appear in the third leaf. However, the first leaf was not affected by stress now and the symptoms were not obvious, so the identification accuracy of K nutrition status by using the information of the first leaf was the worst (74%).
At the invalid tillering stage, following the continues growth of early tillers, the stem of rice starts grow. At this time, because the nutrient supply cannot satisfy the growth of both new tillers and stems, rice leaves suffer from a longer period of K deficiency and the symptoms of the third leaf become more significant. As a result, the identification accuracy of potassium nutrition by using the characteristics of the third leaf increased to 94%. In addition, compared with the productive tillering stage, the first leaf showed more obvious deficiency symptoms because of K deficiency for a long time, so the identification effect was better than that of the productive tillering stage (86%).
At the jointing stage, nutrition was mainly supplied for rice stem and node growth. At this time, due to long-term K nutrition deficiency, the K nutrition in the third leaf began to be easily transferred to the first leaf. Because the first leaf has got the K supply from the old leaf, the symptoms were relatively weakened, and its identification accuracy of K nutrition was lower than that of the invalid tillering stage (78%). For the third leaf, although part of the K nutrition was transferred to the first leaf, but compared with the invalid tillering stage, the deficiency has not become more serious, so the identification accuracy was similar (94%).
At the booting stage, it is the peak time for rice nutrition demand, and nutrient uptake of rice increases rapidly. Because of the inadequate supply of K nutrition for a long time, lots of the K nutrition in the third leaf is transferred to the first leaf for its growth. At this time, the stress symptoms of the third leaf are the most obvious, and the identification effect of K nutrition is the best (96%).
To know the detailed identification results from each K nutrition level, further analysis was conducted on the 3rd leaf, which had the highest overall identification accuracy (Table 6).
The results showed that all the leaf samples of K1, K2, and K3 could be identified successfully at four growth stages, while there were misjudgment in K4 and K5. This is largely due to the yellowish brown leaf margin and necrotic spots mainly appearing under K1, K2, and K3.
In consideration of the reliability and applicability, the diagnosis model was validated by using the top three full expanded leaves of rice (300 samples), which were collected in the four growth stages of 2012 (productive tillering stage, invalid tillering stage, jointing stage, and booting stage), the same as in 2013. Similarly, the same 4 optimal characteristics of rice leaves were used to validate the identification model. The validation accuracies are shown in Table 7.
The results show that the method of object-oriented classification can better identify the K nutrition status of rice, and for other crops, this method also has important potential value. For example, under N deficiency, soybean leaves will have bronze patches; under P deficiency, wheat leaves become dark green and purple, and purple on leaf sheath is particularly prominent; under N deficiency, maize leaves show chlorosis from bottom to top, and symptoms develop from tip to base along the midrib, forming a V-shaped shape. Similar to the rice, these deficiency symptoms can also be captured and identified by machine vision technology so as to diagnose their nutrition status.
4. Conclusions
In this study, the top 3 fully expanded leaves were used to identify the K nutrition status of rice plants. In laboratory conditions, the scanning images of rice leaves were analyzed to get the spectral information. Simultaneously, the special symptoms under K deficiency (the yellowish brown leaf margin area and necrotic spots of leaves) were extracted and quantized by the object-oriented classification method. Our research has presented a highly targeted method which could rapidly and accurately detect rice K nutrition status, with the results proving that the special symptoms plays an important role in identification of K nutrition status.
In this research study, the optimal characteristic set was selected by the SVFS method for establishment of the diagnostic model. The selected set included the leaf G, the leaf tip G, the yellowish brown leaf margin area, and the quantity of necrotic spots. SVM was used to establish the identification model. K nutrition status of rice can be well identified by adopting the four optimal characteristics, and experiments in different years also verify the stability of the method according to the identification accuracy.
This research demonstrates the importance of utilizing the existing technology such as scanning in innovative ways that allow for fast and accurate identification of K nutrition status in rice leaves. At the same time, this study also proposes a new method to extract the specific symptoms of crops under nutrition deficiencies. The application of this method could provide an important tool for farmers and some agricultural corporations in understanding the nutrition status. Further research could focus on applying this technique to other important nutrient elements in rice leaves (such as nitrogen and phosphorus) or nutrition identification of other crops.
Data Availability
The authors confirm that all data underlying the findings are fully available without restriction. Due to the volume of the data, only a selection of images may be found on share‐web (https://share.weiyun.com/53UV1rD). For access to all images, the author may be contacted (Lisu Chen: [email protected]).
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
This research study was funded by the National Natural Science Foundation of China (grant no. 31801255) and the Principal Investigator for Zhejiang Province Postdoctoral Fund (grant no. BSH1502132).