Abstract
Stomata are the main medium of plants for the trade of water, regulate the gas exchange, and are responsible for the process of photosynthesis and transpiration. The stomata are surrounded by guard cells, which help to control the rate of transpiration by opening and closing the stomata. The stomata states (open and close) play a significant role in describing the plant’s health. Moreover, stomata counting is important for scientists to investigate the numbers of stomata that are open and those that are closed to measure their density and distribution on the surface of leaves through different sampling techniques. Although a few techniques for stomata counting have been proposed, these approaches do not identify and classify the stomata based on their states in leaves. In this research, we have developed an automatic system for stomata state identification and counting in quinoa leaf images through the transformed learning (neural network model Single Shot Detector) approach. In leaf imprint, the state of stomata has been determined by measuring the correlation between the area of stomata and the aperture of each detected stoma in the image. The stomata states have been classified through the Support Vector Machine (SVM) algorithm. The overall identification and classification accuracy of the proposed system are 98.6% and 97%, respectively, helping researchers to obtain accurate stomatal state information for leaves in an efficient and simple way.
1. Introduction
The stomata are an essential part of phylogenetic relationship studies and their physical characteristics are important for plant origin and classification studies [1]. Leaf stomata in various plants possess significantly different characteristics such as density, size, and shape [2]. The stomata are surrounded by epidermal cells known as guard cells. When the plant has enough water in its cells, the guard cells swell up, which helps in opening the stomata. Conversely, when the plant cells do not have enough water, the guard cells cannot swell up; therefore, the stomata remain closed. In this state, the photosynthesis process stops, and plants eventually face starvation [3, 4]. Because of stomata’s importance in regulating plant growth and response to the environment, they are the key functional attribute of interest for researchers in plant biology [5]. The programs that are working on the improvement of crops have modified stomata phenotypes to increase the yield [6]. The stomata easily get affected by external environment conditions and their micromorphological characteristics are often used for plant classification [7, 8]. The state of stomata helps plants to reduce water loss by changing their state from open to closed according to environmental conditions. Researchers across various disciplines in plant biology will phenotype stomatal traits for decades to come because of their physiological importance [9]. However, researchers from agriculture and horticulture disciplines are unable to classify the stomata states in leaf. In different environments, stomata have different states. In the stress, environment stomata are commonly present in closure or semiclosure states because different factors affect stomata. The factor drought stress is one of the major causes of crop loss every year and it is also a cause of loss of aesthetic value in ornamentals [10, 11]. Moreover, the scientists count the stomata to investigate their distribution in leaves using different sampling and counting techniques. The purpose of stomata counting is to inspect their numbers density and distribution on leaf surfaces, to know about their numbers that are open or closed at any time, and to see the adaptations of plants to environmental conditions [12]. The first approach to detect stomata through the HOG feature was presented [13, 14]. This study uses a convolutional neural network to classify single-clipped pores as open and closed, and it involves a binary image segmentation process for the automatic pore measurements. This algorithm requires a manually defined range of parameters (like area, spindle, length, solids, and centroid coordinates). Moreover, this study is unable to identify pores when the size or shape is not the same as the predefined parameter range. Recently, a method was presented to determine the stomatal state in microscopic imprints using Mask RCNN [15]. However, this study mainly depends on expert horticulturist knowledge to manually label stomata states (open and close). Furthermore, identification of stomatal states in images through naked eyes leads to inaccurate results due to the fact that the stomatal state is affected by the area of pore aperture in a leaf [16].
A method for the measurement of stomatal pores using image processing software, like ImageJ [17], involves manual measurement of stomata classification and detection system in microscope images for maize cultivars [18, 19]. This type of method wants to manually label points of interest from researchers, such as height, width, and boundaries in pores. The drawbacks of this method are that, first, it requires manual attention and, second, it requires a huge amount of data. Using this method, only some of the data points can be used to build the model, which leads to inaccurate results [20].
In this study, we propose an automatic system for the identification of stomatal state (open and close) and stomata counting in microscopic imprints using a deep convolutional neural network, Single Shot Detection (SSD-MobileNet V2). SSD goes straight from pixels of the image to bounding box coordinates and class probabilities. The architecture of this model is based on the inverted residual structure in which the input and output of residual blocks are thin bottleneck layers as compared to traditional residual models. Moreover, this model also provides a prediction about the category of each pixel along with the bounding box around the object. This method was used for the detection, counting, and state identification of stomata in microscopic imprints of quinoa crop leaf. The results of this approach were evaluated visually. Furthermore, stomatal states were classified through a Support Vector Machine (SVM), which is a supervised machine learning model that uses a classification algorithm for two-group classification problems.
2. Related Work
This section presents machine learning techniques presented by researchers on stomata study using image processing and deep learning. The research on stomata by using image processing techniques started in the early 90s when the first approach [21] was proposed. This technique measures the stomata features in grayscale images using Fourier transform and threshold filter for image segmentation. A study [22] proposed a system of automatic stomata counting in a microscopic image: this technique first converts RGB images into CieLAB to select the best channel for analysis. The detection of stomata is performed by wavelet spot detection and morphological operations. This system achieves 90.6% accuracy by using a watershed algorithm.
An approach was published, which compared tomato cultivars using many morphological features, stomata measurements, and the microscopic images of the different crops, which are obtained by the scanning electron microscope (SEM) method [23]; in this approach, the segmentation was performed by a watershed algorithm resulting in one stoma per image followed by morphological operations. The Sobel kernel filters were used to remove noise and obtain stomatal boundaries. This method uses 100 images of tomato cultivars and a multilayer perceptron algorithm and gets 96.6% accuracy. In 2017, an approach of automatic stomata detection and pore measurements for a grapevine was presented [24]. This approach uses a cascade object detection (COD) algorithm. It consists of two steps: In the first step, the first cascade classifier was trained by using stomata and nonstomata images. In the second step, a sliding window was used on microscopic images to identify the stomata inside the image. The detection of the pore measurements was performed using binary segmentation and skeletonization with ellipse fitting, which estimates complete pore measurement for incomplete stomata. As a result, this method reached 91.6% precision. An automatic stomata counting system using a deep convolutional neural network (DCNN) was proposed [25]. This technique identifies and counts the stomata in a variety of different micro imprints stomata. This network achieves 98.1% identification on the Ginkgo scanning electron micrograph. To improve the accuracy and performance of the traditional Single-Shot Multibox Detector (SSD) object detection model, an approach [26] was presented. This technique works on the defect of a weak correlation between object prediction score and positioning accuracy in the SSD model. The improved model enhanced the correlation between these two by adding an intersection over the union (IoU) prediction loss branch. Deep learning rose to its prominent position in digital image processing and computer vision when neural networks were applied in various types of medical image analysis datasets [27, 28]. Recently, an approach [29] has been published; in this study, the author develops an automated system for stomata detection, which can detect individual stomata boundaries regardless of the plant species. However, this study mainly focuses on stomata detection and segmentation of different species and not providing information about the state of stomata in leaf images.
2.1. Dataset
In this work, the dataset of quinoa microscopic images is photographed through Leitz DMRD light microscope (Leica MikroSkopie & camera (Leica DFC 420) [16, 30]. The surface of the quinoa crop leaf was cleaned using a soft brush. A thin layer of fingernail polish was applied on the surface of the leaf and let it be dry for 10–15 min. A crystal-clear tape was used as the desired area of the leaf surface and drawn off carefully. A 1 mm2 grid was superimposed on the images for the calculation of stomata density [31]. Some example of captured imprint is shown in Figure 1. A total of 400 images are acquired with JPEG format; further, these images are preprocessed using Python libraries in preprocessing, and we resize the image to 512 × 512 and remove noise from the images for the further processes.
(a)
(b)
3. Proposed System
The objective of this study was to develop an automatic system for stomatal state identification, classification, and counting in microscopic imprints. The methodology flow of the proposed system is shown in Figure 2. The output of the system was stomata detection with their state (open and close) and total stomata in a microscopic imprint as shown in Figure 3.
(a)
(b)
The proposed system consists of the following steps:(i)Stomata detection(ii)State identification and counting(iii)State classification
3.1. Stomata Detection
For stomata detection, DCNN (SSD-MobileNet V2) was implemented for creating a learned model using the transfer learning approach. SSD-MobileNet v2 takes one shot to detect multiple objects in the image. Its approach is based on a feed-forward convolutional network that generates the fixed-size collection of bounding boxes and scores for the object present in the bounding boxes [32]; the architecture of SSD is shown in Figure 4. The model consists of two parts: Firstly, it extracts the feature map and secondly applies convolutional filters to detected objects. The first step for training the model is to manually label the stomata region in the microscopic imprints. From labeling, two subfolders (train and test) were created. These subfolders are used to encode the visual properties of labeled objects into feature vectors (train_label and test_label). The ratio of the data in the subfolders is 70 : 30, respectively. In the second step, the feature vector train label is used as the input for the learning model. The learned model for the stomata detection task is finally the classified model of the feature vector test label. The input to the model is stomata microscopic imprints and the output was the stomatal pore anatomy parameters, including the length and width of the stomata and aperture area.
3.2. State Identification and Counting
The stomata state identification task was performed through the three following steps:(i)Detected stomata(ii)Measuring the stomatal and aperture area(iii)Measuring the corelation between stomata and aperture area
The stomata detection is obtained from the learned model, which generates the binary image with the same size as the input image for each stoma in the image. Each pixel in the image is used as the coordinates point. Each point value is either true or false, where true values mean the area of stomata and aperture in the image and false values mean that there is no area of pores. Each input image generates “x” binary maps, where “x” is the number of detected stomata in the image. The learned model extracts the boundary coordinates of stomata and aperture. The coordinates values are Xmin, Xmax, Ymin, and Ymax. The width and height of detected stomata and aperture are calculated using the equations. The area of aperture (AoA) and the area of stomata (AoS) are calculated using equation (2); the area obtained from the coordinates values was in pixel; the area has been converted in µm2 [16]; finally, the corelation between the stomata and their aperture area is measured by using equation (3).
The state of stomata is determined based on corelation between stomata and their aperture area because the stomatal pore aperture is affected by irrigation and biochar treatment [16] and all detected stomata in the image were counted using Python script.
3.3. State Classification
The stomatal state classification was performed through the Support Vector Machine (SVM). This study uses a linear SVM classifier to classify open and closed stomata. The learned model evaluates a total of 1490 stomata in a dataset with their states and save their features (height, width, area, states, and corelation) in CSV format for the classification task. The target values for the classifier were stomata states (open and closed) in the dataset. In SVM, we plot points of data items in n-dimensional space (where n is a number of features in the dataset) with the value of each feature of a particular coordinate. In the end, the stomata classification was performed by finding the hyperplane which differentiates the two classes as illustrated in Figure 5. The support vectors in the graph are the simple coordinates of individual observation. The trained classifier is the frontier that best segregated the two classes as .
4. Results and Discussion
The proposed system has correctly identified, counted, and recognized stomata state when stomata are present in an image.
4.1. Stomata Detection and Counting
The stomata detection and counting results are shown in Figure 6. The proposed model achieved a good performance regarding stomata detection, counting, and state identification. A total of 120 test images were used to evaluate the model performance. Through statistical analysis of 118 images, an average of 1490 stoma were detected by our learned model. To analyze model performance quantitatively, the stomatal pore was manually labeled through the LabelImg tool. The annotation results were used as ground truth (GT). The GT results were compared with results obtained from the proposed model. The average precision (aP) was 98.6%, and average recall (AR) was 97.87 as shown in Table 1.
4.1.1. Stomata State Identification
The proposed system automatically calculates the area of stomata and their aperture by which it recognizes how many stomata are open or closed in the image as shown in Figure 3. The state identification is performed by finding the corelation between stomata and aperture area as described in Section 3.2. The quinoa stomatal parameters of ground truth value and corresponding predicted values of model were compared, and the corresponding relative error of stomatal measurement was calculated as in Table 1. The calculation process was as per the following: (1) Each manually marked stomatal pore was compared to the results produced by the model according to maximum IoU, (2) through the stomatal state identification section and the formulas proposed in Section 3.2, evaluation indices, the anatomical features of the manually labeled stomata region, and the anatomical parameters from the learned model were obtained, and (3) the relative errors between these were calculated and the average relative error for each of the stomata’s anatomical features was summed individually for each respective stomata and divided by the total number of stomata.
4.2. Stomata Classification
The 1490 stomata in the dataset detected by the deep system with the open and closed state are classified by SVM, which classifies the stomata states with 96.5% accuracy, 91.3% F1 score, and 91.5% precision as in Table 1. The confusion matrix is used to determine the performance of a classification model on a set of test data. The classifier predicts 1230 stomata as close and 260 as an open state. The evaluation of the stomata classification model is expressed in Table 2.
4.2.1. Accuracy
The accuracy is the ratio of correctly predicted stomata to the total stomata, and the classifier accuracy was measured using the following equation:
4.2.2. Precision
The precision is also known as a positive predicted value and it is the ratio of positive predicted value to the total predicted values. For the proposed classifier, the precision was calculated for open and close stomata using the two following equations:
4.2.3. Recall
Recall is a sensitivity and probability of detection, that is, true positive rate. It is the ratio of correct positive prediction to the total positive prediction. Recall was also calculated for both open and close stomata prediction by the model using the two following equations:
4.2.4. F1 Score
The F1 score is a measure of model accuracy on a dataset. The F1 score for the proposed classifier is calculated using the following equation:
4.3. Discussion
Stomata are important and functional parts of plant leaves; however, counting stomata is a hard task; scientists used many sampling and counting techniques to count stomata in the leaf. We developed a fully automatic system for stomata state identification and counting. Moreover, the system can be iteratively retrained with new imprints for the performance improvement and for adjusting the need of users. However, the previously proposed methods [25] do not identify the open and closed state of stomata. Our proposed system firstly detects the stomata and secondly it measures the aperture and stomata area, and, on the basis of their corelation, it predicts the stomatal state (open and close). The stomata states (open and close) perform a significant role in describing the plant health. At last, the states of stomata that are identified by the system were classified.
5. Conclusion
By all counts and proven results, we are presenting an automated stomata state identification, counting, and classification system for micrographs of quinoa leaves. The proposed system has identified the state and counted the stomata when stomata are present in the leaf image. The experimental results showed that the proposed system has provided more accurate measures of stomata parameters and provided more accurate information about the state of stomata in leaf micrograph. In addition, this method has an ability to process thousands of stomata in minimum time, reducing the need for researchers to manually measure stomata and speed up the process of analyzing the plant health. The system would be helpful for the plant biology community. In the future, we will develop a generalized toolkit that supports more plant species by applying proposed method for extracting stomata information from leaf image.
Data Availability
The microscopic images data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare no conflicts of interest.
Acknowledgments
The authors thank their colleague Mr. Zulqrrnain Ali from MNS University of Agriculture, Pakistan, who provided insight and shared his expertise with them during this research work.