Abstract
The positioning of the apple growth cycle plays a very important role in predicting the development of apples and guiding fruit farmers in agricultural operations. The traditional method of manually positioning the apple growth cycle has problems such as low efficiency and poor accuracy. Pattern recognition provides support for continuous and rapid positioning during the apple growth process. Under the natural conditions of the orchard, due to the large differences in the individual colors of the apples during the growth process and the influence of factors such as light changes, the photographed apple images are more complex, which brings certain difficulties to the segmentation and recognition of the apples. In this paper, pattern recognition is used to automatically identify and extract the growth stages of apples, a hue intensity (HI) color segmentation algorithm based on a Gaussian distribution model based on prior knowledge is studied, and then an active shape model (ASM) is used to identify each period of apple growth based on pattern recognition. After a series of experimental verifications, the ASM-based automatic identification method proposed in this paper is feasible and can identify the various growth periods of apples, thereby serving the mechanized production of apples.
1. Introduction
Apples have always been recognized as a fruit that can prevent a variety of diseases. Doctors said, “A study by the University of California in the United States has found that apples contain a flavonoid substance that can delay cell aging, protect the heart, and prevent cerebrovascular diseases.” In 2020, China has become the world’s largest apple producer, with apple planting area and output accounting for more than 50% of the world. The apple industry has brought good economic benefits and has become the main source of income for farmers. Precisely positioning the growth cycle of apples to grasp its development status is of great importance for guiding production management such as fertilization and irrigation. In the growth process of apples, it is important to accurately assess the growth of apples and perform appropriate work at the appropriate time.
Smart agriculture is the integration between agriculture and Artificial Intelligence (AI) technology and modern information technology, and it is also a technology that is urgently needed in the development of global agriculture. Nondestructive monitoring of crop growth is one of the important links in smart agriculture. It requires real-time and accurate acquisition of plant growth information to guide the fine management of crops. In particular, the monitoring of crops in different growth periods is an important part of crop growth monitoring. First of all, understanding the growth period of crops can help people analyze the relationship between field crop growth and environmental conditions; in addition, it can effectively guide operations, thereby significantly improve the level of agricultural mechanization and crop yields [1, 2].
Pattern recognition can detect subtle changes in plants due to malnutrition earlier than human vision, providing a reliable basis for timely irrigation and supplementation of nutrients. Under the current circumstances, if apple’s economic benefits and automated production are effectively improved, this will have important practical significance. In the process of mechanized production, orchard management is a very important task. The apple’s growth cycle is divided into six major periods, the first is the growth period of the seedlings; the seedlings need to be carefully maintained and provided with appropriate water and fertilizer management to make the apples grow better. Next is the flower bud differentiation period, and attention should be paid to temperature and light control. The temperature is 12∼18°C; during the flowering period from April to June, stop watering to avoid falling flowers; then in the flowering period of apples, supply enough water before flowering, and stop watering at the flowering period to prevent the flowers from falling off. In the case of poor growth, it is necessary to cut the flower buds in time; cut off the bad flowers and poorly grown flowers to maintain a healthy growth state. This is followed by the physiological fruit drop period and the fruit expansion period of the apple, and the maintenance work should be done well. Finally, it should be fertilized and pruned in time during its maturity period to allow it to grow and produce better results. In actual production, the external morphological structure of the crop has undergone tremendous changes from seedling growth to maturity, and these morphological and structural changes are also very significant for the image. Therefore, it is very necessary and urgent to carry out pattern recognition research on the growth and development cycle of apples with the help of computer vision and image processing theories and methods.
The main contributions of this paper are summarized as follows: (i) we use the HI color based on Gaussian distribution, which is suitable for the image segmentation algorithm of apples at each period of the growth process in the complex environment of the orchard to perform apple image segmentation. (ii) On the basis of correct segmentation, an ASM based on pattern recognition is used to identify apples in the growth process of each period. In other words, the novelty of this paper is mainly to use the HI color based on Gaussian distribution to perform apple image segmentation, and then use ASM to identify the whole apple growth cycle from seedling stage to mature fruit.
The rest of the paper is organized as follows: Section 2 analyzes and summarizes domestic and foreign research work in crop growth and development monitoring, crop image segmentation, and crop automatic identification. Section 3 introduces the collection and segmentation of apple images at each period of the growth process. Section 4 uses ASM to identify apples at each period of the growth process. Experimental results are reported in Section 5, and finally Section 6 concludes this paper.
2. Related Work
Nowadays, labor on farms and orchards depends largely on skilled farmers. Manual labor wastes time and increases the cost of production, while uneducated and inexperienced workers make unnecessary mistakes. In [3], Zhao et al. present a review on some key vision control techniques and their potential applications in fruit or vegetable harvesting robots. In [4], the intelligent agriculture becomes a popular concept. In [5], Wang et al. develop a computer vision-based system for automated, rapid, and accurate yield estimation. In [6], there is increasing interest in the use of image processing techniques for crop detection in intelligent weeding applications. In [7], Lu and Sang propose a recognition method for citrus fruits under varying canopy illumination based on color and contour information. In [8], Linker et al. propose a method for detecting apples in natural lighting. It details an algorithm for estimating the number of apples in color images under natural illumination. With the rapid development of AI, pattern recognition is widely used in agriculture [9]. Deep learning is a special kind of machine learning that can be used to classify crops [10–12], crop image segmentation [13, 14], crop target detection [15, 16], and other missions [17]. In [18], Zhang et al. propose a 13-layer convolutional neural network (CNN) for fruit classification with high accuracy. In [19], Chen et al. use a spot detector based on fully connected CNN to extract candidate regions in the image and segment the target region, and use subsequent CNN counting algorithm to calculate the number of fruits. In [20], Dyrmann et al. present a method for automating weed detection and use CNN to detect the weeds. Faster R-CNN [21] uses the Region Proposal Network (RPN) method to detect the Region of Interest (ROI) in an image. In [22], Inkyu et al. use the faster R-CNN method to detect a variety of fruits and produce good results. The faster R-CNN with VGG16 nets [23] is the most advanced method for fruit detection [9]. In [24], Tian et al. propose an improved YOLO-V3 model for detecting apples during different growth stages; this model is used to detect young apples, expanding apples, and ripe apples in complex backgrounds.
Relying on the continuous development of image processing technology and pattern recognition technology, we can use computer technology for crop shape recognition. At present, in crop morphology recognition, research mainly focuses on the appearance characteristics of crops, that is, the measurement of the geometric shape characteristics of the crops, for example, the classification of certain crops by different leaf shape characteristics, including the rectangularity, circularity, and circularity of the leaves. Geometric parameters such as eccentricity, leaf size and invariant moments, and crop classification and identification can also be achieved through crop flowers, fruits. Most of the work done is focused on the study of specific organ morphology of crops, such as leaves and flowers. Few studies involve the use of computer vision systems to automatically identify the growth and development stages of crops. However, the use of pattern recognition to locate the growth cycle of apples is an indispensable part of practical applications, which provides impetus for the development of this paper. Especially different from the previous work, the novelty of this paper is mainly to use the HI color based on Gaussian distribution to perform apple image segmentation, and then use ASM to identify the whole apple growth cycle from seedling stage to mature fruit.
3. Image Collection and Segmentation
3.1. Data Collection
Computer vision usually includes the following modules: light source, camera or camera image capture card, input and output module, and image processing software. The performance of the image acquisition system will directly affect the quality of the collected data and then affect the subsequent image processing and analysis. Taking into account the actual situation of this research and the goals that need to be completed, the hardware system for apple image segmentation and for identification of each key developmental stage is mainly composed of a computer and an image acquisition system. The image acquisition is usually done by a video camera, ordinary digital or optical camera, and then the digital image in the memory card is converted to the computer. This paper will mainly use digital cameras for data acquisition. The image acquisition system mainly includes tripods, digital cameras, computers, and data transmission equipment. This paper can obtain apple images of different weather conditions and different growth stages through this system and then identify them through image processing related algorithms.
The subject of this paper is apples, and the selected variety is Fuji apple. The image collection time is mainly determined according to the growth status and growth rules of the apple. As far as this research is concerned, it is necessary to consider accurately segmenting apples from images under different weather conditions and automatically identify the four key growth stages of apple flower bud differentiation, apple flowers, physiological fruit drop, and fruit expansion in the orchard. Therefore, we need to shoot continuously at specific times every day. The shooting object is apple (the growth cycle starts from the seedling to the end of the mature period). Apple image collection is mainly concentrated in two time periods. The first time period is mainly to photograph the apples in the orchard. Its function is to analyze the relationship between the green information of the apple and the light under different weather conditions and to lay the foundation for segmentation; the latter time period is mainly to obtain a single apple, and its role is to identify later created training samples. The shooting environment changes include dark light to strong light, weather conditions (sunny, rainy, and cloudy), shooting time (morning, noon, and afternoon), and dry to humid soil. It is also necessary to adjust the height and angle of the camera in time to find a reasonable configuration that can obtain high-quality image data.
3.2. Image Segmentation
The accurate segmentation and extraction of apples in the image constitute a prerequisite for realizing apple growth monitoring. However, the completion of this work is affected by many factors, such as changes in light conditions, changes in surface soil, and complex environments. In order to achieve accurate segmentation of apple images in complex environments, this paper studies an HI color segmentation algorithm based on Gaussian distribution model based on prior knowledge of apples in different weather conditions and different growth and development stages. The algorithm only needs a small number of training samples to complete the segmentation of the apple image.
3.2.1. Existing Crop Segmentation Algorithms
In the algorithm of crop image segmentation, the role of color index is very important. The general color index algorithms mainly include ExG (Excess Green—over-green); ExR (Excess Red—over-red); ExG–ExR (Excess Green minus Excess Red—over-green minus over-red); CIVE (Color Index of Vegetation Extraction—a vegetation color index extraction); and VEG (Vegetation). These algorithms only need to use the three components of red, green, and blue in the RGB color space. The calculation is simple, and it is not sensitive to changes in light intensity; even to complex environments, change has a certain degree of adaptability. However, the prerequisite for the realization of the algorithm is to normalize the color space and finally achieve crop extraction by setting the segmentation threshold.
The method of normalizing the color space is shown in (1), and the values of , , and need to be calculated firstly.
On the basis of , , and , , , and are, respectively, normalized into the interval according to (2), and is assumed.
The calculation of the color index is based on the calculation of , , and values in (2).(i)(ii)(iii)(iv)(v)
3.2.2. HI Color Segmentation Based on Gaussian Distribution Model
The HSI color model expresses the way the human visual system perceives the color of an object, where hue (H) represents the difference in color, saturation (S) represents the purity of the color, and intensity (I) represents the brightness of the color. These characteristic components are used to describe the color. The use of the HSI color model in crop segmentation has the following advantages: compared with the RGB color model, its three color feature components can be directly used for segmentation separately; at the same time, it overcomes the influence of lighting conditions [25].
The color distribution of a single-color object changes with intensity on the hue-saturation plane, where a single color is different in each period of apple growth, and it is green in the growth period of the seedling and the physiological fruit drop period; in the flower bud, it is pink at the differentiation stage, yellow at the flowering stage of the apple, and red at the fruit expansion and maturity stage. Therefore, when building a single-color pixel model in the HSI color model, it is necessary to adapt to changes in external lighting conditions. Therefore, a single Gaussian distribution model can be used to represent the distribution of tone values at a certain intensity. The probability density function of Gaussian distribution is shown in
Here, is the hue value of a single color, is the expected value, and is the variance. The solution of the parameter values and under a certain intensity can be estimated by using the maximum likelihood estimation method on the training sample data set. The specific calculation method is shown in
Here, represents the th hue value of sample pixels under a specific intensity, and refers to the total number of single-color pixel samples.
The HI segmentation algorithm used in this paper is based on the HSI color model, and the type of the apple image collected in the experiment is the RGB color model. Therefore, the image needs to be converted from the RGB color model to the HSI color model when building the tone intensity table and before image segmentation. The basic principle of the conversion from RGB color model to HSI color model is as follows: First, the brightness factor needs to be separated from the RGB color model; the chroma is divided into two parts, hue and saturation; and the angle vector is expressed as hue. On the whole, it is a conversion from a unit cube based on Cartesian coordinates to a double cone based on cylindrical polar coordinates [26]. The conversion formula from RGB color model to HSI color model is defined as follows:(i)(ii)(iii)
The values of , , and in (12) all need to be normalized to the interval before calculation.
Set a pixel , where represent the coordinates of the pixel in the image. According to the HI color segmentation model of Gaussian distribution, represents the deviation (distance) between the hue value of the pixel and its expected value, as follows:
Here, and are, respectively, derived from the expected value and variance of the hue intensity look-up table. If the calculated distance is larger, the possibility that the pixel belongs to a green crop is less. In order to accurately segment the crop image, this paper sets a threshold to judge whether a pixel is a single-color crop; if , ; otherwise, it belongs to the background. It is not difficult to find that the setting of the threshold will directly affect the accuracy of crop image segmentation, and choosing different thresholds will produce different segmentation results. If the threshold is too small, some of the single-color pixels will be mistakenly divided into other-color pixels. If the threshold is too large, some other-color pixels will be mistakenly classified as single-color pixels. Therefore, it is very important to choose a suitable threshold . Through testing, this paper finally determines the best value suitable for image segmentation at each stage of apple growth (the experimental part will be discussed in detail).
4. Recognition and Positioning of Different Periods in the Growth Process of Apples
The identification of each period in the apple growth process is carried out on the basis of accurately segmenting the apples. After image processing, the influence of other factors on the identification effect is reduced. Researchers at home and abroad mainly use information such as shape parameters, color, and texture characteristics to identify single leaves of plants; to identify flower bud differentiation, flowering period, and fruit period, they count on the basis of identifying single leaves and then determine which period the plant is in. In this study, ASM method was used to identify the growth cycle of apples, taking into account the overall geometric shape of the apples in each period and reducing the influence of different leaves on the identification results.
4.1. ASM
ASM is a statistical model method applied to complex shape matching proposed by scholars [27–29]. This research discusses it from three aspects, namely, the point distribution model, the local gray-scale texture model, and the matching target search process. Its basic process is to first select a series of training samples, then use manual calibration to mark the contour (feature points) of the object shape, and then perform alignment operations on these training samples. On this basis, the shape model is established, and the constraint condition of the shape parameter change is obtained. Finally, search for the target that matches the shape model in the tested image, so as to complete the recognition of the target object. The ASM algorithm has the characteristics of fast running speed and high target positioning accuracy, and it is also suitable for recognition under different illumination, complex background, and occlusion. Besides, it can statistically model complex shapes of targets to complete the recognition work. In the matching search process, the shape model is adjusted by parameters to constrain the overall shape of the model.
4.2. Point Distribution Model
We use to denote a specific training set. The training set contains shapes, and each shape has marked points. It can be expressed in another way; that is, there are coordinate points for each shape’s mark point. The training set is represented by : the calibration coordinate point of the th shape and the calibration point of the th shape are represented by a vector in the training set [30].
When manually calibrating the target contour feature points, it is necessary to ensure that the calibration sequence of the feature points in each image is consistent with the calibration position. Otherwise, the alignment of the training samples in the next step will be affected, and the modeling will be unreasonable. There are different shooting angles and different shooting distances during the data acquisition of a certain key growth period of apples. Therefore, the positions and scales of the apples in the images are different, which causes the distribution of training samples in the two-dimensional space to be disordered, resulting in failures. This shows the changing law of the shape of an apple during a key growth period, so it is unreasonable to directly count and process the training samples. Therefore, it is necessary to perform standardized alignment processing on the manually labeled training samples. The main operation steps are as follows: First select a certain sample shape as the basic shape, and other samples are as close as possible to the basic shape through the steps of scale enlargement and reduction, rotation, and translation.
If we only align two samples, their corresponding shape vectors are and , respectively. Let be the basic shape model, then enlarge and reduce by a certain scale , and rotate the angle and translate , so that the shape of is as close as possible to the shape of . Under the framework of least squares, the shape vector is standardized to by adjusting these parameters.
By the least square method, one obtains
Partially differentiate , , , ; set their value to 0; and then solve these equations to get these parameter values.
Therefore, it can be extended to the alignment of samples. First, select the shape vector of a sample as the basic shape vector, and then align other shape vectors to the basic shape vector according to the above calculation method. After all are aligned, find their average shape model and make it the new basic shape vector. Then, align other shape vectors to this new basic shape vector, and repeat this operation until the convergence condition is met.
The average model in the shape model is defined as follows:
The final shape model in the training set can be expressed by the sum of the changes of the average shape model and the weight model.
Here, is the average shape model, is the transformation matrix composed of component eigenvectors, and is the shape weight vector, which is calculated by the following:
The change of the shape weight will change the corresponding shape model, so the shape model of ASM is generated. Searching and matching similar targets in the image can make the shape model approximate the shape of the target image by adjusting the weight vector of different shapes, and the constraint changes within a certain range. The limitation on is to avoid the change of the shape model during the matching process that does not conform to the actual situation. The value of is calculated by the probability distribution model of the parameters.
Here, is eigenvalue, .
4.3. Local Gray Texture Model
The establishment of the local gray-scale texture model firstly needs to analyze the gray-scale information [31]. The calculation process for the gray information of the model feature point in the image is to take this point as the center and take sampling points along the two sides of the model contour normal. Therefore, there are pixel points in total. The gray information is recorded as . In order to reduce the influence of the model in the process of translation and zooming, the gradient value of is calculated:
At the same time, in order to reduce the influence of different lighting conditions, we have done a normalization process on the basis of the gradient value :
Then, calculate the average gray information model corresponding to all the feature points of the model.
Calculate the covariance matrix of the feature point .
According to the calculation of the above formula, we can get the local gray information characteristics of each model feature point: the average gray information model and the covariance matrix , including model feature points.
Mahalanobis distance can describe the degree of similarity between a local feature point and the calibration feature point (similarity measure): the smaller the distance value, the greater the similarity; the greater the distance value, the smaller the similarity. Therefore, in the target image search process, by calculating the corresponding Mahalanobis distance , the feature point with the greatest degree of similarity is obtained, which is the best matching point of the model calibration feature point.
4.4. Matching Target Search Process
4.4.1. ASM Algorithm Single Resolution Search Strategy
When searching for a target in an image, the first step is to use artificially marked feature points to obtain the initial shape model, then use the local gray-scale texture model to select the best matching point, and combine the limitations and changes of shape parameters to obtain a matching result map. The specific process is as follows:(1)The model is initialized and positioned. In the target image, whether there are apple seedlings, flower buds, and fruits at various stages of the growth process, generally determine the approximate position first; on this basis, match the seedling, flower bud, and fruit stage shapes of the initial shape model to obtain the approximate position of the target image; the initial shape is .(2)Discover new model shapes. Search in the vicinity of the feature point according to the normal direction of the feature, and calculate the minimum Mahalanobis distance between the current feature point and the candidate feature point in the current contour neighborhood; that is, the best matching point of the current marked feature point is the minimum Mahalanobis distance ; then, the best matching point corresponding to each feature point of the current shape is calculated in turn, and the new shape vectors are established.(3)The shape parameters are updated. The displacement required by the new model to the characteristic target point is obtained by affine transformation.(4)Shape parameter constraints. From the probability distribution model, the variation range of the shape parameter is , and finally the result is matched with the target contour in the image.(5)Repeat the steps from (2) to (4) until the new shape and the original shape no longer change significantly; we consider that the algorithm converges and the iteration ends.
4.4.2. ASM Algorithm Multiresolution Search Strategy
When the ASM searches for the best matching point, it is often affected by internal and external factors, such as the initial target location and the noise of the image itself. In turn, deviations occur when each feature point is matched with the best candidate point corresponding to it, and the search performance is changed. The result of the search becomes worse as the number of deviation points increases or the deviation increases, which in turn affects the matching effect of the ASM algorithm. The traditional ASM algorithm faces the following contradictions: In the process of feature point matching, from one perspective, the expected search range is large enough to ensure that the best matching position of the feature point can be searched. However, if the search interval is too large, the number of nonoptimal feature points similar to ideal feature matching points in the search interval will increase, the computational efficiency of the algorithm will be greatly reduced, and the possibility of errors will continue to increase. From another perspective, in general, the smaller search interval is expected to reduce the amount of calculation and finally improve the efficiency. In response to the abovementioned contradictions, Cootes and other research scholars proposed a new ASM search method, namely, multiresolution search strategy. The basic idea is “from low to high” multilevel search: The first step is to search for the approximate position of the target in the lowest resolution image. Then, in the image environment with the resolution gradually increasing, the precise position of the target is continuously improved until the original image that is the highest resolution image is found. This method can speed up the running speed and reduce the probability that the local structure of the image interferes with the shape model.
In the process of multiresolution search strategy, the first step is to build a pyramid of Gaussian images. The images in each layer are the same, but the difference between them is different resolutions. The lowest layer which is the original image is image layer 0, and the upper layer of lower resolution image is image layer 1. The method of acquisition is to process the lowest layer image with a 5 5 image mask and take a sample of two pixels; that is, the length and width scale of the image obtained in each layer is half of the previous layer; then, get the pyramid of the n-layer image. In this way, a pyramid of Gaussian images at different resolutions is established.
5. Experiment Design and Result Analysis
5.1. Image Segmentation Experiment Results: Value Selection
In order to determine the appropriate threshold in the HI color segmentation of the Gaussian distribution model to accurately segment the apples in the growth process of each period, this paper selected 45 different weather conditions (sunny, cloudy, and rainy), different environments, and different growth stages. The apple image of the period was used as a sample for testing. At the same time, the relationship between the selection of value and the segmentation result is discussed. As shown in Figure 1 (the periods are marked with orange numbers), the six images are apple in the seedling stage, flower bud differentiation stage, apple flowering stage, physiological fruit drop stage, fruit expansion stage, and mature stage. The results show that the setting of value has a great influence on the results of apple image segmentation. When = 1, the lack of apples is serious, and some pixels belonging to apples are wrongly classified as background, as shown in Figure 2. When = 4, most of the pixels belonging to the background are wrongly classified as apple flower buds, apple flowers, or their fruits, as shown in Figure 3. After a lot of experimental verification, it is found that the threshold segmentation effect is the best within [2.4, 2.6], which meets the goal of correctly extracting apples. Figure 4 shows that when = 2.5, the effect of image segmentation is much better than when the values are 1 and 4, respectively. The value at this time shows that the HI color segmentation algorithm of the Gaussian distribution model extracts the apples in the growth process of each period. It adapts to changes in different environmental conditions and has good robustness. Therefore, this paper sets the threshold to 2.5 as the parameter of the HI color segmentation method of the Gaussian distribution model to segment the apple image during each period of growth.
In order to further prove the excellent performance of the HI color segmentation algorithm of the Gaussian distribution model in extracting apples in the growth process of each period, this paper again selects typical data from the obtained apple image data: different weather conditions (sunny, cloudy, and rainy), different growth stages, and different levels of complexity are used as test samples.
In this study, the experimental results of the four algorithms of CIVE, ExG–ExR, ExG, and VEG are compared with the experimental results of the method in this paper. The threshold of the HI segmentation algorithm of the Gaussian distribution model is set to = 2.5. It can be seen from the experimental results that the HI algorithm has the best segmentation effect and can adapt to changes in lighting and complex environments. Compared with the three algorithms of CIVE, ExG–ExR, and VEG, the ExG algorithm has better adaptability and can adapt to general light intensity changes, but the image segmentation effect is inferior to the HI algorithm. The VEG algorithm has the worst segmentation effect and is most susceptible to the influence of different light intensity; the noisy point is serious. The ExG–ExR algorithm has a high error rate. The CIVE algorithm can handle the segmentation of green crops and soil better, but when there are interference factors (such as shadows) in the image, the performance is not good. In summary, the HI algorithm selected in this article has the best effect.
5.2. The Positioning Results of Each Period in the Apple Growth Process
The ASM algorithm is used to accurately describe the overall two-dimensional shape of the apple during the growth process, and the multiresolution search strategy is used to determine the position of the apple during the growth process to identify it. The training samples are shown in Figure 1. In order to overcome the shortcomings of traditional methods, this paper directly describes the overall outline of each stage of apple growth. The specific operation is to mark the contours in each image. As shown in Figure 5, the green points are the marked contour points.
In order to improve the accuracy and efficiency of recognition at each period of the apple growth process, we should pay attention to the following issues. The outline needs to be marked in order, and the calibration points should be distributed as far as possible on the edge of the contour. Use these ordered calibration points to describe the contour, and then you can convert them from the image to the two-dimensional space and record it to describe the overall contour.
Although the contour points for calibration at each period of apple growth process are marked in order, there are still certain errors. The description of the overall geometric shape will still be affected by three aspects:(1)When obtaining the contour calibration points at each period of apple growth process, the original calibration points are in their respective coordinate systems. Therefore, the apples of the same stage and different periods have different coordinate systems, and there will be differences when they are unified to the same coordinate system.(2)When the data is collected, the distance between the camera and the apple is different, which causes the scales of apples in different stages and different periods to be inconsistent in the image, so they cannot be directly compared.(3)The apple and the imaging plane are not completely parallel, which causes distortion in the image of the geometry of the apple at each stage of the growth process. In order to overcome the influence of the abovementioned unfavorable factors, it is necessary to align the sequence points of the apple contour calibration at each stage in different images. The main purpose of alignment is to be able to compare the same marked feature points of training samples from different coordinate systems and build a statistical shape model on this basis to objectively describe the trend and law of shape changes.
After the calibration points of the training samples are aligned, the average sequence calibration points of the training samples are obtained according to (17). The final shape model is denoted by . The initial model is used to search for the matching target, and the best matching effect is achieved by limiting the shape weight parameter .
In this paper, we select 100 apple trees in the orchard and segment the HI image based on the Gaussian distribution model. The optimal value for image segmentation in each period of apple growth is 2.5, and the weight parameter = 0.03 in the shape model.
The success rate of apple identification and positioning at each period of the growth process is shown in Figure 6.
6. Conclusions
Based on pattern recognition, this paper studies an HI color segmentation algorithm based on Gaussian distribution, which is suitable for apple image segmentation at each period of the growth process in the complex environment of the orchard. It is based on prior knowledge to segment the apple image. First, the apple image with the background removed under different weather conditions is obtained as the training sample, the HI look-up table is established, and then each pixel in the image is judged on whether it belongs to an apple. Morphological processing and contour tracking are performed on the binary image of the segmented apple image, and the appropriate threshold is determined in the HI color segmentation of the Gaussian distribution model to accurately segment the apples in the growth process of each period. On the basis of correct segmentation, the method based on pattern recognition, ASM, is used to identify the apples in the growth process of each period; the overall outline is calibrated to describe their shape; and then the training samples are counted and calculated. Finally, the initial shape model is obtained. This paper shows, through a series of experimental results, that our algorithms are effective and feasible; they can meet actual production needs, provide decision support for real-time management of orchards, and finally lay the foundation for the application of smart agriculture. Although this paper has achieved some satisfactory results, it also has shortcomings. Using the Gaussian distribution HI color segmentation algorithm to segment the apple image depends on the selection of the value. If the value is selected incorrectly, it will have a great impact on the segmentation result. Therefore, a clustering method should be added on this basis to achieve a better segmentation effect; in addition, the initial positioning of the ASM algorithm should be improved to make the search matching effect better.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (no. 31860331), YEFICRC Project of Yunnan Provincial Key Programs (no. 2019ZG009), and Yunnan Agricultural Joint Youth Fund (no. 2018FG001-091).