Abstract
The detection of image edges plays an important role for image processing. In view of the fact that these existing methods cannot effectively detect the edge of the image when facing the image with rich details. This paper proposes a novel method of asymmetric spike-timing-dependent plasticity (STDP) image edge detection based on the visual physiological mechanism. In the proposed method, the original image is preprocessed by the Gabor filter to simulate the visual physiological orientation characteristics to obtain the image information in different directions, and the orientation feature fusion is used to reconstruct the primary edge feature information of the image. Then, based on the mechanism of the visual nervous system, a neuron network composed of dynamic synapses based on the asymmetric STDP mechanism is constructed to further process it to obtain impulse response images. In order to eliminate disturbance of the neuron’s system noise on the impulse response image, the impulse response image is filtered by a Gaussian filter. Then, the lateral inhibition between neurons is applied to refine the filtered image edges. Finally, the result is normalized, and the final edge of the experimental image is obtained. Experimental results based on the colony image data set collected in the laboratory indicate that the proposed method achieved better performance than these state-of-the-art methods; meanwhile, the AUC value remains above 0.6.
1. Introduction
At present, for the low-level visual feature processing of images based on bionic vision, many related theories have been proposed and good experimental results have been obtained [1–3]. The edges and contours are the dominant features to describe an image; hence, these two features are usually employed for higher-level image processing. Detection of edges and contours is a hot topic in the field of image processing. It is widely used in computer vision fields such as image classification, target detection, and image segmentation [4–7]. How to effectively and accurately detect the edges and contours of the image is of great significance to the subsequent image analysis, recognition, and understanding.
A number of methods have been proposed for image edge detection. This mainly includes the following: (1) the method based on the space domain, which is mainly based on spatial calculations, uses a relatively primitive differential operator, and judges the position of the edge based on the extreme value of the first derivative of edge gray value and the zero-crossing point of the second derivative. The template of the edge detection operator is convolved with the input image to directly perform edge detection on the acquired image. The Roberts operator [8] achieved high edge positioning accuracy, and the detected edges are relatively delicate, but it is sensitive to noise and of poor robustness. It is easy to cause local edge loss and cause the edge contour of the detected object to be discontinuous. Although the Sobel operator [9] and Prewitt operator [10] can suppress noise, the detection boundary line is wider. Compared with the above-mentioned method, the Canny operator [11] is relatively insensitive to noise but is susceptible to the influence of gradient amplitude and double thresholds and detects false edges and edge discontinuities. The Laplace operator [12] is of isotropy, linearity, and displacement invariance, but it needs to perform two-level difference processing while obtaining the edge, which produces a double-pixel edge and doubles the noise and affects the detection accuracy. The Log operator [13] is implemented on the basis of the Laplace operator. First, the Gaussian function is used to low-pass filter the noise existing in the original image, and then, the Laplace operator is used for edge detection. Compared with the Laplace operator, although the noise in the image is suppressed, it weakens some low-intensity edges and causes a discontinuity in edge detection. (2) Another is a method based on the transform domain, which transforms the image to the corresponding transform domain through various image transformations, obtains the coefficient matrix, and performs a certain correction on the coefficient matrix to obtain the result. For example, wavelet transform [14] uses the transformed high-frequency components to eliminate the sudden change information and noise in the image. However, wavelet is not optimal in terms of the sparsity of the representation function, and the scale of wavelet transform is difficult to be unified, which will cause the contradiction between edge positioning accuracy and noise. Mathematical morphology [15] is a method that uses nonlinear filtering. By introducing the basic features and structure of the image, the problem of image processing such as noise suppression, feature extraction, and edge detection is solved; meanwhile, it balances off the detection accuracy and antinoise performance. However, there are shortcomings such as the problem of a single structural element and poor performance of edge detection in the context of rich details.
The increasements of the complexity and diversity of images require more effective edge detection methods. With the advancement of the physiological experiments of the visual mechanism in recent years, a large number of results have been obtained, which enables people to have a certain understanding of the cognitive process of vision. Given the near-perfect ability of the human visual system in processing complex image tasks, it can eliminate noise well and has extremely strong fault tolerance, which is unmatched by any existing image processing technology. Therefore, the human visual system currently provides inspiration and guidance while proposing novel models for image processing [16, 17]; e.g., image edge detection based on PCNN, which simulates the distribution and transmission of neuron pulse information flow, gives full play to the nonlinear modeling ability of neuron network in edge detection [18]. As well as the Gabor filter that simulates the direction selectivity of the visual nervous system, it has also been better applied in edge detection [19]. The paper [20] studied the experimental and theoretical methods for searching for effective local training rules for unsupervised pattern recognition through high-performance memristor spike neural networks. The paper [21] proposed a temporal preprocessing model of video frames using a biologically inspired vision model, and the bioinspired model consists of multiple layers of processing analogous to the photoreceptor cells in the visual system of small insects. There are also some deep learning-based methods for edge detection. For example, the paper [22] uses a spherical camera and two personal computers to build a remote apple growth monitoring hardware system and obtain apple images regularly. A fusion convolutional feature (FCF) edge detection network is designed to segment apple images for remote estimation of the apple size throughout the growth period. The paper [23] proposes an edge detection model with improved performance based on the convolutional neural networks and Laplacian filters, and the proposed method successfully detected the fuzzy defects on the noisy X-ray image. However, these methods lack an in-depth study of the related physiological mechanisms; moreover, the experimental objects are also homogeneous.
In order to address these issues in these existing methods, this paper studies the application of image edge detection based on neuron pulse emission coding under the asymmetric STDP mechanism of the visual pathway from the perspective of biological vision. Consider that the synapse is the key physiological structure for the effective transmission of impulse information between biological neurons [24]. It will be subject to changes in the intensity of the stimulation signal inside and outside of the biological organism, constantly self-adjust and change, and reshape the connection strength between neurons to meet the needs of biological nervous system information processing and action guidance. The asymmetric STDP information processing mechanism of the visual pathway of excitatory and inhibitory synapses is studied. This is in view of the fact that the neurotransmitters found in physiological anatomy experiments can be divided into two types: excitatory neurotransmitters and inhibitory neurotransmitters [25]. These different types of neurotransmitters also play a very important role in the process of biological visual information processing and play a decisive role in regulating the synaptic connections between neurons. Therefore, the time windows of long-term potentiation (LTP) and long-term inhibition (LTD) based on dynamic synaptic plasticity are asymmetric. This paper proposes that the Izhikevich neurons are used as the basic nodes of the network, and the information flow between neurons is transmitted through the physiological structure of dynamic synapses, and the information flow pulses are coded in time series. At the same time, the lateral inhibition mechanism of information transmission between neurons based on physiological experiments can be used to improve the contrast between the edge pixels of the image and the background information of the image. This can make the edges of the image richer and provide better basic feature information of the main content of the image for subsequent higher-level image-related tasks.
The rest of the present study is organized as follows. In Section 2, the experimental materials and methods of this article are introduced. In Section 3, experimental results are discussed and analyzed. The conclusion is drawn in Section 4.
2. Materials and Methods
In this paper, by simulating the visual processing mechanism, a dynamic synaptic neuron network based on the asymmetric STDP mechanism is constructed to realize the effective detection of image edges. According to physiological experiments, neurons in the visual cortex have direction selectivity for input stimuli. In view of the fact that the frequency and direction of the Gabor filter are similar to the human visual system, the original image is preprocessed by using the Gabor filter to simulate the human visual mechanism to obtain image features in different directions. After the feature fusion is carried out, it will be transmitted to a neuron network composed of dynamic synapses based on the asymmetric STDP mechanism. By recording the first pulse firing time of the neuron, the information flow processing scheme based on time sequence coding is given. As the neurons in the visual pathway are disturbed by a lot of noise, Gaussian filtering is performed on the pulse information stream based on timing coding. In addition, considering the physiological mechanism of lateral inhibition between neurons, this paper simulates the mechanism of lateral inhibition, further processes the image after Gaussian filtering, and finally obtains the image after neuron lateral inhibition. Finally, the normalization process is performed to obtain the final edge of the image. The specific process is shown in Figure 1.
According to related physiological research, this paper constructs a dynamic synaptic network with the Izhikevich neurons as the basic unit and uses time series coding for the information flow pulse [26]. At the same time, the important physiological significance of excitatory synapses/inhibitory synapses in the process of visual information processing and processing is considered, as well as visual physiological mechanisms such as asymmetric STDP mechanism and lateral inhibition (the specific structure is shown in Figure 2). Neurons promote or inhibit each other through the formation of excitatory synaptic transmitter AMPA/inhibitory synaptic transmitter GABA and dynamically adjust the weight of synaptic connections between neurons. And based on the asymmetric STDP mechanism, it realizes the effective transmission and processing of various sensory information [27]. In this article, the processing of visual information is mainly considered.
The neuron model is an important foundation of the neuron network. Taking into account the computational efficiency, complexity, and mathematical analysis performance of the existing neuron model. In this paper, the Izhikevich neuron model is used to construct a pulsed neuron network, and its mathematical model is shown in where are model parameters. is the time variable. is the membrane potential of neuron . is the recovery variable of neuron . is Gaussian white noise with intensity . is the threshold value of the membrane voltage. is the strength of the synaptic connection from the -th neuron to the -th neuron. The external input is , where where is a binary factor, which indicates whether the -th neuron has input, and is the strength of the external input signal.
If the system parameters are inconsistent, the Izhikevich neuron model will show different firing patterns. In this paper, we take , , , and , and the intensity of white noise is .
2.1. Asymmetric STDP Mechanism
Synapse is the key structure for information transmission between neurons, and it is constantly changing and remodeling its connection strength to meet the needs of all aspects of the body due to changes in the body and outside of the body [28]. A complete synapse is composed of the presynaptic membrane, postsynaptic membrane, and synaptic cleft in between neurons. The presynaptic membrane has vesicles that store neurotransmitters, and there are receptors for the corresponding neurotransmitter on the postsynaptic membrane. After the presynaptic membrane is electrically or chemically stimulated, the vesicles release neurotransmitters to the synaptic cleft and bind to the corresponding receptors on the postsynaptic membrane to generate various electrical activities (local potentials). Then, it spreads to the corresponding neural circuits in a short period of time, producing different neurobehavioral activities [29].
The two main types of neurotransmitters in the brain are excitatory transmitters and inhibitory transmitters [30]. These transmitters also play an important role in the process of visual information processing. AMPA is the vast majority of excitatory synaptic transmitters in the brain. Synaptic plasticity, that is, the dynamic changes of neuron synaptic performance, is considered to be the basis of information encoding and storage in learning and memory. One of the most important mechanisms is that the regulation of synaptic strength is closely related to the regulation of AMPA receptor transport in the synapse [31]. GABA is the most widely distributed inhibitory neurotransmitter in the central nervous system. The specific mechanism of GABA is that GABA released from the presynaptic membrane binds to the GABA receptors of the postsynaptic membrane to form a receptor complex and undergo configuration changes, activate ion channels, allow ions to pass selectively, and cause neuronal hyperpolarized. It inhibits the excessive discharge of excitatory neurons and finally plays a role in hindering the transmission of nerve signals [32]. The existence of excitatory synapse/inhibitory synapse is an important part of information transmission in the nervous system, and it is also of great significance to synaptic plasticity.
Studies have found that the time sequence of presynaptic spikes and postsynaptic spikes affects the strength of the connections between presynaptic and presynaptic neurons. Hebb first interpreted this phenomenon from a mathematical point of view and proposed the Hebb learning rule. The principle is to increase or decrease the connection weight of the synapse according to the correlation of the firing of the neurons before and after the synapse.
According to the experimental results [33], the relationship between the time difference between the two neurons to produce nerve impulses on the excitatory post-synaptic current (EPSC). Aiming at the time asymmetry of synaptic plasticity changes, Bi and Poo further proposed the “spike-timing-dependent plasticity” mechanism. According to the length of the time course, it can be divided into short-term plasticity and long-term plasticity (mainly including LTP and LTD). They can unsupervised and autonomously adjust the synaptic weights of neural networks, more accurately describe the changes in the weight connections of neurons in biology, and amend the Hebb learning rule.
The LTP/LTD change time window of asymmetric STDP synaptic plasticity is asymmetric, and its essence is based on the interval of neuron firing time, reflecting the causal relationship between neurons in the form of directional connections. The typical asymmetric STDP learning mechanism is shown in Figure 3. When the presynaptic neuron pulse firing time is before the postsynaptic neuron pulse firing time , that is, , the strength of the synaptic connection between them will increase; on the contrary, for , the strength of synaptic connections is weakened. Its function expression is shown in where represents the connection strength between neuron and neuron . is the difference between the time when the presynaptic cell produces spike and the time when the postsynaptic neuron produces spike, that is, . is the STDP adjustment function. The parameters and affect the adjustment range. The larger their value is, the larger the synaptic connection strength increases or decreases within one step. and are the delay constants of STDP adjustment parameters, and , , , and are used in this article.
2.2. Lateral Inhibition
Hartline discovered the phenomenon of lateral inhibition for the first time when conducting monocular electrophysiological experiments on Limulus. According to further in-depth experiments on visual physiology, it is found that visual lateral inhibition is carried out in the analog signal part, which has an important manifestation in horizontal cells. When horizontal cells receive a signal from a light information pathway, they are affected by glutamate released by photoreceptor cells and release GABA, which inhibits the release of glutamate from receptor cells in other light information pathways, thereby weakening the response of adjacent pathways.
According to this physiological phenomenon, this article introduces the lateral inhibition between neurons in the cerebral cortex when constructing the interconnection of neuronal networks. When a neuron is excited, it will inhibit the excitement of other neurons in the area, which can significantly enhance the recognition of the visual system during target recognition and enhance the recognition of image edge information. This article considers the sequence of neuron firing and adopts the neuron lateral inhibition method as shown in Figure 4. If the neuron at the center of the receptive field is excited earlier than other neurons, it will have an inhibitory effect on the corresponding neuron. If the peripheral neurons are excited first, the central neurons will be inhibited. In this way, not only the edge points of the image can be highlighted, but also different levels of information can be processed in a targeted manner, so as to express rich image edge information.
To reduce complexity and facilitate calculations, a receptive field window is constructed, and the lateral inhibition of adjacent neurons is shown in where , respectively, represent the central element before and after the update in the receptive field window and , respectively, represent the noncentral element before and after the update in the receptive field window.
3. Experimental Results and Discussion
In order to verify the effectiveness of the method in this paper, the image data sets such as Lena, which are commonly used in edge detection, and the colony image data sets collected by this research group in the laboratory are used as the experimental objects. Among them, the colony map is obtained by the laboratory using the imitating natural light suspension dark-field system, through the F/1.4 large aperture lens, and the colony after the Petri dish culture is obtained by imaging at the level of tens of millions of pixels. This article installs Matlab R2016b version on Ubuntu 20.04 LTS version for experiment. The main hardware includes AMD Ryzen5 5600H CPU and 16GB memory.
And compare the results of the method proposed in this article with traditional edge detection methods such as the Sobel, Roberts, Prewitt, Canny, Log, PCNN, and Gabor; the experimental results are shown in Figure 5. Among them, the first row is the original picture. The second row is the test result of the Sobel method. The third row is the test result of the Roberts method. The fourth row is the detection result of the Prewitt method. The fifth row is the result of the Canny method. The sixth row is the test result of the Log method. The seventh row is the PCNN method. The eighth row is the test result of the Gabor method. The ninth row is the test result of the method in this paper. From the experimental results in Figure 5, it can be seen that the Sobel and other methods can detect the more obvious edge information of the salient target when targeting simpler pictures such as Peppers but will lose most of the edge information of the background. In addition, for images with richer details such as Lena, although the overall edge information of the image can be outlined, most of the detailed edge information will also be lost, resulting in discontinuous edge detection.
In Figure 5, it can be intuitively found that although the detected edge information of Log is richer, it has certain shortcomings. It will cause excessive segmentation during detection, which is too sensitive to noise points, and the detected image edge information is too redundant. This affects the subsequent processing of the image and is not conducive to observation. From the experimental results, it can be found that the edge continuity detected by the algorithm in this paper is good, and a better single-pixel edge can be obtained after refinement. The edge detection accuracy is high, and the edge information can be highlighted, for example, for images such as Lena. In the case of preventing excessive segmentation, their detailed edge information can still be well characterized, and it has a better detection effect than other edge detection methods.
The 1st row is the original image. The 2nd row is the detection result of the Sobel method. The 3rd row is the result of the Roberts method. The 4th row is the detection result of the Prewitt method. The 5th row is the test result of the Canny method. The 6th row is the detection result of the Log method. The 7th row is the detection result of the PCNN method. The 8th row is the detection result of the Gabor method. The 9th row is the test result of the method in this paper. On the basis of qualitative analysis, in order to better quantitatively compare the experimental results of different methods, this article uses the ROC/AUC indicators commonly used in machine learning to evaluate the experimental effects of various edge detection methods. The ROC curve mainly includes two indicator parameters: false positive rate (FPR) and true positive rate (TPR). A series of target values can be obtained by changing the threshold, and the ROC curve can be drawn with TPR as the ordinate and FPR as the abscissa. AUC is the sum of the ROC curve and the accumulated area under the horizontal axis. The larger the area, the better the edge detection effect of the image, and vice versa, the poorer the edge detection effect. The specific calculation is shown in (6). In Figure 5, it can be intuitively found that although the detected edge information of Log is richer, it has certain shortcomings. It will cause excessive segmentation during detection, which is too sensitive to noise points, and the detected image edge information is too redundant. This affects the subsequent processing of the image and is not conducive to observation. From the experimental results, it can be found that the edge continuity detected by the algorithm in this paper is good, and a better single-pixel edge can be obtained after refinement. The edge detection accuracy is high, and the edge information can be highlighted, for example, for images such as Lena. In the case of preventing excessive segmentation, their detailed edge information can still be well characterized, and it has a better detection effect than other edge detection methods. where represents the pixel set that correctly classifies the positive example as the positive example under different thresholds. represents the pixel set that correctly classifies negative examples as negative examples under different thresholds. represents the set of pixels that misclassify negative examples as positive examples under different thresholds. represents a set of pixels that incorrectly classify a positive example as a negative example under different thresholds.
In the process of calculating AUC, it is necessary to obtain the ground truth of the image. Since manual hand-drawing is inefficient and subjective, an objective and automatic judgment method is needed to obtain the reference map. In this paper, edge images are obtained by taking multiple thresholds for different edge detection methods, and the number of edge points at the same position in the edge images is counted, so as to obtain candidate edge images. Then, the best candidate edge image is determined by combining ROC statistical indicators and the diagnosis line, and this image is used as the edge reference image. Then, according to the above-mentioned index calculation method, this paper, respectively, calculated the Sobel, Roberts, Prewitt, Canny, Log, and the ROC/AUC index value of this method relative to the edge reference image. The AUC values of this method in Lena, Cameraman, Mandril, and Peppers can reach 0.8863, 0.8049, 0.7519, and 0.7949, respectively, which are significantly higher than the comparative experimental method, indicating the effectiveness of the edge detection method in this paper. The specific results are shown in Table 1. It can be intuitively found from Figure 6 that the ROC curve of the method proposed in this paper is closer to the upper left corner of the coordinate axis, indicating that the method proposed in this paper is superior to the existing traditional methods in the effect of image edge detection.
At the same time, this article counts the processing speed of different algorithms in image detection. According to the experimental results, it can be found that the edge detection speed of the Sobel, Log, and Gabor images based on traditional mathematical methods is relatively faster, but their edge detection accuracy is lower. Compared with the PCNN algorithm, which is also based on biological inspiration, the method in this paper has a faster processing speed while maintaining a higher edge detection accuracy.
At the same time, in order to further illustrate the effectiveness of this method, in this paper, the experimental objects are further processed, and the experimental results are further analyzed by using information entropy as the evaluation index. The larger the entropy value, the more edge detail information contained in the research object. The calculation is shown in where represents the gray value of the image and represents the probability of the pixel with the gray value in the image appearing in the image.
The experimental results are shown in Figure 7. By comparing the experimental images with rich details such as the colony, it can be intuitively found that the edge detection of the colony in this article has more obvious contrast and the continuity of the colony is more perfect. At the same time, it also has a very good detection effect under a small detection target. In addition, through formula (7), the method in this paper and several comparative experimental methods are used to calculate the information entropy of the edge detection results. The specific results are shown in Table 2. According to the quantitative analysis, the information entropy value of the method in this paper has a relatively large advantage over the methods such as the Sobel, Log, PCNN, and Gabor, reaching 0.7309, 0.7763, and 0.9862, respectively.
Through the visualization in Figure 8, it is obvious that the method in this paper can achieve higher information entropy in general and has greater advantages. This also shows that the method in this paper can obtain richer image edge details.
In addition, in order to further verify the effectiveness and scalability of the method proposed in this paper, this paper applies it to the segmentation and extraction of fundus blood vessels while keeping the parameters unchanged. Through the experimental results, it can be found that for different types of fundus blood vessels, the method in this paper can better extract the main blood vessel segmentation results, which also provides a good foundation for subsequent blood vessel processing tasks. The specific experimental results are shown in Figure 9.
Through the above qualitative analysis of the experimental results of different experimental methods, it can be found that the method in this paper can ensure the completeness and coherence of edge detection as much as possible while preventing the oversegmentation of the image and at the same time highlight the edge details of the image. This is of great significance for edge detection applied on images with richer details such as colonies. It is also necessary to improve the performance of this method through further research.
In addition, through further quantitative analysis of the experimental results, the AUC value of this method on different experimental images is relatively better than the existing traditional methods, which shows that the accuracy of edge detection is higher. The statistical analysis of the information entropy of the experimental results also shows that the method in this paper can retain more edge detail information when performing image edge detection. According to the qualitative and quantitative analysis of the experimental results, it consistently shows that the method in this paper has greater advantages over the existing traditional methods.
4. Conclusion
Different from traditional image edge detection methods based on spatial and exchange domains, this paper introduces asymmetric STDP, excitatory synapses/inhibitory synapses, time coding, and lateral inhibition based on physiological experiments related to visual physiological mechanisms. Through the introduction of its principle and function, the corresponding calculation model is established, and a method based on asymmetric STDP image edge detection is proposed. At present, the method proposed in this paper is mainly used in the extraction of low-level visual features of the image and has a good effect in the edge detection of the colony image collected in the laboratory. The later application of this method in the field of image preprocessing has certain practical significance and at the same time provides some ideas for image processing methods based on vision mechanisms. How to further explain and simulate the characteristics of biological visual pathways will also be the focus of our next research work.
Data Availability
The image data sets used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the Ministry of Education Industry-University Cooperation Collaborative Education Project (202102019039), Zhejiang University City College Scientific Research Cultivation Fund Project (J-202223), ROIS NII Open Collaborative Research (22S0601), and JSPS KAKENHI (grant numbers 20H00592 and 21H03424).