Abstract
At present, the segmentation method for the heart is one of the most difficult problems and a hot topic in medical imaging segmentation and image analysis. In order to achieve high-precision recognition of heart and blood vessel image, a single target auxiliary blood vessel image recognition algorithm, based on Markov Random Field (MRF) and ash fusion residual network information, is proposed in this paper. The corresponding features are extracted by the residual network depth, and the heart and blood vessels are segmented with high precision through MRF gray information to assist blood vessel image recognition. Thus, an algorithm combining Markov Random Field and Grayscale Information can complete such auxiliary recognition for single target ship image. The paper will also discuss auxiliary recognition techniques for identifying images of the heart and blood vessels. Added to that, it presents current medical image algorithms with a particular emphasis on cardiac image algorithms. Through a series of analysis, better algorithms are proposed. Thus, the research contents of a single target present the vascular image recognition algorithm. Therefore, a blood vessel image recognition algorithm is proposed for the research content of a single target. A series of validation experiments are conducted on the DRIVE and star datasets. The experimental results show that the method proposed in this paper has high segmentation accuracy, can achieve high-precision segmentation, and can achieve high-precision single target blood vessel image recognition. By observing and analyzing two-dimensional cardiac images, such as diabetic arterioles, hemorrhages, hard exudates, and so on, the difficulty of diagnosing cardiovascular disease characteristics can be reduced, and the diagnosis and treatment can be facilitated.
1. Introduction
CardioVascular Disease (CVD), also known as circulatory system illness, has become the most common cause of death. The diagnosis of CVD is usually carried out in the late stage of symptoms, and the cost of late intervention is elevated. As for the therapeutic effect, it can greatly reduce this disease. Therefore, it is very important to quantitatively evaluate and diagnose cardiac function in the early stage. Therefore, it is very important for the early quantitative assessment of cardiovascular images and diagnosis of cardiac function. The experimental results show that the segmentation algorithm for cardiovascular images has high accuracy and can achieve high-precision segmentation, thereby providing early help for the treatment of cardiovascular diseases. The heart problem will be vital in clinical cardiology. Clinical parameters such as ventricular volume, stroke volume, ejection fraction, and myocardial quality can be derived; thus, a quantitative analysis of overall and local cardiac function is determined [1]. The calculation of these parameters is based on the accurate description for the cardiac parts as the left ventricle (LV) and the right ventricle (RV).
At present, the imaging techniques for clinical evaluation of cardiac function mainly include echocardiography, cardiac radionuclide imaging, and so on. The Cardiac Magnetic Resonance Imaging (CMRI) is generally collected according to the cardiac tomography standard and such collected images mainly include lots of information. A succession of 2D images taken from the mitral valve in an apical direction and vertically toward the typical four-chamber ventricular septum makes up the short axis image [2]. The CMRI can clearly show the circular LV and crescent RV as it is the standard image for detecting cardiac function. In medical imaging technology, magnetic resonance imaging technology has the characteristics of high temporal and spatial resolution with the absence of trauma and radiation. Added to that, this method can clearly display the structure and function of the heart [3] as it has become the initial tool for judging the structure and function of the heart in clinic.
With the aim for high-precision cardiac vascular image recognition, a single target auxiliary vascular image recognition algorithm, integrating residual network feature extraction and Markov Random Field (MRF) gray information, will be proposed in this paper. The corresponding features are extracted by depth residual network, and the auxiliary recognition of vascular image is realized by using the MRF gray information.
This paper will be divided into five sections. In Section 2, the effective methods of cardiovascular image recognition as well as analysis the feasibility of the MRF gray information feature extraction and the residual network in cardiovascular auxiliary recognition will be proposed. Section 3 presents the research methodology; thus, the vascular image auxiliary recognition algorithm for feature extraction and the MRF gray information will be introduced. The obtained results along with their analysis will be discussed in Section 4. Finally, Section 5 will conclude this work and propose some future ideas [4].
2. State of the Art
At present, the main methods of vessel segmentation in the literature include vessel tracking, matched filtering, and machine learning. Martin et al. adopted such method for particle filter, which selected some points from the optic disc boundary as initial seed points to further track blood vessels, but this method is easy to be tracked by blood vessel branches or intersections [5]. In the study by Wang et al., combining Bayesian theory with multiscale detection, multiscale tracking of blood vessels is realized [6]. Sato et al. proposed a novel technique for image filtering and segmentation in their paper; the obtained results for filtering were superimposed, and a double threshold blood vessel segmentation is realized [7]. Sato et al.first used the matched filter to estimate some parameters and they finally used the histogram with the aim of automatically selecting the threshold to complete the blood vessel segmentation [8]. Lei et al. proposed a feature-based automatic blood vessel segmentation method, which uses AdaBoost classifier. They also used the machine learning method to realize the segmentation of blood vessel network. They first extracted a 23-dimensional feature vector (including Hessian matrix, gray co-occurrence matrix, and so on) for each pixel in the color heart image, then trained the blood vessel classifier by using the random forest method, and finally obtained better results. The effect is increased by 8.5%. [9]. Garcia developed an algorithm based on the principle of the distribution; however, this method generates noise in the background area easily. As a result, it generates lots of false positive values and performs poorly in detecting the lesion area [10].
In the study by Mittal et al., they applied a filter to images at different scales to obtain standardized images and then they incorporated the images. As a result, the proposed method is suitable for the analysis of the lesion images [11]. Merinopoulos et al. proposed a method where spatial dependence and probabilistic statistics are used to roughly approximate the vessel structure. Their method decreases interference in lesion areas while also accurately approximating the vascular anatomy [12]. A contrast-sensitive segmentation method, proposed by Iida et al., consists of three steps: background normalization, second-order Gaussian filtering, and regional growth digitization. Their approach has improved sensitivity and exceled in vessel segmentation in low-contrast regions [13]. Paltrow et al. proposed a new algorithm to detect small blood vessels, in order to better segment the blood vessels; they kept the blood and blood vessel parts in part of the image. As a result, the proposed approach has greater advantages for the segmentation of some low-contrast regions than other unsupervised methods [14]. Mittal first preprocessed the images by vector flow map cutting and, finally, obtained good segmentation results [15]. A segmentation method based on the Hessian matrix and the threshold entropy is proposed by Nguyen et al. They used morphological feature-based spectral clustering techniques to enhance blood vessels, by eventually combining the results with binary images, based on entropy maximization of threshold segmentation [16].
Yuan et al. treated the segmentation task as a pixel classification problem. In this way, the grayscale of the image can be maintained at the moment each pixel is recorded by the feedforward network. Thus, the proposed method consists of constructing a 39-dimensional discriminant feature vector for each pixel, such as local features, Hessian matrix, and divergence of vector fields, and classifying each pixel using the Extreme Learning Machine (ELM) classifier. The method proposed by Acla et al. extracted binary images after preprocessing the green channel images, while extracting the green channel to reconstruct another binary image and then comparing the regions of the two binary images to get the main vessels [17]. The final segmentation results were obtained using the combination of the main vessels and the classified obtained vessels [18]. Considering that some existing methods are not good enough to identify vascular structure and fine vessels, Shi proposed a new approach, based on trainable conditional random fields [19].
To sum up, the research for efficient automatic vascular segmentation mode is of great significance. There are many automatic segmentation methods of fundus vessels. These methods can extrapolate blood vessels without experts and can deliver reliable and efficient segmentation results.
In 2017, Stacom proposed a data set, called Automated Cardiac Diagnosis Challenge (ACDC), that was used later in the same year to organize the 2017 MICCAI international challenge. The data sets have been confirmed by experts and the left ventricles images have enclosed epicardial contours of myocardium (myo). ACDC regroups patients into five categories according to cardiac physiological parameters, with clear characteristics. The dataset is composed of 150 exams (all from different patients) divided into 5 evenly distributed subgroups (4 pathological plus 1 healthy subject groups). The challenge results show that the segmentation method will be applied for 120 images. Although the effect of the top-level deep learning segmentation method seems to be within the range of human expectations in the segmentation results on LV, it is difficult to achieve this accuracy in the segmentation of RV and myocardiogram. Compared to the image segmentation of pathological cases, the image segmentation of healthy subjects is more difficult. Segmentation, at the end, is a difficult problem for medical experts and requires deep learning methods. It should be noted that the use of a larger database may help solve these issues.
In order to solve the problem of cardiovascular segmentation, this paper proposes a single target auxiliary blood vessel image recognition algorithm, integrating residual network feature extraction and MRF gray information, and applies it to the auxiliary image recognition of fundus blood vessels. The whole algorithm flowchart is shown in Figure 1. Firstly, the depth residual neural network is used for feature extraction, and then the gray information obtained by MRF is used to assist vascular memory recognition. The combination of both features has achieved good results.
3. Methodology
3.1. Segmentation Algorithm Based on MRF Grayscale Information
The MRF is a probabilistic undirected graph model for ordered data annotation and partitioning, proposed by Cahn et al. [20]. Its foundation is mainly based on maximum entropy theory and hidden Markov model, and it has been widely used in natural language processing, bioinformatics, and image segmentation due to its powerful expression ability.
The nature of Markov random fields is a representation of the relations between the variables. As shown in Figure 2, each node in the graph represents a random variable, and the relationship between different nodes is expressed by the probability of another random variable. The Markov property consists of linking the nodes together and calculating the distance between them in such a way that the current node is mainly determined by the nodes around it, and the spatial distant nodes have little influence.
Each node in the Markov Random Field corresponds to an observation, and the main objective is to solve the corresponding Markov random fields when these observables are known. Thus, the mathematical definition form is as follows:where and represent the junction in the graph, indicates the connection between the junction and the edges, and represents the junction that is not connected to the junction .
In image segmentation, images need to be divided into multiple different regions according to different targets that are being searched. Pattern for finding the essential characteristics of components from the original data is one of the most important objectives. The traditional way, based on feature extraction and classifier classification, requires the artificial design of features according to the characteristics of the task. As shown in Figure 3, the neural network is composed of input layer, hidden layer, and output layer, which can complete end-to-end recognition tasks. The specific approach is to determine the category to which each pixel belongs when certain features of the image are known, mainly the color and the location relationship. Then, two pixels with similar color values have a higher probability of dividing into the same category, while pixels with large difference in color values and different positions have less probability of belonging to the same category. The observed and marked values of the image meet the conditional probability distribution represented as follows:
Then, the image segmentation process can be regarded as the process of solving its marker field under the known conditional random field. According to Bayes’s law, the conditional probability distribution is equivalent to
If the conditional probability distribution is modeled directly using the Gibbs distribution, then can be viewed as a conditional random field. The solution of this field is regarded as the process of solving the maximum posterior probability as follows:
According to the Hammersley-Clifford theorem, the posterior probability of the marker field y follows the Gibbs distribution shown as follows:
Among them, is the normalization factor of the posterior probability, and the function with parameters is the potential energy function defined within C. Generally, conditional random fields construct potential energy functions by combining Gaussian kernel functions with multiple features. Each pixel belongs to a field when certain features of the image are known. The ultimate goal of fundus vascular segmentation is to find a marker field with the known observation field, eventually obtaining the maximum posterior probability. The energy function contains two terms, of first- and second-order potential functions, which measure the dependence of the observed and labeled values of individual pixels at position i and the consistency of markers between different pixels adjacent to this position. Thus, the energy function can be expressed as follows:
The first-order and the second-order potential functions are, respectively, expressed in the following equations:
The conditional random field considers the spatial context information of the fundus image, which can partly overcome the defects of the neural network and be used to further refine the segmentation results of the residual neural network.
3.2. Feature Extraction Based on Residual Networks
A residual network is a convolutional neural network proposed by four scholars from Microsoft Research. It is easy to optimize and can improve accuracy by adding considerable depth. Its internal residual block uses jump connection to alleviate the gradient disappearance problem caused by the increasing depth in deep neural network. The convolutional neural network (CNN) is a kind of deep one, and it aims to provide an end-to-end learning in the deep learning field in recent years. Neural networks originated from the concept of receptive field were proposed in the 1960s by Hubel et al. In 1980, Fukishima proposes a multilayer neural network with convolutional and subsampling operations whereas, in 1998, the LeNet-5 network structure is launched, as designed by Lecun et al. The network uses a gradient-based backpropagation algorithm to train a neural network, and the model is applied to the classification of the ImageNet database, which reduces the TOP5 error rate from 25.8% to 16.4%, thus making the convolutional neural network academia focus. Patterns were defined to find the essential characteristics of the objects from the original data. The traditional way, based on feature extraction and classifier organization, requires the design of features according to the characteristics of the task. As shown in Figure 3, the neural network represented by the CNN deep learning algorithm has achieved a series of breakthrough research results in image feature detection, speech recognition, and so on. The network can automatically extract effective features from the data to make it widely used. Compared to the traditional shallow learning, CNN does not rely on empirical artificial selection features as it can automatically learn deep features according to the structure of neural network, with strong feature learning and classification abilities.
The network from the bottom to the top, respectively, extracted different levels and scales. We need to design the appropriate network structure for each level of such network and to better describe our goals. The multiscale convolutional neural network structure framework used for fundus vascular segmentation is shown in Figure 3. First, a convolution operation of 21 × 1 × 1 is added after all the intermediate convolution layers of each stage; second, the 21 feature maps, obtained from each stage, are sampled to the original picture size; then, these feature maps are convolved through 1 × 1 matrix. Finally, through the loss function of each stage, the activation function of the segmentation for the network fused such maps for each level, taking into account the features of each intermediate layer. In this way, the rich information at multiscale and multilevel is fully used to segment blood vessels. As the characteristics of the algorithm can well meet the requirements of single-object assisted vascular image recognition, this paper uses the algorithm for optimization.
3.3. Feature Extraction and MRF Grayscale Information Identification Algorithm
The analysis of cardiac vascular structure will be the screening as well as the diagnosis of clinical large-scale heart diseases. Usually, in order to better analyze the changes of cardiac vascular structure, experts will manually segment blood vessels. However, manual segmentation depends on the expertise of the experts and is very time-consuming. In terms of time and cost, manual segmentation may not be suitable for it. Therefore, the research for efficient automatic vascular segmentation mode is of great significance. There are many automatic segmentation methods of fundus vessels. In the absence of experts, these methods can complete the task of segmentation of blood vessels and obtain equally reliable segmentation results and efficient segmentation efficiency.
The network is widely used. Some of these applications aim at the research of vascular segmentation. At present, most vascular segmentation methods, based on convolutional neural network, whether pixel by pixel segmentation or graphics segmentation, have reached the segmentation accuracy of the current mainstream methods. However, the effect of some methods of fine vessel segmentation in vascular edge and low-contrast areas still needs to be improved. In order to further improve the segmentation of detail information and difficult samples, we propose an algorithm based on residual network and conditional random field as shown in Figure 4.
(a)
(b)
(c)
(d)
(e)
(f)
As shown in Figure 3, the residual or vascular segmentation algorithm consists of three steps: pretreatment, rough segmentation, and resplit. The learning direction of network parameters will be controlled by the majority of non-vascular pixels, and the final result will tend to segment the majority of non-vascular pixels.
For the training phase, due to the limited database size, data amplification is used to increase the number of training images. Then, the image data will be input to be trained into the multiscale convolutional neural network, the network loss will be calculated according to the improved cross-entropy loss function, the network parameters will be interactively updated until the network converges, and the coarse segmentation probability map from the network will be outputted. Furthermore, the probability map will be refined as a function and the final binary vessel segmentation map will be delivered. Adopting a deep residual network facilitates the segmentation of elongated vascular structures. Figure 4 gives the probability distributions of the networks with or without residues, respectively. The residual network is more concentrated at both ends of the probability information, and the contrast improvement of the probability map can better distinguish between vessels and non-vessels data. In the test phase, the probability graph is obtained directly using the network-trained parameters and then the conditional random fields. Therefore, the research for efficient automatic vascular segmentation mode is of great significance. The three specific steps of vessel segmentation are given as follows:(1)Pretreatment: without enough training images, a large number of convolutional neural network parameters can easily lead to over fitting the network. Considering the relatively small scale of the existing database, we enlarge the training data by scaling, rotating, and flipping.(2)Rough segmentation: the probability map of blood vessels is obtained by using the convolution neural network with the increase of remaining blocks.(3)Resplit: finally, the gray information of fully connected conditional random field is segmented.
4. Result Analysis and Discussion
4.1. Experimental Dataset and Preprocessing
In this paper, a series of validation experiments are carried out on two commonly used vascular segmentation databases, DRIVE and star. The database source is reliable, the data are available, and the sample size is enough to support the experimental demonstration of this paper. Therefore, this paper selects this sample.
DRIVE database comes from Holland diabetes heart disease screening project. The screening included 400 diabetic patients aged between 25 and 90 years. The DRIVE database randomly selected 40 images, of which 33 did not show any signs of diabetic optic heart disease and 7 had mild early diabetic heart disease. These ones will be inflatable ones using an 8-bit color picture size of 768584 pixels. The image is saved in JPEG compressed format.
The star database contains 20 images. The image is an 8-bit color map of 35 fields obtained by TOPCON trv-50 camera, which can be saved in portable bitmap ppm format. The image size is 605 × 700 pixels with a field diameter of about 605 × 700 pixels. Each image in the database gives the artificial segmentation results of two observers. With the segmentation of fine ships, the results are manually annotated by the first observer as the basic fact of the evaluation algorithm.
Generally speaking, the original blood vessel images collected by the camera inevitably have the disadvantages of low contrast and high noise. To solve these problems, preprocessing is to enhance the collected color image, improve the image processing technology of the blood vessel part, and remove all kinds of noise interference, so as to facilitate the subsequent processing.
Cardiovascular images consist usually of three channel color images. Figure 5 shows a color image of the heart. Through observation, it can be found that the main parts of the heart blood vessels are relatively clear, but the contrast with the background is low, and many of them are difficult to distinguish with the naked eye. Combined with the color theory and the observation of the component map of each channel, it is found that the cardiac vessels of channel have rich edges and strong contrast compared with other channels, as shown in Figure 4. Therefore, this paper selects -channel component map for cardiac vascular segmentation.
4.2. Feature Extraction and MRF Grayscale Information
The residual network of the previous section aims to extract the features of heart images. For heart images, only about 10% of pixels are blood vessels, and the proportion of vascular and non-vascular pixels is seriously unbalanced. If this section is treated equally, the direction of network parameters will be controlled by the majority of non-vascular pixels, and the final result will tend to segment the majority of non-vascular pixels.
The cross entropy is an important concept in Shannon information theory, which is mainly used to measure the difference information between two probability distributions. First, the cross-entropy loss function is constructed as the loss function of the neural network:
The convolutional neural network learning process is to update the parameter in the direction that leads the probability value to get closer to the real value. This is the resulting pixel of label value; it is correct to judge the category to which the pixel belongs. Through analysis, most of the correctly identified vascular pixels have probability values larger than the threshold, and most of the correctly identified non-vascular pixels have a smaller probability than the threshold. At the same time, most of the pixels are misdivided.
Figure6(a) shows that the horizontal axis represents the pixels. Accordingly, the vertical axis of Figure 6(b) represents the misclassified pixels. If the probability function is close enough to the true label value, the improved loss function will ignore the loss at that pixel. We achieved the optimal segmentation performance by adjusting the values of λ and is shown in Figure 7 after determining at each probability value.
(a)
(b)
(a)
(b)
For some computer vision processing tasks, the depth of the network is usually increased in order to perform the task. At the same time, for the detection of images, local details are also indispensable; in particular, it facilitates the detection of vascular details, while one of abstract vascular graphics. Adopting a deep residual network facilitates better segmentation of elongated vascular structures (as shown in Figures 8 and 9). Figure 10 gives the probability distributions of the networks with or without residues, respectively. The residual network is more concentrated at both ends of the probability information, and the contrast improvement of the probability map can better distinguish between vessels and non-vessels outputs. Therefore, it can be concluded from Figure 10 that the algorithm has good accuracy.
5. Conclusion
Nowadays, the whole heart oriented segmentation method is one of the difficult problems and hot topics in medical image segmentation and image analysis. As cardiovascular diseases have become one of the main diseases threatening human health, early noninvasive diagnosis of cardiovascular diseases is highly needed and it is put forward by many clinicians, which will play a very important role in prolonging human life expectancy and improving human quality of life. However, the entire heart structure must be removed in order to apply early auxiliary diagnostics, interventional guided therapy, and cardiac surgery navigation. Efficient segmentation of cardiovascular will talk about auxiliary recognition ways of counting to recognize the heart and blood vessel image and evaluating the recognition effect. Therefore, it is of great clinical value to accurately extract the complete region and the edge of the heart with the help of cardiac MR image. The corresponding features are extracted by the depth of residual network, and the heart and blood vessels are segmented with high precision. In this paper, the existing medical image algorithms are discussed, focusing on the algorithms of heart image. Thus, a series of analysis and improved algorithms are proposed. The research contents of a single target-assisted vascular image recognition algorithm integrating residual network feature extraction and MRF gray information. The heart blood vessels are roughly segmented by depth residual network, and then the MRF gray information of fundus blood vessels is used to realize the auxiliary recognition of the fundus blood vessels at different scales. A series of validation experiments were carried out on DRIVE and star datasets. Experimental results show that this method has high segmentation accuracy and can realize high-precision segmentation. In this study, high-precision single target blood vessel image recognition is realized. By observing and analyzing two-dimensional heart images, such as diabetic arterioles, bleeding, and hard exudates, it is less difficult to diagnose the characteristics of cardiovascular diseases and help diagnose and treat them. Compared to other literature results, this research overcomes the problems of lacking and segmentation difficulties in image segmentation and has certain application value.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 82160053) (for Hong Lang).