Mathematical Aspects Behind Deep Learning and Transfer Learning Approaches for Medical Image AnalysisView this Special Issue
Blood Vessel Segmentation of Fundus Retinal Images Based on Improved Frangi and Mathematical Morphology
An improved blood vessel segmentation algorithm on the basis of traditional Frangi filtering and the mathematical morphological method was proposed to solve the low accuracy of automatic blood vessel segmentation of fundus retinal images and high complexity of algorithms. First, a global enhanced image was generated by using the contrast-limited adaptive histogram equalization algorithm of the retinal image. An improved Frangi Hessian model was constructed by introducing the scale equivalence factor and eigenvector direction angle of the Hessian matrix into the traditional Frangi filtering algorithm to enhance blood vessels of the global enhanced image. Next, noise interferences surrounding small blood vessels were eliminated through the improved mathematical morphological method. Then, blood vessels were segmented using the Otsu threshold method. The improved algorithm was tested by the public DRIVE and STARE data sets. According to the test results, the average segmentation accuracy, sensitivity, and specificity of retinal images in DRIVE and STARE are 95.54%, 69.42%, and 98.02% and 94.92%, 70.19%, and 97.71%, respectively. The improved algorithm achieved high average segmentation accuracy and low complexity while promising segmentation sensitivity. This improved algorithm can segment retinal vessels more accurately than other algorithms.
Blood vessel segmentation of fundus retinal images can help doctors in diagnosing multiple eye diseases. Segmenting blood vessels integrally and accurately is necessary for accurate analysis of main blood vessels and branches . Currently, physicians mark blood vessels manually according to experiences, which is characterized by low efficiency and easy interference by subjective factors [2, 3]. Therefore, the automatic segmentation of retinal vessels is of important significance [4, 5].
Gray distribution of fundus retinal images is uneven due to influences of noises, artifacts, and illuminations, accompanied by low contrast between blood vessels and background. Moreover, arteries and veins in images cross over and superpose mutually, thereby resulting in difficulties of segmentation. The existing blood vessel segmentation methods of fundus retinal images include supervised and unsupervised learning. The former requires training according to the provided standard training set and uses the trained classifier to segment blood vessels in unknown images. The latter requires no training but segments blood vessels through thresholding the filtering response or depending on methods on the basis of certain rules.
Without artificial prior marking information, the retinal vessel segmentation method on the basis of unsupervised learning has a small workload and high working efficiency. Currently, unsupervised segmentation methods include methods on the basis of windows [6–13], vessel tracking [14–20], and morphological operations [21–23].
Chaudhuri et al.  were the first to propose blood vessel segmentation by using the Gaussian filter. Subsequently, Li et al.  proposed a segmentation technique by combining a multiscale matched filter and dual-threshold method. Kaur and Sinha  proposed a segmentation algorithm on the basis of Gabor filter and gray cooccurrence matrix. Wang et al.  processed the coarse and fine blood vessels by using the multiscale 2D Gabor wavelet. Singh and Srivastava  proposed a method on the basis of the extended matched filter. Cruz-Aceves et al.  segmented blood vessels by using a multiscale Gabor filter and threshold segmentation method on the basis of multiobjective optimization. Aguirre-Ramos et al.  enhanced the blood vessel structure and its profile by using the Gabor filter and Gaussian distribution derivatives. Singh and Srivastava  proposed a matched filtering technique centered at the Gumbel probability distribution function to improve retinal vessel segmentation performances. These methods, which are based on windows, can maintain the original vessel structure but require the processing of each pixel, thus leading to heavy computational workloads and long segmentation time.
Frangi et al.  introduced the Hessian matrix into the extraction of characteristic directions of images. Kumar et al.  extracted blood vessels from retinal images on the basis of inherent characteristics of LoG and MF, which avoided misclassification of nonvascular pixels. Fathi and Naghsh-Nilchi  proposed a multiscale vessel segmentation algorithm based on continuous wavelet transform. Qian Zhao et al.  put forward an algorithm on the basis of level set and regional growth. Rezaee et al.  proposed an optimized segmentation technique of retinal vessels that combined adaptive filtering, fuzzy entropy, and skeletonization. Ghoshal et al.  proposed a method to enhance fine blood vessels by vascular area and axial ratio. Zhao et al.  developed an infinite activity profile model for retinal vessel segmentation through the information of the image hybrid region. These methods on the basis of vessel tracking can acquire local features of blood vessels. However, they cannot realize continuous tracking due to branches or intersections, resulting in poor segmentation accuracy.
Rodrigues and Marengoni  proposed an optic disc detection algorithm on the basis of wavelet transform and mathematical morphology. The proposed algorithm segmented veins and arteries in retinal images according to tubular characteristics of blood vessels. Rodrigues and Bezerra  applied morphology and topology extractor to extract pixels of the vascular tree and discovered topological vascular characteristics and connectivity. Neto et al.  proposed a coarse-to-fine vascular detection method of retinal images. First, this method implemented general vessel segmentation. Second, refined vessel segmentation was performed on the basis of curvature analysis and morphological reconstruction. These methods on the basis of morphology are fast, highly efficient, and satisfactorily inhibit noise. However, they rely highly on the selection of structural elements.
In this study, a vessel segmentation method of fundus retinal images on the basis of the improved Frangi and mathematical morphology was proposed by combining vessel tracking and morphological operation. First, the fundus retinal image was preprocessed by global enhancement. Second, blood vessels were enhanced by the improved Frangi filtering method. Third, noise interferences surrounding the fine blood vessels were eliminated by the improved mathematical morphological method. Last, vessel segmentation of the retinal image was performed through the Otsu threshold segmentation method.
2. Vessel Segmentation of Retinal Images
2.1. Image Preprocessing
Fundus retinal images are in RGB format, which generally have to be transformed to single-channel images for the convenience of computer processing. Images in R, G, and B channels were compared (Figure 1). Blood vessels in the G channel image have high contrast with the background. Therefore, the G channel image was chosen for preprocessing.
Enhancement of fundus retinal images can distinguish blood vessels from other background regions. Histogram equalization processing can enhance the contrast of each object in a specific image, in which the scope of image intensity will be extended. As fundus retinal images have low contrast and the vascular region is dark with low contrast, the global histogram equalization method fails to achieve the ideal enhancement of blood vessels (Figure 2(a)).
Adaptive histogram equalization (AHE) implements histogram enhancement on each pixel by calculating the transformation function of each pixel neighbor domain. AHE is more appropriate to the local contrast of images and enhanced image edges to gain additional details. For retinal vascular images, AHE may amplify noises surrounding fine blood vessels in images while enhancing the contrast. Therefore, using contrast-limited adaptive histogram equalization (CLAHE) algorithm is essential to enhance retinal vessels because it can inhibit noise enhancement. Moreover, the CLAHE algorithm has a simple calculation and determines only one parameter of the amplitude limit. Figure 2(b) shows the processing result of the CLAHE algorithm.
The enhancement results of Figure 2 were compared. The AHE algorithm increased the noises surrounding the fine blood vessels and in the image background while enhancing blood vessels. This feature resulted in the uneven brightness distribution of the image. On the contrary, the CLAHE algorithm decreased noise interferences while enhancing the contrast between blood vessels and background effectively.
2.2. Improved Frangi Filter Algorithm
Blood vessels present a linear tubular structure in fundus retinal images, and the diameter range falls within a limit. From Gaussian function and original image convolution, most structures that are smaller than scale in the image can be inhibited, whereas those with a width equivalent to are enhanced. Therefore, vascular responses under different scales could be expressed by convolution of the original image and Gaussian kernel , whose scale is the radius of the vessels: where the Gaussian function is
The original Hessian matrix of a two-dimensional image is defined as
where , , , and are second-order partial derivatives of the image. Numerical values of these four parameters can be approximate to the linear convolution of the image and the scale-normalized derivative of Gaussian kernel function :
Therefore, the Hessian matrix of the point in the image is defined as
In a two-dimensional image, the Hessian matrix is a two-dimensional positive definite matrix that has two eigenvalues and corresponding eigenvectors. When blood vessels are low-dark tubular structures relative to the background, the Hessian matrix of the pixels at blood vessels has high positive eigenvalue , small eigenvalue , and . Moreover, is higher when the brightness of the pixel at the blood vessel is lower and the blood vessel diameter is larger. Therefore, an equivalence factor was introduced into the vascular response function to normalize responses of blood vessels under different scales.
In retinal images, eigenvalues of the Hessian matrix represent curvature intensity of blood vessels, whereas eigenvectors represent the curvature direction of blood vessels. As and are two eigenvalues of the matrix and , the eigenvector corresponding to reflects the direction of maximum curvature (perpendicular to the axial direction of the blood vessel). By contrast, that of reflects the direction of minimum curvature (axial direction of the blood vessel). Hence, another parameter of that represents the direction angle of the eigenvectors was introduced in [25–28]. Therefore, the proposed new vascular response function is
The output result under multiscale is where and are the minimum and maximum radii of blood vessels in fundus retinal images, respectively.
Figure 3 depicts the blood vessel enhancement effects on the basis of the traditional and improved Frangi filter. The vascular response function on the basis of the traditional Frangi filter will strengthen noises to a large extent while enhancing blood vessels. Thus, many false-positive pixels are generated in the marginal area of field of view (FOV), which influences follow-up elimination of margins of the FOV region through masking. However, the improved Frangi filter can increase the overall performance of images significantly while enhancing blood vessels. The improved Frangi filter enhances coarse blood vessels and maintains the fine ones to the maximum extent, without influencing edges of the FOV region.
2.3. Otsu Segmentation Algorithm on the Basis of Improved Mathematical Morphology
2.3.1. Improved Mathematical Morphology
After the fundus retinal image is enhanced by the improved Frangi filter, small branches of blood vessels are enhanced. At the same time, the noises in the background are increased. Therefore, the primary goal is to eliminate noise interferences in the background region of the image while retaining fine blood vessels as much as possible before image segmentation.
Basic operations of mathematical morphology include dilation and erosion, which are defined, respectively, as where is an image for processing, is a structural element of morphology, and and are images after dilation and erosion, respectively.
Open operation in morphology refers to the erosion operation of the image by using the structural element and then implementing the dilation operation. Open operation can eliminate bright regions of the image that are smaller than structural elements without affecting other details. Open operation is defined as
The structural element in mathematical morphology is very important to image processing. The linear structure was chosen as the structural element considering that blood vessels in fundus retinal images have tubular structures. This structural element has two parameters, namely, length and angle. Traditional mathematical morphology can only use the same linear structural elements (length and angle in the structural element are fixed) to process the entire image. Blood vessels in the fundus retinal image are in network distribution and have different diameters and directions. Therefore, traditional mathematical morphological processing fails to achieve the ideal effect (Figure 4(a)). Thus, an improved mathematical morphological method was proposed [29–31].
First, the lengths of linear structural element increase from the minimum diameter (2 pixels) of blood vessels to the maximum diameter (12 pixels) for every 1 pixel. Second, the angles of the linear structural element are determined every 10° from 0° to 170° given that the direction of blood vessel sections ranges between 0° and 360° and is symmetric. In this way, open operation results of 198 templates could be gained. If the gray value of the open operation results of th template at the pixel point is , then the gray value of each pixel point after final morphological processing chooses the maximum of 198 operation results at the corresponding position: where is the image processed by the improved mathematical morphology. Figure 4(c) shows the result.
Figures 4(a) and 4(c) depict the processing results of the traditional and improved mathematical morphologies, respectively. Moreover, Figures 4(b) and 4(d) show the magnifications of corresponding subregions. Traditional mathematical morphology decreases partial noises but causes serious distortion of the image, and some vascular information is lost. However, the improved mathematical morphology decreases image noises significantly and sharpens image profiles and details, without causing loss of image information. These features are beneficial for subsequent segmentation.
2.3.2. Otsu Segmentation Algorithm
Otsu algorithm is a high-efficiency and simple algorithm for image binarization. According to gray characteristics of images, the Otsu algorithm divides an image into background and foreground. Then, gray histograms of the background and foreground pixels were calculated, and their variances were compared to find the optimal threshold. This threshold refers to the threshold at the maximum variance and is used to distinguish background and foreground pixels.
For an image, is the segmentation threshold of the background and foreground pixels. refers to the proportion of foreground pixels in the image, and pertains to the average gray level. is the proportion of background pixels in the image, and is the average gray level.
The overall average gray level of the image is
The variances of foreground and background pixels are
When the variance is at maximum, the difference between the foreground and background also reaches the maximum. At this moment, the gray level is the optimal threshold.
3. Experimental Results and Analysis
In this study, the proposed algorithm was tested on the test set of public fundus retinal images by the DRIVE and STARE data sets. Qualitative and quantitative contrast analyses among segmentation results of the proposed method and manual segmentation of two experts and traditional Frangi filter processing were carried out. For comparative assessment, segmentation effects of the proposed method were compared with those of new algorithms. The traditional Frangi filtering process is also performed after the CLAHE-enhanced image, and then, improved mathematical morphology operations and Otsu algorithm are used for segmentation.
3.1. Segmentation Effect
The manual segmentation results of the two experts are the most important indicators to assess the segmentation effect of fundus retinal images. Figure 5 shows the comparison of the vascular segmentation results of the proposed method, the manual segmentation results of two experts, and the results of traditional Frangi filtering. These images were chosen randomly from the DRIVE data set. Figure 5(a) illustrates the original colorful fundus retinal images. Figures 5(b) and 5(c) show the manual segmentation results of the first and second experts, respectively. Moreover, Figures 5(d) and 5(e) depict the segmentation results of the traditional Frangi filter and the proposed method, respectively.
The results of manual segmentation by experts, traditional Frangi filtering segmentation, and the proposed method segmentation of three fundus retinal images which were chosen randomly from the DRIVE data set are shown in Figure 5. Figure 6 illustrates the 1D cross sections of the middle row of marked subarea (the framed part in the first column of images in Figure 5) in the above three segmentation results. This figure also shows that the width of blood vessels segmented by the proposed algorithm is consistent with that of the manual segmentation of experts, whereas the blood vessels segmented by the traditional Frangi filter are coarse. When using the traditional Frangi filter to enhance blood vessels, optic disk pixels are very easy to be determined as vascular pixels compared with those of Figure 5. Moreover, the traditional Frangi filter tends to coarsen veins and arteries in the center, thus generating false-positive pixels surrounding blood vessels. However, the proposed method shows a good segmentation effect of retinal vessels and assures the integrity and accuracy of vessel segmentation.
Figure 7 shows the comparison of the detailed segmentation effects of the above three methods. After the traditional Frangi filtering segmentation, many vascular pixels at ends of blood vessels are lost, and many fine blood vessels that are important to analyze retinal images are missing. Comparatively, the proposed method maintains many fine blood vessels, and the whole retinal vessels are still in good structure and connectivity.
3.2. Performance Analysis
For the objective evaluation of the segmentation effect of blood vessels, accuracy (Acc), sensitivity (Se), and specificity (Sp) are generally applied in quantitative assessment. Acc refers to the proportion of accurately classified pixels in total pixels of fundus retinal image. Se and Sp refer to the proportion of vascular and nonvascular pixels that are recognized accurately in the segmentation result, respectively. Table 1 shows the calculation formulas of these three indicators.
In Table 1, TP and FP refer to vascular and background pixels that are judged as a blood vessel in the segmentation result, respectively. FN and TN refer to vascular and background pixels that are judged as a background in the segmentation result, respectively.
Figure 8 shows the comparisons of accuracy and sensitivity between the traditional Frangi filter and the proposed method on the test set. The proposed method is superior to the traditional Frangi filter in terms of the segmentation effect of most fundus retinal images. The two images in the proposed method have lesser sensitivity but significantly higher sensitivity on the remaining images compared with those in the traditional Frangi filter. Moreover, accuracy is increased while significantly improving sensitivity.
For a better assessment, the proposed method was compared with other relatively new algorithms in terms of accuracy, sensitivity, and specificity. Performance comparison of the proposed method with some of the existing methods on both the DRIVE and STARE data sets was also conducted, as shown in Tables 2 and 3.
From the performance comparison of the methods in Tables 2 and 3, the method proposed by Budak et al.  has the highest accuracy in the DRIVE and STARE data sets, respectively, 0.9685 and 0.9735, which are higher than the proposed method’s 0.0131 and 0.0243, respectively. The method proposed by Guo et al.  has the highest sensitivity, respectively, 0.9589 and 0.9861, which are much higher than all other methods, but this method tends to make the specificity very low, only 0.7046 and 0.5628, respectively.
Fu et al. , Guo et al. , Budak et al. , and Guo et al.  are all based on deep learning methods for blood vessel segmentation. The disadvantage is that a large amount of data is required for learning. For medical images such as the fundus retina, obtaining enough fundus retina images is itself a challenging problem. The amount of data in the existing fundus image database is also very small, and there are large differences between different data sets. The proposed method is the same as the methods proposed by Chaudhuri et al. , Li et al. , and Barkana et al. . It only uses image features and attributes to process. Compared with the methods of Chaudhuri et al. , the proposed method has the highest accuracy in the DRIVE data set, and the method is simple and easy to implement. Under the condition of ensuring the sensitivity, the overall accuracy is high, and the balance of the three indicators is ensured.
Vessel segmentation of fine blood vessels for fundus retinal images is difficult. Compared with other classical methods, the proposed method achieves higher accuracy because it can segment fine blood vessels well while protecting the integrity of the trunk after segmentation. Using the improved Frangi Hessian model in enhancing blood vessels not only extracts vascular feature maps under multiscale but also enhances fine branches of blood vessels effectively. In addition, the proposed method eliminates noise interferences surrounding fine blood vessels through the improved mathematical morphological operation. Hence, the proposed method highlights fine blood vessels, thus enabling accurate segmentation.
A blood vessel segmentation algorithm of fundus retinal images on the basis of the improved Frangi and mathematical morphology is proposed in this study. The proposed method uses the improved Frangi Hessian model to enhance blood vessels, thereby achieving the extraction of blood vessel feature maps under multiscale conditions and enhancing small blood vessels. Moreover, an improved mathematical morphological operation is used to eliminate noise interferences surrounding the fine blood vessels, considering the diameter and direction of blood vessels. Hence, fine blood vessels can be recognized as blood vessel pixels accurately in the final Otsu segmentation. The proposed method is tested by the public DRIVE and STARE data sets. According to the test results, the average segmentation accuracy, sensitivity, and specificity of retinal images in DRIVE and STARE are 95.54%, 69.42%, and 98.02% and 94.92%, 70.19%, and 97.71%, respectively. Moreover, the proposed method can maintain relatively high segmentation accuracy under the premise of ensuring segmentation sensitivity and shows good overall performances. However, the optic disk may interfere, thus influencing the segmentation effect. Hence, future studies may focus on eliminating such influences on the segmentation effect.
In this study, the proposed algorithm was tested on the test set of public fundus retinal images by the DRIVE database.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This work was financially supported by the Project of Science and Technology of Shaanxi (No. 2020GY-029).
M. Niemeijer, J. Staal, B. van Ginneken, M. Loog, and M. D. Abramoff, “Comparative study of retinal vessel segmentation methods on a new publicly available database. Medical Imaging 2004: Image Processing,” International Society for Optics and Photonics, vol. 5370, pp. 648–656, 2004.View at: Google Scholar
F. Al-Turjman, M. H. Nawaz, and U. D. Ulusar, “Intelligence in the Internet of Medical Things era: a systematic review of current and future trends,” Computer Communications, vol. 150, pp. 644–660, 2019.View at: Google Scholar
J. Kaur and H. P. Sinha, “Automated detection of retinal blood vessels in diabetic retinopathy using Gabor filter,” International Journal of Computer Science and Network Security (IJCSNS), vol. 12, no. 4, p. 109, 2012.View at: Google Scholar
X. H. Wang, Y. Q. Zhao, M. Liao et al., “Automatic retinal vessel segmentation based on multiscale 2D Gabor wavelet,” Journal of Automation, vol. 41, no. 5, pp. 970–980, 2015.View at: Google Scholar
N. P. Singh and R. Srivastava, “Extraction of retinal blood vessels by using an extended matched filter based on second derivative of Gaussian,” Proceedings of the National Academy of Sciences, India Section A: Physical Sciences, vol. 89, no. 2, pp. 269–277, 2019.View at: Publisher Site | Google Scholar
I. Cruz-Aceves, F. Oloumi, R. M. Rangayyan, J. G. Aviña-Cervantes, and A. Hernandez-Aguirre, “Automatic segmentation of coronary arteries using Gabor filters and thresholding based on multiobjective optimization,” Biomedical Signal Processing and Control, vol. 25, pp. 76–85, 2016.View at: Publisher Site | Google Scholar
H. Aguirre-Ramos, J. G. Avina-Cervantes, I. Cruz-Aceves, J. Ruiz-Pinales, and S. Ledesma, “Blood vessel segmentation in retinal fundus images using Gabor filters, fractional derivatives, and expectation maximization,” Applied Mathematics and Computation, vol. 339, pp. 568–587, 2018.View at: Publisher Site | Google Scholar
A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever, Multiscale Vessel Enhancement Filtering. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Berlin, Heidelberg, 1998.
Y. Zhao, L. Rada, K. Chen, S. P. Harding, and Y. Zheng, “Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images,” IEEE Transactions on Medical Imaging, vol. 34, no. 9, pp. 1797–1807, 2015.View at: Publisher Site | Google Scholar
G. Yang, Q. Yang, and H. Jin, “A novel trust recommendation model for mobile social network based on user motivation,” Electronic Commerce Research, pp. 1–22, 2019.View at: Google Scholar
F. Tian, H. Li, and L. Yuan, “Design and implementation of AD9361-based software radio receiver,” EURASIP Journal on Wireless Communications and Networking, vol. 1, 14 pages, 2019.View at: Google Scholar
J. E. Solís-Pérez, J. F. Gómez-Aguilar, R. F. Escobar-Jiménez, and J. Reyes-Reyes, “Blood vessel detection based on fractional Hessian matrix with non-singular Mittag–Leffler Gaussian kernel,” Biomedical Signal Processing and Control, vol. 54, p. 101584, 2019.View at: Publisher Site | Google Scholar
H. Fu, Y. Xu, D. W. K. Wong, and J. Liu, “Retinal vessel segmentation via deep learning network and fully-connected conditional random fields,” in IEEE 13th international symposium on biomedical imaging (ISBI), pp. 698–701, Prague, Czech Republic, 2016.View at: Google Scholar