Abstract
Stroke is a kind of cerebrovascular disease that heavily damages people’s life and health. The quantitative analysis of brain MRI images plays an important role in the diagnosis and treatment of stroke. Deep neural networks with massive data learning ability supply a powerful tool for lesion detection. In order to study the property of the stroke lesions and complete intelligent automatic detection, we collaborated with two authoritative hospitals and collected 5,668 brain MRI images of 300 ischemic stroke patients. All the lesion regions in the images were accurately labeled by professional doctors to ensure the authority and effectiveness of the data. Three categories of deep learning object detection networks including Faster R-CNN, YOLOV3, and SSD are applied to implement automatic lesion detection with the best precision of 89.77%. Meanwhile, statistical analysis of the locations, shapes of the lesions, and possible related diseases is conducted with valid conclusions. The research contributes to the intelligent assisted diagnosis and prevention and treatment of ischemic stroke.
1. Introduction
As the second most deadly disease in the world, stroke has always been one of the major causes which damage human beings’ life and health. It is characterized by high morbidity, disability rate, mortality rate, and recurrence rate and brings a heavy burden to the society and the families of patients. At present, the incidence of stroke shows a rapid increase in low-income groups and younger groups [1]. With the increasing importance of medical images in clinical diagnosis, MRI has become an important basis for the diagnosis and treatment of stroke, especially for ischemic stroke, which is hardly distinguished from CT scans compared with hemorrhagic stroke.
Most hospitals all over the world still rely on doctors for medical image diagnosis, which faces up the following problems: (1) It costs much time and effort for radiologists and doctors to examine the MRI images, and patients usually need to wait for over 24 hours to get the imaging conclusion. This will probably delay the valuable best treatment hours of stroke and reduce the recovery opportunity of patients. (2) The workload is too heavy. When the doctors get tired, sort of misdiagnosis or missed diagnosis will occur and influence patients’ treatment. Therefore, automatic and intelligent computer-aided diagnosis of MRI has become a priority.
The traditional machine learning (ML) method for object detection or analysis requires image denoising, segmentation, manual setting of features, combined with classifiers for recognition. The steps are cumbersome, time-consuming, and laborious, with low accuracy and poor diagnostic effect. Deep learning (DL) is a new research direction in the field of machine learning, which is closer to the original purpose of machine learning-artificial intelligence. It can simulate the hierarchical structure of the human brain, learn the inherent principles and presentation levels of data samples, automatically extract eigenvalues, and have a strong learning ability. In recent years, the practicality of deep learning has greatly improved with the help of high-performance GPU servers and massive datasets. Deep learning technology represented by a convolutional neural network (CNN) automatically extracts characteristic values from a large number of samples to obtain more advanced abstract features for classification, detection, and segmentation, making it possible for intelligent MRI interpretation.
In order to further study automatic diagnosis and prevention of ischemic stroke, we cooperated with two local Grade III A hospitals and collected 5,668 brain MRI images and their clinical imaging reports from 300 cases, with all the lesion areas accurately labeled by professional neurologists. Three kinds of object detection networks, Faster R-CNN, YOLOv3, and SSD, were designed and implemented to carry out automatic lesion detection on MRI images. Statistical analysis of lesion locations, shapes of lesions, and distributions of suspected diseases are carried out and reliable conclusions are given. The research can supply effective data and methods for automatic diagnosis and prevention of ischemic stroke, which is beneficial to people’s life and health.
2. Related Work
2.1. Disease Recognition and Classification
The medical image lesion detection and auxiliary diagnosis system based on deep learning can extract the advanced features of the lesion in the medical image, and the combination with clinical practice will greatly reduce the workload of doctors. In the computer-based lesion detection methods, the characteristics of body parts or organs are calculated and examined through supervised learning methods or classical image processing techniques (such as filtering and mathematical morphology). Among them, the training data samples used in the machine learning method based on supervised learning need to be provided by professional physicians with comprehensive pathological images and manual annotation.
In the past few decades, deep learning technology has developed rapidly and found wide applications in the medical field. In particular, CNN has been applied to lesion detection, with an accuracy improvement of 13–34% [2]. Sirinukunwattana et al. [3] used a spatially constrained convolutional neural network (SC-CNN) to detect and classify colorectal adenocarcinoma cells. It adopts the neighboring ensemble predictor (NEP) method for classification and recognition, which can achieve higher accuracy compared with the traditional feature classification methods. Dou et al. [4] used 3D CNN to automatically detect cerebral hemorrhage (CMBS) from MR images, which can extract more representative advanced features. Compared with 2D CNN and traditional manual feature extraction, 3D CNN detection accuracy can be as high as 93.16%. Different types of CNN architectures have made great success in diagnosing various diseases. Therefore, the rapid development of deep learning has brought big prospects in the field of medicine. Another important application of deep learning in medical images is lesion recognition. Deep learning can effectively mine useful information from the training data and improve the accuracy and speed of medical diagnosis. Kooi et al. [5] used CNN to identify malignant breast lesions.
2.2. Lesion Detection Networks
Lesions detection in medical images belongs to a special kind of object detection task [2]. Before the formal involvement of deep learning in object detection, the traditional detection methods are region selection, feature extraction, classification, and regression, which have two difficult problems to solve. Firstly, the strategy of regional selection has a poor effect and high time complexity. The other is the poor robustness of feature extraction by hand. With the fast development of deep learning, object detection algorithms are mainly divided into two factions, one-stage and two-stage.
The two-stage method, represented by the RNN system, solves the problem in two steps: (1) generating region proposals and using CNN to extract features and (2) putting them in the classifier to classify and correct the position. This series of algorithms are inseparable from region proposal. R-CNN system makes full use of the value of feature maps, in which Faster R-CNN [6] and Mask R-CNN [7] are the good representatives. Compared with the previous network, Faster R-CNN not only improves the detection accuracy but also improves the detection speed. It truly realizes the end-to-end target detection framework and shows excellent performance. Cai et al. [8] used one-stage lesion detection to detect different lesions in CT images. Yap et al. [9] pretrained on the ImageNet dataset and used the prior information of natural images for breast tumor detection. Therefore, this paper first chooses Faster R-CNN as the lesion detection network in brain MRI images of ischemic stroke.
The one-stage method is represented by YOLO and SSD. Regression is performed directly on the predicted target object. Among them, YOLOv3 [10] introduces the FPN structure, and its detection layer is integrated by three-level feature layers. YOLOv3 focuses on solving the problem of small object detection and achieves better performance. Liu et al. [11] adopted simplified VGGNet and multiscale output to SSD, making the network more robust for the processing of target scale. SSD was also used for breast cancer lesion detection and significantly got higher performance than other similar algorithms [12]. Therefore, YOLOv3 and SSD are also selected as two MRI lesion detection networks for ischemic stroke in this paper.
In addition, utilizing multimodal images is also quite common. Ben-Cohen et al. [13] used PET images to help lesion detection in CT scans of the liver. Zhang et al. [14] developed a strategy to detect breast masses from digital tomosynthesis by fine-tuning the model pretrained on mammography datasets. Zhao et al. [15] also used multimodal data for liver tumor detection.
2.3. Current Problems with Medical Data in Deep Learning
The current medical image data in deep learning mainly has the following three problems:(1)Available medical image datasets are in great demand. The reason why deep learning has such strong expressive ability is that many useful features are extracted from massive data. If the dataset is relatively small, the model learning is not sufficient, the recognition accuracy might be very low.(2)The resolution and dimension of medical images need to be improved. At present, the medical images are mainly 2D gray images, which is difficult to distinguish between pulmonary vascular cross-sectional and pulmonary nodules. In addition, medical images are affected by the differences between devices and patients, which brings difficulties to deep learning.(3)The authoritative labeling of medical images is lacking. Only if the images are marked by professional doctors, can the dataset be used. However, doctors are busy with clinical work. They need to spend extra time and effort to build valuable datasets.
For the above reasons, we are making effort to build a special ischemic stroke MRI dataset.
3. Data Collection and Statistical Analysis
3.1. Data Collection and Preprocessing
In order to systematically and deeply study the pathological changes of ischemic stroke, our research team cooperated with two local Grade III A hospitals including Qilu Hospital of Shandong University (Qingdao) and Qingdao Municipal Hospital to collect the brain MRI images of 300 ischemic stroke patients and the corresponding clinical diagnosis reports. The database contained a total of 5,668 DWI sequence images. Furthermore, all lesion images are accurately marked by experienced neurologists.
Currently, there are two main storage formats for object detection labels: VOC format and COCO format. This dataset uses the former format. Neurologists use the LabelImg tool to label the lesions in the images and mark the Bounding Box of the lesions. The annotation is stored in an xml file, which includes not only the width, height, and channels of the images but also the coordinates and category of the bounding boxes. The entire database contains four files, as shown in Table 1.
3.2. Data Analysis and Visualization
The images and reports in the database are randomly selected from the clinical data of ischemic stroke in the two hospitals from 2017 to 2019, which can objectively reflect the comprehensive characteristics of ischemic stroke. The images and reports were anonymized before being used by the researchers. The collected MRI data and reports are statistically analyzed and visualized by Excel and Python for further exploring the internal useful information about ischemic stroke. The process of data analysis is divided into tabulation and statistical analysis. The purpose of tabulation is to summarize important medical data of patients for easy browsing, query, and storage. Statistical analysis on the MRI images and clinical imaging reports includes the following five aspects: (1) gender, (2) age, (3) possible related diseases, (4) lesion location, and (5) lesion shape.
3.2.1. Gender
Figure 1 shows that the proportion of males and females of the patients is about 9/8, reflecting that males are more likely to suffer from ischemic stroke and should take more precautions to keep their health. The gender characteristic of stroke in China is that the morbidity of men is higher than women [1]. Men are more likely to suffer from stroke than women [16]. A similar ratio has also appeared in world-wide research of stroke [17]. The above references come to the same conclusion.
3.2.2. Age
The second index is the age distribution of the 300 patients. Their average age is 66.527, the minimum is 19, the maximum is 92, and the standard deviation of age is 12.107. In addition, we also calculated the number of cases of different age groups, as shown in Figure 2. Obviously, more than 80% of the patients are older than 50 and the age section between 60 and 70 has the biggest risk. Therefore, the elderly need to be more cautious about ischemic stroke. Their family members should keep a good eye on them and help with their cerebrovascular health.
3.2.3. Possible Related Diseases
We conducted data analysis on clinical diagnosis or Conclusion and counted the five most common diseases of stroke patients. The result is shown in Figure 3. Since the database is about ischemic stroke, also known as cerebral infarction, the diagnosis result of cerebral infarction is 100%. The second-highest stroke symptom is senile degeneration of the brain, accounting for approximately 74%. Senile degeneration of the brain is a kind of degenerative changes in the brain. Brain atrophy as a type of senile degeneration of the brain is the most common chronic disease in middle age and older age. The high proportion of senile degeneration of the brain indicates that the elderly are more likely to suffer from stroke. The third place is sinusitis. Among all the 300 stroke patients, 169 patients have sinusitis, which is approximately 56%. Sinusitis is a chronic inflammation of the sinuses. According to the statistical results, we can see that the incidence of the disease is very high. Ethmoiditis is also a kind of sinusitis.
3.2.4. Lesion Location
We also analyzed the vocabulary in the MRI description, namely, Finding, and studied the high incidence location and the shape of the lesion in stroke patients. The results of this data analysis will improve the reliability of the prevention, diagnosis, and treatment of ischemic stroke. Finding word segmentation is performed through the Jieba library in natural language processing, and the position and shape word frequency of the lesion are calculated. The Jieba library is an excellent third-party Chinese word segmentation library in Python and is suitable for word frequency statistics. The location distribution of the lesion is displayed as a histogram, indicating high-risk areas, as shown in Figure 4.
From Figure 4, we can see, the most vulnerable pathological areas are basal ganglia, frontal lobe, radiation crown, pons, occipital lobe, parietal lobe, temporal lobe, and thalamus. Among them, the number of lesions in the basal ganglia was the largest, with 61 cases, followed by the frontal lobe with 46 cases. There were 40 cases of radiation crown. In addition, more than 20 cases were pons, occipital lobe, parietal lobe, and temporal lobe. The greater the number of lesions in a brain area, the higher the risk. From this point of view, the basal ganglia, frontal lobe, radial coronal area, pons, occipital lobe, parietal lobe, and temporal lobe are the most common sites for cerebral infarction. The thalamus and cerebellar hemispheres also have the possibility of stroke. The locations of the brain in the high incidence areas are shown in Figure 5.
3.2.5. Lesion Shape
The fifth index is the statistics of the shape of the lesion area. Among them, patchy accounts for about 50% of the total lesion, as shown in Figure 6. Because the shape of the lesion in the diagnosis report is affected by the radiologists’ subjective factors, they may offer different shape descriptions for the same lesion.
4. Experimental Results and Analysis
Based on the collected data, automatic lesion detection is implemented using three categories of object detection networks. The experiment is deployed on Ubuntu16. 04 system. The server is equipped with NVIDIA GTX TITAN X and the CPU is Intel Xeon E5-2620 v4. The deep learning framework is PyTorch.
4.1. Lesion Detection Network Design
In this paper, three kinds of better-performing target detection networks (Faster R-CNN, YOLOv3, and SSD) are applied to automatically detect the lesions of ischemic stroke on the collected data. Faster R-CNN may use VGG-16 or ResNet-101 for feature extraction. Therefore, four object detection networks are experimented overall.
The Faster R-CNN algorithm uses a two-stage detection architecture. First, the Region Proposal Network (RPN) is used to generate the Region of Interest (ROI), and then the generated ROI is classified and regressed. The feature extraction network uses the weights of the VGG-16 or ResNet-101 models trained on ImageNet for initialization. The Faster R-CNN (VGG-16) network structure designed and implemented in this paper is shown in Figure 7:
YOLOv3 uses the idea of regression to directly complete object detection using a one-stage network. YOLOv3 brings forth the network structure of DarkNet-53, which draws on the residual idea of ResNet to make the model easier to converge. At the same time, it utilizes multilayer feature maps and does not use a pooling layer. In order to ensure the efficiency of object detection, we used Tiny-DarkNet. YOLOv3 makes multiscale prediction similarly to FPN, replacing the Softmax function with the Logistic function to process the category’s prediction score. Logistic classifiers are independent of each other and can achieve multicategory prediction.
Based on the ideas of Faster R-CNN and YOLOv3, SSD utilizes a fixed frame for region generation based on a one-stage network and adopts multilayer feature information. The SSD algorithm also uses VGGNet for feature extraction. On this basis, several convolutional layers are added, and then a 3∗3 convolution kernel is called to predict on 6 feature layers with different sizes and depths to obtain the classification. The predicted value of regression of the preselection box is then calculated, and the final result is got with the network loss. The difference of SSD from YOLOv3 is that the features of SSD are predicted separately from shallow to deep layers, without depth fusion.
The training parameters of the four object detection networks used in our experiment are shown in Table 2.
4.2. Comparison and Analysis of the Detection Networks
The entire database is divided into a train set and a test set according to the 4 : 1 ratio. Since not all images contain lesions, the amount of image data used for object detection is less than 5,668. Totally, there are 1,137 images with lesion labeling. The train set includes 910 images and the test set includes 227 images. Table 3 shows the quantized experimental results.
From Table 3, it can be seen that the detection accuracy (mAP) obtained by Faster R-CNN (VGG-16), Faster R-CNN (ResNet-101), and YOLOv3 networks is similar, while SSD performs the best. This reflects SSD has the best learning ability and adaptability to a variety of lesions of different sizes, types, shapes, and gray levels. And the robustness of the database annotation is also proved.
In addition, we further studied the changes in the accuracy of YOLOv3 and SSD as the number of iterations increases. When the number of YOLOv3 training iterations was 2000, the obtained mAP was 60.1%; when it increased to 20,000, mAP also increased to 74.9%. When the number of the SSD training iterations was 2,000, the mAP was 70.8%. Similarly, when it increases to 20,000, the entire network tends to stabilize, and the maximum mAP is 89.77%. It shows that, as the number of iterations increases, the detection accuracy of the network also increases. And the SSD increment is larger.
4.3. Visualized Analysis of Lesion Detection Results
The visualized lesion detection results obtained by the above networks on the test set are shown in Figure 8. The first row is the original manual labeling ground truth, and the second to fifth rows are lesion detections results from Faster R-CNN (VGG-16), Faster R-CNN (ResNet-101), SSD, and YOLOv3, respectively. Figures 8(a)–8(g) show different brain MRI image layers. Figure 8 shows that the four detection networks can all perform well as to single clear lesion and large-area lesions, in which SSD is still the best one. For example, in column (g), the left bright line, although similar to a lesion part observed from the gray scale with eyes, is not wrong mistaken by the networks. This bright line is actually an MRI machine scanning trace and easy to cause confusion even for professional doctors. The four networks have obtained accurate detection results, which show the effectiveness of the object detection algorithms on our database.
Figure 9 shows the detection results of the four networks as to the more difficult lesions in the database. Faster R-CNN (VGG-16) and Faster R-CNN (ResNet-101) in Figures 9(a)–9(d) detect some normal regions as lesions, leading to redundant inspections. Faster R-CNN generates multiscale ROIs through RPN, which achieve better results for small target detection and, on the other hand, increase the risk of redundant inspections. In column (e), the two Faster R-CNN networks generate multiple small lesion regions compared with one entire area in GT. This is because some areas have multiple adjacent lesions and it is difficult to separate these lesions. When data labeling is performed, the areas are marked as one lesion. The YOLOv3 network has fewer parameters; thus, the test speed of YOLOv3 is the fastest, reaching 30FPS. By combining the regression idea from YOLO and the anchor mechanism from Faster R-CNN, SSD regresses multiscale regional features and achieves more accurate experimental results for lesions in the image. Therefore, SSD is more accurate for the detection of ischemic stroke lesions, which also verifies the effectiveness of our database.
5. Conclusion
In order to further study the disease of ischemic stroke, we collected image data from two local Grade III A hospitals and completed professional labeling for afterward open research. Statistical analysis is carried out aimed at five indexes and valuable findings and suggestions are given for ischemic stroke precaution and treatment. Deep learning methods including Faster R-CNN, SSD, and YOLOv3 networks are conducted for automatic lesion detection with a precision of 89.77%.
In future, we will continue to collect more data on ischemic stroke and hope to offer a better open research platform. Meanwhile, we may also divide the dataset according to different attributes. The research can improve the intelligence level of computer-aided diagnosis of stroke and promote the development of the theory and practice of artificial intelligence in the medical field.
Data Availability
The database used to support the findings of this study is still being collected and is expected to be released when the data size reaches 1000 cases.
Disclosure
Shujun Zhang and Hongyan Wang are co-corresponding authors.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (61702295 and 61672305) and the Key Research and Development Plan Project of Shandong Province (2017GGX10127).