Research Article | Open Access
Microvascular Ultrasonic Imaging of Angiogenesis Identifies Tumors in a Murine Spontaneous Breast Cancer Model
The purpose of this study is to determine if microvascular tortuosity can be used as an imaging biomarker for the presence of tumor-associated angiogenesis and if imaging this biomarker can be used as a specific and sensitive method of locating solid tumors. Acoustic angiography, an ultrasound-based microvascular imaging technology, was used to visualize angiogenesis development of a spontaneous mouse model of breast cancer (). A reader study was used to assess visual discrimination between image types, and quantitative methods utilized metrics of tortuosity and spatial clustering for tumor detection. The reader study resulted in an area under the curve of 0.8, while the clustering approach resulted in the best classification with an area under the curve of 0.95. Both the qualitative and quantitative methods produced a correlation between sensitivity and tumor diameter. Imaging of vascular geometry with acoustic angiography provides a robust method for discriminating between tumor and healthy tissue in a mouse model of breast cancer. Multiple methods of analysis have been presented for a wide range of tumor sizes. Application of these techniques to clinical imaging could improve breast cancer diagnosis, as well as improve specificity in assessing cancer in other tissues. The clustering approach may be beneficial for other types of morphological analysis beyond vascular ultrasound images.
Clinical detection and diagnosis of breast cancer typically begins with X-ray mammographic identification of a suspicious lesion, followed by diagnostic imaging with other imaging modalities, often ultrasound. Mammography and grayscale ultrasound utilize anatomic features of breast tissue in order to discriminate malignant from benign breast tissue. The Breast Imaging-Reporting and Data System (BI-RADS) provides a framework for lesion ranking by probability of disease based on a number of radiographic features [1, 2]. Although reported numbers are variable, the sensitivity of screening mammography typically lies between 70% and 90% . Ultrasound imaging can improve this sensitivity, although the specificity of breast imaging using this combination is poor in women with dense breasts, as low as 40% . Furthermore, this limitation is even more significant with women with prosthetic implants, where X-ray mammography cannot penetrate silicone or saline effectively. Universally, uncertain imaging results require biopsy, which provides additional emotional stress, cost, and possibility of complication to a large percentage of women who end up with negative biopsy results after inconclusive imaging [4, 5]. Additional imaging methods that could better identify malignancies could improve breast cancer detection and ultimately reduce the number of unnecessary biopsies performed each year.
Microvascular imaging approaches have been proposed as an approach for cancer detection and diagnosis due to the prevalence of pathological angiogenesis, a process in which growing tumors quickly outpace their blood supply and trigger the growth of new vasculature . Angiogenesis is recognized as a hallmark of cancer and a pervasive feature of solid tumors. Notably, the newly formed vessels are distinct from healthy vasculature in several ways . Cancer neovasculature lacks regular hierarchical branching structures, leading to unpredictable flow and frequent arteriovenous shunts. The vessels are also immature and leaky and form tortuous, sinuous, and erratically shaped vessels [8–10]. Thus, perfusion patterns and abnormal vascular geometry make likely biomarkers for improving sensitivity to disease.
Acoustic angiography is a method of contrast-enhanced ultrasound imaging, and it detects broadband superharmonic emissions from microbubbles by transmitting at a low frequency and receiving with a higher frequency transducer, several harmonics above that of the transmitted signal . Tissue produces much less energy than microbubbles in the superharmonic range, so imaging microbubbles with superharmonics enables high contrast-to-tissue ratios and high resolution due to the high frequency receive [12, 13]. The resulting images reveal that the morphology of vessels resolved at 100-200 μm in diameter and contrast signal from vasculature with very little background signal. Acoustic angiography has been utilized in small animal models where it has enabled visualization of cancer-induced microvascular angiogenesis and invasion. Using this technique, researchers have observed microvascular patterns which exhibit significant differences in both morphology and density of microvasculature, in both rat and mouse tumor models [14–21]. It was also recently demonstrated to be feasible for imaging microvasculature in humans, depicting breast vasculature as small as 200 μm to a depth of approximately 15 mm .
Other ultrasound imaging approaches have been used for visualizing tumor vasculature in clinical applications. Previous studies with Doppler and contrast-Doppler images of breast lesions have found that the detection of penetrating vessels is highly suggestive of malignancy [23–25]. Other features visible in Doppler imaging that suggested malignancy were hypervascularity and tortuosity. However, these studies lacked specificity and several false positives occurred (primarily in cases of fibroadenomas). Contrast-enhanced ultrasound imaging has also been investigated for breast cancer diagnosis, and features of contrast-enhanced ultrasound (CEUS) imaging have been found to be correlated with histological tumor features, especially microvessel density . Additionally, reader studies investigating clinical CEUS imaging in tumors of other organs have shown that CEUS is superior to unenhanced imaging in the liver [27–31], pancreas [32, 33], kidney , and bladder . Though the majority of ultrasound contrast studies are performed in 2D, Luo et al. acquired 3D CEUS images of liver lesions and processed them to create “sonographic angiograms” to display vascular structure . They reported good reader agreement, sensitivity, and specificity using diagnostic features including ring-like enhancement, peripheral nodular enhancement, and spoke-and-wheel arterial arrangements, combined with the presence of early-phase enhancement.
Acoustic angiography can display 3D vascular morphology of vessels as small as 100-200 μm in diameter with high contrast and provide information about vascular density for vessels of all sizes. Therefore, we hypothesize that acoustic angiography images may provide a sensitive and specific diagnostic tool for cancer through the evaluation of vascular features that are common in tumors.
This hypothesis has been tested here using two detection approaches in a genetically engineered mouse model of breast cancer. First, visual assessment with a reader study mimics clinical practices for image-based diagnosis. Second, a quantitative method uses spatial and geometric information (tortuosity) to detect tumor presence using a statistical clustering algorithm. The sensitivity and specificity of each method are reported using receiver operating characteristic (ROC) curves, and the relationships between robust lesion identification and tumor size are explored.
2. Materials and Methods
The C3(1)/Tag mouse is a model of basal breast cancer in which tumors develop spontaneously in the mammary pads of female mice, becoming palpable around 16 weeks of age . Age-matched control mice were female wild-type (FVB/NJ) littermates. This study included 48 mice: 31 C3(1)/Tag (tumors) and 17 FVB/NJ (controls). Mice were monitored for tumor development using palpation and B-mode imaging, and contrast images were acquired in mice between 12 and 18 weeks old, as tumors emerged at different times. The Institutional Animal Care and Use Committee approved all protocols to ensure ethical standards for the treatment of animals and compliance with federal regulations.
2.1. Image Acquisition
Images of the mammary pads in the lower abdomen were collected while mice were anesthetized with 1.5% vaporized isoflurane in oxygen, and temperature was maintained with a heated stage. A 27-gauge catheter was inserted into a tail vein for the administration of contrast using a syringe pump ( microbubbles per minute, Harvard Apparatus, PHD2000). Fur was removed using clippers and chemical depilation, and ultrasound gel was applied to the imaging region.
Images were acquired with a modified Vevo 770 (VisualSonics, Toronto, ON, CA) using a custom dual-element RMV 707 which included a confocal 4 MHz annular element transducer for transmission in addition to a 30 MHz receive element. The driving waveform was a single-cycle, 1.2 MPa, 4 MHz pulse. Microbubbles used for this study were lipid-shelled, perfluorocarbon agents similar to Definity (Lantheus Medical Imaging 2017), approximately 1 μm in diameter, made in-house as described previously .
Contrast images resulted from transmitting with the low frequency element and receiving the superharmonic signals from excited microbubbles with the high frequency element, and three-dimensional images were acquired using a linear translation stage. Two images were averaged at each location, with 100 μm separating each frame. The frame rate was 4 frames per second, which resulted in an acquisition time of approximately 1-2 minutes for the 3D images encompassing in the axial and lateral directions and 25-30 mm in the elevation direction. Tumor dimensions were measured in 3 orthogonal axes from the B-mode images. Tumor volumes were computed assuming ellipsoidal form, and the geometric mean was used to represent the average tumor diameter.
2.2. Qualitative Assessments
Visual image classification was performed using a reader study design, and seven readers with experience visualizing ultrasound images rated the likelihood of malignancy on a scale of 1-6. A user interface was created using GUIDE (Graphical User Interface Development Environment, MATLAB, The MathWorks, Inc.) to display 3 views of each contrast image, which were presented to readers in random order. Two views were 2D images in the axial and coronal planes with scrollbars to allow for interactive viewing of individual slices from the 3D image volume. The third view was a maximum intensity projection of the data in the coronal plane. Each image also displayed a small red dot, indicating the suspicious region for the readers to evaluate. In tumor images, the dot was placed at the center of the tumor. In control images, the dot was positioned randomly in the mammary pad. Localization was performed on the B-mode images prior to study initiation, and coordinates were transferred to the coregistered angiography images.
Readers were presented with the acoustic angiography contrast images in random order and did not have access to B-mode images. Readers used patterns of high vascular density, peripheral enhancement, tortuosity, tissue distortion, and branching as indicators of tumor presence. The presence of multiple features or very strong features increased the readers’ confidence of malignancy.
Statistical analysis included calculation of receiver operating characteristic curves and interreader agreement and was performed using STATA/SE 14.2, treating reader assessments as ordinal variables. To characterize the relationship between the readers’ performance and tumor diameter, we applied ROC regression. The general ROC regression model can be written as whereis the survival function for controls with covariate , , and and are common and disease-specific covariates, respectively. In the present study, we considered the ROC regression model where is the empirical survival function for the controls estimated for each reader . The regression model was performed using the rocreg fuction, and standard errors were obtained using bootstrapping with 1000 replicates.
2.3. Quantitative Tortuosity Analysis
The images were also analyzed using quantitative methods to assess vascular tortuosity. First, contrast images were exported and linearly interpolated in the elevation direction to create isotropic voxels. Then, vessels were segmented using semiautomated multiscale ridge regression, using Aylward and Bullitt’s algorithm . Finally, tortuosity was calculated from the centerline of each vessel using two metrics known as the distance metric and the sum of angles metric after resampling the centerlines to 50 μm spacing between points .
The distance metric () is defined as the cumulative distance between consecutive points, , along the centerline, divided by the distance between the endpoints:
The sum of angles metric () is the sum of the angles between consecutive trios of points along the centerline, normalized by the path length. The angle is calculated by using the dot product of the two vectors defined by three points, normalized to unit length, then calculating the angle in radians using inverse cosine:
Summary statistics of tortuosity parameters and ROC curves were calculated in R using the pROC and ROCR packages [40–42]. Clustering analysis was performed using the DBSCAN algorithm using the DBSCAN package in R , with a minimum number of points per cluster of 4 and a neighborhood size () of 27.
3. Results and Discussion
The tumors included in this study ranged from 0.8 mm to 8.2 mm in diameter with a mean and standard deviation of mm. The median size was 2.3 mm, and the size distribution was skewed toward small tumors (<3 mm) in order to assess the efficacy of ultrasound vascular imaging for tumor identification and to determine the influence of tumor size on sensitivity. Tumors smaller than 3 mm in diameter are barely palpable in the mammary pads of mice but visible in high frequency (30 MHz) B-mode imaging, as shown in Figure 1, which illustrates a typical image in the abdomen of a healthy control as well as images from mice with tumors 0.8, 2.1, and 5.1 mm in diameter.
Acoustic angiography contrast images, such as those shown in Figures 2 and 3, reveal vascular cross-sections in the axial and coronal planes and show longer vessel segments in coronal planes and maximum intensity projections. Figure 2 illustrates typical contrast images of small tumors less than 2 mm in diameter. Figures 2(a) and 2(b) show axial cross-sections through each tumor, with dense vasculature and peripheral enhancement apparent. Figure 2(c) shows a coronal projection, showing vascular tortuosity and branching. These features of vascular geometry are not seen in control images, such as those shown in Figure 3.
3.1. Reader Study Analysis
Readers independently completed their assessments for all images in approximately two hours. There was strong agreement between the seven readers, reflected by a value of 0.778 for Kendall’s coefficient of concordance, and no significant difference was found between reader effects.
A receiver operator characteristic curve was constructed from the aggregate results of all seven readers, which revealed an overall area under the curve of 0.798 with a 95% confidence interval of 0.702-0.894, shown in Figure 4(a). The overall AUC value indicates the classification performance for this sample of data, which included tumors from a wide size range of 0.8 to 8 mm in diameter. The optimal cutoff, with sensitivity and specificity weighed equally, resulted in a specificity of 0.86 and a sensitivity of 0.65, at a reader assessment rank of 3.
The overall performance of the study, represented by the ROC curve, is dependent on the distribution of tumor sizes included in the study. Therefore, in order to characterize the relationship between reader performance and tumor diameter, we applied ROC regression, as described in equation (2). Regression analysis indicated that tumor diameter had a significant effect on the ROC curve, with and a coefficient of 0.343. Figure 4(b) shows the expected ROC curves for 3 different tumor sizes, chosen from the quartiles of the tumor diameters: 1.37, 2.32, and 5.09 mm. The area under the ROC curve and 95% confidence intervals for these tumor diameters are 0.670 (0.593, 0.766), 0.777 (0.723, 0.860), and 0.956 (0.907, 0.997), also listed in Table 1.
a95% confidence interval.
The average rating for a control image was , with most control images consistently rated between 1 and 3, while the mean rating for tumors among all readers was , with ratings for tumors being noticeably more variable across tumor diameters. Figure 5 shows the mean reader assessment rank for the 31 tumors versus tumor diameter. Scores for the 21 tumors smaller than 3 mm in diameter received a wide range of scores between 1.57 and 5.86, with a mean and standard deviation of . However, the 10 tumors larger than 3 mm in diameter received consistently higher scores with a mean and standard deviation of . These results suggest that tumor size strongly influenced the ability of readers to identify the vascular geometry tumors.
Overall, the specificity of the reader study was better than the sensitivity, meaning that false positives are uncommon using the qualitative reader study approach for classification. Different clinical applications may require different levels of sensitivity or specificity to provide useful classification. Table 2 lists the sensitivity and specificity values that result from requiring either a sensitivity or a specificity level of 0.9. For 1.37 and 2.3 mm tumors, sensitivity and specificity are expected to be suboptimal when the other metric is required to be 0.9, but for tumors approximately 5 mm in diameter, both the sensitivity and specificity are greater than 0.9.
3.2. Tortuosity Analysis and Clustering
Quantitative analysis of tortuosity included calculations of the distance metric and sum of angles metric for all vessels segmented in each image. Each image was summarized using mean tortuosity metrics in order to compare tumor and control images using the Wilcoxon rank sum. The mean and standard deviation of the distance metric was in tumors and in controls, with no significant difference between tumor images and controls (). The mean and standard deviation of the sum of angles metric was rad/mm for tumors and rad/mm for controls, resulting in a significant difference between the two groups, with .
The mean tortuosity metric can be used to classify an image by setting a simple threshold to determine if the image is more likely to belong to the tumor or the control group. ROC analysis was used to determine the efficacy of classification using mean tortuosity across the range of possible thresholds. The AUC for the distance metric was 0.682, and 0.824 for the sum of angles metric and listed in Table 3. These results are similar to the overall AUC found using the qualitative reader study approach (0.778). These results support the hypothesis that tortuosity is higher in tumors than controls and therefore can be used to determine if an individual image contains a tumor, whether using visual or quantitative methods of discrimination.
a95% confidence interval.
However, classification based on mean tortuosity diminishes the likelihood of detecting small tumors due to use of the mean value to represent the population of vessels within an entire image. Additionally, the use of summary statistics neglects the predictive value of heterogeneity and spatial information provided in the vascular images. Therefore, a clustering approach known as Density-Based Spatial Clustering in Applications with Noise (DBSCAN) was used to produce a classifier that accounts for spatial heterogeneity.
The vessels in each image were grouped into clusters using the DBSCAN algorithm, and maximum cluster size was chosen as the most predictive parameter for image classification. Other potential classification metrics generated by DBSCAN, such as total number of vessels clustered and number of clusters generated, had lower predictive value than the maximum cluster size. The mean and standard deviation of the maximum cluster size were much greater in tumor images () than control images (). ROC analysis of maximum cluster size for image classification resulted in an area under the curve of 0.948 (95% CI: 0.829-0.977), as seen in Figure 6(a) and Table 3.
Weighing sensitivity and specificity equally, the optimal cutoff is a minimum cluster size of 8 vessels, resulting in a sensitivity of 0.871 and a specificity of 0.941, with 95% confidence intervals of 0.667-1.0 and 0.765-1.0, respectively. Maximum cluster size is positively correlated to tumor diameter, with a significant linear regression resulting in and , as seen in Figure 6(b). Density-based spatial clustering of vessel-level tortuosity metrics provides the most accurate detection of tumors in these acoustic angiography images, with higher area under the ROC curve (0.95) than both image-level statistical analysis and reader study-based identification. The reader study approach resulted in an AUC of approximately 0.8, with high reader agreement (), and no significant differences between readers. These results are consistent with the trends presented by a previous study of 3D vascular morphology in liver tumors . Though reader agreement was good, there was greater variation between the ratings of tumor images than those of controls, especially in small tumors. The tumors with the most variability in assessment level tended to have sparser vascular patterns and features that were visible in some of the 2D image frames but obscured in the maximum intensity projections. This suggests that images with vascular abnormalities visible in the maximum intensity projection resulted in simpler and more consistent ratings as tumors.
Three-dimensional tortuosity in small vessels can be difficult to identify in maximum intensity projections (such as those seen in Figure 2), due to multiple vessels overlapping each other in the projected volume, which can obscure the morphology of individual vessels. Therefore, image segmentation and quantitative analysis were also performed to calculate vascular tortuosity, which can be displayed as 3D objects instead of image projections, as seen in Figure 7. Two metrics of tortuosity, the sum of angles metric and distance metric, were computed, and the revealed a significant difference between tumor and control images while the did not. However, mean values do not consider spatial information, biasing the detection toward larger tumors. Therefore, the most reliable classification method was using the Density-Based Spatial Clustering in Applications with Noise (DBSCAN) to generate clusters using vessel-level tortuosity data and using the size of the cluster as a prediction metric. In this study of 48 images, tortuosity clustering-based classification resulted in an excellent area under the curve of 0.95.
This study demonstrates the performance of acoustic angiography for distinguishing tumors from healthy control tissue through a reader study approach and using quantitative analysis of vascular geometry. Both methods resulted in AUCs of 0.8 or better, but classification using clustering of quantitative tortuosity metrics yielded the highest AUC of 0.95. Unsurprisingly, both methods showed that the sensitivity of detection depended on the tumor diameter, with larger tumors detected with more accuracy. However, performance was still notable for tumors on the order of 5 mm, which would be considered very small in clinical diagnostics.
These results support the potential diagnostic benefit of microvascular imaging techniques in improving sensitivity and specificity over grayscale ultrasound. In addition to acoustic angiography, the technique of ultrasound localization microscopy (ULM), also referred to as super resolution (ultrasound) imaging, can provide even higher resolution microvascular images than described here [19, 44, 45] and can be combined with acoustic angiography in order to reduce the ULM preprocessing to separate microbubbles from the tissue clutter . Although ULM still faces challenges with long acquisition and processing times for 3D data sets, it has the potential to resolve microvasculature smaller than 20 microns. Similarly, we hypothesize that photoacoustic imaging, which has similar capability to image the microvasculature in vivo , could also be used as a tool to identify the presence of malignant angiogenesis. A discussion of acoustic angiography relative to photoacoustic imaging has been provided previously by Gessner et al. .
An imaging technique with high resolution and high contrast to display vascular morphology has the potential to improve tumor diagnosis through the identification of the vascular features unique to tumor angiogenesis. Acoustic angiography, a contrast-enhanced ultrasound image technique, uses microbubble superharmonics to selectively image microvasculature as small as 100-200 μm. These images can be used to identify images of tumors in a mouse model of breast cancer. Sensitivity was correlated to tumor diameter using visual assessment and quantitative approaches. The best discrimination was generated using a statistical clustering algorithm to identify clusters of tortuous vessels in images. This method resulted in an excellent area under the curve of 0.95 in a sample of 48 images with tumors ranging from 0.8 to 8 mm in diameter.
The ultrasound image data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
P.A. Dayton declares that he is an inventor on a patent describing dual-frequency superharmonic imaging and an equity holder in SonoVol, Inc., a company that has licensed that patent.
We thank the seven readers who generously gave their time to participate in this study and Dr. Yueh Lee for his advice on study design. The collaboration of F. Stuart Foster, Marc Lukacs, Mike Lee, and Emmanuel Cherin are appreciated for the development of the custom RMV probes. This study was funded by the National Institute of Health through R01CA170665, R01CA189479, and F99CA212227, as well as a pilot grant from the UNC Lineberger Comprehensive Cancer Center.
- American College of Radiology BI-RADS Committee, Breast imaging reporting and data system, American College of Radiology, 1998.
- American College of Radiology, Breast Imaging Reporting and Data System BI-RADS, D. O. CJ, S. EA, M. EB, and M. EA, Eds., American College of Radiology, Reston, VA, 5th edition, 2013.
- M. Heijblom, J. M. Klaase, F. M. van den Engh, T. G. van Leeuwen, W. Steenbergen, and S. Manohar, “Imaging tumor vascularization for detection and diagnosis of breast cancer,” Technology in Cancer Research & Treatment, vol. 10, no. 6, pp. 607–623, 2011.
- J. Rey, S. Gardner, and R. Cushing, “Determinants of surgical site infection after breast biopsy,” American Journal of Infection Control, vol. 33, no. 2, pp. 126–129, 2005.
- M. Montgomery, “Uncertainty during breast diagnostic evaluation: state of the science,” Oncology Nursing Forum, vol. 37, no. 1, pp. 77–83, 2010.
- J. Folkman, “Angiogenesis,” Annual review of medicine, vol. 57, no. 1, pp. 1–18, 2006.
- D. Hanahan and R. A. Weinberg, “Hallmarks of cancer: the next generation,” Cell, vol. 144, no. 5, pp. 646–674, 2011.
- P. Carmeliet and R. K. Jain, “Angiogenesis in cancer and other diseases,” Nature, vol. 407, no. 6801, pp. 249–257, 2000.
- P. Carmeliet and R. K. Jain, “Principles and mechanisms of vessel normalization for cancer and other angiogenic diseases,” Nature reviews: Drug discovery, vol. 10, no. 6, pp. 417–427, 2011.
- R. K. Jain, “Determinants of tumor blood flow: a review,” Cancer research, vol. 48, no. 10, pp. 2641–2658, 1988.
- R. Gessner, M. Lukacs, M. Lee, E. Cherin, F. S. Foster, and P. A. Dayton, “High-resolution, high-contrast ultrasound imaging using a prototype dual-frequency transducer: in vitro and in vivo studies,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 57, no. 8, pp. 1772–1781, 2010.
- A. Bouakaz, S. Frigstad, F. J. Ten Cate, and N. de Jong, “Super harmonic imaging: a new imaging technique for improved contrast detection,” Ultrasound in Medicine & Biology, vol. 28, no. 1, pp. 59–68, 2002.
- D. E. Kruse and K. W. Ferrara, “A new imaging strategy using wideband transient response of ultrasound contrast agents,” IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control, vol. 52, no. 8, pp. 1320–1329, 2005.
- R. C. Gessner, S. R. Aylward, and P. A. Dayton, “Mapping microvasculature with acoustic angiography yields quantifiable differences between healthy and tumor-bearing tissue volumes in a rodent model,” Radiology, vol. 264, no. 3, pp. 733–740, 2012.
- J. M. Dunleavey, L. Xiao, J. Thompson et al., “Vascular channels formed by subpopulations of PECAM1+ melanoma cells,” Nature communications, vol. 5, no. 1, p. 5200, 2014.
- S. E. Shelton, Y. Z. Lee, M. Lee et al., “Quantification of Microvascular Tortuosity during Tumor Evolution Using Acoustic Angiography,” Ultrasound in Medicine & Biology, vol. 41, no. 7, pp. 1896–1904, 2015.
- S. R. Rao, S. E. Shelton, and P. A. Dayton, “The 'fingerprint' of cancer extends beyond solid tumor boundaries: assessment with a novel ultrasound imaging approach,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 5, pp. 1082–1086, 2016.
- B. D. Lindsey, S. E. Shelton, K. H. Martin et al., “High resolution ultrasound superharmonic perfusion imaging: in vivo feasibility and quantification of dynamic contrast-enhanced acoustic angiography,” Annals of biomedical engineering, vol. 45, no. 4, pp. 939–948, 2017.
- F. Lin, S. E. Shelton, D. Espindola, J. D. Rojas, G. Pinton, and P. A. Dayton, “3-D ultrasound localization microscopy for identifying microvascular morphology features of tumor angiogenesis at a resolution beyond the diffraction limit of conventional ultrasound,” Theranostics, vol. 7, no. 1, pp. 196–204, 2017.
- S. K. Kasoji, J. N. Rivera, R. C. Gessner, S. X. Chang, and P. A. Dayton, “Early assessment of tumor response to radiation therapy using high-resolution quantitative microvascular ultrasound imaging,” Theranostics, vol. 8, no. 1, pp. 156–168, 2018.
- B. D. Lindsey, S. E. Shelton, F. S. Foster, and P. A. Dayton, “Assessment of molecular acoustic angiography for combined microvascular and molecular imaging in preclinical tumor models,” Molecular imaging and biology: MIB: the official publication of the Academy of Molecular Imaging, vol. 19, no. 2, pp. 194–202, 2017.
- S. E. Shelton, B. D. Lindsey, P. A. Dayton, and Y. Z. Lee, “First-in-human study of acoustic angiography in the breast and peripheral vasculature,” Ultrasound in Medicine & Biology, vol. 43, no. 12, pp. 2939–2946, 2017.
- S. Raza and J. K. Baum, “Solid breast lesions: evaluation with power Doppler US,” Radiology, vol. 203, no. 1, pp. 164–168, 1997.
- G. Santamaria, M. Velasco, X. Farre, J. A. Vanrell, A. Cardesa, and P. L. Fernandez, “Power Doppler sonography of invasive breast carcinoma: does tumor vascularization contribute to prediction of axillary status?” Radiology, vol. 234, no. 2, pp. 374–380, 2005.
- D. Stanzani, L. F. Chala, N. Barros, G. G. Cerri, and M. C. Chammas, “Can Doppler or contrast-enhanced ultrasound analysis add diagnostically important information about the nature of breast lesions?” Clinics, vol. 69, no. 2, pp. 87–92, 2014.
- W. R. Jia, W. M. Chai, L. Tang et al., “Three-dimensional contrast enhanced ultrasound score and dynamic contrast- enhanced magnetic resonance imaging score in evaluating breast tumor angiogenesis: Correlation with biological factors,” European journal of radiology, vol. 83, no. 7, pp. 1098–1105, 2014.
- T. Albrecht, A. Oldenburg, J. Hohmann et al., “Imaging of liver metastases with contrast-specific low-MI real-time ultrasound and SonoVue,” European radiology, vol. 13, Supplement 3, no. S3, pp. N79–N86, 2003.
- E. Leen, W. J. Angerson, S. Yarmenitis et al., “Multi-centre clinical study evaluating the efficacy of SonoVue™ (BR1), a new ultrasound contrast agent in Doppler investigation of focal hepatic lesions,” European journal of Radiology, vol. 41, no. 3, pp. 200–206, 2002.
- M. X. Lin, H. X. Xu, M. D. Lu et al., “Diagnostic performance of contrast-enhanced ultrasound for complex cystic focal liver lesions: blinded reader study,” European Radiology, vol. 19, no. 2, pp. 358–369, 2009.
- E. Quaia, V. Alaimo, E. Baratella et al., “Effect of observer experience in the differentiation between benign and malignant liver tumors after ultrasound contrast agent injection,” Journal of Ultrasound in Medicine, vol. 29, no. 1, pp. 25–36, 2010.
- E. Quaia, F. Calliada, M. Bertolotto et al., “Characterization of focal liver lesions with contrast-specific US modes and a sulfur hexafluoride-filled microbubble contrast agent: diagnostic performance and confidence,” Radiology, vol. 232, no. 2, pp. 420–430, 2004.
- M. Kitano, M. Kudo, K. Maekawa et al., “Dynamic imaging of pancreatic diseases by contrast enhanced coded phase inversion harmonic ultrasonography,” Gut, vol. 53, no. 6, pp. 854–859, 2004.
- A. Sofuni, H. Iijima, F. Moriyasu et al., “Differential diagnosis of pancreatic tumors using ultrasound contrast imaging,” Journal of Gastroenterology, vol. 40, no. 5, pp. 518–525, 2005.
- E. Quaia, M. Bertolotto, V. Cioffi et al., “Comparison of contrast-enhanced sonography with unenhanced sonography and contrast-enhanced CT in the diagnosis of malignancy in complex cystic renal masses,” American Journal of Roentgenology, vol. 191, no. 4, pp. 1239–1249, 2008.
- G. Caruso, G. Salvaggio, A. Campisi et al., “Bladder tumor staging: comparison of contrast-enhanced and gray-scale ultrasound,” American Journal of Roentgenology, vol. 194, no. 1, pp. 151–156, 2010.
- W. Luo, K. Numata, M. Morimoto et al., “Three-dimensional contrast-enhanced sonography of vascular patterns of focal liver tumors: pilot study of visualization methods,” American Journal of Roentgenology, vol. 192, no. 1, pp. 165–173, 2009.
- J. E. Green, M. A. Shibata, K. Yoshidome et al., “The C3(1)/SV40 T-antigen transgenic mouse model of mammary cancer: ductal epithelial cell targeting with multistage progression to carcinoma,” Oncogene, vol. 19, no. 8, pp. 1020–1027, 2000.
- S. R. Aylward and E. Bullitt, “Initialization, noise, singularities, and scale in height ridge traversal for tubular object centerline extraction,” in IEEE Transactions on Medical Imaging, vol. 21, no. 2, pp. 61–75.
- E. Bullitt, G. Gerig, S. M. Pizer, W. Lin, and S. R. Aylward, “Measuring tortuosity of the intracerebral vasculature from MRA images,” IEEE transactions on medical imaging, vol. 22, no. 9, pp. 1163–1171, 2003.
- R. D. C. Team, R: a language and environment for statistical computing, R Foundation for Statistical Computing, 2018.
- T. Sing, O. Sander, N. Beerenwinkel, and T. Lenguaer, “ROCR: visualizing classifier performance in R,” Bioinformatics, vol. 21, no. 20, pp. 3940-3941, 2005.
- X. Robin, N. Turck, A. Hainard et al., “pROC: an open-source package for R and S+ to analyze and compare ROC curves,” BMC Bioinformatics, vol. 12, no. 1, p. 77, 2011.
- M. Hahsler, M. Piekenbrock, and D. Doran, “dbscan: fast density-based clustering with R,” Journal of Statistical Software, vol. 91, no. 1, pp. 1–30, 2019.
- K. Christensen-Jeffries, R. J. Browning, M. X. Tang, C. Dunsby, and R. J. Eckersley, “In vivo acoustic super-resolution and super-resolved velocity mapping using microbubbles,” IEEE transactions on medical imaging, vol. 34, no. 2, pp. 433–440, 2015.
- C. Errico, J. Pierre, S. Pezet et al., “Ultrafast ultrasound localization microscopy for deep super-resolution vascular imaging,” Nature, vol. 527, no. 7579, pp. 499–502, 2015.
- T. M. Kierski, D. Espindola, I. G. Newsome et al., “Super harmonic ultrasound for motion-independent localization microscopy: applications to microvascular imaging from low to high flow rates,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, p. 1, 2020.
- L. V. Wang and J. Yao, “A practical guide to photoacoustic tomography in the life sciences,” Nature Methods, vol. 13, no. 8, pp. 627–638, 2016.
Copyright © 2020 Sarah E. Shelton et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.