Table of Contents Author Guidelines Submit a Manuscript
Abstract and Applied Analysis
Volume 2014, Article ID 973147, 7 pages
Research Article

Biopsy Needle Localization and Tracking Using ROI-RK Method

Université de Lyon, CREATIS, CNRS UMR5220, Inserm U1044, INSA-Lyon, Université Claude Bernard Lyon 1, 69100 Lyon, France

Received 29 June 2014; Revised 9 August 2014; Accepted 9 August 2014; Published 14 October 2014

Academic Editor: Shen Yin

Copyright © 2014 Yue Zhao et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


ROI-RK method is a biopsy needle localization and tracking method. Previous research work has proved that it has a robust performance on different series of simulated 3D US volumes. Unfortunately, in real situations, because of the strong speckle noise of the ultrasound image and the different echogenic properties of the tissues, the real 3D US volumes have more complex background than the simulated images used previously. In this paper, to adapt the ROI-RK method in real 3D US volumes, a line-filter enhancement calculation only in the ROI is added to increase the contrast between the needle and background tissue, decreasing the phenomenon of expansion of the biopsy needle due to reverberation of ultrasound in the needle. To make the ROI-RK method more stable, a self-correction system is also implemented. Real data have been acquired on an ex vivo heart of lamb. The result of the ROI-RK method shows that it is capable to localize and track the biopsy needle in real situations, and it satisfies the demand of real-time application.

1. Introduction

In medical examinations and surgeries, minimally invasive technologies are widely used. However, the localization and tracking of biopsy needles using 3D ultrasound (US) are always a problem, and robustness of localization and real-time property of the algorithms represent important challenges. Even though, in the last two decades, modern large-scale processes have become more and more perfect within all the possible industrial applications, it is still difficult to obtain the physical models for complicated processes such as the microtool visualization in 3D space [1]. So, currently most clinicians do the biopsy examinations using a 2D US medical imaging system to achieve the visualization of the biopsy needle and tissue. However, the situation to be observed is 3D; it requires that the clinicians have extensive experience on operating the US probe and the biopsy needle simultaneously on both hands so that the 2D scanning plane includes the needle. Unfortunately, it is extremely hard to locate the 2D US plane which corresponds best to the needle and in most situations only a part of the needle is visible, which could lead to an unpredictable injure of the human tissue.

To avoid the problem, different technologies can be implemented to assist the clinician during the procedure. For example, electromagnetic solutions can be implemented to increase the localization accuracy [2]. Several microtool localization algorithms have been proposed. Ding et al. have proposed a biopsy needle segmentation and localization method based on two orthogonal planes in the 3D US volume [3]. However, it takes 280 s to complete the segmentation and localization procedure, which is not adapted to the real-time nature of ultrasound imaging. The 3D Hough transform and its improvements are also proposed as tool localization methods using 3D US technology [47]. The challenge of the Hough transform based tool localization method is the balance between the calculation time and the localization accuracy. To avoid the exhausting search of parameters, iterative algorithms such as random sample consensus (RANSAC) [8] have been applied. It is firstly implemented in a model-fitting segmentation method on 2D acoustic image by Tao et al. [9]. Barva et al. have proposed an R-RANSAC based curvilinear object localization method in 3D US volume [10]. Then, Uherčík et al. have proposed an efficient model-fitting RANSAC (MF-RANSAC) algorithm for straight needle localization in 3D US volume [11]. Though the calculation time of the RANSAC based localization method is decreased, it cannot meet the demand of real-time tracking. What’s more, the localization accuracy of RANSAC localization algorithm is reported to be less than 1 mm using simulated 3D US volumes with homogeneous background [11], but it fails to detect the needle with an inhomogeneous background. Indeed, in real 3D US volumes, tissue with highly echogenic structures exists around the needle. For this reason, the risk of a poorly fitting model from the RANSAC algorithm is quite important in real situations.

To solve this problem, the usual solution is to increase the number of iterations in order to increase the probability of reaching the best model. This is done at the cost of an increase in computation power and time. So, Zhao et al. have proposed an automatically determined region of interest (ROI) strategy together with the RANSAC and Kalman filter algorithm (ROI-RK) localization and tracking method to decrease the calculation time and improve the robustness of localization [12]. However, it is a pity that the verification of this method using real 3D US data is not presented in [12]. The objective of this paper is to provide such a complementary experimental evaluation of ROI-RK method. It presents the localization and tracking result of ROI-RK method using the real US data volume obtained from a heart of lamb.

This paper is organized as follows. Section 2 briefly presents the procedure of the ROI-RK method, Section 3 explains the experimental settings and procedures, and Section 4 gives the localization and tracking results. A short discussion and the conclusion are given in Section 5.

2. ROI-RK Method

The ROI-RK method is a two-step microtool localization and tracking method which can be adapted to 2D US images or 3D US volumes. The first step is an initialization step, which allows the initialization of ROI using a tubularness structure enhancement method proposed by Frangi et al. [13]. Once the ROI is initialized, the second step of ROI-RK, the tracking method itself begins. In the tracking loop, the RANSAC algorithm is used to detect the position of the biopsy needle in the ROI. Since the ROI size is limited, while the needle is inserted, a Kalman filter is implemented to update the ROI meanwhile decreasing the measurement error of the RANSAC algorithm. Since the needle insertion is controlled by hand, a motion estimation strategy is also applied to estimate the inserting velocity of the needle. In the following subsections, more details of ROI-RK method are presented.

2.1. Step I ROI Initialization

The substantial speckle noise of the 3D US volume leads to an extremely large number of outliers. In this situation, the RANSAC algorithm fails to estimate a good fitting model for the needle. To reduce the influence of the background noise, a limited ROI is defined. The shape of ROI is defined as a cylindrical region since the needle can be considered as a thin, long cylinder. Its radius is set at five times the real radius of the needle. Suppose is the subset of all the voxels within the ROI, and it satisfies the condition: is the subset of voxels whose intensity is larger than the threshold, and is the Euclidian distance from the voxel to the estimated axis . To further reduce the speckle noise and improve the contrast between the needle and background, a tubularness volume is calculated from the original 3D US volume using the Hessian based line filtering method [13]. Figure 1 gives an example of the original 3D volume and its tubularness volume. After using Frangi’s line filter, the original US volume is calculated as a tubularness volume; thus, it is easier for RANSAC to locate the needle. For the subsequent volumes, the ROI is chosen automatically according to the last result of the RANSAC algorithm.

Figure 1: The original 3D US volume (a) and its tubularness volume (b).
2.2. Step II: Tracking Algorithm
2.2.1. RANSAC Algorithm

Unlike the partial least square (PLS) data driven method [14], the RANSAC algorithm is a model based iterative method. It is efficient to separate the inliers data from the outliers. Here, the RANSAC is used to estimate the optimal axis of the biopsy needle. The localization procedure contains four steps [11].

Step 1. Threshold is as follows: the threshold strategy is first implanted to reduce the number of voxels using the assumption that the intensity of the needle voxels is higher than the background tissue.

Step 2. Axis localization is as follows: the RANSAC algorithm is used to estimate an approximate position of the needle axis.

Step 3. Local optimization is as follows: a principle component analysis (PCA) based local optimization algorithm is used to determine a more accurate axis position using the inlier voxels located from Step 2.

Step 4. Tip localization is as follows: the needle tip is identified along the optimal needle axis.

2.2.2. Motion Estimation

The speckle tracking method [15] is used to estimate the velocity of the needle tip. First, a small 3D speckle pattern around the estimated tip position is chosen as the kernel region in the previous US volume. Then, a larger region is chosen as a searching region in the next US volume around the known tip position. Then, the kernel region slides voxel by voxel in the searching region, and the normalized cross-correlation (NCC) is calculated for each position as a matching criterion between the kernel and the searching region. The fast normalized cross-correlation (FNCC) algorithm [16] is used to reduce the NCC calculation time. The largest NCC is considered as the best corresponding one with the kernel region. The tip velocity can be calculated by dividing the interframe displacement by the interframe time interval.

2.2.3. Mathematical Model for the Kalman Filter

The Kalman filter is an optimal recursive data processing algorithm, and it is capable to estimate the past, present, and even the future state of a system. Thus, it is an excellent candidate to predict the ROI position and refine the needle one in the 3D US.

To fix a line in 3D space, a minimum of four parameters is needed [17]. Considering the parameter complexity and the estimation result of the RANSAC algorithm, five parameters are chosen to represent the 3D line: two direction angles (the angle between the needle and the plane ) and (the angle between the needle and the positive direction of the -axis) and the Cartesian coordinate of the tip . Figure 2(b) gives the definition of these two angles. During the insertion procedure, the needle moves along its axis direction. Using the constant velocity model, the state vector of the Kalman filter is where and are the angle velocities, which are supposed to be zero. is the expected inserting velocity. The corresponding state transition matrix is and, here, is the time interval between the two US volumes.

Figure 2: The sketch of the definitions of angles and .

Thanks to the RANSAC algorithm and the speckle tracking method, the position and motion information of the needle can all be measured, so the measurement vector and the corresponding measurement matrix are

Using the set of equations of Kalman filter: The information obtained above is connected and used in the tracking loop.

2.2.4. Tracking Loop

During the clinical procedure, the 3D US probe and the biopsy needle are all manually controlled. Movements of the probe, such as rotating and pressing, or the relative movement between the probe and the needle, could bring the needle outside of the ROI; thus, it leads to the failure of the RANSAC algorithm. Therefore, an ROI innovation strategy using Kalman filter and a self-correction system using feed-back are implemented in the tracking loop. Once the ROI is initialized, the tracking system only runs in it. The line-filter enhancement calculation runs in the ROI to decrease the expansion phenomenon of the needle in the images due to reverberation. With the tip velocity estimated using speckle tracking and the previous needle position information, with the time update equations of Kalman filter ((5) and (6)), the ROI position can be innovated in the coming volume. Then, the RANSAC algorithm runs in the updated ROI to locate the biopsy needle. Using the new position information and the measurement update equations of Kalman filter ((7), (8), and (9)), the needle position is refined and the measurement error is effectively decreased.

To ensure the robustness of the ROI-RK method, an automatic error detection method is also implemented. In real situation, the needle insertion procedure is always along the same direction. The observer model as mentioned in [18] can be later implemented for a better error correction. In this case, if the axis accuracy (Figure 3) of the present volume and the previous volume is larger than 3 mm, it is treated as a localization error, and the ROI will be reinitialized.

Figure 3: The diagram for the definition of axis accuracy.

3. Experimental Setting

A series of 3D US volumes are acquired using the Sonix RP scanner equipped with a 4DC7-3/40 3D probe. The center frequency of the probe is 5 MHz. A three-dimensional freedom motor has been used to fix a biopsy needle and perform the insertion procedure. The diameter of the needle is 17 gauges (1.2 mm). A heart of lamb is water-bathed. The whole experimental system is set as in Figure 4. The motor is manually controlled step by step during the insertion procedure. The inserting step is 2 mm. A series of ten 3D US volumes are acquired using the US machine and processed using a laptop computer.

Figure 4: The experimental setting of the real data acquisition.

In each 3D US volume, there are 55 planes, 64 beams per plane, and 224 samples per beam. The sector angle of one plane in the US volume is 34.5° and the total tilt angle of the scan plane is 38°. The axial resolution is about 0.3 mm, the lateral resolution is about 0.7 mm, the azimuthal resolution is about 0.9 mm. In real US volumes, it is hard to define the ground truth of the needle position. In our application, the needle is segmented by hand for five times, and the ground truth position is the mean of the results of manual segmentations.

4. Results

To verify the localization and tracking results of ROI-RK method, the axis accuracy, the deviation of the two directional angles , , and the deviation of the needle tip are calculated. The axis accuracy (Figure 3) is defined as The deviation of tip is the Euclidian distance between the estimated tip and the tip segmented by hand. The localization and tracking procedure are repeated thirty times and a curve is fitted using the root mean square error (RMSE) of each parameter. The RMSE is calculated as

Figure 5 shows localization and tracking results of the ROI-RK method using the series of 3D US volumes of lamb heart. From the fitting curve of Figures 5(a) and 5(b), it is obvious that, using the Kalman filter, the measurement error of axis localization and angle are efficiently decreased. For the tip localization, the fitting curve of RMSE remains the same. However, for the RMSE of angle , the fitting curve rises a little. This is not because the Kalman filter becomes invalid, but the drawback of our US volumes. From Figures 2(a) and 3(a), there is a reflection phenomenon below the real image of the biopsy needle, which leads to a high intensity influence region of the needle localization. With the insertion of needle, this reflection region expands and thus leads to an inaccuracy of angle . The maximum axis accuracy is smaller than 1.8 mm. The maximum RMSE of is within 8 degrees, and that of is smaller than 5 degrees. The RMSE of tip localization is smaller than 1.6 mm. The average localization time in one 3D US volume is 0.1 s. To summarize, the ROI-RK method can accurately locate and robustly track the needle using real 3D volumes within an acceptable range. It also satisfies the demand of real-time processing.

Figure 5: The RMSE of axis accuracy and angles , and the tip position and their fitting curves.

For better observation, Figure 6 gives the localized tool plane and its neighbor planes. Note that the tool plane does not correspond to the acquisition plane of the 3D volume, so these planes are the interpolated planes using the localization result of the ROI-RK method. From the series of planes, the structures of needle in Plane 2, Plane 3, and Plane 4, are the most intense. The located tool plane (Plane 3) is the plane in the middle, which means that the ROI-RK method well locates the position of needle in the needle voxels cluster.

Figure 6: The detected tool plane and its neighbor planes.

5. Discussion and Conclusion

Though the US imaging technology has been further improved, the real-time navigation system using US still faces a great challenge. Firstly, because of the different echogenic properties of the different tissues, the intensity of the same biopsy needle can be different in different tissue. So, a dynamic threshold estimation and diagnosis model could be built using the modern data-driven methods [19]. Secondly, for several reasons, the structure of the biopsy needle appears in the US volume like a reverberating object that is much larger than the needle itself. With the multiple reflection problems, the fake images of needle influence the localization performance of the ROI-RK method. Therefore, better US imaging systems as well as biopsy needles with high echogenic properties are required to lead to better performances of this totally automatic localization method. Thirdly, since ROI-RK method depends on a stochastic adaptive complex system, some fault data diagnostic systems, as mentioned in [20, 21], could be implemented to further decrease the localization error.

It proves that this method performs well not only in the series of simulation US volumes but also in the real tracking situations. Further experiments will be done using different organs and tissues and also with various biopsy needle. An application system which implemented the ROI-RK method should also be developed to meet the requirement of clinical applications.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.


  1. S. Yin, X. Li, H. Gao, and O. Kaynak, “Data-based techniques focused on modern industry: an overview,” IEEE Transactions on Industrial Electronics, 2014. View at Publisher · View at Google Scholar
  2. A. Hakime, A. Barah, F. Deschamps et al., “Prospective comparison of freehand and electromagnetic needle tracking for US-guided percutaneous liver biopsy,” Journal of Vascular and Interventional Radiology, vol. 24, no. 11, pp. 1682–1689, 2013. View at Publisher · View at Google Scholar · View at Scopus
  3. M. Ding, Z. Wei, L. Gardi, D. B. Downey, and A. Fenster, “Needle and seed segmentation in intra-operative 3D ultrasound-guided prostate brachytherapy,” Ultrasonics, vol. 44, supplement 1, pp. e331–e336, 2006. View at Publisher · View at Google Scholar · View at Scopus
  4. H. Zhou, W. Qiu, M. Ding, and S. Zhang, “Automatic needle segmentation in 3D ultrasound images using 3D Hough transform,” in Proceedings of the in SPIE 5th International Symposium on Multispectral Image Processing and Pattern Recognition, 67890R, 2007.
  5. H. Zhou, W. Qiu, M. Ding, and S. Zhang, “Automatic needle segmentation in 3D ultrasound images using 3D improved Hough transform,” in Proceedings of the Medical Imaging: Visualization, Image-guided Procedures, and Modeling, 691821, vol. 6918, March 2008. View at Publisher · View at Google Scholar · View at Scopus
  6. W. Qiu, M. Ding, and M. Yuchi, “Needle segmentation using 3D quick randomized hough transform,” in Proceeding of the 1st International Conference on Intelligent Networks and Intelligent Systems (ICINIS '08), pp. 449–452, Washington, DC, USA, November 2008. View at Publisher · View at Google Scholar · View at Scopus
  7. W. Qiu, M. Yuchi, M. Ding, D. Tessier, and A. Fenster, “Needle segmentation using 3D Hough transform in 3D TRUS guided prostate transperineal therapy,” Medical Physics, vol. 40, no. 4, Article ID 042902, 2013. View at Publisher · View at Google Scholar · View at Scopus
  8. M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the Association for Computing Machinery, vol. 24, no. 6, pp. 381–395, 1981. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  9. L. Tao, U. Castellani, A. Fusiello, and V. Murino, “3D acoustic image segmentation by a RANSAC-based approach,” in Proceedings of the OCEANS, pp. 1098–1101, 2003.
  10. M. Barva, J. Kybic, J.-M. Mari, C. Cachard, and V. Hlaváč, “Automatic localization of curvilinear object in 3D ultrasound images,” in Medical Imaging 2005-Ultrasonic Imaging and Signal Processing, Proceedings of SPIE, pp. 455–462, USA, February 2005. View at Publisher · View at Google Scholar · View at Scopus
  11. M. Uherčík, J. Kybic, H. Liebgott, and C. Cachard, “Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 8, pp. 1907–1916, 2010. View at Publisher · View at Google Scholar · View at Scopus
  12. Y. Zhao, C. Cachard, and H. Liebgott, “Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method,” Ultrasonic Imaging, vol. 35, no. 4, pp. 283–306, 2013. View at Publisher · View at Google Scholar
  13. A. F. Frangi, W. J. Niessen, K. L. Vincken, and M. A. Viergever, “Multiscale vessel enhancement filtering,” Proceedings of the Medical Image Computing and Computer-Assisted Interventation (MICCAI '98), vol. 1496, pp. 130–137, 1998. View at Google Scholar
  14. S. Yin, G. Wang, and X. Yang, “Robust PLS approach for KPI-related prediction and diagnosis against outliers and missing data,” International Journal of Systems Science, vol. 45, no. 7, pp. 1375–1382, 2014. View at Publisher · View at Google Scholar
  15. D. E. Robinson, F. Chen, and L. S. Wilson, “Measurement of velocity of propagation from ultrasonic pulse-echo data,” Ultrasound in Medicine & Biology, vol. 8, no. 4, pp. 413–420, 1982. View at Publisher · View at Google Scholar · View at Scopus
  16. D.-M. Tsai and C.-T. Lin, “Fast normalized cross correlation for defect detection,” Pattern Recognition Letters, vol. 24, no. 15, pp. 2625–2631, 2003. View at Publisher · View at Google Scholar · View at Scopus
  17. K. S. Roberts, “A new representation for a line,” in Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '88), pp. 635–640, 1988.
  18. J. Liu, S. Laghrouche, M. Harmouche, and M. Wack, “Adaptive-gain second-order sliding mode observer design for switching power converters,” Control Engineering Practice, vol. 30, pp. 124–131, 2014. View at Publisher · View at Google Scholar
  19. S. Yin, G. Wang, and H. R. Karimi, “Data-driven design of robust fault detection system for wind turbines,” Mechatronics, vol. 24, no. 4, pp. 298–306, 2014. View at Google Scholar
  20. S. Yin, X. Yang, and H. R. Karimi, “Data-driven adaptive observer for fault diagnosis,” Mathematical Problems in Engineering, vol. 2012, Article ID 832836, 21 pages, 2012. View at Publisher · View at Google Scholar · View at MathSciNet
  21. J. Liu, S. Laghrouche, and M. Wack, “Observer-based higher order sliding mode control of power factor in three-phase AC/DC converter for hybrid electric vehicle applications,” International Journal of Control, vol. 87, no. 6, pp. 1117–1130, 2014. View at Publisher · View at Google Scholar · View at MathSciNet