Abstract
ROI-RK method is a biopsy needle localization and tracking method. Previous research work has proved that it has a robust performance on different series of simulated 3D US volumes. Unfortunately, in real situations, because of the strong speckle noise of the ultrasound image and the different echogenic properties of the tissues, the real 3D US volumes have more complex background than the simulated images used previously. In this paper, to adapt the ROI-RK method in real 3D US volumes, a line-filter enhancement calculation only in the ROI is added to increase the contrast between the needle and background tissue, decreasing the phenomenon of expansion of the biopsy needle due to reverberation of ultrasound in the needle. To make the ROI-RK method more stable, a self-correction system is also implemented. Real data have been acquired on an ex vivo heart of lamb. The result of the ROI-RK method shows that it is capable to localize and track the biopsy needle in real situations, and it satisfies the demand of real-time application.
1. Introduction
In medical examinations and surgeries, minimally invasive technologies are widely used. However, the localization and tracking of biopsy needles using 3D ultrasound (US) are always a problem, and robustness of localization and real-time property of the algorithms represent important challenges. Even though, in the last two decades, modern large-scale processes have become more and more perfect within all the possible industrial applications, it is still difficult to obtain the physical models for complicated processes such as the microtool visualization in 3D space [1]. So, currently most clinicians do the biopsy examinations using a 2D US medical imaging system to achieve the visualization of the biopsy needle and tissue. However, the situation to be observed is 3D; it requires that the clinicians have extensive experience on operating the US probe and the biopsy needle simultaneously on both hands so that the 2D scanning plane includes the needle. Unfortunately, it is extremely hard to locate the 2D US plane which corresponds best to the needle and in most situations only a part of the needle is visible, which could lead to an unpredictable injure of the human tissue.
To avoid the problem, different technologies can be implemented to assist the clinician during the procedure. For example, electromagnetic solutions can be implemented to increase the localization accuracy [2]. Several microtool localization algorithms have been proposed. Ding et al. have proposed a biopsy needle segmentation and localization method based on two orthogonal planes in the 3D US volume [3]. However, it takes 280 s to complete the segmentation and localization procedure, which is not adapted to the real-time nature of ultrasound imaging. The 3D Hough transform and its improvements are also proposed as tool localization methods using 3D US technology [4–7]. The challenge of the Hough transform based tool localization method is the balance between the calculation time and the localization accuracy. To avoid the exhausting search of parameters, iterative algorithms such as random sample consensus (RANSAC) [8] have been applied. It is firstly implemented in a model-fitting segmentation method on 2D acoustic image by Tao et al. [9]. Barva et al. have proposed an R-RANSAC based curvilinear object localization method in 3D US volume [10]. Then, Uherčík et al. have proposed an efficient model-fitting RANSAC (MF-RANSAC) algorithm for straight needle localization in 3D US volume [11]. Though the calculation time of the RANSAC based localization method is decreased, it cannot meet the demand of real-time tracking. What’s more, the localization accuracy of RANSAC localization algorithm is reported to be less than 1 mm using simulated 3D US volumes with homogeneous background [11], but it fails to detect the needle with an inhomogeneous background. Indeed, in real 3D US volumes, tissue with highly echogenic structures exists around the needle. For this reason, the risk of a poorly fitting model from the RANSAC algorithm is quite important in real situations.
To solve this problem, the usual solution is to increase the number of iterations in order to increase the probability of reaching the best model. This is done at the cost of an increase in computation power and time. So, Zhao et al. have proposed an automatically determined region of interest (ROI) strategy together with the RANSAC and Kalman filter algorithm (ROI-RK) localization and tracking method to decrease the calculation time and improve the robustness of localization [12]. However, it is a pity that the verification of this method using real 3D US data is not presented in [12]. The objective of this paper is to provide such a complementary experimental evaluation of ROI-RK method. It presents the localization and tracking result of ROI-RK method using the real US data volume obtained from a heart of lamb.
This paper is organized as follows. Section 2 briefly presents the procedure of the ROI-RK method, Section 3 explains the experimental settings and procedures, and Section 4 gives the localization and tracking results. A short discussion and the conclusion are given in Section 5.
2. ROI-RK Method
The ROI-RK method is a two-step microtool localization and tracking method which can be adapted to 2D US images or 3D US volumes. The first step is an initialization step, which allows the initialization of ROI using a tubularness structure enhancement method proposed by Frangi et al. [13]. Once the ROI is initialized, the second step of ROI-RK, the tracking method itself begins. In the tracking loop, the RANSAC algorithm is used to detect the position of the biopsy needle in the ROI. Since the ROI size is limited, while the needle is inserted, a Kalman filter is implemented to update the ROI meanwhile decreasing the measurement error of the RANSAC algorithm. Since the needle insertion is controlled by hand, a motion estimation strategy is also applied to estimate the inserting velocity of the needle. In the following subsections, more details of ROI-RK method are presented.
2.1. Step I ROI Initialization
The substantial speckle noise of the 3D US volume leads to an extremely large number of outliers. In this situation, the RANSAC algorithm fails to estimate a good fitting model for the needle. To reduce the influence of the background noise, a limited ROI is defined. The shape of ROI is defined as a cylindrical region since the needle can be considered as a thin, long cylinder. Its radius is set at five times the real radius of the needle. Suppose is the subset of all the voxels within the ROI, and it satisfies the condition: is the subset of voxels whose intensity is larger than the threshold, and is the Euclidian distance from the voxel to the estimated axis . To further reduce the speckle noise and improve the contrast between the needle and background, a tubularness volume is calculated from the original 3D US volume using the Hessian based line filtering method [13]. Figure 1 gives an example of the original 3D volume and its tubularness volume. After using Frangi’s line filter, the original US volume is calculated as a tubularness volume; thus, it is easier for RANSAC to locate the needle. For the subsequent volumes, the ROI is chosen automatically according to the last result of the RANSAC algorithm.
(a)
(b)
2.2. Step II: Tracking Algorithm
2.2.1. RANSAC Algorithm
Unlike the partial least square (PLS) data driven method [14], the RANSAC algorithm is a model based iterative method. It is efficient to separate the inliers data from the outliers. Here, the RANSAC is used to estimate the optimal axis of the biopsy needle. The localization procedure contains four steps [11].
Step 1. Threshold is as follows: the threshold strategy is first implanted to reduce the number of voxels using the assumption that the intensity of the needle voxels is higher than the background tissue.
Step 2. Axis localization is as follows: the RANSAC algorithm is used to estimate an approximate position of the needle axis.
Step 3. Local optimization is as follows: a principle component analysis (PCA) based local optimization algorithm is used to determine a more accurate axis position using the inlier voxels located from Step 2.
Step 4. Tip localization is as follows: the needle tip is identified along the optimal needle axis.
2.2.2. Motion Estimation
The speckle tracking method [15] is used to estimate the velocity of the needle tip. First, a small 3D speckle pattern around the estimated tip position is chosen as the kernel region in the previous US volume. Then, a larger region is chosen as a searching region in the next US volume around the known tip position. Then, the kernel region slides voxel by voxel in the searching region, and the normalized cross-correlation (NCC) is calculated for each position as a matching criterion between the kernel and the searching region. The fast normalized cross-correlation (FNCC) algorithm [16] is used to reduce the NCC calculation time. The largest NCC is considered as the best corresponding one with the kernel region. The tip velocity can be calculated by dividing the interframe displacement by the interframe time interval.
2.2.3. Mathematical Model for the Kalman Filter
The Kalman filter is an optimal recursive data processing algorithm, and it is capable to estimate the past, present, and even the future state of a system. Thus, it is an excellent candidate to predict the ROI position and refine the needle one in the 3D US.
To fix a line in 3D space, a minimum of four parameters is needed [17]. Considering the parameter complexity and the estimation result of the RANSAC algorithm, five parameters are chosen to represent the 3D line: two direction angles (the angle between the needle and the plane ) and (the angle between the needle and the positive direction of the -axis) and the Cartesian coordinate of the tip . Figure 2(b) gives the definition of these two angles. During the insertion procedure, the needle moves along its axis direction. Using the constant velocity model, the state vector of the Kalman filter is where and are the angle velocities, which are supposed to be zero. is the expected inserting velocity. The corresponding state transition matrix is and, here, is the time interval between the two US volumes.
(a) An example of 3D US volume of heart
(b) The definition of angles and
Thanks to the RANSAC algorithm and the speckle tracking method, the position and motion information of the needle can all be measured, so the measurement vector and the corresponding measurement matrix are
Using the set of equations of Kalman filter: The information obtained above is connected and used in the tracking loop.
2.2.4. Tracking Loop
During the clinical procedure, the 3D US probe and the biopsy needle are all manually controlled. Movements of the probe, such as rotating and pressing, or the relative movement between the probe and the needle, could bring the needle outside of the ROI; thus, it leads to the failure of the RANSAC algorithm. Therefore, an ROI innovation strategy using Kalman filter and a self-correction system using feed-back are implemented in the tracking loop. Once the ROI is initialized, the tracking system only runs in it. The line-filter enhancement calculation runs in the ROI to decrease the expansion phenomenon of the needle in the images due to reverberation. With the tip velocity estimated using speckle tracking and the previous needle position information, with the time update equations of Kalman filter ((5) and (6)), the ROI position can be innovated in the coming volume. Then, the RANSAC algorithm runs in the updated ROI to locate the biopsy needle. Using the new position information and the measurement update equations of Kalman filter ((7), (8), and (9)), the needle position is refined and the measurement error is effectively decreased.
To ensure the robustness of the ROI-RK method, an automatic error detection method is also implemented. In real situation, the needle insertion procedure is always along the same direction. The observer model as mentioned in [18] can be later implemented for a better error correction. In this case, if the axis accuracy (Figure 3) of the present volume and the previous volume is larger than 3 mm, it is treated as a localization error, and the ROI will be reinitialized.
(a) The original 3D US volume of heart of lamb
(b) The definition of axis accuracy
3. Experimental Setting
A series of 3D US volumes are acquired using the Sonix RP scanner equipped with a 4DC7-3/40 3D probe. The center frequency of the probe is 5 MHz. A three-dimensional freedom motor has been used to fix a biopsy needle and perform the insertion procedure. The diameter of the needle is 17 gauges (1.2 mm). A heart of lamb is water-bathed. The whole experimental system is set as in Figure 4. The motor is manually controlled step by step during the insertion procedure. The inserting step is 2 mm. A series of ten 3D US volumes are acquired using the US machine and processed using a laptop computer.
In each 3D US volume, there are 55 planes, 64 beams per plane, and 224 samples per beam. The sector angle of one plane in the US volume is 34.5° and the total tilt angle of the scan plane is 38°. The axial resolution is about 0.3 mm, the lateral resolution is about 0.7 mm, the azimuthal resolution is about 0.9 mm. In real US volumes, it is hard to define the ground truth of the needle position. In our application, the needle is segmented by hand for five times, and the ground truth position is the mean of the results of manual segmentations.
4. Results
To verify the localization and tracking results of ROI-RK method, the axis accuracy, the deviation of the two directional angles , , and the deviation of the needle tip are calculated. The axis accuracy (Figure 3) is defined as The deviation of tip is the Euclidian distance between the estimated tip and the tip segmented by hand. The localization and tracking procedure are repeated thirty times and a curve is fitted using the root mean square error (RMSE) of each parameter. The RMSE is calculated as
Figure 5 shows localization and tracking results of the ROI-RK method using the series of 3D US volumes of lamb heart. From the fitting curve of Figures 5(a) and 5(b), it is obvious that, using the Kalman filter, the measurement error of axis localization and angle are efficiently decreased. For the tip localization, the fitting curve of RMSE remains the same. However, for the RMSE of angle , the fitting curve rises a little. This is not because the Kalman filter becomes invalid, but the drawback of our US volumes. From Figures 2(a) and 3(a), there is a reflection phenomenon below the real image of the biopsy needle, which leads to a high intensity influence region of the needle localization. With the insertion of needle, this reflection region expands and thus leads to an inaccuracy of angle . The maximum axis accuracy is smaller than 1.8 mm. The maximum RMSE of is within 8 degrees, and that of is smaller than 5 degrees. The RMSE of tip localization is smaller than 1.6 mm. The average localization time in one 3D US volume is 0.1 s. To summarize, the ROI-RK method can accurately locate and robustly track the needle using real 3D volumes within an acceptable range. It also satisfies the demand of real-time processing.
(a)
(b)
(c)
(d)
For better observation, Figure 6 gives the localized tool plane and its neighbor planes. Note that the tool plane does not correspond to the acquisition plane of the 3D volume, so these planes are the interpolated planes using the localization result of the ROI-RK method. From the series of planes, the structures of needle in Plane 2, Plane 3, and Plane 4, are the most intense. The located tool plane (Plane 3) is the plane in the middle, which means that the ROI-RK method well locates the position of needle in the needle voxels cluster.
(a) Plane 1
(b) Plane 2
(c) Plane 3
(d) Plane 4
(e) Plane 5
(f) Plane 6
5. Discussion and Conclusion
Though the US imaging technology has been further improved, the real-time navigation system using US still faces a great challenge. Firstly, because of the different echogenic properties of the different tissues, the intensity of the same biopsy needle can be different in different tissue. So, a dynamic threshold estimation and diagnosis model could be built using the modern data-driven methods [19]. Secondly, for several reasons, the structure of the biopsy needle appears in the US volume like a reverberating object that is much larger than the needle itself. With the multiple reflection problems, the fake images of needle influence the localization performance of the ROI-RK method. Therefore, better US imaging systems as well as biopsy needles with high echogenic properties are required to lead to better performances of this totally automatic localization method. Thirdly, since ROI-RK method depends on a stochastic adaptive complex system, some fault data diagnostic systems, as mentioned in [20, 21], could be implemented to further decrease the localization error.
It proves that this method performs well not only in the series of simulation US volumes but also in the real tracking situations. Further experiments will be done using different organs and tissues and also with various biopsy needle. An application system which implemented the ROI-RK method should also be developed to meet the requirement of clinical applications.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.