Development of Automatic 3D Blood Vessel Search and Automatic Blood Sampling System by Using Hybrid Stereo-Autofocus Method

Abstract

We developed an accurate three-dimensional blood vessel search (3D BVS) system and an automatic blood sampling system. They were implemented into a point-of-care system designed for medical care, installed in a portable self-monitoring blood glucose (SMBG) device. The system solves problems of human error caused by complicated manual operations of conventional SMBG devices. We evaluated its accuracy of blood-vessel position detection. The 3D BVS system uses near-infrared (NIR) light imaging and the stereo and autofocus hybrid method to determine blood vessel locations accurately in three dimensions. We evaluated the accuracy of our 3D BVS system using a phantom of human skin, blood vessels, and blood. Additionally, we established an automatic blood sampling system for SMBG and assessed its performance in relation to punctures, blood suction, transport, and discharge on an enzyme sensor. The 3D BVS and automatic blood sampling system are adequate for use in a portable SMBG device.

1. Introduction

Recently, the rapidly increasing number of diabetic patients has [1] become a major social problem throughout the world. Diabetic patients must independently measure blood glucose concentrations using a self-monitoring blood glucose (SMBG) device and self-inject insulin several times each day [2, 3]. Diabetes patients and candidates might constitute up to one-fifth of the world population [1]. Therefore, SMBG devices are expected to play an increasingly important role in daily medical care. The commercial SMBG devices of various kinds that are on the market today can be categorized as either noninvasive or minimally invasive. Noninvasive devices adopt infrared spectroscopy [4], FT-IR-ATR method [5], photoacoustic spectroscopy [6], and so on. However, these methods present severe problems of low accuracy and unreliability. In contrast, minimally invasive devices [7, 8] can measure blood glucose concentrations with high accuracy [9], but puncture with a needle or knife edge is painful. In addition, such devices frequently induce human error because of their complicated manual operations during puncture, treatment of small blood samples, self-measurement, and so on. Therefore, many improvements are necessary for SMBG devices. Several groups have developed minimally invasive SMBG devices equipped with a blood vessel search system to reduce human error. For example, Yamakoshi et al. developed a device to detect the positions of blood vessels in the arm using reflected near-infrared (NIR) light [10], but the blood vessel depth could not be determined accurately. Another example is a bio-medical-chip developed by Horiike et al., which can detect the position of a blood vessel in three dimensions using electric signals [11]. However, their chip showed low accuracy of blood vessel location detection, and the device entailed operations that were time-consuming and painful for patients. Although Kandani et al. developed a three-dimensional blood vessel search (3D BVS) system using the stereo method [12], the value of the standard deviation of measured positions was higher for deeper blood vessel locations. In this study, we set our goal as integration of a 3D BVS system and an automatic blood sampling system. To accomplish that goal, we combined the autofocus method with the original stereo method to establish a hybrid method. We set the allowed value of 100 μm for the targeted blood vessel, which had diameter of 500–1000 μm. An automatic blood sampling system for collection and transport of the blood for blood glucose measurements was connected with 3D BVS to produce the portable medical care system. For patients with diabetes, this system is expected to improve quality of life to a considerable degree.

As described in this paper, we first discuss the hybrid stereo-autofocus method for 3D BVS systems by introducing images processed with NIR light. Then we examine the accuracy of the 3D BVS system. Secondly, to evaluate our newly developed automatic blood sampling system for blood glucose measurement, we verify its capabilities of (1) accurate puncture,(2) stable blood transport, and(3) blood discharge on a glucose enzyme sensor. In light of those results, we assess the performance of this integrated portable medical care system.

2. Hybrid Stereo-Autofocus Method and Blood Vessel Visualization

2.1. Autofocus Method and Stereo Method

We used a hybrid stereo-autofocus method to detect the position of a blood vessel located between the first and second joints of the finger using transmitting NIR light. Our previous studies, which were done using a stereo method, did not determine the blood vessel depths accurately [12]. The measurement error was high for blood vessels located deep in the finger. Therefore, we adopted an autofocus method to obtain a very clear image of blood vessels using the template search method in the region of blood vessel [13]. This method improved the accuracy considerably. As presented in Figure 1(a), we took blood vessel images using different camera positions in relation to the finger based on the autofocus method. D means the length between the initial set position and the fixed base, and 𝑆 the slide length as indicated in Figure 1(a). Fixed base means the upper surface of the glass plate, which is located at the position 𝐷_𝑠 and on which the finger was set. We evaluated the blood vessel image sharpness based on the template-matching algorithm and selected the clearest image at the camera position 𝐷_𝑠 as shown in Figure 1(a) [13]. In the auto focus procedure, we fix the incline angle 𝜃 as indicated in Figure 1(b).

(a) Schematic diagram of movement of the cameras in our system

(b) Schematic view of the stereo optical system embedded in a 3D BVS system

(c) 8 bit Pixel images of CMOS camera 1 and 2 (three rows) and deviation lengths 𝐵1and 𝐵2

Figure 1 

Schematic diagram of the autofocus and stereo hybrid methods embedded in a 3D BVS system.

Next, as shown in Figure 1(b), we measured blood vessel positions using the stereo method [12]. In Figure 1(b), L stands for the distance between the two CMOS cameras, f represents the focal length, 𝐷_𝑏 signifies the vertical distance between the center of the blood vessel and the line drawn between two center points of the lenses, and 𝐷_𝑠 denotes the vertical distance between the reference surface of the fixed base and the line drawn between respective center points of the lenses. In Figure 1(b), Δ𝐷 is the depth of the blood vessel center from the reference surface of fixed base. Figure 1(b) shows 𝜃₁ and 𝜃₂ as the angles of the light axes from the center of the blood vessel against the horizontal line between the respective centers of the two lenses. As shown in Figure 1(c), the center of the blood vessel can be detected from the luminance distribution of the blood vessel images obtained using the two CMOS cameras. Furthermore, 𝜃₁ and 𝜃₂ are calculable using trigonometric functions. The size of image plane pixel, 5.55 μm, was used to evaluate the blood vessel center position with the same length ratio as the pixel. Therefore, we obtain the deviation lengths 𝐵₁ and 𝐵₂ as indicated in Figure 1(c), which can be calculated as a number of deviated pixels times by pixel size 5.55 μm. Finally, 𝐷_𝑏 is expressed as 𝐷_𝑏=tan𝜃₁⋅tan𝜃₂tan𝜃₁+tan𝜃₂𝐿, (1)here,𝜃₁=𝜋−𝜃−𝜃^′₁,𝜃₂=𝜋−𝜃−𝜃^′₂, 𝜃^′₁=tan⁻¹(𝑓/𝐵₁), 𝜃^′₂=tan⁻¹(𝑓/𝐵₂),𝐿=𝐿₁+𝐿₂=(𝐷_𝑏/tan𝜃₁)+(𝐷_𝑏/tan𝜃₂₁).

Length 𝐷_𝑠 is already determined by using the autofocus method as shown in Figure 1(a). Therefore, the blood vessel depth (Δ𝐷) can be calculated asΔ𝐷=𝐷_𝑏−𝐷_𝑠. (2)

2.2. 3D BVS System and Blood Vessel Visualization

We incorporated the hybrid stereo-autofocus components into a 3D BVS system, which included four NIR LED lights (peak wavelength of 870 nm, KED871M51A; Kyosemi Corp.) as the light sources, two CMOS cameras (PPV404C; Asahi Denshi Inc.), and two image-processing units (PP-DEB-007; Asahi Electronics Laboratory Co., Ltd.). The NIR LEDs were set behind the finger to irradiate the finger with NIR light. We used two CMOS cameras to take images using the NIR transmitted light. Details of the CMOS cameras were the following: 5.32 mm focal length, 8 × 8 × 5.55 mm size, and 5.55 × 5.55 μm pixel size. Images were of 8-bit digital gray-scale values ranging from black (0) to white (255) for one pixel. The 640 × 480 pixel images were stored as JPEG image files. The images were processed by an image processing unit (AT401; Asahi Denshi Inc.) to calculate the blood vessel depth using the autofocus and stereo algorithms. Figure 2 presents a photograph of our 3D BVS system. The respective centers of CMOS cameras and LEDs are set on the same horizontal plane. The length between the cameras (𝐿 value: Figure 1(b)) was set to 30 mm. The length between the object and the camera lens (𝐷_𝑠 value: Figure 1) was set to 18.8 mm. The CMOS cameras were inclined at 33.5 deg using a rotating stage (KSP-256; Sigma Koki Co., Ltd.). The size of our 3D BVS device became 40 × 30 × 10 mm. Therefore, our system had size and features that allowed its installation into our new portable SMBG device. We used the transmitted NIR light to visualize blood vessels located in the human finger [14–16]. We selected small veins as our measurement targets and employed NIR light with wavelengths of 650–950 nm. The NIR light was transmitted through the finger and was recorded in images by the cameras in the positions presented in Figures 1(b) and 2. Figure 3 shows images obtained using the visible light and NIR light of the wavelengths shown in the figures. According to the steepest gradient search of the luminance distribution, especially in the close area to the blood vessel, finally we chose the wavelength of 870 nm [11, 13]. Figure 4 presents blood vessel images and the luminance distribution along line A–A′ as indicated in Figure 4(b). We concluded that our imaging system can provide a clearer image of the blood vessel to determine its location in three dimensions.

Figure 2 

Photograph of the 3D BVS system.

(a) Visible light

(b) 625 nm

(c) 660 nm

(d) 765 nm

(e) 780 nm

(f) 850 nm

(g) 870 nm

(h) 940 nm

(i) 950 nm

Figure 3 

Blood vessel photos in cases of NIR lights, wavelengths 625–950 nm, and visible light.

Figure 4 

Image of the blood vessel in the human finger. The NIR light wavelength is 870 nm.

Next we discuss the image processing associated with blood vessel detection. We adopted template matching to determine the image sharpness. The templates were produced on the assumption that the luminance distributions of blood vessels are convex-downward quadratic curves, as shown by the red curve in Figure 5, where the luminance distribution is portrayed as a blue line. Figures 6(a)–6(d) show results of template matching. Figure 6(a) presents the distribution of residual error obtained by template matching calculation. It shows two local maximum points, which correspond to the blood vessel position. Figure 6(b) portrays the distribution of the derivative of residual error. Figures 6(c) and 6(d) show the point at which the derivative value becomes zero, which is the center of the blood vessel. Moreover, we assumed the inclination of the diagram of derivative values to be the rate of luminance change. Therefore, we confirmed that the blood vessel can be detected clearly as the rate of luminance change. The sharpness can be evaluated using the angle of inclination of derivatives, as shown in Figures 6(c) and 6(d). Results show that we can select a clear image of a blood vessel taken by the camera located at distance 𝐷_𝑠 shown in Figure 1(a).

Figure 5 

Luminance distribution of the image including the blood vessel and the template to search the blood vessels.

(a) Residual error of template matching

(b) Derivative of residual error

(c) Derivative of residual error around blood vessel 1

(d) Derivative of residual error around blood vessel 2

Figure 6 

Residual errors calculated by template matching algorithm and its derivatives distributions to determine the sharpness.

3. Evaluation of Blood Vessel Position Detection

We evaluated the accuracy of our 3D BVS system by carrying out experiments using finger phantoms containing artificial blood vessels. The photograph and the finger phantom size are depicted in Figure 7. Small veins of a human finger, with diameters of 0.5–1.0 mm, exist at depths of 0.5–2.0 mm from the surface. Therefore, we used the diameter of 0.75 mm for artificial blood vessels. We generated finger phantoms of seven types with artificial blood vessels embedded at different depths of 0.5–2.0 mm with 0.25 mm intervals. The finger phantoms were made of the polyacetal resin (TPS-POM; Toyo Plastic Seiko Co., Ltd.), which had optical characteristics similar to those of human skin [17]. We included colored water in the artificial blood vessels.

(a) Photograph of the finger

(b) Size of finger phantom

Figure 7 

Finger phantom image and its dimensions.

We first increased the input power of NIR light to reduce the influence of scattering and thereby obtain a clear image. The scattering of NIR light reduces both the intensity of the transmitted light and the contrast between the artificial blood vessel and the tissue. We adopted four levels of input power of the NIR light: 10, 30, 50, and 60 mW. Figures 8(a) and 8(b) show photographs of blood vessels and luminance distributions of finger phantom images with the four input powers. Through comparison with the slope and concentration of the luminance distribution, we finally selected 50 mW to overcome the scattering and provide the clearest image of artificial blood vessels.

(a) Photographs of artificial blood vessel with the described light powers

(b) Luminance distributions of artificial blood vessel for the described input light powers

Figure 8 

Influence of light power on the luminance distributions.

We measured the depths of the artificial blood vessels using our 3D BVS system, which is based on the hybrid stereo-autofocus method. To avoid the influence of light from the environment, we covered the whole system with a light-proof cloth. First, we analyzed the images and determined the vertical distance between the reference surface of the fixed base and the line drawn between respective center points of the lenses as 𝐷_𝑠, presented in Figure 1(a), using autofocus methods. Secondly, we measured the position of the artificial blood vessel based on the stereo method. Figure 9(a) shows stereo images of the artificial blood vessels. The rows of pixels on the center lines represented as A-A′and B-B′, as portrayed in Figure 9(a), were used to determine the center and the blood vessel depth. Pixels located on the line (X, Y) from value (0, 240)–(640, 240) provide luminance according to the eight-byte digital number, where X-Y coordinates were defined as portrayed in Figure 9(a). Figure 9(b) shows that a rapid decrease in brightness is observed at the area of the blood vessel. Therefore, we are able to determine the artificial blood vessel position. The center of the artificial blood vessel position of camera 1 was set to the 327th pixel and that of camera 2 to the 316th pixel. According to Figures 1(b) and 1(c), the deviations between the center position of blood vessel and the fixed base plate in the image planes of CMOS camera 1 and 2 were 7 pixels and 4 pixels, respectively. Because the pixel size was 5.55 μm, the actual deviation lengths were, respectively, 64.8 μm in camera 1 and 93.6 μm in camera 2. Next, the angles of light axes, 𝜃₁ and 𝜃₂, as indicated in Figure 1(b), were obtained using these deviation length 𝐵₁ and 𝐵₂ and focal length 𝑓 as shown in Figure 1(c). We then calculated the blood vessel depth Δ𝐷 using (1) and (2). Figure 10 shows stereo images of the artificial blood vessels embedded at the indicated depths, where the left image was taken by camera 1 and the right image by camera 2,𝜃=sin⁻¹sin𝜃𝑛,𝜙=tan⁻¹𝑓𝑠,𝑑=𝑑^′tan𝜃.tan𝜙 (3)

(a) Stereo images of the artificial blood vessel

(b) Luminance distributions on lines 𝐴-𝐴′and 𝐵- 𝐵′

Figure 9 

Stereo images and luminance distributions obtained using two cameras, wavelength of 870 nm.

Figure 10 

Stereo images of artificial blood vessels embedded in the finger phantom at the indicated depth.

Next, to assess the position detection accuracy of this 3D BVS detection device, we strove to eliminate the influence of the refraction of NIR light in the visualization procedure. Because the refractive index of polyacetal differs from that of air, NIR light is refracted at the interface and the difference occurs between the blood vessel position of the image and that of the actual value. Therefore, we corrected the result of the measurements based on Snell’s law using the refractive index of the polyacetal as 1.48. Figure 11 shows the trajectory of NIR light from the finger phantom to the camera. Figure 11 shows that the measured depth of the blood vessel becomes smaller than the actual depth because of the refraction of NIR light. Formulas for the incidence angle, refraction angle, and the blood vessel depth are shown in (3). Using these values, we corrected the measured depth obtained using (1) and (2). Figure 12 shows the relation between the corrected depth and the actual one, which we named “true” depth. We calculated the standard deviations in seven cases of “true” depths. In each “true” depth, we measured more than ten times and calculated the standard deviation by using the differences between the “true” depth and measured depths. It shows that the deeper the depth, the larger the standard deviation. The maximum standard deviation was 110 μm, and the average value of standard deviations was 63 μm. The average of all deviations was 29.8 μm, which means that average value of differences between measured and true depths in seven cases. Good correlation (𝑅² = 0.946) was found between the corrected depths and the actual depths. These errors were less than the allowance value of 150 μm for the targeted blood vessel with diameter greater than 500 μm.

Figure 11 

Refraction of light in the skin tissue and depth correction.

Figure 12 

Relation between the estimated depths and true depth of artificial blood vessel.

4. Automatic Blood Sampling System

We fabricated an automatic blood sampling system that had been designed to be built in a portable SMBG device for use in conjunction with the 3D BVS system. In addition to two small stepping motors, we used a glucose enzyme sensor made by Horiba, Ltd. for accurate and reliable blood glucose measurement purposes [18], which was expected to perform accurate and quick blood collection. Figure 13 shows a prototype of the newly developed automatic blood sampling system, with size of 47 × 30 × 20 mm. Regarding performance evaluation, we used an artificial finger made of silicone rubber (RTV2 VP7550; Asahi Chemical Ind. Co. Ltd.), which had similar elastic properties to those of human skin. The artificial blood vessel diameter was set as 1.0 mm; the blood vessel was embedded to 1.0 mm depth in the artificial finger. We used a 50% glycerin solution, which had similar viscosity to that of human blood, as artificial blood. This artificial blood was kept in the artificial blood vessel. We examined the capabilities for puncture, blood extraction, transport, and discharge on the enzyme sensor. The glucose enzyme sensor required at least 5.0 μL of blood for blood glucose measurements. Figures 14(a)–14(e) show four stages of blood sampling procedures, such as puncture, extraction, transport, discharge, and measurement. We analyzed blood sampling and measurement capabilities to provide 5.0 μL of blood on the enzyme sensor. Figure 15 presents the relation between the artificial blood volume on the enzyme sensor and the piston speed for blood transport and discharge. As the piston speed decreases, the amount of artificial blood increases. We concluded that an adequate piston speed to provide over 5.0 μL of blood for measurement was 1.0 mm/s.

Figure 13 

Automatic blood sampling system equipped with artificial finger and blood vessel.

(a) Initial position

(b) Puncture and blood extraction

(c) Blood transport

(d) Discharge of artificial blood

(e) Enzyme sensor to measure the blood glucose

Figure 14 

Photos of automatic blood sampling procedure and the enzyme sensor used to measure the blood glucose.

Figure 15 

Relation between the volume of artificial blood on the sensor and the piston travel speed (±SD).

5. Conclusions

We developed a 3D BVS system and an automatic blood sampling system to fabricate a portable SMBG device. We adopted the hybrid stereo-autofocus method to detect the position of blood vessels in three dimensions. We also examined the performance of our automatic blood sampling system, which features puncture, blood extraction, transport, and discharge for an enzyme sensor to measure the blood glucose. Results of the evaluation of our 3D BVS and automatic blood sampling devices are summarized as follows. (1) Accuracy evaluation of 3D BVS system was confirmed. The maximum standard deviation was 111 μm and the average of the standard deviation was 63 μm when detecting 0.5 mm-diameter blood vessels embedded at depths of 0.5–2.5 mm with 0.25 mm intervals. Results show very good correlation with actual measurements: 𝑅² = 0.946.(2) Blood sampling performance evaluation was done with the piston speed for blood suction and discharge selected as 1.0 mm/s to provide 0.5 μL artificial blood on the glucose enzyme sensor. That amount was sufficient for measuring the blood glucose concentration. Our integrated system, combined with 3D BVS and automatic blood sampling systems, can be used effectively as a portable medical care system.