Abstract

Background. Traditional invasive hemoglobin (Hb) detection led to delayed diagnosis, operational inefficiency, incorrect critical decision making, and uncomfortable patient experience. To facilitate real-time total hemoglobin (tHb) monitoring, a portable prototype of a noninvasive Hb detection system was developed, and the accuracy of Hb predicted based on partial least squares (PLS) as well as backpropagation artificial neural network (BP-ANN) models was validated. Results. The prototype was combined with a signal processing circuit and a spectrophotometric probe containing 8 wavelength LEDs light source and photodiode array. Laboratory invasive Hb (Lab_tHb) and spot check Hb measurements with PLS (SpHb_PLS) and BP (SpHb_BP) methods were obtained simultaneously by hematology analyzer and the designed system. The invasive and noninvasive estimates of the Hb levels were analyzed using Spearman correlation as well as Bland–Altman plot and receiver operating characteristic (ROC) curve analysis. A total of 238 volunteers had attempted laboratory invasive and noninvasive spot check Hb measurements. Mean Lab_tHb, SpHb_PLS, and SpHb_BP were 13.6 ± 1.80 g/dL, 13.5 ± 1.07 g/dL, and 13.6 ± 1.06 g/dL, respectively. Noninvasive SpHb_PLS (r = 0.61, ) and SpHb_BP (r = 0.62, ) had a strong correlation with invasive tHb values. The Bland–Altman plot showed excellent consistency between the proposed noninvasive methods and laboratory invasive reference. In ROC analysis, PLS and BP models were good at predicting Hb ≥ 12 g/dL with area under the curve of 0.828 and 0.824, respectively. Observed differences between invasive and noninvasive Hb measurements displayed no significant correlation with perfusion index values. Conclusions. The result confirmed that noninvasive Hb monitoring had an excellent correlation with traditional invasive Hb measurement. Furthermore, it is suggested that the developed prototype has the potential for the noninvasive detection of Hb concentration with the methods of PLS and BP-ANN.

1. Introduction

The hemoglobin (Hb) level in human blood is crucial to evaluate the physiological status and diagnose chronic and acute anemic conditions both in daily care and during surgery [1]. Hb measurement is an essential step in determining blood loss and can help guide therapeutic decisions including operative intervention and blood transfusion [2]. However, Hb is currently measured with an invasive gold standard reference method, in which the blood sample is taken from the patient and then analyzed by blood or hematology analyzers [3]. This has subsequently led to the delayed diagnosis, operational inefficiency, incorrect critical decision making, and uncomfortable patient experience. The traditional invasive methods cannot meet the needs of real-time patient monitoring in emergency situations. Furthermore, repeated blood sampling may cause potential injury to the patient and increase the risk of infection and exposure to blood borne pathogens [4].

With the development of noninvasive detection and point-of-care testing technology, noninvasive Hb (SpHb) measurement is recently introduced that provides the capability to monitor Hb levels in a real-time, continuous, and noninvasive mode [57]. The online Hb monitoring by photoplethysmography (PPG) related techniques allows for the immediate clinical reaction to the detected results without pain and risk of infection. The Pulse Co‐oximetry, NBM 200 system, and mobile handset Haemospect are the modalities detecting Hb concentration in the blood continuously and noninvasively commercially available today [810]. Aside from the above commercial products, there are numerous studies focused on developing spectrophotometric systems and prediction algorithms for noninvasive Hb monitoring in human blood. Suzaki et al. applied pulse spectrophotometry at seven wavelengths to develop a noninvasive, continuous, and highly accurate instrument, which showed potential for measuring tHb [11]. Based on the PPG method, Timm et al. used three wavelengths to design the optical sensor system to measure the Hb concentration, pulse, and oxygenation [1,12]. Soon afterward their improved sensor system was able to determine Hb level by using the water absorption coefficient above 1150 nm [13]. Similarly, Doshi et al. developed an optical noncontact type sensor, with the help of which it is possible to measure Hb with two wavelengths, 660 nm and 940 nm [14]. Al-Baradie and Bose also designed a novel LED-based Hb sensor system for measuring PPG signals continuously at the wavelengths of 670 nm and 810 nm [15]. In 2014, with a couple of PPG signals at two different wavelengths, Nirupa and Kumar achieved the measurement of Hb concentration, oxygen saturation, heart rate, and heart rate variability [16]. On the basis of the formed nine wavelengths broadband light source and silicium photodiode array, Ding et al. constructed a high-performance spectrophotometric system for noninvasive Hb monitoring with principal component analysis combined with BP-ANN model [17]. With eight regression machine learning algorithms, Reşit Kavsaoğlu proposed a noninvasive method for the prediction of the Hb values using the characteristic features of the PPG signals and the results were promising [18]. In 2017, Konyukhov et al. developed a system consisting of computer-based software and a PPG signal acquisition module (including LED with wavelengths of 658, 776, 974, and 1348 nm) for noninvasive assessment of Hb level [19]. Noninvasive Hb measurement is still growing popularity and allows an immediate estimate of clinically significant blood loss and has significantly improved perioperative transfusion practices and even has helped enhance the quality of care. However, studies focused on the accuracy of noninvasive hemoglobin monitoring showing inconsistent or even contradictory results [47]. There were still some difficulties and obstacles in the clinical promotion of noninvasive Hb detection technology before the problems of reliability and accuracy were solved.

Although thousands of noninvasive Hb monitoring devices have been used clinically in hospitals around the world, the technology and its accuracy are still improving. The problems of not-high detection accuracy, expensive optical sensor consumables, insufficient miniaturization, and lack of portability are still the obstacles for the technology to be applied in clinics extensively [4, 20]. In the present study, eight wavelength LEDs with broadband light source were used in relatively portable, miniaturized, and inexpensive PPG spectrophotometric system designs. Combined temporally constrained independent component analysis (cICA) and adaptive filters, high-quality PPG signals were derived from the motion artifacts corrupted PPG signals with the amplitude information reserved [21]. Furthermore, models based on partial least squares (PLS) regression and backpropagation artificial neural network (BP-ANN) related to characteristics of the PPG signals have been established to predict the total Hb concentration (SpHb) continuously and noninvasively. The accuracy of Hb estimation for the developed prototype was validated in ostensibly healthy volunteers by comparison with the traditional invasive laboratory reference.

2. Materials and Methods

2.1. System Design

The portable continuous and noninvasive Hb monitoring system developed consists of three units as shown in Figure 1, a probe sensor, a front-end signal acquisition module, and a signal processing module with control software. The probe sensor includes eight LEDs covering an effective wavelength range from 610 to 940 nm, with light radiation transmitting to a finger of a person. Upon passing through the finger, transmission light is detected by a particular high specification photodiode array, which has a bandwidth of 14 kHz and a minimum static current of 120 μA. Proportional to attenuation by finger tissues, the output current of the photodiode array contains different frequency components and is low in amplitude. Then, the current is applied to transimpedance amplifiers and is converted into voltage signals. Multiplexers and their corresponding low pass filters (LPF) with specific cutoff frequencies are used to obtain the signal in the pulse wave frequency range of 0–10 Hz. After the analog to digital converters (ADCs), the PPG signals are transmitted to a microcontroller unit (MCU) for preprocessing. The MCU is also responsible for controlling the light intensity and switching sequences of LEDs through driving circuit and multiplex switch to produce eight channels of PPG signal. The multichannel PPG signals are then serially transmitted to the signal processing module for further processing. Various processing steps including filtering, cICA combined with adaptive filtering, and wavelet transform are used to eliminate the noise, motion artifacts, and baseline wander in the PPG signals after their reception [21]. Then 56 characteristic properties are obtained from the PPG signals during the feature extraction stage. The PLS and BP-ANN models are first established by measuring Hb levels via traditional laboratory invasive method together with the developed noninvasive optical measurements in a representative population. After that, for a given person, PPG features for the unknown Hb values obtained from in vivo measurements are compared with the PLS or BP-ANN derived calibration model and predicted Hb values are subsequently calculated and displayed as shown in Figure 2.

2.2. Data Modelling

In the present study, 56 features were derived from the eight channels of the PPG signal for each individual. Subsequently, these 56 characteristic properties were entered into the modelling algorithms and Hb levels of the person were estimated. The applied modelling methods are PLS regression and BP-ANN.

The PLS regression is a multivariate statistical method to regress the predictor or estimation variable X and the response or observable variable Y [22]. This method constructs components linearly based on the original estimation variables while considering the response values to produce a parsimonious model on a new space. The PLS regression model reflects the covariance between the predictors X and the response variables Y. Therefore, the PLS regression model is able to predict Y from X and to find latent components which produce the maximum covariance between the two matrices and is applicable when there are more variables in feature predictions than the observations leading to reliable predictive power.

The artificial neural network (ANN), inspired by the structure and the way the human brain works, is extensively applied for classification and prediction in many fields. Particularly, with self-learning, adaptive and self-organizing characteristics, the BP-ANN model is very suitable for recognition of patterns in complex systems and has a strong ability to deal with nonlinear problems [2326]. Considering the correlation between the input variables, principal component analysis (PCA) was used for dimension reduction before BP-ANN modelling [27]. The normal BP-ANN model generally consists of an input layer, a hidden layer and an output layer as functional processing units. The nodes or neurons between adjacent layers are connected together with different weights to form a network. The BP-ANN model consists of feature dimension reduction step, input, hidden, and output layers as shown in Figure 3. The BP-ANN model can be trained by adjusting the weights between nodes within the network structure. Based on the above learning steps, the objective function, which quantified by network error sum of squares is minimized between the actually observed values and the desired output of the network.

In our study, a 10-fold cross-validation method was applied to construct more flexible PLS and BP-ANN models. At first, traditional laboratory invasive Hb values and PPG features of 238 healthy volunteers were divided into 10 almost equal subgroups to train the models. One of the 10 subgroups was used for testing and the rest were used for training. The same training and testing data were applied to both PLS and BP-ANN classifiers. After 10 times of training and testing, the relative reliable Hb prediction models were established.

2.3. Experimental Procedures

A prospective study, which was approved by the Institutional Review Board from the Chinese PLA General Hospital with a requirement to obtain participant consent, was conducted to validate the accuracy of the newly developed noninvasive Hb monitoring system. PPG signals were collected noninvasively from the right index finger of ostensibly healthy volunteers who participate in a physical examination in the clinic and SpHb values were predicted based on previously established PLS (SpHb_PLS) and BP-ANN (SpHb_BP) models, respectively. The healthy volunteers kept on fast on the day of the physical examination. Each of the volunteers was asked to sit quietly for about 5 minutes before the noninvasive Hb monitoring to achieve a relatively placid state. Afterward, volunteers put their right index finger into the finger clip and kept their arm relaxed and stable for about 2 minutes for Hb levels detection. During this period, the probe was covered with an optical shield to avoid interference by ambient light. Perfusion index (PI) and oxygen saturation (SpO2) were also recorded by our developed PPG system. Exclusion criteria were age >75 or <16 years and any injury, infection, or skin breaks hindering the application of the device probe. Immediately following the noninvasive detection, a capillary blood sample was taken and the gold standard Hb measurement (Lab_tHb) was obtained according to the standard practice of the hospital using a fully automated hematology analyzer (Sysmex XS-1000i, Japan) at the clinical laboratory in the Chinese PLA General Hospital to match each noninvasive SpHb measurement. The hematology analyzer used in the experiment was calibrated and monitored daily to ensure its performance.

2.4. Statistical Analysis

Data were presented as mean ± standard deviation (SD) for continuous descriptive variables. Accuracy of SpHb detected by the developed PPG system was assessed by computing bias and SD of each prediction method compared with traditional laboratory reference Lab_tHb at the same time point. Based on the absolute bias of SpHb and Lab_tHb, the outcome measures were subdivided into 5 groups of <0.5, 0.5 to 1.0, 1.1 to 1.5, 1.6 to 2.0, and >2.0 g/dL, and the group membership proportions were summarized with 95% confidence interval. Scatter plots and linear regression model were applied to compare between two continuous variables (Lab_tHb vs. SpHb_PLS and Lab_tHb vs. SpHb_BP). The beta coefficients and intercepts were estimated by the linear regression model, and the goodness of fit test and reference measurement regression line was quantified by the coefficient of determination (R2). Spearman’s correlation analysis was performed to assess the correlation between the invasive Lab_tHb and the two noninvasive SpHb values. Furthermore, the relationship among SpHb measurements and the absolute differences between tHb and SpHb and PI were also analyzed by Spearman’s correlation method. The coherence or agreement of the traditional laboratory invasive tHb and the presented noninvasive SpHb was further analyzed by plotting Bland–Altman. Two methods were considered to be in agreement if the majority of values fell within ±1.96 SD. The receiver operating characteristic curve (ROC) was constructed to test the predictive value of the noninvasive Hb measurements. The area under the ROC curve (AUCs) gave an estimate of the overall discriminate ability. All statistical analyses were two-sided and performed using SPSS version 20 software package (SPSS, Chicago, Ill, USA), with a value of indicating statistical significance.

3. Results

A total of 238 invasive and noninvasive Hb data pairs were collected from 238 ostensibly healthy volunteers. Of the 238 subjects, 145 (60.9%) were female and had an average age of 43 ± 25 years (Table 1). The mean Lab_tHb, SpHb_PLS, and SpHb_BP were 13.6 ± 1.8 g/dL (range 6.8 to 17.2 g/dL), 13.5 ± 1.1 g/dL (range 9.2 to 16.2 g/dL), and 13.6 ± 1.1 g/dL (range 9.4 to 16.3 g/dL), respectively. In addition, the bias estimated as the absolute mean of the paired differences for SpHb_PLS vs. tHb and SpHb_BP vs. tHb was for both 1.1 g/dL.

To demonstrate the range accuracy of the developed noninvasive Hb monitoring system, 238 paired absolute differences were divided into 5 subgroups according to their magnitude (Table 2). Table 2 showed that the range accuracy of observed absolute differences is similar in PLS predicted measurements to those calculated by BP_ANN method. Overall, 198 (83.2%) noninvasive SpHb measurements corresponded with invasive tHb values by <2.0 g/dL. 173 (72.7%) of SpHb_PLS values were <1.5 g/dL different from the tHb and the SpHb_BP is 168 (70.6%). The corresponding percentages of tHb values differing from SpHb measurements by >2.0 g/dL for both PLS and BP-ANN models were 16.8%.

As shown in Figure 4, Spearman correlation analysis linear regression model revealed a moderate positive relationship between invasive tHb and noninvasive SpHb (Lab_tHb vs. SpHb_PLS: R2 = 0.43, r = 0.61, ; Lab_tHb vs. SpHb_BP: R2 = 0.42, r = 0.62, ). The p-values for the correlation between tHb and SpHb for both models are less than the significance level of 0.05, which indicates that the correlation coefficients are significant. Figure 5 showed the Bland–Altman plot for gold standard invasive tHb and noninvasive SpHb measurements. The mean absolute bias was 0.04 g/dL for PLS model, with an upper agreement limit at 2.72 g/dL and a lower agreement limit at -2.64 g/dL. For the BP-ANN model, the mean absolute difference, upper agreement limit, and lower agreement limit were 0.02, 2.71, and −2.66 g/dL, respectively. In addition, 96.7% of readings fell within 2 SDs of the mean difference for SpHb_PLS measurements while that for SpHb_BP values is 97.1%.

Figure 6 demonstrated the ROC for the ability of the newly developed noninvasive Hb monitor to predict a measured Hb greater than 12 g/dL. The characteristics are good with AUCs of 0.83 and 0.82 for PLS and BP-ANN models, respectively. Furthermore, the Lab_tHb (), SpHb_PLS () and SpHb_BP (p = 0.017) were observed to increase significantly with increasing PI. However, the absolute differences of tHb and SpHb for PLS and BP-ANN methods were not significantly related to either PI or SpO2 (all ).

4. Discussion

The benefits of immediate, accurate, and noninvasive detection of Hb levels in human blood are obvious. Moreover, the development of a portable, miniaturized, and inexpensive system for Hb monitoring to avoid the need for blood sampling and to improve measurement accuracy has been the focus of this field. In the present study, a new noninvasive spectrophotometry-based Hb monitoring system was developed based on a probe sensor consisting of broadband (610–940 nm) light source and a matched photodiode array, a front-end signal acquisition module, and a signal processing module. PLS and BP-ANN were used for establishing Hb prediction models, with which the validation study was conducted based on 238 ostensibly healthy volunteers. The preliminary experimental results indicate that the newly developed system and established models enable noninvasive detection of human Hb concentration with similar accuracy and adaptability compared to existing commercial products.

The principles of continuous and noninvasive Hb detection used for the developed prototype in the present study are similar to those of SpO2 measurement by a conventional pulse oximeter [28]. Emissions from different wavelengths of LEDs are made to travel through the fingertip, and the transmitted light is captured with a photodiode sensor, which changed the light signal to electronic signal. Then the electrical signal is digitized and processed to obtain the corresponding target parameters (pulse wave, pulse rate, SpO2, etc.). The biggest difference between the developed prototype and a conventional pulse oximeter is that the prototype applies eight wavelengths of LED that pass through the fingertip, whereas a pulse oximeter for SpO2 measurement uses only two wavelengths. Furthermore, whether it is a commercial product or a prototype of laboratory research, the wavelengths of LED light sources for Hb monitoring devices are not exactly the same and have their own characteristics. According to the previous studies, the number of wavelengths for LEDs used for noninvasive Hb detection ranges from two to nine, while the wavelength range covers substantially 660 nm to 1348 nm [1119]. In fact, the choice of wavelength range is very important. Because it not only affects the capacity to identify the target molecule based on its known absorption characteristics but also influences optical path length due to scattering phenomena, which is associated with wavelength [29, 30]. In the present study, considering that the wavelength of the LED is easily affected by the temperature, the wavelength of the light source is preferably selected in a region where the Hb absorption coefficient is with flat changes. At the same time, in order to increase the sensitivity of Hb detection, the wavelength interval should be selected as small as possible in a region where the absorption coefficient changes drastically, and the wavelength interval can be slightly larger in a region where the variation is blunt. Therefore, we eventually developed the probe sensor, which consists of eight LEDs with wavelengths from 610 nm to 940 nm, is capable of deriving eight channels of PPG signals with sufficiently high speed, and is different from previous studies in the choice of wavelength [1119].

PLS regression and ANN have been used for noninvasive blood chemical measurements in recent years. Combined with principal component analysis and BP-ANN, Ding et al. obtained an optimal 9-11-1 (number of nodes for input layer, hidden layer, and output layer) network structure and proved that the spectrophotometric system they had developed was able to detect the Hb concentration noninvasively [17]. With extracted time-domain characteristics of PPG signal, Reşit Kavsaoğlu et al. used eight machine learning methods including PLS regression, generalized regression neural network, and support vector regression for the measurement of Hb level. According to the performance validation, the support vector regression was found to be the best machine learning method in the noninvasive prediction of Hb values based on the PPG signal [18]. Near-infrared transmission spectral analysis in conjunction with PLS regression calibration models was also applied in several studies to predict Hb concentrations in whole blood samples and the results were promising [18, 31, 32]. Similar to previous studies using spectrophotometry for noninvasive Hb measurement [17, 18], we have used the BP-ANN and multivariate statistical method of PLS regression. The nonlinear processing capability of BP-ANN is very strong but at the expense of computational complexity. In order to reduce the complexity of calculation, the input variables were reduced in dimension by principal component analysis. Then the number of nodes for the hidden layer in the present study was achieved and determined through a trial-and-error process to avoid the scenarios of underfitting and overfitting. Based on the training data set, the BP-ANN was trained using 10-fold cross-validation method and the lowest root mean square error corresponded to an optimum input layer of three nodes, hidden layer of six nodes, and an output layer of one node. As such, a network structure of 3-6-1 was applied for the BP-ANN model. The results show that the established model can predict the Hb concentration well. However, the BP-ANN model has the disadvantage of slow learning speed. In order to improve the accuracy of the calibration model, it is necessary to increase the sample size of the training set. PLS regression can perform regression modelling under the condition that there are serious multiple correlations of independent variables [22]. A similar prediction was achieved by the PLS model in our study considering proper characteristics selection while extracting the principal component. In addition, we also applied cross-validity to avoid overfitting in the process of extracting components, thereby improving the generalization of the established PLS model.

Previous studies have discussed the correlation and accuracy of continuous noninvasive Hb measurement. However, inconsistent results were obtained in these studies based on different types of noninvasive Hb monitoring devices and patient groups [3338]. Therefore, we may draw a preliminary conclusion that the bias between SpHb and tHb was small but the limits of agreement value were wide through meta-analyses. Similar to previous reports, the present study demonstrated the accuracy of the newly developed prototype with a bias close to 1.1 g/dL and a large limit of agreement higher than 2.0 g/dL in both prediction models. Huang et al. conducted a study to evaluate the accuracy of SpHb measured by Radical-7 in patients with liver transplantation. In addition to an insufficient trending ability, they found that the correlation was moderate (r = 0.59) but bias and limits of agreement were high (2.28 g/dL, -0.78 to 5.34 g/dL) between SpHb and tHb [39]. A recent study by Erdogan Kayhan et al. revealed that the absolute bias and limits of agreement between tHb and SpHb detected by Radical-7 in preanhepatic phase were 1.11 g/dL and −2.08 to 4.31 g/dL, respectively [40]. The accuracy of the prototype developed in our study is comparable to the aforementioned two recent reports, and the limits of agreement may even be better. Moreover, the ability of the prototype to predict high values of Hb (greater than 12 g/dL) was also good. However, we still should note that perfusion status, basal Hb concentration, and levels of SpO2 may affect the accuracy of SpHb measurement. Inconsistent findings including the present study may also be attributed to the different patient groups, laboratory reference, invasive blood samples, and types of sensors. Considering the Radical-7 and Pronto-7 monitors, the bias was reported to be negatively associated with PI, with thresholds of 2.0% and 1.4%, respectively [6, 41]. Although PI in the present study is positively associated with SpHb values detected by both models, the PI and SpO2 are no longer correlated to the bias. Along with the reduction in limits of agreement, this is an improvement related to the newly developed prototype compared to previous studies [6, 4143].

There are several limitations to the present study. First, the established models for Hb prediction were constructed with few subjects presenting extreme Hb concentrations. A separate analysis should be conducted in the anemia to improve the generalization of the prediction models. Second, the performance of the newly developed prototype and existing commercial products (Radical-7, Pronto-7, or NBM-200, etc.) was not compared in the same experiment, for the noninvasive Hb monitoring product was not available in our laboratory. Third, the range and sample size of subjects were limited and subsequently affected the predictive power and generalization of the calibrated models. Large sample size and multicenter studies recruiting different patient groups (different medical conditions such as anemia, trauma, and anesthesia) are warranted. We have developed a portable, miniaturized, and inexpensive spectrophotometric system along with methods for the noninvasive Hb prediction. Future comprehensive studies are needed to improve the performance of the system before it meets the requirements of clinical application.

5. Conclusions

A portable, miniaturized, and inexpensive spectrophotometric prototype has been developed based on a broadband light source composed of 8 LEDs for PPG signal acquisition with the wavelength range from 610 to 940 nm. PLS regression and BP-ANN were applied for establishing prediction models, with which the Hb levels of 238 healthy subjects were estimated by extracting features of multichannel PPG signals. The preliminary results indicate that the prototype can predict Hb values noninvasively with acceptable accuracy and satisfaction. Since it is a first stage to validate the developed instrument, there are still much to be done in the future to improve the overall performance of the prototype to achieve requirements for clinical application.

Abbreviations

Hb:Hemoglobin
tHb:Total hemoglobin
PLS:Partial least squares
BP-ANN:Backpropagation artificial neural network
PCA:Principal component analysis
ROC:Receiver operating characteristic curve
SpHb:Noninvasive hemoglobin
PPG:Photoplethysmography
LPF:Low pass filter
ADC:Analog to digital converter
MCU:Microcontroller unit
PI:Perfusion index
SpO2:Oxygen saturation.

Data Availability

The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Ethical Approval

We confirm that we have read the journal’s position on issues involved in ethical publication and affirm that this report is consistent with those guidelines.

All the authors have given consent for publication of this paper.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Hongyun Liu and Fulai Peng contributed equally to this work. HL and FP were responsible for developing the prototype and writing the manuscript; MH, JS, and HA conducted the clinical experiments; GW was responsible for data analysis; WW was involved in revising the manuscript critically for important intellectual content and have given final approval of the version to be published. All authors read and approved the final manuscript.

Acknowledgments

This study was supported by the National Key Technology R&D Program, Ministry of Science and Technology (2016YFC1305703, 2013BAI03B03, 2015BAI01B08), Chinese PLA General Hospital (2016-tm-042), and the National Natural Science Foundation of China (61372047, 61701540). The authors thank PLAGH for the support to carry out this research.