Research Article  Open Access
Suxian Cai, Shanshan Yang, Fang Zheng, Meng Lu, Yunfeng Wu, Sridhar Krishnan, "Knee Joint Vibration Signal Analysis with Matching Pursuit Decomposition and Dynamic Weighted Classifier Fusion", Computational and Mathematical Methods in Medicine, vol. 2013, Article ID 904267, 11 pages, 2013. https://doi.org/10.1155/2013/904267
Knee Joint Vibration Signal Analysis with Matching Pursuit Decomposition and Dynamic Weighted Classifier Fusion
Abstract
Analysis of knee joint vibration (VAG) signals can provide quantitative indices for detection of knee joint pathology at an early stage. In addition to the statistical features developed in the related previous studies, we extracted two separable features, that is, the number of atoms derived from the wavelet matching pursuit decomposition and the number of significant signal turns detected with the fixed threshold in the time domain. To perform a better classification over the data set of 89 VAG signals, we applied a novel classifier fusion system based on the dynamic weighted fusion (DWF) method to ameliorate the classification performance. For comparison, a single leastsquares support vector machine (LSSVM) and the Bagging ensemble were used for the classification task as well. The results in terms of overall accuracy in percentage and area under the receiver operating characteristic curve obtained with the DWFbased classifier fusion method reached 88.76% and 0.9515, respectively, which demonstrated the effectiveness and superiority of the DWF method with two distinct features for the VAG signal analysis.
1. Introduction
The knee is the largest and most complex joint in the human body [1]. A pair of knees support nearly the entire weight of the human body and help the body perform different locomotion functions. Knee osteoarthritis is a common form of rheumatic disorder caused by the degeneration or damage of articular cartilage in the knee joint. Detection of knee joint pathology at an early stage can help clinicians apply appropriate therapeutical or surgical procedures to retard the degenerative process in the affected knee joint [2, 3].
Arthroscopy is usually performed as a semiinvasive surgical procedure for knee joint disorder detection [1]. Physicians can inspect the interior of a joint with an arthroscope fiber that is inserted into the knee through a small incision. Although arthroscopy is often used as the gold standard for relatively lowrisk assessment of joint surfaces [4], it cannot be applied to patients whose knees are in a highly degenerated state due to arthritis, ligamentous instability, meniscectomy, or patellectomy. Furthermore, arthroscopy is not suited for routine examinations of the articular cartilage because the same incision may not undergo repeated arthroscope fiber invasions due to bacterial infection.
Magnetic resonance imaging can assist in the characterization of in vivo orthopaedic condition of articular cartilage and is also a popular noninvasive method for the assessment of knee joint degeneration [5]. The weakness of the magnetic resonance imaging is that such a technique is only able to display anatomical morphology [6]. The imaging protocol cannot support functional condition detection of the knee joint during leg movement, because the subject has to lay down throughout the magnetic resonance scanning procedure.
Vibration arthrometry is an alternative technology for noninvasive detection of knee pathologies [4, 7]. The knee joint vibration arthrographic (VAG) signal can be recorded by accelerometer or electrostethoscope sensors attached on the surface of the knee cap [8–10]. For the healthy adults, their articular surfaces in the knee are smooth and slippery without any cartilage friction or collision. However, the vibrations generated due to the friction between the degenerative articular cartilages are expected to present anomalous patterns in the amplitude and frequency scales [2]. The vibration arthrometry not only provides a clinical option of the noninvasive and lowcost routine detection for knee joint disorders but also supports the functional study on the knee joint during leg movements [1].
Computeraided analysis of knee joint VAG signals is very useful for screening and monitoring of articular cartilage disorders at an early stage [11–13]. Based on the noninvasive detection results, the computational algorithms may effectively help the medical experts make an accurate decision, so that the frequency of the diagnostic open surgery with arthroscope can be reduced [8, 14–16]. Adaptive filtering techniques based on the leastmeansquare (LMS) and recursive leastsquares lattice (RLSL) algorithms were used to remove muscle contraction interference present in VAG signals [17]. Tavathia et al. [18] and Moussavi et al. [19] proposed different linear prediction models and adaptive segmentation methods for parameterization and screening of VAG signals. Jiang et al. [20] extended the application of vibration arthrometry to artificial knee joints in vitro and analyzed the VAG signal with the rootmeansquared (RMS) value and the parameters of an autoregressive (AR) model. Matching pursuit (MP) timefrequency distributions (TFDs) of VAG signals were studied by Krishnan et al. [9], and a modified local discriminant bases algorithm was proposed by Umapathy and Krishnan [14] to distinguish the abnormal VAG signals from the normal ones. In order to simplify the procedures of signal processing and decision making, Rangayyan and Wu [11, 12] proposed the statistical parameters including form factors, skewness, kurtosis, probability density function entropy, variance of meansquared values, and turns count with adaptive threshold, for the screening of VAG signals based on the radialbasis function network (RBFN). Mu et al. [21] used the linear and nonlinear strict 2surface proximal classifiers to test a subset of the aforementioned statistical parameters. In this paper, the number of atoms derived from the wavelet MP decomposition and the turns count detected with the fixed threshold in the waveform variability analysis are extracted as features, and a classifier fusion system based on the dynamic weighted fusion (DWF) method is proposed for the classification of the VAG signals.
2. Data Set
The VAG data were collected by the research group of Rangayyan, University of Calgary, Canada [9]. The experimental protocol was approved by the Conjoint Health Research Ethics Board of the University of Calgary. Each subject was requested to sit on a rigid table in a relaxed position with the leg being tested freely and suspended in midair. The knee joint vibration was measured by placing a miniature accelerometer at the middle position of the patella [8]. Minute electrical charges were generated by the accelerometer based on the acceleration and deceleration of the knee movement when the subject swung the leg over an approximate angle range of 135° to 0° and back to 135° in the duration s. Each VAG signal was conditioned by an isolation preamplifiers to prevent the aliasing effects. The signal was then amplified and digitized with a data acquisition board and the National Instruments LabVIEW software at the sampling rate kHz with 12bit resolution per sample. Auscultation of the knee joint using a stethoscope was also performed, and a qualitative description of the sound intensity and type was recorded.
In the present study, we used a total of 89 VAG signals (recorded from 51 healthy volunteers and 38 subjects with knee joint pathologies), the same as investigated in a few previous related studies [11, 12, 14]. The normal signals were recorded from the healthy subjects identified by physical examinations. The abnormal signals, an example of which is shown in Figure 1(b), were collected from symptomatic patients scheduled to undergo arthroscopic examinations independent of the VAG studies. The knee joint disorders were associated with chondromalacia of different grades, meniscal tear, tibial chondromalacia, and anterior cruciate ligament injuries, which were confirmed in arthroscopy. Compared with the normal VAG signal displayed in Figure 1(a), we can observe that the abnormal signal exhibits a higher degree of variability in the time domain, as illustrated in Figure 1(b).
(a)
(b)
3. Feature Description
3.1. Number of Wavelet Matching Pursuit Decomposition Atoms
VAG signals are nonstationary in nature, that is, such signals exhibit timevarying spectral characteristics, and they cannot be accurately represented by common signal processing techniques such as Fourier transform and AR modeling [9]. It is therefore better to use a joint timefrequency approach for VAG analysis. For timefrequency representation of a signal, the methodology of MP introduced by Mallat and Zhang [22] is able to decompose the signal using basis functions with good timefrequency properties which are referred to as atoms.
The MP method is a socalled “greedy” algorithm that successively approximates a signal of samples with orthogonal projections onto elements from a waveform dictionary of vectors, in which . Gabor function, local cosine trees, and wavelet packets are often applied to build up dictionaries for MP applications. In this investigation, we implemented the Daubechies wavelet MP decomposition [23], because the Daubechies wavelets are a family of orthogonal wavelets that have a support of minimum size for a given number of vanishing moments [24], and such wavelets can be used for signal decomposition with excellent time and scale properties [25]. The projection of VAG signal using the dictionary of wavelet packet bases, , calculated with a Daubechies 8 (db8) filter can be formulated as where are the expansion coefficients and denotes the iterations of decomposition. And the wavelet MP decomposition can be implemented as follows. In the beginning, is projected on a vector , and the residue is computed, that is, where denotes the inner product (projection). Since the first atom is orthogonal to , we have In order to minimize , is chosen such that is maximum, that is, The MP iterates this procedure by subdecomposing the residue. And the VAG signal after iterations of decomposition is then expressed as where and . Since the residue term can be regarded as noise after sufficient iterations, one common approach to stop the iterative process depends on the convergence of residual energy, . In the previous related studies [9, 26], Krishnan et al. used the decay parameter defined by Mallat and Zhang [22] to end the MP decomposition based on the Gabor function dictionary. Although the decay parameter as an iterative indicator is well devised in theory, for a wavelet packet dictionary that contains vectors, each MP iteration then requires operations. For the VAG data analyzed, each signal consists of samples, such that the wavelet MP decomposition is computationally expensive.
In the present study, we propose using a signaltonoise ratio (SNR) as the alternative indicator to determine the iterations of the wavelet MP decomposition. According to the VAG signals in Figure 1, it can be inferred that, with a given SNR, the number of the wavelet MP decomposition iterations for a normal signal will be fewer than that for an abnormal signal, because the abnormal signal is much more noisy and also contaminated by a larger amount of artifacts such as muscle contraction interference. Thus, the number of MP iterations can be considered as a potential feature for classification applications. On the other hand, the MP decomposition with many great iterations can provide an excellent value of SNR and is also suited for denoising applications [26], but such an implementation is time consuming as mentioned above. It is therefore necessary to search for an appropriate value of SNR that makes a tradeoff between efficiency and effectiveness of the wavelet MP decomposition. After testing the SNR with different values, we found that the SNR of 15 dB could be an excellent indicator to determine the wavelet MP iterations. The value (0.0002) of the number of atoms (Natom) obtained with the Student’s ttest [27] indicated that the Natom values (numerically equal to the number of decomposition iterations ) were significantly different between the normal and abnormal VAG signals. Such results also confirm our assumption mentioned above.
From Figure 2, we can observe that the much noise has been reduced in the normal and abnormal VAG signals, reconstructed with 145 and 784 MP atoms, respectively. The results of noise removal in the VAG signals, as depicted in Figure 2, are even better than those in the previous study using the Gabor function dictionary and the energy decay parameter [26].
(a)
(b)
3.2. Turns Count with the Fixed Threshold
Besides the timefrequency MP decomposition, the waveform variability analysis in the time domain may be useful for classification as well. According to Rangayyan [28], a signal sample can be identified as a “turn” if it satisfies the following two conditions at the same time: (1) it represents a change in direction in the signal, that is, an alteration in the sign of the derivative (from positive to negative or vice versa); (2) the difference between its amplitude and that of the preceding sample is over a certain threshold. Willison [29] used the turns count method to analyze the electromyographic signal. The experiments showed that the electromyographic signal recorded from a patient with myopathy usually possesses more turns than the signal of a healthy subject at a comparable level of volitional effort. In the present work, we first normalized each VAG signal in the amplitude range from zero to unity, the same as in the recent studies [11, 12]. In each signal, the amplitude of all samples was amplified with the same scale, so that the variability information of the signal can be preserved. Before applying the turns count method in the signal, we implemented a filtering procedure using a 10thorder lowpass Butterworth filter (cutoff frequency: 50 Hz) with unit gain at direct current (DC) [23]. This lowpass Butterworth filter causes a delay of 100 samples (or 0.05 s), which was calibrated after the filtering procedure in our experiments. The reason that we used the lowpass Butterworth filter instead of the signal reconstructed with the MP atoms, as shown in Figure 2, is that the MP method is unable to eliminate the artifacts such as the interference caused by muscle contractions or 50 or 60 Hz powersupply lines.
In the past work of Rangayyan and Wu [12], the threshold to determine the significance of a turn was adaptively set to be , where denotes the standard deviation of the VAG signal analyzed. Although the turns detected with the adaptive threshold provide good discriminant information for classification of VAG signals, the number of turns counted from a normal signal is larger than that of an abnormal one, because the standard deviation of the normal VAG signal is usually smaller than that of the abnormal signal [12]. Such a result, however, somewhat deviates from our expectation that the turns associated with an abnormal VAG signal would be larger in number due to a higher degree of variability. In the present study, we fixed the amplitude threshold at 0.2 to compute the turns over the normalized and filtered VAG signals.
Figure 3 shows the results of the turns count with the fixed threshold (TCFT) method for the VAG signals in Figure 1, which were normalized and processed with the Butterworth filter. We can observe that more significant turns have been identified in the abnormal signal, as marked in Figure 3(b), in comparison with the normal signal shown in Figure 3(a). In addition, the value of the TCFT obtained with the Student’s ttest is 0.0013 (significance level: ), which indicates a significant difference between the normal and abnormal signals.
(a)
(b)
3.3. Statistical Features
In addition to two aforementioned features, we also considered the other five features extracted from the same VAG data set in our previous work [11–13, 30]. These features included the form factors computed for the first half (FF1) and the second half (FF2) of each VAG signal [11]; the variance of the meansquared value (VMS) of the each signal [12]; the mean value () of the Parzenwindow probability density function of each signal [13]; the fractal dimension (FD) estimated by the power spectral analysis [30]. Total seven features were combined in the vector form for the following pattern analysis task.
4. Classification
To perform the signal classifications, we applied a single leastsquares support vector machine (LSSVM) and the ensembles of several component LSSVM classifiers, the details of which are presented as follows.
4.1. LeastSquares Support Vector Machine
The support vector machine (SVM) proposed by Cortes and Vapnik [31] is a type of universal approximator, the learning of which follows the structural risk minimization criterion [32]. To optimize the SVM model parameters, a subset of the representative training data is selected to be the support vectors, which are considered to be the most informative for the classification task. By choosing the nonlinear innerproduct kernels in the network, the SVM is able to perform the same function as the polynomial learning machine, radial basis function network, or multilayer perceptron with a single hidden layer [33, 34]. The LSSVM was proposed by Suykens et al. [35] as a reformulation to the standard SVM, with an improvement of the moderate complexity. The learning of the LSSVM is implemented by minimizing a regularized leastsquares cost function with equality constraints, under the KuhnTucker condition [36]. Recently, the LSSVM has also been widely used in a number of biomedical applications [37–39].
To determine the kernel function suited for the VAG signal classification, we implemented the LSSVM using the linear, polynomial, sigmoid, and Gaussian kernels, one by one specified by different model parameters, and then evaluated each LSSVM with the leaveoneout (LOO) method. As a type of crossvalidation approach, the LOO partition procedure repeatedly used each signal once for the validation and the remaining signals for training [34]. By checking the accuracy and the optimal separating hyperplane provided by each LSSVM, we chose the polynomial kernel function, the degree and intercept parameters of which equal to 2 and 1, respectively, and set the regularization parameter of the LSSVM to be 5.
4.2. Ensemble of LSSVMs with the Bagging Algorithm
As an emerging machine learning methodology based on the “divide and conquer” principle [33], ensemble of classifiers was widely used in the literature [40–44], with the aim to achieve a better performance versus a single classifier. By combing a finite number of component neural networks (CNNs) with a welldevised combination rule [41–43] or fusion strategy [44–46], a neural network ensemble is expected to provide an informative overall decision that is supposedly superior to that attained by any one of the CNNs acting solely [33].
The most popular ensemble algorithms are AdaBoost [47] and Bagging [48]. The AdaBoost works by repeatedly training a given type of weaklearning machine from different distributed training data sets and then combining their outputs. The distribution of training data for the current CNN is boosted depending on the performance of previous CNNs, that is, the training data that are incorrectly predicted by previous CNNs will be chosen with priority to train the current CNN. In spite of the effectiveness, the AdaBoost is very sensitive to outliers and sometimes results in overfitting [49]. On the other hand, the Bagging algorithm introduces the bootstrap approach [50] into the training data resampling procedure [48] and aggregates the CNNs with the simple average strategy [33]. In the bootstrap procedure, each data sample was selected separately at random from the original data set such that a particular data sample could appear multiple times in a bootstrapgenerated data set. The bias of the Bagging ensemble would converge by averaging, while the variance falls much smaller than that of each CNN.
Since the LSSVM is not a weaklearning machine [35], we used the Bagging algorithm rather than the AdaBoost for the ensemble of 5 component LSSVMs (CSVMs) which were labeled from CSVM1 to CSVM5 in numerical sequence. The CSVMs combined with the Bagging algorithm were trained by different bootstrapgenerated data sets. The number of signals in each bootstrapgenerated data set was of equal size to the original VAG data set. The testing data set for each CSVM was the same as the original VAG data set. Because the training data for each CSVM were generated using the bootstrap approach, it is not necessary to apply the LOO method to the ensemble system any more.
4.3. Ensemble of LSSVMs with the Dynamic Weighted Fusion Rule
According to the linear combination rule of the Bagging, the CSVMs are simply averaged in the ensemble, so that the effectiveness of the ensemble would be affected by some of the CSVMs with poor performance, because the simple average strategy treats all the CSVMs equally. With the aim to utilize the diverse knowledge generated by the CSVMs, we applied a dynamic weighted fusion (DWF) rule to adaptively combine the CSVMs in the classifier fusion system [51].
Suppose that a total of CSVMs are linearly combined in the classifier fusion system. The local decision generated by the th CSVM is denoted as , with regard to the feature vector of the th VAG signal, . The classifier fusion system then provides the overall classification decision, , by linearly combining the CSVMs with the weights that are varied from one signal to another. Thus, the DWFbased ensemble output can be formulated as
The nonnegative and normalization constraints on the fusion weights, as widely accepted in the literature [41, 43, 45], can be written as
The task of the DWF is to determine the fusion weights that help the ensemble system provide an overall classification decision with higher accuracy. To achieve this goal, let us study the error term of the CSVMs and the DWFbased ensemble. Concerning the kth CSVM, the squared error that characterizes the difference between the local decision and the desired class label, , in relation to the th VAG signal is
Then, the squared error of the ensemble, , is estimated in an analogous manner to that of each CSVM. Consider that the fusion weights are normalized, as presented in (7), the class label can be split by multiplying the fusion weights, so that is derived as follows: where represents the instantaneous error of the th CSVM.
Considering (7) and (9), the minimization of the squared error of the ensemble is equivalent to the constrained quadratic programming (CQP) problem specified as follows:
In order to solve the CQP problem presented in (10), we applied the Lagrange multiplier method [36] and defined the cost function as where the nonnegative coefficient represents the Lagrange multiplier, which varies from one signal to another.
According to the weak Lagrangian principle [36], the optimum solution to the CQP problem, , is the stationary point of the cost function presented in (11) and satisfies the following unique equations:
The optimal weights of the DWF that minimize the squared error of the ensemble system can be obtained by solving (12), that is,
Because the error term of the CSVM can be estimated when the th VAG class label is given and the CSVM model parameters are specified, the optimal fusion weights can be directly computed according to (13).
Now let us divert our attention to the DWFbased ensemble error term. Considering (9) and (13), we have
It is clear that both of and are nonnegative, that is, and . To compare with the squared error of the CSVM, we may employ the division operator such that
Therefore, the optimal fusion weights derived in (15) guarantee that the DWFbased ensemble system more or less likely outperforms a single CSVM.
For a fair performance comparison, the CSVMs combined by the DWFbased ensemble were with the same bootstrapgenerated training data and model parameters as those CSVMs combined by the Bagging algorithm. The testing data for the CSVMs were the entire data set of the VAG signals.
In addition, we also selected some subsets of fixed size 15 to train the CSVMs, and would like to evaluate the generalization capability of the proposed DWFbased ensemble in the case of smallsize training data. The subset for each CSVM was actively selected according to the quadratic Renyi entropy maximization criterion [52] as follows. In the experiments, we first randomly partitioned the entire VAG data set into two subsets: the working subset containing 15 VAG signals and the candidate subset containing the remaining 74 signals. In each iteration step, we randomly selected one VAG signal from the candidate subset to replace an arbitrary signal in the working subset. If the quadratic Renyi entropy of the new working subset increases, then the working signal should be replaced by the candidate signal ; otherwise, remains in the working subset, and the candidate signal should be rejected and returned to the candidate subset [35]. After a few iteration steps, the quadratic Renyi entropy would become stable when reaching the maximum value, then the active selection of the fixedsize subset is terminated. The subsets for the 5 CSVMs were selected independently, and the corresponding quadratic Renyi entropy values with respect to the iteration steps are showed in Figure 4. After the model parameters of the CSVMs were optimized by the corresponding subsets of the fixed size, the original VAG data were input to the CSVMs and the DWFbased ensemble for testing and comparison purpose.
5. Classification Results
The classification accurate rate in percentage of the single LSSVM with the LOO method (LSSVM/LOO) was 87.64%, which was better than the result (accuracy: 61.8%) obtained by the Fisher’s linear discriminant analysis using the same feature set. In addition, the receiver operating characteristic (ROC) curve technique was implemented to test the overall diagnostic performance for all classification methods. According to Table 1, the area under the ROC curve () obtained with the LSSVM/LOO was 0.7523 with a standard error (SE) of 0.0536, as illustrated in Figure 5.

(a)
(b)
In the ensemble experiments, the highest accuracy provided the CSVMs was 83.15%, whereas the lowest one was only 76.41%. Despite that the value of the CSVM4 was slightly smaller than that of the LSSVM/LOO, any of the other four CSVMs outperformed the LSSVM/LOO with higher values under the ROC curves (see Figure 5).
Referring to Table 1 and Figure 5, the superiority of the DWFbased ensemble in the diagnostic performance was prominent. The DWFbased ensemble provided the highest overall accuracy of 88.76%, the best value of 0.9515, and the lowest SE of 0.0244. It can be observed from Figure 5 that the ROC curve obtained with the DWFbased ensemble was consistently over either of that provided by the Bagging or the single LSSVM/LOO classifier. In comparison with the single LSSVM/LOO classifier, the Bagging ensemble can improve the diagnostic ROC curve, but its accurate rate was even 8.99% lower than that of the LSSVM/LOO classifier.
Regarding the experiments with smallsize training data input, the CSVMs optimized by the training data of fixed size 15 provided relatively poor diagnostic results on the testing data (the entire VAG data set), as depicted in Figure 6. The highest value produced by CSVM4 was only 0.692 (SE: 0.0571), and the worst value produced by CSVM2 was 0.5026 (SE: 0.0637), the latter of which was just slightly better than a wild guess. However, the DWFbased ensemble was not affected by its CSVMs which had difficulty in the case of lacking enough training data. The value under the ROC curve provided by the DWFbased ensemble still reached as high as 0.9494, with the SE value of 0.0345. Such results indicated that the DWFbased ensemble had a better generalization capability when coping with the training data of smaller size.
6. Discussion
The Natom and TCFT features derived in the present study indicated the significance of difference () between the normal and abnormal VAG signals, and helped the classifiers improve their diagnostic performance. In the previous work [12], the signal turns were adaptively determined by the threshold that was a half of the standard deviation of the VAG signal analyzed. As the variance of an abnormal VAG signal is commonly larger than that of a normal signal, the number of signal turns detected from the abnormal signal is smaller, which cannot describe the oscillations in the abnormal signal. To overcome such a drawback, the lowpass Butterworth filter and the fixed threshold method were introduced in the present study. With the fixed threshold, more signal turns would be detected from the abnormal VAG signals, which revealed the degree of higher variability in amplitude resulting from different knee joint disorders. Compared with the adaptive threshold, the fixed threshold could help physicians establish a straightforward but effective discriminant criterion in knee joint VAG signal monitoring.
Regarding the signal classification, the results of the single LSSVM/LOO are much better than the previous studies using the logistic regression analysis with AR coefficients as features (accuracy: 68.9%) [17]; or with the energy, energy spread, frequency, and frequency spread features derived from the Gabor MP method (accuracy: 68.9%, : 0.68) [9]. The improvement of classification accuracy was largely contributed by the Natom () and TCFT () features developed in the present study.
Although the accurate rate of the Bagging 8.99% was lower than the LSSVM/LOO, the ensemble provided an of 0.8483, which was much larger than that of the LSSVM/LOO (: 0.7523), which implied that the Bagging performed better on the prediction of the true positive (abnormal) signals associated with the knee joint disorders. On the other hand, it is worth noting that the Bagging ensemble did not outperform the five CSVMs. The Bagging ensemble was worse in accurate rate than either CSVM2 or CSVM5, and its value was lower than that of CSVM1. The reason why the Bagging did not achieve the better performance may be related to the robustness of the LSSVM. According to the remarks of Breiman’s work [48], “Bagging stable classifiers is not a good idea”, because the Bagging ensemble with stable classifiers can only slightly improve the accuracy, but leading to more computational complexity.
The DWFbased ensemble produced higher classification accuracy and better ROC diagnostic performance than any of the CSVMs, along with the Bagging ensemble and the single LSSVM/LOO. The ROC curve result of the DWFbased ensemble is consistently better than that of the RBFN classifier with the features of form factors, skewness, kurtosis, and entropy (: 0.8172) [11] or with the features of variance of meansquared values and turns count with adaptive threshold (: 0.9174) [12] in the recent related studies. In addition, the DWF method is also comparable to the nonlinear strict 2surface proximal classifier (: 0.95) proposed by Mu et al. [21]. The fusion weights in the DWFbased ensemble were dynamically optimized, which guaranteed the superiority of diagnostic performance.
7. Conclusion
Analysis of VAG signals using advanced digital signal processing and pattern recognition techniques is able to provide distinct indicators of degenerative articular cartilage surfaces, and has high potential for noninvasive detection of knee joint pathology [53, 54]. In the feature extraction experiments, the features of Natom and TCFT, respectively, derived from the timefrequency wavelet MP decomposition and timedomain signal variability analysis are separable with significant values. Using these features, the classification by means of the LSSVM/LOO is superior to the logistic regression analysis used in the previous studies. In addition, we utilized the ensembles of classifiers to effectively ameliorate the overall classification performance. Compared with the most popular Bagging algorithm, the DWFbased ensemble used in the present study can significantly improve the classification accuracy and the ROC curve with higher and lower SE values, over the entire VAG data set.
Acknowledgments
The authors would like to thank Dr. Ranagaraj M. Rangayyan, Dr. Cyril B. Frank, and Dr. Gordon D. Bell from University of Calgary, for the work of data acquisition. This work was supported in part by the National Natural Science Foundation of China under Grant no. 81101115, the Natural Science Foundation of Fujian Province of China under Grant no. 2011J01371, the Fundamental Research Funds for the Central Universities of China under Grant no. 2010121061, the Natural Sciences and Engineering Research Council of Canada (NSERC), and the Canada Research Chairs Program. The authors are also grateful to the anonymous reviewers for the constructive comments. The preliminary works were presented at the 16th International Conference on Digital Signal Processing and the 2009 International Conference on Computational Intelligence for Measurement Systems and Applications.
References
 Y. Wu, S. Krishnan, and R. M. Rangayyan, “Computeraided diagnosis of kneejoint disorders via vibroarthrographic signal analysis: a review,” Critical Reviews in Biomedical Engineering, vol. 38, no. 2, pp. 201–224, 2010. View at: Google Scholar
 C. B. Frank, R. M. Rangayyan, and G. D. Bell, “Analysis of knee joint sound signals for noninvasive diagnosis of cartilage pathology,” IEEE Engineering in Medicine and Biology Magazine, vol. 9, no. 1, pp. 65–68, 1990. View at: Publisher Site  Google Scholar
 V. J. Vigorita, Orthopaedic Pathology, Lippincott Williams and Wilkins, Philadelphia, PA, USA, 1999.
 G. F. McCoy, J. D. McCrea, and D. E. Beverland, “Vibration arthrography as a diagnostic aid in diseases of the knee. A preliminary report,” Journal of Bone and Joint Surgery B, vol. 69, no. 2, pp. 288–293, 1987. View at: Google Scholar
 B. J. Manaster, J. Crim, and Z. S. Rosenberg, Diagnostic and Surgical Imaging Anatomy: Knee, Ankle, Foot, Lippincott Williams and Wilkins, Philadelphia, PA, USA, 2007.
 F. Eckstein, C. Adam, H. Sittek et al., “Noninvasive determination of cartilage thickness throughout joint surfaces using magnetic resonance imaging,” Journal of Biomechanics, vol. 30, no. 3, pp. 285–289, 1997. View at: Publisher Site  Google Scholar
 W. G. Kernohan, D. A. Barr, G. F. McCoy, and R. A. B. Mollan, “Vibration arthrometry in assessment of knee disorders: the problem of angular velocity,” Journal of Biomedical Engineering, vol. 13, no. 1, pp. 35–38, 1991. View at: Publisher Site  Google Scholar
 R. M. Rangayyan, S. Krishnan, G. D. Bell, C. B. Frank, and K. O. Ladly, “Parametric representation and screening of knee joint vibroarthrographic signals,” IEEE Transactions on Biomedical Engineering, vol. 44, no. 11, pp. 1068–1074, 1997. View at: Publisher Site  Google Scholar
 S. Krishnan, R. M. Rangayyan, G. D. Bell, and C. B. Frank, “Adaptive timefrequency analysis of knee joint vibroarthrographic signals for noninvasive screening of articular cartilage pathology,” IEEE Transactions on Biomedical Engineering, vol. 47, no. 6, pp. 773–783, 2000. View at: Google Scholar
 S. Krishnan, R. M. Rangayyan, G. D. Bell, and C. B. Frank, “Auditory display of kneejoint vibration signals,” Journal of the Acoustical Society of America, vol. 110, no. 6, pp. 3292–3304, 2001. View at: Publisher Site  Google Scholar
 R. M. Rangayyan and Y. F. Wu, “Screening of kneejoint vibroarthrographic signals using statistical parameters and radial basis functions,” Medical and Biological Engineering and Computing, vol. 46, no. 3, pp. 223–232, 2008. View at: Publisher Site  Google Scholar
 R. M. Rangayyan and Y. Wu, “Analysis of vibroarthrographic signals with features related to signal variability and radialbasis functions,” Annals of Biomedical Engineering, vol. 37, no. 1, pp. 156–163, 2009. View at: Publisher Site  Google Scholar
 R. M. Rangayyan and Y. Wu, “Screening of kneejoint vibroarthrographic signals using probability density functions estimated with Parzen windows,” Biomedical Signal Processing and Control, vol. 5, no. 1, pp. 53–58, 2010. View at: Publisher Site  Google Scholar
 K. Umapathy and S. Krishnan, “Modified local discriminant bases algorithm and its application in analysis of human knee joint vibration signals,” IEEE Transactions on Biomedical Engineering, vol. 53, no. 3, pp. 517–523, 2006. View at: Publisher Site  Google Scholar
 K. S. Kim, J. H. Seo, J. U. Kang, and C. G. Song, “An enhanced algorithm for knee joint sound classification using feature extraction based on timefrequency analysis,” Computer Methods and Programs in Biomedicine, vol. 94, no. 2, pp. 198–206, 2009. View at: Publisher Site  Google Scholar
 Y. Wu and S. Krishnan, “Combining leastsquares support vector machines for classification of biomedical signals: a case study with kneejoint vibroarthrographic signals,” Journal of Experimental and Theoretical Artificial Intelligence, vol. 23, no. 1, pp. 63–77, 2011. View at: Publisher Site  Google Scholar
 S. Krishnan, R. M. Rangayyan, G. D. Bell, C. B. Frank, and K. O. Ladly, “Adaptive filtering, modelling and classification of knee joint vibroarthrographic signals for noninvasive diagnosis of articular cartilage pathology,” Medical and Biological Engineering and Computing, vol. 35, no. 6, pp. 677–684, 1997. View at: Google Scholar
 S. Tavathia, R. M. Rangayyan, C. B. Frank, G. D. Bell, K. O. Ladly, and Y. T. Zhang, “Analysis of knee vibration signals using linear prediction,” IEEE Transactions on Biomedical Engineering, vol. 39, no. 9, pp. 959–970, 1992. View at: Publisher Site  Google Scholar
 Z. M. K. Moussavi, R. M. Rangayyan, G. D. Bell, C. B. Frank, K. O. Ladly, and Y. T. Zhang, “Screening of vibroarthrographic signals via adaptive segmentation and linear prediation modeling,” IEEE Transactions on Biomedical Engineering, vol. 43, no. 1, pp. 15–23, 1996. View at: Publisher Site  Google Scholar
 C. C. Jiang, J. H. Lee, and T. T. Yuan, “Vibration arthrometry in the patients with failed total knee replacement,” IEEE Transactions on Biomedical Engineering, vol. 47, no. 2, pp. 219–227, 2000. View at: Publisher Site  Google Scholar
 T. Mu, A. K. Nandi, and R. M. Rangayyan, “Screening of kneejoint vibroarthrographic signals using the strict 2surface proximal classifier and genetic algorithm,” Computers in Biology and Medicine, vol. 38, no. 10, pp. 1103–1111, 2008. View at: Publisher Site  Google Scholar
 S. G. Mallat and Z. Zhang, “Matching pursuits with timefrequency dictionaries,” IEEE Transactions on Signal Processing, vol. 41, no. 12, pp. 3397–3415, 1993. View at: Publisher Site  Google Scholar
 Y. Wu and S. Krishnan, “Classification of kneejoint vibroarthrographic signals using timedomain and timefrequency domain features and leastsquares support vector machine,” in Proceedings of the 16th International Conference on Digital Signal Processing (DSP '09), pp. 361–366, Santorini, Greece, July 2009. View at: Publisher Site  Google Scholar
 I. Daubechies, “Wavelet transform, timefrequency localization and signal analysis,” IEEE Transactions on Information Theory, vol. 36, no. 5, pp. 961–1005, 1990. View at: Publisher Site  Google Scholar
 S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, San Diego, CA, USA, 2nd edition, 1999.
 S. Krishnan and R. M. Rangayyan, “Automatic denoising of kneejoint vibration signals using adaptive timefrequency representations,” Medical and Biological Engineering and Computing, vol. 38, no. 1, pp. 2–8, 2000. View at: Google Scholar
 G. J. Hahn and S. S. Shapiro, Statistical Models in Engineering, Wiley, Hoboken, NJ, USA, 1994.
 R. M. Rangayyan, Biomedical Signal Analysis: A CaseStudy Approach, IEEE and Wiley, New York, NY, USA, 2002.
 R. G. Willison, “Analysis of electrical activity in healthy and dystrophic muscle in man,” Journal of Neurology, Neurosurgery, and Psychiatry, vol. 27, no. 5, pp. 386–394, 1964. View at: Google Scholar
 R. M. Rangayyan, F. Oloumi, Y. Wu, and S. Cai, “Fractal analysis of kneejoint vibroarthrographic signals via power spectral analysis,” Biomedical Signal Processing and Control, vol. 8, no. 1, pp. 23–29, 2013. View at: Google Scholar
 C. Cortes and V. Vapnik, “Supportvector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. View at: Publisher Site  Google Scholar
 V. N. Vapnik, Statistical Learning Theory, Wiley, New York, NY, USA, 1998.
 S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall PTR, Englewood Cliffs, NJ, USA, 2nd edition, 1998.
 R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, Wiley, New York, NY, 2nd ed edition, 2001.
 J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines, World Scientific Publishing, Singapore, 2002.
 S. G. Nash and A. Sofer, Linear and Nonlinear Programming, McGrawHill, Columbus, OH, USA, 1995.
 Y. Wu and S. Krishnan, “Computeraided analysis of gait rhythm fluctuations in amyotrophic lateral sclerosis,” Medical and Biological Engineering and Computing, vol. 47, no. 11, pp. 1165–1171, 2009. View at: Publisher Site  Google Scholar
 Y. Wu and S. Krishnan, “Statistical analysis of gait rhythm in patients with Parkinson’s disease,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 2, pp. 150–158, 2010. View at: Google Scholar
 Y. Wu and L. Shi, “Analysis of altered gait cycle duration in amyotrophic lateral sclerosis based on nonparametric probability density function estimation,” Medical Engineering and Physics, vol. 33, no. 3, pp. 347–355, 2011. View at: Publisher Site  Google Scholar
 L. K. Hansen and P. Salamon, “Neural network ensembles,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 10, pp. 993–1001, 1990. View at: Publisher Site  Google Scholar
 S. Hashem and B. Schmeiser, “Improving model accuracy using optimal linear combinations of trained neural networks,” IEEE Transactions on Neural Networks, vol. 6, no. 3, pp. 792–794, 1995. View at: Publisher Site  Google Scholar
 J. Kittler, M. Hatef, R. P. W. Duin, and J. Matus, “On combining classifiers,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 226–239, 1998. View at: Google Scholar
 N. Ueda, “Optimal linear combination of neural networks for improving classification performance,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 2, pp. 207–215, 2000. View at: Google Scholar
 L. I. Kuncheva, “A theoretical study on six classifier fusion strategies,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 2, pp. 281–286, 2002. View at: Publisher Site  Google Scholar
 Y. Wu, C. Wang, S. C. Ng, A. Madabhushi, and Y. X. Zhong, “Breast cancer diagnosis using neuralbased linear fusion strategies,” in Proceedings of the 13th International Conference on Neural Information Processing (ICONIP '06), vol. 4234 of Lecture Notes in Computer Science, pp. 165–175, Hong Kong, October 2006. View at: Publisher Site  Google Scholar
 A. Sinha, H. Chen, D. G. Danu, T. Kirubarajan, and M. Farooq, “Estimation and decision fusion: a survey,” Neurocomputing, vol. 71, no. 1315, pp. 2650–2656, 2008. View at: Publisher Site  Google Scholar
 Y. Freund and R. E. Schapire, “A DecisionTheoretic Generalization of OnLine Learning and an Application to Boosting,” Journal of Computer and System Sciences, vol. 55, no. 1, pp. 119–139, 1997. View at: Google Scholar
 L. Breiman, “Bagging predictors,” Machine Learning, vol. 24, no. 2, pp. 123–140, 1996. View at: Google Scholar
 G. Ridgeway, “The State of boosting,” Computing Science and Statistics, vol. 31, pp. 172–181, 1999. View at: Google Scholar
 B. Efron and R. Tibshirani, An Introduction to the Bootstrap, Chapman and Hall, New York, NY, USA, 1993.
 Y. Wu and S. Krishnan, “An adaptive classifier fusion method for analysis of kneejoint vibroarthrographic signals,” in Proceedings of the IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA '09), pp. 190–193, Hong Kong, May 2009. View at: Publisher Site  Google Scholar
 M. Girolami, “Orthogonal series density estimation and the kernel eigenvalue problem,” Neural Computation, vol. 14, no. 3, pp. 669–688, 2002. View at: Publisher Site  Google Scholar
 S. Cai, Y. Wu, N. Xiang et al., “Detrending knee joint vibration signals with a cascade moving average filter,” in Proceedings of the 34th Annual International Conference of IEEE Engineering in Medicine and Biology Society (EMBC '12), pp. 4357–4360, San Diego, CA, USA, August 2012. View at: Google Scholar
 Y. Wu, S. Cai, F. Xu, L. Shi, and S. Krishnan, “Chondromalacia patellae detection by analysis of intrinsic mode functions in knee joint vibration signals,” in IFMBE Proceedings of 2012 World Congress on Medical Physics and Biomedical Engineering (WC '12), vol. 39, pp. 493–496, Beijing, China, May 2012. View at: Google Scholar
Copyright
Copyright © 2013 Suxian Cai et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.