Simultaneous-Fault Diagnosis of Automotive Engine Ignition Systems Using Prior Domain Knowledge and Relevance Vector Machine

Abstract

Engine ignition patterns can be analyzed to identify the engine fault according to both the specific prior domain knowledge and the shape features of the patterns. One of the challenges in ignition system diagnosis is that more than one fault may appear at a time. This kind of problem refers to simultaneous-fault diagnosis. Another challenge is the acquisition of a large amount of costly simultaneous-fault ignition patterns for constructing the diagnostic system because the number of the training patterns depends on the combination of different single faults. The above problems could be resolved by the proposed framework combining feature extraction, probabilistic classification, and decision threshold optimization. With the proposed framework, the features of the single faults in a simultaneous-fault pattern are extracted and then detected using a new probabilistic classifier, namely, pairwise coupling relevance vector machine, which is trained with single-fault patterns only. Therefore, the training dataset of simultaneous-fault patterns is not necessary. Experimental results show that the proposed framework performs well for both single-fault and simultaneous-fault diagnoses and is superior to the existing approach.

1. Introduction

1.1. Background of Engine Ignition Patterns

Although automotive engine ignition systems vary in construction, they are similar in basic operation. All of them have a primary circuit that causes a spark in the secondary circuit, which is then delivered to the correct spark plug at the proper time. The conditions inside the ignition system and the cylinder also affect the ignition pattern in the secondary circuit. Consequently, the ignition patterns reflect the conditions within the ignition system and help pinpoint their faults [1], such as wide or narrow spark-plug gaps and open spark-plug cables. After capturing the ignition pattern, the automotive mechanic compares the features of the captured pattern with samples from handbooks for diagnosis [2, 3]. This procedure is called ignition system diagnosis. However, there are several challenges for the automotive mechanic which are as follows.(1)The engine ignition pattern is time dependent. Different engine models produce the ignition patterns of various amplitude and duration for the same kind of fault. Even for the same engine, it may produce slightly different shapes of ignition patterns for each engine cycle due to engine speed fluctuation and various testing conditions. Therefore, there is no exact scale and duration for sample patterns in the handbooks. Hence, the traditional diagnosis merely relies on prior domain knowledge and the engineer’s experience.(2)Practically, the engine ignition-system diagnosis is a simultaneous-fault problem, but many handbooks only provide single-fault patterns for reference. To determine simultaneous faults, the engineer can only extract and analyze some specific features of single-fault patterns from a simultaneous-fault pattern, such as frequency, firing voltage, and burn time, and make a decision about the presence of simultaneous faults according to their experience and knowledge.(3)As suggested in the existing literature [1–3], the ignition-system diagnosis based on the shape features and the prior domain knowledge of the ignition pattern cannot conclude a definite answer. It is because many possible faults may occur individually or simultaneously. The handbooks do not provide the rank of the probability of each possible fault. Therefore, to find out a fault based on ignition patterns, many trials for disassembling and assembling of engine parts are often necessary unless the engineer has very rich experience.

To tackle these challenges, an effective feature extraction method for engine ignition patterns is required, which combines domain knowledge (DK), time-frequency decomposition, and dimensional reduction techniques. Moreover, an advanced probabilistic classifier is necessary to provide the rank of each possible fault and reliable diagnostic results. In recent years, some intelligent diagnostic methods based on pattern recognition have been developed for multiclass fault diagnosis (i.e., single-fault diagnosis because only a single fault is identified) of mechanical systems [4–9]. Generally, these methods include two steps: feature extraction and classification.

1.2. Feature Extraction Methods

Feature extraction is very important because the in-depth and hidden features of single-fault patterns can be detected through frequency subband decomposition. Referring to the existing literature, many classical feature extraction techniques were applied to fault diagnosis; the most typical one is the fast Fourier transform (FFT) [10–13]. However, its main drawback is the unsuitability for nonstationary patterns. Wavelet packet transform (WPT) [1, 4, 14–19] is another popular time frequency localization analysis method that received a widespread utilization in the past decade. By means of multiscale analysis, WPT can be successfully applied to nonstationary patterns, based on subband coding and a systematic decomposition of a pattern into its subband levels for pattern analysis. Therefore, WPT is employed in this research for feature extraction.

Nevertheless, one drawback for WPT is that the size of the extracted features is larger or equal to that of its original pattern. If the original pattern is of a high dimension, there is a large amount of extracted features that may incur two issues: (1) the high complexity of the trained classifiers because of the huge amount of inputs; (2) there may be many redundant and unimportant extracted features so that noise can be induced. Both of the issues can degrade the classifier performance. Therefore, compensating the drawback by employing dimensional reduction technique such as principal component analysis (PCA) [20–22] is suggested. In this research, PCA is selected as the dimensional reduction technique for a simple illustration purpose. More advance techniques could be considered in the future. Compared to other dimensional reduction techniques, PCA has three advantages: (1) it has no hyperparameter; (2) PCA eliminates the interaction of variables because the principal components are independent of each other; (3) the principal components are sorted by their information weights, so some unimportant principal components can be further reduced. Then, the feature extraction approach of WPT+PCA can transform an original ignition pattern into a reduced dimensional feature vector while retaining most of the information content.

1.3. Classification Methods

For classification, a fault can be considered as a label, no matter whether it is a single fault or simultaneous fault. To date, there are only a few researches on simultaneous-fault diagnosis. The typical classification method for simultaneous-fault diagnosis is to build a number of classifiers according to the combination of all possible faults; this method is called monolabel classification [23]. However, it is practically difficult to obtain the training data of all possible combinations particularly for ignition patterns. Normally, the number of combination of all faults in an engineering problem is very large that affects the diagnostic accuracy because the complexity of the classifiers will also be immensely increased. Moreover, if a new single fault is added in the future, the number of required training simultaneous-fault patterns grows significantly. To overcome this drawback, Yélamos et al. [23] proposed a binarization strategy using support vector machine (SVM) and applied to simultaneous-fault diagnosis of a simulated chemical process based on time-independent data, in which the labels of the single faults or simultaneous faults were processed as binary vectors, that is, 0 or 1 only. For each label, a binary classifier was constructed using SVM with one-versus-all splitting strategy. Given an unknown pattern, the classifier would output a vector of binary results (0 or 1). From this approach, only single-fault patterns are used for training the classifiers while simultaneous-fault patterns are not necessary. The experimental results showed that the overall accuracy of their binarization approach is almost the same as that of the traditional monolabel approach. This kind of binarization approach sounds good but still suffers from several drawbacks: (1) the approach assumes that informative features are obvious and available that is not always the case for time-dependent signal patterns, so this approach cannot be suitable for ignition patterns; (2) the one-versus-all strategy ignores the pairwise correlation between the labels and hence the classification accuracy is mostly degenerated; (3) the approach only considers the presence of a fault, if its corresponding output is close to the classification margin which lacks confidence of correct classification, that is, the degree of belief of faults.

From the practical point of view, a proper classifier has to offer the probabilities of all possible faults. Then the user can at least trace the other possible faults according to the rank of their probabilities when the predicted fault(s) from the classifier is incorrect in the problem. Therefore, it is better to employ probabilistic classifier for simultaneous-fault diagnosis. The probabilistic structure is also suitable for the fault with uncertainty such as engine ignition-system diagnosis. Typically, probabilistic neural network (PNN) [24, 25] was employed as a probabilistic classifier. It was shown in [24] that the performance of PNN is superior to SVM based method for multilabel classification. However, the main drawback of PNN lies in the limited number of inputs because the complexity of the network and the training time are heavily related to the number of inputs. Recently, Widodo et al. [6] proposed to apply an advanced classifier, namely, relevance vector machine (RVM) to fault diagnosis of low speed bearings. They showed that RVM is superior to SVM in terms of diagnostic accuracy. Besides, RVM can also handle regression problem [26]. RVM is a statistical learning method proposed by Tipping [27], which trains a probabilistic classifier with sparser model using Bayesian framework. RVM can be extended to multiclass version using one-versus-all (1vA) strategy. However, this strategy was verified to produce a large region of indecision [28, 29]. In view of this drawback, this research is the first in the literature to incorporate pairwise coupling, that is, one-versus-one (1v1) strategy, into RVM, namely pairwise coupled relevance vector machine (PCRVM). As PCRVM considers the correlation between every pair of fault labels, a more accurate estimate of label probabilities for simultaneous-fault signals can be achieved.

1.4. Decision Threshold Optimization

If a probabilistic classification is applied to fault detection, the predicted fault is usually inferred as the one with the largest probability. The other alternative approach is that the probabilistic classifier ranks all the possible faults according to their probabilities and lets the engineer make a decision. These inference approaches work fine with single-fault detection but fail to determine which faults occur simultaneously in the simultaneous-fault problem. It is because the engineer cannot identify the number of simultaneous faults based on the output probability of each label. For instance, an output probability vector for five labels is given as [0.21,0.5,0.69,0.01,0.6]. In this example, it is difficult for the engineer to judge whether the simultaneous faults are labels 2, 3, and 5. To identify the number of simultaneous faults, a decision threshold must be introduced and thus a new step of decision threshold optimization is proposed in the current framework other than feature extraction and probabilistic classification.

1.5. Research Objectives and the Proposed Framework

Currently, very little research examines whether the features of single-fault ignition patterns can be reflected in the ignition patterns of some simultaneous faults. If it is feasible, some rational (not all) simultaneous faults are likely to be identified based on the prior domain knowledge and the features of single-fault ignition patterns. In other words, the features about the single faults in a simultaneous-fault pattern could be detected and then classified using the probabilistic classifier trained with the single-fault patterns only. Under this concept, the simultaneous-fault patterns are not necessary for training the classifiers. Once a new single fault is added in the future, the diagnostic system can be easily extended because the issue of combinatory single faults has been eliminated. To verify the feasibility and determine the best feature extraction method, this research proposes to extract the important knowledge-specific, time-domain, and frequency-domain features of the single-fault patterns using the combination of WPT+PCA, FFT, and DK. Then the pairwise coupled probabilistic classifier is trained using a training dataset of these extracted single-fault features in order to identify simultaneous faults for reasonable unseen patterns. Therefore, a feasibility study on this idea for simultaneous-fault diagnosis is an important contribution of this research. Another important contribution of the research is the reduction of required training patterns for simultaneous-fault diagnosis.

This paper is organized as follows. The proposed framework and the related techniques are described in Section 2. In Section 3, the experimental setup is presented, followed by the results and a comparison with latest approach [23] in Section 4 and discussion in Section 5. Finally, a conclusion is given in Section 6.

2. Proposed Framework and Related Techniques

The proposed diagnosis framework (Figure 1) includes three steps: feature extraction, classification, and threshold optimization. The framework is general so that different feature extraction, probabilistic classification, and threshold optimization techniques could be adopted. In this paper, FFT, WPT, and PCA are examined in the step of feature extraction and their detailed descriptions can be, respectively, found in [22, 30, 31]. In addition, these techniques are combined, respectively, with time-related domain knowledge (DK) for a comprehensive comparison.

Figure 1 

Proposed framework of the simultaneous engine ignition-fault diagnosis system and its evaluation.

2.1. Formulation of the Proposed Framework

Given a sample dataset 𝐃={(𝐱_𝑖,𝐥_𝑖)} of (single-fault or simultaneous-fault) patterns, 𝑖=1 to 𝑁_𝐷, 𝐱_𝑖∈𝐑^𝑛 and 𝐥_𝑖=[𝑙_𝑖1,𝑙_𝑖2,…,𝑙_𝑖𝑑] is a vector of labels of the corresponding single-fault pattern of 𝐱_𝑖 and 𝑑 is the number of single faults. Here there may be more than one fault in 𝐥_𝑖 so that ∑^𝑑_𝑔=1𝑙_𝑖𝑔≥1, 𝑙_𝑖𝑔∈{0,1} for 𝑔=1 to 𝑑. In Figure 1, the sample dataset is divided into three groups: training dataset, validation dataset, and test dataset where training dataset only involves single-fault patterns.

After applying feature extraction techniques to the patterns {𝐱_𝑖}, a set of feature vectors 𝐅={(𝐟_𝑖,𝐥_𝑖)} is produced. A training dataset of single-fault patterns only (no simultaneous-fault patterns are necessary) is selected to train a multilabel classifier 𝑓_class by using probabilistic classification algorithm. Then 𝑓_class takes an unknown feature vector 𝐟 as input and outputs a probability vector 𝝆=[𝜌₁,𝜌₂,…,𝜌_𝑑] where 𝑑 is the number of the single-fault labels. Here 𝜌_𝑗=𝑃(𝑙_𝑗∣𝐟)∈[0,1] denotes the probability that 𝐟 belongs to the 𝑗th label for  𝑗=1 to 𝑑. Since every 𝜌_𝑗 is an independent probability, Σ𝜌_𝑗 is not necessarily equal to one. At this stage, the diagnostic system can provide the probability vector 𝝆 to the user as a quantitative measure for reference and further use. Afterwards, the multilabel decision vector 𝐲=[𝑦₁,𝑦₂,…,𝑦_𝑑] is constructed from 𝝆 using (1): 𝑦_𝑗=𝜀𝜌_𝑗=1,if𝜌_𝑗≥𝜀,0,otherwise,for𝑗=1to𝑑, (1) where 𝜀∈(0,1) is a user-defined decision threshold and 𝑦_𝑗 indicates that 𝐟 belongs to the 𝑗th label or not (Figure 2). For example, if 𝜀=0.5 and 𝝆=𝑓_class(𝐟)=[0.72,0.42,0.82,0.28,0.86], then 𝐲=𝜀(𝝆)=[1,0,1,0,1]. Therefore, 𝐟 is diagnosed as a simultaneous-fault (1,3,5). Notice that 𝐲=[0,0,0,0,0] indicates that no fault has been found, and hence the unseen instance 𝐟 is diagnosed as a normal pattern.

Figure 2 

Decision function based on threshold 𝜀.

2.2. Extraction of Prior Domain Knowledge Features for Ignition Patterns

When an engine starts firing, its secondary coil produces a rapid high voltage to cause spark plug to produce spark. This high voltage is called the firing voltage. Then the spark voltage decreases until zero. The spark voltage represents the voltage required to maintain spark for the duration of the spark line. The duration is called the burn time. After the burn time, the energy in the ignition coil nearly exhausts, and the residual energy forms slight oscillation in the ignition coil. The entire procedure is shown in Figure 3. Using the ignition pattern to diagnose the engine fault is a common diagnostic method for automotive engineers. With reference to some handbooks [2, 3], the following prior domain knowledge for a pattern can be observed for engine fault diagnosis (Figure 3): (1)firing voltage (𝐹₁);(2)burn time (𝐹₂);(3)average spark voltage (𝐹₃).

Figure 3 

Key domain knowledge features of the normal engine ignition signal.

In this study, all patterns start from the firing voltage (𝐹₁) which is at the first sampling point: 𝐹₁=𝑥₁, (2) where 𝑥₁ is the voltage of the first sampling point. Ideally, the burn time (𝐹₂) starts from the spark voltage and ends at the position where the spark voltage falls to zero. However, in practice, the voltage could slightly oscillate after the burn time so that exact zero value may not be reached. In this study, when the voltage falls to 0.1% of the firing voltage, it is considered as zero and the burn time ends. The feature 𝐹₂ can be obtained as illustrated in Figure 4, where 𝑎 indicates the end point of burn time, and 𝐿_𝑃 is the length of patterns. With the index 𝑎 and time step 𝑞, the average spark voltage of the spark line (𝐹₃) can be calculated as follows: 𝐹₃=1𝑎^𝑎_𝑞=1𝑥_𝑞. (3)

Figure 4 

Procedure for calculating the burn time (𝐹2).

2.3. Feature Extraction Using WPT and PCA and Combined Feature Vector

WPT is a generalization of wavelet decomposition that offers a richer signal analysis [31]. It is well known that WPT can extract time-frequency features of a signal pattern. Given a set of patterns 𝐗={𝐱_𝑖}, 𝑖=1 to 𝑁_𝐷, WPT transforms an ignition pattern 𝐱_𝑖∈𝑅^𝑠 into a set of 2^𝐽 coefficient packets 𝑣_𝑖,𝑏∈𝑅^𝑚, and 𝑚 is the ceiling function of 𝑠/2^𝐽 at level 𝐽(𝑏=1 to 2^𝐽). Then, these packets 𝑣_𝑖,𝑏 are concatenated as 𝐯_𝑖=[𝑣_𝑖,1,𝑣_𝑖,2,𝑣_𝑖,3,…,𝑣_𝑖,2^𝐽] as the extracted features of the pattern 𝐱_𝑖. It is believed that the in-depth and hidden features of the single fault patterns can be detected through the coefficient packets 𝐯_𝑖 after WPT decomposition. WPT is applied to every 𝐱_𝑖 to form a set of features 𝐕={𝐯_𝑖}, 𝑖=1 to 𝑁_𝐷.

Usually, the dimension of 𝐯_𝑖 is large and a certain amount of the features may be redundant. Therefore, PCA is employed for dimension reduction of 𝐯 while retaining its important information. The details of PCA can be found in [22]. After applying PCA to 𝐕, a set of eigen vectors 𝐡_𝑗 and eigen values 𝑒_𝑗 are returned, which represent the transformation vectors and the importance of the transformed dimensions, where 𝑗=1 to 𝑝,  𝑒₁≥𝑒₂≥⋯≥𝑒_𝑝≥0.  The  𝑘  (𝑘<𝑝) most important dimensions are selected based on the criterion of ∑^k_𝑗=1𝑒^′_𝑗≤0.99, that is, a 1% of information loss is allowed, where 𝑒^′_𝑗=𝑒_𝑗/∑^𝑝_𝑗=1𝑒_𝑗 is a normalized eigen value. Knowing the value of 𝑘, the corresponding transformation matrix 𝐇=[𝐡₁𝐡₂⋯𝐡_𝑘] is then formed. So 𝐅^′=𝐇^𝑇𝐕 is the reduced feature dataset. For any unseen ignition pattern 𝐱 in the future, its feature vector can be obtained by 𝐟^′=𝐇^𝑇𝐯, where 𝐯=WPT(𝐱). By combining the prior domain knowledge, the final feature vector as the classifier inputs is given in the following: 𝐟=𝐟^′,𝐹₁,𝐹₂,𝐹₃. (4)

2.4. Relevance Vector Machine

Relevance vector machine [27] is a statistical learning method utilizing Bayesian learning framework and popular kernel methods. In fault diagnosis, RVM is designed to predict the posterior probability of the binary class membership (i.e., either positive or negative) for an unseen input 𝐟, given a set of training data (𝐅,𝐭)={𝐟_𝑛,𝑡_𝑛}, 𝑛=1 to 𝑁, 𝑡_𝑛∈{0,1}, and 𝑁 is the number of training data. It follows the statistical convention and generalizes the linear model by applying the logistic sigmoid function 𝜎(𝑧(𝐟))=1/(1+exp(−𝑧(𝐟))) to the predicted decision 𝑧(𝐟) and adopting the Bernoulli distribution for 𝑃(𝐭∣𝐅). The likelihood of the data is written as follows [27]:𝑃(𝐭∣𝐅,𝐰)=^𝑁_𝑛=1𝜎𝑧𝐟_𝑛;𝐰^𝑡𝑛1−𝜎𝑧𝐟_𝑛;𝐰^1−𝑡𝑛where𝑧(𝐟;𝐰)=^𝑁_𝑖=1𝑤_𝑖𝐾𝐟,𝐟_𝑖+𝑤₀, (5) where 𝐰=(𝑤₀,𝑤₂,…,𝑤_𝑁)^𝑇 are the adjustable parameters, and a radial basis function (RBF) is typically chosen for 𝐾(⋅).

The current objective is to find the optimal weight vector 𝐰 in (5) for the given dataset 𝐅, which is equivalent to find 𝐰 so as to maximize the probability 𝑃(𝐰∣𝐭,𝐅,𝜶)∝𝑃(𝐭∣𝐅,𝐰)𝑃(𝐰∣𝜶), with 𝜶=[𝛼₀,𝛼₁,…,𝛼_𝑁] a vector of  𝑁+1  hyperparameters. However, it is impossible to determine the weights analytically. Hence, closed-form expressions for either the marginal likelihood 𝑃(𝐰∣𝜶) or equivalently the weight posterior 𝑃(𝐰∣𝐭,𝐅,𝜶) are denied. Thus, the following approximation procedure is chosen [32], which is based on Laplace’s method.(a) For the current fixed values of 𝜶, the most probable weights 𝐰_MP are found, which is the location of the posterior mode. Since 𝑃(𝐰∣𝐭,𝐅,𝜶)∝𝑃(𝐭∣𝐅,𝐰)𝑃(𝐰∣𝜶), this step is equivalent to the following maximization: 𝐰_MP=argmax_𝐰log𝑃(𝐰∣𝐭𝐅,𝜶)=argmax_𝐰log{𝑃(𝐭∣𝐅,𝐰)𝑃(𝐰∣𝜶)}=argmax_𝐰^𝑁_𝑛=1𝑡_𝑛log𝑠_𝑛+1−𝑡_𝑛1−log𝑠_𝑛−12𝐰^𝑇𝐀𝐰with𝑠_𝑛=𝜎z𝐟_𝑛,𝛼;𝐰𝐀=diag₀,𝛼₁,…,𝛼_𝑁. (6)(b) Laplace’s method is simply a Gaussian approximation to the log-posterior around the mode of the weights 𝐰_MP. Equation (6) is differentiated twice to give ∇_𝐰∇_𝐰log𝑃(𝐰∣𝐭,𝐅,𝜶)_𝐰MP=−Φ^𝑇𝐁Φ+𝐀, (7) where 𝐁=diag(𝛽₁,…,𝛽_𝑁) is a diagonal matrix with 𝛽_𝑛=𝜎{𝑧(𝐟_𝑛;𝐰)}[1−𝜎{𝑧(𝐟_𝑛;𝐰)}], and Φ is a 𝑁×(𝑁+1) design matrix with Φ_𝑛𝑚=𝐾(𝐟_𝑛,𝐟_𝑚−1) and Φ_𝑛0=1, 𝑛=1 to 𝑁, and 𝑚=1 to 𝑁+1. By inverting (7), the covariance matrix 𝚺=−(∇_𝐰∇_𝐰 log𝑃(𝐰∣𝐭,𝐅,𝜶)|_𝐰MP)⁻¹= (Φ^𝑇𝐁Φ+𝐀)⁻¹ can be obtained.(c) The hyperparameters 𝜶 are updated using an iterative reestimation equation. Firstly, randomly guess 𝛼_𝐢 and calculate 𝛾_𝑖=1−𝛼_𝑖𝚺_𝑖𝑖, where 𝚺_𝑖𝑖 is the 𝑖th diagonal element of the covariance matrix 𝚺. Then reestimate 𝛼_𝐢 as follows: 𝛼^new_𝑖=𝛾_𝑖𝐮²_𝑖, (8) where 𝐮=𝐰_MP=𝚺Φ^𝑇𝐁𝐭. Set 𝛼_𝑖←𝛼^new_𝑖 and reestimate 𝛾_𝑖 and 𝛼^new_𝑖 again until convergence. Then 𝐰=𝐰_MP is estimated so that the classification model 𝑧(𝐟;𝐰)=∑^𝑁_𝑖=1𝑤_𝑖𝐾(𝐟,𝐟_𝑖)+𝑤₀ is obtained.

2.5. Pairwise Coupled RVM

The traditional RVM formulation is designed only for binary classification; that is, the output is either positive (+1) or negative (−1). In order to resolve the current simultaneous-fault problem, multiclass strategies of one-versus-all (1vA) and one-versus-one (1v1, or specifically named as pairwise coupling) [28] can be employed. Traditionally 1vA strategy constructs a group of classifiers 𝑓_class=[𝐶₁,…,𝐶_𝑑] in a 𝑑-label classification problem. For any unknown input 𝐟, the classification vector 𝐲=[𝑦₁,𝑦₂,…,𝑦_𝑑], where 𝑦_𝑖=1 if 𝐶_𝑖(𝐟)=+1 or 𝑦_𝑖=0 if 𝐶_𝑖(𝐟)=−1. The 1vA strategy is simple and easy to implement. However, it generally gives a poor result [29, 33, 34] since 1vA does not consider the pairwise correlation and hence induces a much larger indecisive region than 1v1 as shown in Figure 5.

(a)

(b)

Figure 5 

Indecisive regions (shaded area) using 1vA (a) and pairwise coupling (1v1) (b) [29].

On the other hand, pairwise coupling (1v1) also constructs a group of classifiers 𝑓_class=[𝐶₁,…,𝐶_𝑑] in a 𝑑-label classification problem. However, each 𝐶_𝑖=[𝐶_𝑖1,…,𝐶_𝑖𝑗,…,𝐶_𝑖𝑑] is composed of a set of 𝑑−1 different pairwise classifiers 𝐶_𝑖𝑗, 𝑖≠𝑗. Since 𝐶_𝑖𝑗 and 𝐶_𝑗𝑖 are complementary, there are totally 𝑑(𝑑−1)/2 pairwise classifiers in 𝑓_class (Figure 6(b)).

(a)

(b)

Figure 6 

Architecture of 1vA classifier (a) and 1v1 classifier (b).

In this study, each 𝐶_𝑖𝑗 can be an RVM classifier which estimates the pairwise probability that an unknown instance 𝐟 belongs to the 𝑖th label against the 𝑗th label, that is, 𝐶_𝑖𝑗(𝐟)=𝑃(𝑙_𝑖∣𝐟,𝑙_𝑖 or 𝑙_𝑗). There are several methods for pairwise coupling strategy [28], which are, however, suitable for multiclass diagnosis only because of the constraint Σ𝜌_𝑖=1. Note that the nature of simultaneous-fault diagnosis is that Σ𝜌_𝑖 is not necessarily equal to 1. Therefore, the following simple pairwise coupling strategy for simultaneous-fault diagnosis is proposed.

Every 𝐶_𝑖𝑗 is trained only by the training data with the 𝑖th and 𝑗th labels. Let 𝜌_𝑖𝑗=𝐶_𝑖𝑗(𝐟)=𝑃(𝑙_𝑖∣𝐟,𝑙_𝑖 or 𝑙_𝑗) be the pairwise probability of the 𝑖th label against the 𝑗th label for an unknown instance 𝐟, where 𝐶_𝑖𝑗(𝐟) is estimated using RVM. Then, 𝜌_𝑖 is calculated as 𝝆_𝑖=𝐶_𝑖(𝐟)=∑^𝑑_{𝑗=1∶𝑗≠𝑖}𝑛_𝑖𝑗𝐶_𝑖𝑗(𝐟)∑^𝑑_{𝑗=1∶𝑗≠𝑖}𝑛_𝑖𝑗=∑^𝑑_{𝑗=1∶𝑗≠𝑖}𝑛_𝑖𝑗𝜌_𝑖𝑗∑^𝑑_{𝑗=1∶𝑗≠𝑖}𝑛_𝑖𝑗, (9) where 𝑛_𝑖𝑗 is the number of training data with the 𝑖th and 𝑗th labels. Hence, the probability 𝜌_𝑖  can be more accurately estimated from 𝜌_𝑖𝑗=𝐶_𝑖𝑗(𝐟) because the pairwise correlation between the labels are taken into account. With the above pairwise coupling strategy, PCRVM can more accurately estimate the probability vector 𝝆 and hence generate a higher classification accuracy for simultaneous-fault diagnosis.

2.6. Decision Threshold Optimization and 𝐹-Measure

PCRVM can only provide the probability vector 𝝆=[𝜌₁,𝜌₂,…,𝜌_𝑑] of the single-fault labels but the desired result is the classification vector 𝐲=[𝑦₁,𝑦₂,…,𝑦_𝑑]=[𝜀(𝜌₁),𝜀(𝜌₂),…,𝜀(𝜌_𝑑)]. It is obvious that the value of decision threshold 𝜀 will greatly affect the classification accuracy. For a situation without any prior information, the best estimate of 𝜀 may be simply set to 0.5, that is, the presence of a fault is considered if its probability is at least 0.5. However, the value of 𝜀 should be optimized according to the classification accuracy. In other words, the value 𝜀 should be chosen to produce the highest classification accuracy over a validation dataset.

Besides, the traditional evaluation of classification accuracy only considers exact matching of the predicted label vector 𝐲 against the true label vector 𝐥. This evaluation is however not suitable for simultaneous-fault diagnosis where partial matching is preferred. Therefore, a common evaluation called 𝐹-measure is employed.

𝐹-measure [35] is commonly used as performance evaluation for information retrieval systems where a document may belong to a single or multiple tags simultaneously. This is very similar to the current application that contains a mixture of single-fault and simultaneous-fault patterns. With 𝐹-measure, the evaluation of single-fault and simultaneous-fault test patterns can be appropriately done at one time. To define 𝐹-measure 𝐹_me, two concepts of precision (𝜋) and recall (𝜏) are used so that 𝐹_𝑚=2𝜋𝜏,𝜋+𝜏 (10) where 𝜋 and 𝜏 are originally designed for single-fault patterns only but can be extended to handle simultaneous-fault patterns. For 𝑁_𝑇 single-fault and simultaneous-fault test data, ∑𝜋=^𝑑𝑖=1∑^𝑁𝑇_𝑗=1𝑦^𝑗_𝑖l^𝑗_𝑖∑^𝑑_𝑖=1∑^𝑁𝑇_𝑗=1𝑦^𝑗_𝑖∑,𝜏=^𝑑𝑖=1∑^𝑁𝑇_𝑗=1𝑦^𝑗_𝑖l^𝑗_𝑖∑^𝑑_𝑖=1∑^𝑁𝑇_𝑗=1l^𝑗_𝑖, (11) where 𝑦^𝑗_𝑖 and 𝑙^𝑗_𝑖 are, respectively, the 𝑖th predicted label and the 𝑖th true label in the 𝑗th test data, 𝑦^𝑗_𝑖 and 𝑙^𝑗_𝑖∈{0,1}. Substituting (11) into (10), the final 𝐹-measure equation is given in (12). The larger the 𝐹-measure value, the higher the diagnostic accuracy is 𝐹_𝑚=2×∑^𝑑_𝑖=1∑^𝑁𝑇_𝑗=1𝑦^𝑗_𝑖𝑙^𝑗_𝑖∑^𝑑_𝑖=1∑^𝑁𝑇_𝑗=1𝑦^𝑗_𝑖+∑^𝑑_𝑖=1∑^𝑁𝑇_𝑗=1𝑙^𝑗_𝑖∈[0,1]. (12) With 𝐹-measure, the value 𝜀 can be optimized using typical direct search techniques such as Genetic Algorithms (GA) and Particle Swarm Optimization (PSO) [36].

2.7. Principle of Detection of Single Faults and Simultaneous Faults

After an unknown instance 𝐟 is passed to the above system, a probability vector 𝝆 is produced. If 𝐟 is caused by a single fault (e.g., the 𝑗th fault), 𝐟 contains only the symptoms of the 𝑗th fault. Then, in 𝝆, the corresponding probability 𝜌_𝑗≥𝜀^∗ so that 𝑦_𝑗=1 in the decision vector 𝐲 while all other 𝑦_𝑘=0, 𝑘≠𝑗. In other words, ∑𝑦_𝑗=1 and hence a single fault is detected.

For the case that 𝐟 is caused by two simultaneous faults (e.g., the 𝑗th and 𝑘th faults), 𝐟 is constituted by the symptoms of the 𝑗th and 𝑘th faults. These symptoms may be overlapping or interdistorted. In the current diagnostic system, probabilities are employed to give the similarity of 𝑓 against the 𝑗th and 𝑘th faults by 𝐶_𝑗 and 𝐶_𝑘, respectively. If their symptoms are not highly overlapping or interdistorted, there is a high chance that the corresponding probabilities 𝜌_𝑗,𝜌_𝑘≥𝜀^∗. Under this circumstance, 𝑦_𝑗=1 and 𝑦_𝑘=1, making ∑𝑦_𝑗≥1 so that a simultaneous fault can be detected. The mechanism is similar for three or more simultaneous faults. By combining these cases, the proposed system can diagnose both single fault and simultaneous faults using classifiers trained with single faults only.

2.8. Summary of Proposed Framework and Techniques

The previous framework and techniques are summarized in Algorithm 1. Figure 7(a) shows the workflow of using DK and WPT+PCA as feature extraction. Every dataset for training, validation and test requires going through the step of feature extraction. Figure 7(b) shows the construction of the classifier 𝑓_class. The classifier has the architecture of pairwise coupling as depicted in Figure 6(b). Then the classifier is passed to an optimizer to search for the optimal decision threshold based on a validation set 𝑉𝐴𝐿𝐼𝐷_𝐹 and 𝐹-measure 𝐹_me as shown in Figure 7(c), where 𝑓_class outputs the probability vector 𝝆=[𝜌₁,𝜌₂,…,𝜌_𝑑] for each case in 𝑉𝐴𝐿𝐼𝐷_𝐹. To optimize the threshold, the 𝐹-measure 𝐹_me over 𝑉𝐴𝐿𝐼𝐷_𝐹 can be evaluated as the fitness value. Since the direct search technique is easily stuck by local minima, it is necessary to run 𝑀 different times of the optimization step in Figure 7(c) to avoid this issue. For testing and running, the step in Figure 7(d) is very similar to Figure 7(c) except the optimal threshold 𝜀_opt that has been determined. The choice of parameters of the feature extraction, classification, and direct search techniques are discussed in Section 4.

Given a training dataset TRAIN_F of single-fault patterns only, a validation dataset VALID_F and a test dataset TEST_F of single- fault and simultaneous-fault patterns (all datasets have been preprocessed by the combination of DK and WPT and PCA, as presented in Figure 7(a)):  (i) Train the probabilistic classifier 𝑓class   𝑓class includes 𝑑(𝑑−1)/2 pairwise classifiers 𝐶𝑖𝑗 as shown in Figure 6(b).  (ii) ∀ f  ∈  VALID_F, prepare the probability vector 𝝆    Calculate 𝑓class (f) = 𝝆 = [𝜌1,𝜌2,…,𝜌𝑑] (Figure 7(c)).  (iii) For k = 1 to 𝑀// Run a direct search technique, such as GA or PSO, M times    Produce an initial population 𝜖for the decision threshold 𝜀    (a) ∀𝜀 ∈ 𝜖, find the classification vector y(f) = y = [𝑦1,𝑦2,…,𝑦𝑑]= [𝜀(𝜌1),𝜀(𝜌2),…,𝜀(𝜌𝑑)] according to (1).    (b) Calculate the 𝐹-measure 𝐹me with y(f) and l(f) using (12), that is, find 𝐹me over VALID_F,      where l(f) = [𝑙1,𝑙2,…,𝑙𝑑] is the true classification vector for input f provided from VALID_F.    (c) Produce next generation of 𝜀    Until convergence or matching stopping criteria, return the best solution ε as the 𝜀𝑘.  (iv) Among all 𝜀𝑘, k = 1 to M, choose the one producing the highest F-measure 𝐹me as the optimal decision threshold 𝜀opt.  (v) Return the trained probabilistic classifier 𝑓class and the optimized decision threshold 𝜀opt as the main components of the   intelligent diagnostic system.  (vi) The performance of 𝑓class and 𝜀opt can be evaluated with TEST_F and 𝐹me as illustrated in Figure 7(d).

Algorithm 1 

Algorithm of the proposed framework for simultaneous-fault diagnosis of time-dependent ignition patterns.

(a)  Feature extraction

(b) Training

(c) Decision threshold optimization

(d) Test∖run

Figure 7 

Workflow of feature extraction, decision threshold optimization, diagnostic system training, and testing.

3. Experimental Setup

To verify the effectiveness of the proposed methodology, an experiment was set up for sample data acquisition and evaluation tests. The details of the experimental setup and preparation of datasets are presented in the following subsections.

3.1. Data Sampling

In total, a set of single faults and simultaneous faults were imitated and selected as demonstration examples. There are 10 kinds of single faults as described in Tables 1 and 4 kinds of simultaneous faults as described in Table 2. However, there is an issue that the simultaneous-fault patterns are not caused by a random combination of single faults but some reasonable combinations (e.g., it is impossible to have wide spark-plug gap and narrow spark-plug gap at the same time). Moreover, the experimental data show that a simultaneous-fault ignition pattern is caused by a combination of at most three single faults. Beyond these constraints, the ignition patterns cannot be captured due to engine stall. Some sample ignition patterns of these single faults and reasonable simultaneous faults are shown in Figures 8 and 9, respectively.

Table 1 

Sample single faults of engine trouble reflected by ignition patterns.

Table 2 

Sample possible simultaneous faults of engine trouble reflected by ignition patterns.

Figure 8 

Sample ignition patterns and their corresponding single engine faults.

(a)

(b)

(c)

(d)

(e)

Figure 9 

Sample ignition patterns and their corresponding simultaneous engine faults.

In this study, five well-known inline 4-cylinder electronic ignition engines, namely, HONDA B18C, HONDA D15B, HONDA K20A, TOYOTA 2NZ-FE, and MITSUBISHI 4G15, were employed as the experimental platforms, and a computer-linked automotive scope meter was used (Figure 10) to capture raw ignition patterns. Different models of engines were used for training in order to enhance the generalization of the classifier. To capture ignition patterns, the sampling frequency of the scope meter was set to a high rate of 100 kHz, that is, 100,000 sampling points per second. Under the software provided by the scope meter, ignition patterns were recorded in a PC and converted into a file of excel format for processing and analysis.

Figure 10 

Collection of ignition patterns from a test engine using a computer-linked automotive scope meter.

For each case (single fault or simultaneous faults in Tables 1 and 2) in every test engine, sixteen ignition patterns (four patterns for each cylinder) were captured over two different engine testing conditions according to the standard procedure in [3] (1200 rpm and 2000 rpm). As the pattern obtained in each cylinder per engine cycle is somewhat unrepeatable, four patterns per cylinder are required. The reason for causing unrepeatable patterns is that a constant engine speed is difficult to hold during sampling. Furthermore, each cylinder has its own manufacturing error, different inlet and exhaust flow characteristics, and so forth. Finally there were 1600 ignition patterns of single faults (i.e., 10 labels × 4 patterns × 4 cylinders × 2 testing conditions × 5 engines) and 800 ignition patterns of simultaneous faults (i.e., 5 labels × 4 patterns × 4 cylinders × 2 testing conditions × 5 engines).

3.2. Data Normalization

As the number of sampling points of every captured pattern is not exactly the same due to engine speed fluctuation and various testing conditions, all patterns were normalized within the same range in order to match the number of inputs of the classifier. Normalization of the ignition patterns was done in terms of duration. In this study, the number of sampling points for every pattern was less than 17,000. For the sake of conservation, a standard number of sampling points for all patterns was set to 18,000 in order not to lose any exceptional information. To standardize the duration of all patterns, steady-state values can be appended to the rear part of the patterns if necessary. Normally, the steady-state value for the ignition pattern is equal to zero (0 V). For those patterns having fewer than 18,000 data points, zeros can be appended. Therefore, the durations of all sample patterns were normalized before feature extraction using WPT+PCA.

3.3. Allocation of Datasets

In order to test the diagnostic performance for both single faults and simultaneous faults, about 3/4 of the single-fault patterns were taken as training data 𝑇𝑅𝐴𝐼𝑁. There were 1/16 of the single-fault patterns and 1/5 simultaneous-fault patterns in the validation dataset 𝑉𝐴𝐿𝐼𝐷, while the remaining 3/16 of the single-fault patterns and 4/5 simultaneous-fault patterns were used as test dataset 𝑇𝐸𝑆𝑇.

4. Experimental Results

To select the best combination of the techniques for feature extraction, classification, and threshold optimization, many experiments based on the sample dataset were conducted. The sample dataset was separated into 3 groups: 𝑇𝑅𝐴𝐼𝑁 for training the classifier, 𝑉𝐴𝐿𝐼𝐷 for the threshold optimization and selection of direct search techniques, and 𝑇𝐸𝑆𝑇 for evaluating the performance of different combinations of the feature extraction, classification, and threshold optimization techniques. The performance evaluation over 𝑇𝐸𝑆𝑇 is based on 𝐹-measure that can evaluate single-fault and simultaneous-fault patterns at one time according to partial matching criterion. All experiments were carried out under a PC with Core i5 @ 3.20GHz and 4GB RAM. All the proposed techniques mentioned were implemented using Matlab R2008a.

4.1. Results of Various Combinations of Feature Extraction and Classification Techniques

The reasonable combinations of DK, FFT, and WPT+PCA for feature extraction were tested as shown in Table 3 along with the corresponding evaluation. The classification techniques used in the experiment include PNN, RVM, and PCRVM. PNN [24] was selected for comparison because it is a traditional probabilistic classification using radial basis (Gaussian) kernel. The input dimension 𝑠 of the classifiers for evaluation is subject to the feature extraction technique. In terms of WPT, PCA, and DK, WPT transforms the original patterns of 18000 points into different packets at level 𝐽. The value of 𝐽 can be determined using entropy information. A built-in function bestlev (meaning best level) is available in Matlab wavelet toolbox for this purpose. After carrying out many experiments using the function bestlev, 𝐽 was tested to be 9 for the sample dataset of ignition patterns. In this study, the common mother wavelet, Haar wavelet, was selected for the purposes of illustration and comparison of different feature extraction techniques. For better performance, different types of mother wavelets could be evaluated in the future. After PCA, the most 22 important dimensions were selected as described in Section 2.3. Therefore, the size of 𝑠 is equal to 22 plus the three domain features, that is, 25 totally. For FFT and DK, the sizes of 𝑠 are equal to 18000 and 3 features, respectively.

Table 3 

Evaluation of different combinations of techniques.

Table 4 

GA operators and parameters.

In the construction of the intelligent engine diagnostic systems with different techniques for comparison, each feature extraction technique was firstly employed to preprocess the training dataset 𝑇𝑅𝐴𝐼𝑁, and then different classification techniques were applied. The performance of every combination was evaluated over 𝑇𝐸𝑆𝑇 using 𝐹-measure. In order to reflect the effectiveness of the feature extraction, the classification techniques under 𝑇𝑅𝐴𝐼𝑁 without any preprocessing were also examined. Therefore, there were totally 18 combinations of feature extraction and classification techniques as shown in Table 3.

For classification techniques of PNN, RVM, and PCRVM, several simple settings are necessary. PNN requires a hyperparameter called smoothing factor or spread, which is equivalent to the width of the Gaussian kernel within PNN. If the value of spread is set too high, the trained classifier may easily overfit the training patterns and hence a lower generalization. In the case study, the value of spread for PNN was simply set to be 0.2 according to rule of thumb [37]. The RVM and PCRVM employ different classification strategies (1vA versus 1v1) but they share the same set of hyperparameters, namely, type of kernel functions and the corresponding kernel parameters. For illustration purpose, Gaussian was selected as the kernel function 𝐾(⋅) and its kernel width was set to be 1.0 in order to calculate the design matrix Φ in (7). The experimental results of various combinations of feature extraction and classification techniques are shown in Table 3. In order to evaluate the 𝐹-measures under different combinations of preprocessing and classifications, the decision threshold was simply set to 0.5 for a simple and fair comparison in this phase.

4.2. Results with Threshold Optimization

Genetic Algorithms (GA) are the most classical direct search technique, while Particle Swarm Optimization (PSO) is another popular choice. Both of them were tested for the optimization of the decision threshold and they share the same objective function. Since the 𝐹-measure 𝐹_me∈(0,1), the objective function of optimization can be simply set as follows: min1−𝐹_me. (13) The higher the 𝐹_me, the better the optimization result will be. The optimization procedure follows the proposed algorithm in Algorithm 1, where the number of runs 𝑀 was set to be 20. Tables 4 and 5 show the detail settings of the GA and PSO operators and parameters, respectively, according to the literature [36]. Therefore, among 20 runs of the proposed algorithm for every combination of feature extraction and classification techniques under GA and PSO optimization, the optimized threshold 𝜀_opt and its corresponding 𝐹_me value of different combinations of techniques are shown in Tables 6 and 7, respectively.

Table 5 

PSO operators and parameters.

Table 6 

Evaluation of different combinations of techniques using GA-optimized threshold.

Table 7 

Evaluation of different combinations of techniques using PSO-optimized threshold.

4.3. Individual Result of Single- and Simultaneous-Fault Diagnosis

The objective of this research is to train a probabilistic classifier using single-fault patterns and then predict both single and simultaneous faults. However, it is unclear whether the performance of the trained probabilistic classifier on simultaneous faults in Section 4.2 is correct or not because the classification results of different combinations of techniques were all evaluated over the whole test dataset 𝑇𝐸𝑆𝑇, which contains single-fault and simultaneous-fault patterns. To better illustrate the performance of the proposed method, 𝑇𝐸𝑆𝑇 was further separated into two groups, one for purely single-faults 𝑇𝐸𝑆𝑇₁, and another for purely simultaneous-faults 𝑇𝐸𝑆𝑇_𝑠. All evaluation tests were done using the combination of DK+WPT+PCA as feature extraction and the PSO-optimized threshold of 0.7147 because Tables 6 and 7 show that this combination produces the best 𝐹-measure. The 𝐹-measures of purely single faults and purely simultaneous faults are shown in Tables 8 and 9, respectively, which were calculated using (12) with the related faults. For example, for Fault 1, 𝐹_me is evaluated on the test cases of Fault 1 only. For simultaneous faults of the combination (5,7), after prediction there is a classification vector 𝐲=[𝑦₁,…,𝑦₁₀], and a true vector 𝐥=[𝑙₁,…,𝑙₁₀], then 𝑦₅ and 𝑦₇ with the true values 𝑙₅ and 𝑙₇ from the test cases are employed to compute the two separate 𝐹_me values for detail analysis.

Table 8 

Single-fault diagnosis under 𝐹-measure (with DK+WPT+PCA, 𝜀_opt = 0.7147).

Table 9 

Simultaneous fault diagnosis under 𝐹-measure with classifiers trained with single faults (with DK + WPT + PCA and 𝜀_opt = 0.7147).

4.4. Results Comparison with the Latest Technique

To further verify the effectiveness of the presented framework, the existing binarization approach using SVM [23] was applied to the ignition system diagnosis for comparison. The binarization approach builds classifiers directly based on raw ignition patterns, so there is no feature extraction step. In this approach, a number of binary classifiers 𝐵𝑓_𝑗(⋅) were constructed, respectively, using support vector machines (SVM) with one-versus-all splitting strategy where 𝑗=1 to 𝑑, 𝑑 is the number of single faults again. A decision vector 𝐲=[𝜃(𝐵𝑓₁(𝐱))⋯𝜃(𝐵𝑓_𝑑(𝐱))] can be obtained for an unknown pattern 𝐱, where 𝐵𝑓_𝑗(𝐱)∈𝑅 is the raw output value of the 𝑗th SVM classifier, and 𝜃(𝐵𝑓_𝑗(𝐱))=1 if 𝐵𝑓_𝑗(𝐱)≥0 and 𝜃(𝐵𝑓_𝑗(𝐱))=0 Otherwise. From this framework, only single-fault patterns were used for training the binary classifiers while simultaneous-fault patterns are also not necessary. Since there is no probabilistic output but only a binary decision vector is generated in the binarization approach, no decision threshold optimization is necessary in this experiment. The results using the binarization approach is shown in Table 10.

Table 10 

Evaluation results of the binarization approach and the proposed framework for fault diagnosis.

5. Discussion of Results

5.1. Effect of Feature Extraction and Pairwise Probabilistic Classification

The experimental results presented in Section 4 are discussed in this section. Table 3 illustrates that the step of feature extraction is effective. DK is the time-related features of an ignition pattern but only improves the overall classification accuracy about 1% as compared with the methods without any feature extraction, while FFT and WPT+PCA give about 4.4% and 4.8% improvement, respectively. When combining both time-related and frequency-related features by DK and WPT+PCA, the overall classification accuracy is about 7% higher than that without any feature extraction. Table 3 also indicates that no matter which classification technique is employed, the integration of DK and WPT+PCA as feature extraction gives the best accuracy. In addition, the three classification techniques are compared by using 𝐹-measure as well. Both PNN and RVM employ 1vA strategy for probabilistic classification. In other words, only 𝑑 binary classifiers were constructed for 𝑑 labels so that there are large indecision regions between pairs of classes. Therefore, when a test case lies on these regions, PNN and RVM mostly fail to classify the faults correctly. However, PCRVM employs 1v1 strategy, which minimizes those indecision regions. Table 3 verifies the effectiveness of the 1v1 strategy because PCRVM outperforms the other two classification techniques. This situation is almost the same as the tests with optimized decision threshold as shown in Tables 6 and 7. Therefore, the proposed PCRVM is a very effective and promising classification technique.

5.2. Effect of Decision Threshold Optimization

Tables 3, 6, and 7 illustrate that the GA and PSO can improve the overall accuracy by 3.48% and 3.5% as compared with the fixed decision threshold of 0.5, but these two techniques give nearly the same threshold and 𝐹_me. The reason is that the experiment was run for 20 times for both the GA and PSO, and then the pair of results with the highest 𝐹_me was returned. However, it is found that the standard deviations of the 20 results for the GA and PSO are 1.02E-3 and 3.23E-4, respectively. For the GA, the standard deviation is larger than PSO in this case study. This result indicates that PSO is more stable than the GA and theoretically requires a fewer number of runs to obtain a suboptimal result than the GA. This is because PSO is somehow insensitive to the initial values, whereas the GA is initialized with random start points within the search space and the search result is very sensitive to the initial values [36]. Consequently, PSO is recommended for this application.

5.3. Diagnosis of Simultaneous Faults

Table 8 reveals that the trained classifiers using PNN, RVM, and PCRVM perform well because the test cases contain single-fault patterns only. Due to the advantage of pairwise coupling, PCRVM performs the best among the three classification techniques.

For the test cases of simultaneous-fault patterns, there are only five reasonable combinations of simultaneous faults because not every combination is possible. Since a simultaneous-fault pattern is caused by different single faults, some of the time-related and frequency-related features may be distorted or even vanished. Therefore, the feature extraction using DK and WPT+PCA cannot work very well and hence the values of 𝐹_me in Table 9 drop a little bit as compared with the values in Table 8, but they can still provide an accuracy ranging from 0.49 to 0.8. Once again, PCRVM outperforms the other classified techniques because of pairwise coupling strategy. Within the simultaneous-fault diagnosis, the most misclassified fault is Fault 10, because the ignition pattern of Fault 10 is almost distorted by Fault 3. Nevertheless, the experimental results can still verify the following:(1)the proposed framework can alleviate the problem of exponential growth of training dataset for simultaneous-fault ignition patterns by training the probabilistic classifier using single-fault patterns only. This evidence can be found in Tables 8 and 9 that the single-fault patterns can be almost correctly classified, while the overall classification accuracy for simultaneous-fault ignition patterns is still satisfactory;(2)the feature extraction techniques of DK combined with WPT+PCA can effectively capture the time-related and frequency-related features from single-fault and simultaneous-fault ignition patterns;(3)the features of single-fault ignition patterns can really be detected in some feasible simultaneous-fault ignition patterns; this feasibility will create a new research direction for automotive engine diagnosis;(4)RVM is more robust than PNN for probabilistic classification;(5)the pairwise coupling (1v1) strategy can improve the accuracy for common probabilistic classification techniques.

5.4. Comparison with the Latest Approach

Table 10 reveals that the binarization approach works badly on ignition pattern classification. In other words, the binarization method does not work for engine ignition-system diagnosis. In addition, after feature extraction, the performance of the binarization can be generally raised about 50% as well. Therefore, the effectiveness of feature extraction is verified under all frameworks and techniques tested in this paper. It is highly believed that this feature extraction can also work well in many other practical applications.

6. Conclusions

One of the challenges in ignition system diagnosis is that more than one single fault may appear at a time. Another challenge is the acquisition of large amount of costly simultaneous-fault ignition patterns for constructing the diagnostic system because the number of the training patterns depends on the combination of different single faults. In this paper, simultaneous-fault diagnosis for automotive engine ignition patterns was studied and a new framework combining feature extraction, probabilistic classification, and decision threshold optimization based on a fair multilabel assessment, 𝐹-measure, has successfully been developed. With the proposed diagnosis framework, the acquisition of large amount of simultaneous-fault patterns can be avoided.

In this study, the combination of feature extraction techniques of DK, FFT, and WPT+PCA have been tried along with the classification techniques of PNN, RVM, and PCRVM to tackle the simultaneous-fault diagnosis. The experimental results reveal that PCRVM combined with WPT+PCA and DK performs the best for both single-fault and simultaneous-fault diagnoses. Its average accuracy for single-fault diagnosis is about 0.95 while the average accuracy for simultaneous faults is only about 0.76. It implies that the feature extraction technique based on DK and WPT+PCA for simultaneous-fault detection may not be perfect. Alternative approach, such as the integration of feature extraction, classification, and multiexpert reasoning, could be studied in the future.

This study also shows that the decision threshold for identifying the number of simultaneous faults can be optimized over 𝐹-measure using direct search techniques, such as GA and PSO. Both the GA and PSO generate almost the same decision threshold but PSO requires less computational time and is more stable because of its lower standard deviation in multiple runs. Moreover, PSO has fewer operators and hence fewer adjustable parameters that can further reduce the user burden. Overall speaking, PSO should be the first choice of the threshold optimization technique in the current application.

To further verify the effectiveness of the proposed framework, the latest method, binarization method using SVM, was also employed to diagnose the simultaneous faults. The results show that the diagnosis accuracy of the binarization method is worse than that of the proposed framework. Therefore, the proposed framework is very suitable for engine ignition-system fault diagnosis. Since the proposed framework for simultaneous-fault diagnosis is general, it can be adapted to other similar applications. Finally, the original contributions of the research are summarized as follows.(1)The research is a first attempt at integrating DK+WPT+PCA, PCRVM, and direct search techniques into a general framework for simultaneous-fault diagnosis of automotive ignition systems. (2)The proposed diagnostic system is the first in the literature that can be trained with single-fault signal patterns (i.e., single-fault time-dependent patterns) only, while it can diagnose simultaneous-fault signal patterns too. (3)This paper is also the first in the literature that reports that the features of single-fault ignition patterns can be detected in some feasible simultaneous-fault ignition patterns. This fact is an important contribution to automotive engine diagnosis.(4)The integration of the pairwise coupling (1v1) strategy into RVM is original, and the 1v1 strategy can really improve the classification accuracy of RVM.

Notation

𝐀:	Diagonal matrix of hyperparamters
𝑎:	End point of burning time
𝚩:	Diagonal matrix in RVM
𝐵𝑓𝑗(⋅):	𝑗th binary classifier
𝐶𝑖:	𝑖th probabilistic classifier
𝐶𝑖(𝐟):	Probability of 𝐟 belonging to the 𝑖th label
𝐶𝑖𝑗:	Pairwise classifier
𝐶𝑖𝑗(𝐟):	Pairwise probability of 𝐟 belonging to the 𝑖th label against the 𝑗th label
𝑐1:	Cognitive parameter of PSO
𝑐2:	Social parameter of PSO
𝐷:	Sample dataset
𝑑:	Number of labels (faults)
𝑒𝑗:	𝑗th eigen value
𝑒′𝑗:	𝑗th normalized eigen value
𝐅:	Set of feature vectors
𝐅′:	Set of feature vectors created by WPT and PCA
𝐹1:	Firing voltage
𝐹2:	Burn time
𝐹3:	Average spark voltage of spark line
𝐹me:	𝐹-measure
𝐟:	Feature vector
𝐟′:	Feature vector created by WPT and PCA
𝐟ℎ:	ℎth feature vector
𝑓class:	Probabilistic classifier
𝐇:	PCA transformation matrix
𝐡𝑗:	𝑗th eigen vector
𝐽:	Decomposition level of WPT
𝐾(⋅):	Kernel function in RVM
𝐿𝑃:	Length of ignition pattern (i.e., number of data point in ignition pattern)
𝐥:	True label vector
𝐥𝑖:	𝑖th true label vector
𝑙𝑖:	𝑖th label in 𝐥
𝑙𝑖𝑔:	𝑔th label in 𝐥𝑖
𝑙𝑗𝑖:	𝑖th label in the 𝑗th test data
𝑁:	Number of training data
𝑁𝐷:	Number of cases in sample dataset
𝑁𝑇:	Number of test data
𝑛𝑖𝑗:	Number of training data with the 𝑖th and 𝑗th labels
𝑃(𝑙𝑗∣𝐟):	Probability of 𝐟 belonging to 𝑙𝑗
𝑃(⋅):	Probability
𝑠:	Input dimension of classifier to be evaluated
𝐭:	Set of faulty labels in training dataset
𝑡𝑛:	Faulty label of the 𝑛th training case
𝑇𝐸𝑆𝑇:	Original test dataset
𝑇𝐸𝑆𝑇1:	Single-fault cases in test dataset
𝑇𝐸𝑆𝑇𝑠:	Simultaneous-fault cases in test dataset
𝑇𝐸𝑆𝑇_𝐹:	Test dataset after feature extraction
𝑇𝑅𝐴𝐼𝑁:	Original training dataset
𝑇𝑅𝐴𝐼𝑁_𝐹:	Training dataset after feature extraction
𝐕:	Set of coefficient vectors
𝑉𝐴𝐿𝐼𝐷:	Original validation dataset
𝑉𝐴𝐿𝐼𝐷_𝐹:	Validation dataset after feature extraction
𝐯:	Coefficient vector
𝐰:	Optimal vector in RVM
𝑤𝑛:	𝑛th optimal parameter in RVM
𝐰MP:	Most probable weight vector in RVM
𝑤𝑐:	Inertial weight of PSO
WPT(·):	Wavelet packet transform function
𝐗:	Set of ignition pattern vectors
𝐱:	Unseen ignition pattern
𝑥𝑖:	𝑖th data point in 𝐱
𝐲:	Predicted label vector
𝑦𝑖:	𝑖th predicted label
𝑦𝑗𝑖:	𝑖th predicted label in the jth test data
𝑧(𝐟):	Predicted decision
𝜶:	Hyperparameter vector of RVM
𝛼𝑛:	𝑛th hyperparameter of RVM
𝜀:	Decision threshold
𝜀𝑘:	𝑘th tentative threshold produced in optimization process
𝜀opt:	Optimized threshold
𝜃(⋅):	Decision function of binarization approach
𝜋:	Precision
𝝆:	Probability vector
𝜌𝑖:	Probability of the 𝑖th label
𝜌𝑖𝑗:	Pairwise probability of the 𝑖th label against the 𝑗th label
𝚺:	Covariance matrix in RVM
𝚺𝑖𝑖:	𝑖th diagonal element of covariance matrix Σ
𝜎(⋅):	Logistic sigmoid function
𝜏:	Recall
𝜖:	Initial population
Φ:	Design matrix in RVM.

Acknowledgment

The research is supported by the University of Macau Research Grant nos. MYRG075(Y2-L2)-FST12-VCM, MYRG141(Y2-L2)-FST11-IWF, and MYRG149(Y2-L2)-FST11-WPK.