Robust Matching Pursuit Extreme Learning Machines

Yuan, Zejian; Wang, Xin; Cao, Jiuwen; Zhao, Haiquan; Chen, Badong

doi:https://doi.org/10.1155/2018/4563040

Scientific Programming

On this page

Abstract Introduction Preliminaries and Related Works Experimental Results Conclusion Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Programming Foundations for Scientific Big Data Analytics

View this Special Issue

Research Article | Open Access

Volume 2018 | Article ID 4563040 | https://doi.org/10.1155/2018/4563040

Robust Matching Pursuit Extreme Learning Machines

Zejian Yuan,¹Xin Wang,¹Jiuwen Cao,²Haiquan Zhao,³and Badong Chen¹

Academic Editor: Wenbing Zhao

Received25 Aug 2017

Revised23 Nov 2017

Accepted07 Dec 2017

Published01 Feb 2018

Abstract

Extreme learning machine (ELM) is a popular learning algorithm for single hidden layer feedforward networks (SLFNs). It was originally proposed with the inspiration from biological learning and has attracted massive attentions due to its adaptability to various tasks with a fast learning ability and efficient computation cost. As an effective sparse representation method, orthogonal matching pursuit (OMP) method can be embedded into ELM to overcome the singularity problem and improve the stability. Usually OMP recovers a sparse vector by minimizing a least squares (LS) loss, which is efficient for Gaussian distributed data, but may suffer performance deterioration in presence of non-Gaussian data. To address this problem, a robust matching pursuit method based on a novel kernel risk-sensitive loss (in short KRSLMP) is first proposed in this paper. The KRSLMP is then applied to ELM to solve the sparse output weight vector, and the new method named the KRSLMP-ELM is developed for SLFN learning. Experimental results on synthetic and real-world data sets confirm the effectiveness and superiority of the proposed method.

1. Introduction

Extreme learning machine [1] is a kind of single hidden layer feedforward network (SLFN) [2]. In the past decade, ELM became popular and attractive in the machine learning and pattern recognition communities for its fast adaptability and good generalization performance [3]. In general, ELM has the following advantages: (i) It not only has the ability of estimating the unknown mathematical model embedded in a mass of training samples but also possesses parallel schemes to be efficiently implemented in parallel for training and testing; (ii) it uses randomly generated input weights and hidden biases without tuning during the training phase, and therefore, the output weights can be analytically obtained by solving the standard least squares (LS) problem. Thus, extremely fast learning ability and efficient computation cost can be achieved, especially for big data applications. In view of these remarkable superiorities, ELM has been widely applied in many applications, such as face recognition [4], series compensated transmission line protection [5], time series analysis [6], and nonlinear model identification [7].

However, ELM still has several drawbacks. First, ELM encounters the problem of irrelevant variables when handling real-world data sets [8]. Second, choosing a proper hidden nodes number is an open problem for all ELM algorithms. An ELM network with too few hidden nodes may not be accurate for modeling the input data, whereas a network with too many hidden nodes tends to generate an overfitting model [9]. Moreover, when the number of hidden nodes is more than the input data, ELM might have the singularity problem [4]. Third, the original ELM learns the model with an -norm based loss function, which is very vulnerable to noise. It is well known that the -norm can magnify the bad effects of outliers associated with large deviations [10]. The presence of non-Gaussian noises or outliers in the training data may thus lead to an unreliable model with degraded performance.

To overcome the first and second limitations, several methods have been proposed in the regularization framework [9, 11–13]. Furthermore orthogonal matching pursuit (OMP) is a plain and efficient iterative algorithm which chooses an atom in the dictionary with the best correlation to the remaining elements at each iteration [14]. As such, OMP has been embedded to ELM (OMP-ELM) to overcome the singularity problem and led to more stable solution than the original ELM [15]. Most of the existing methods learn the model with an -norm based loss function, which may perform poorly in the presence of non-Gaussian noises (which exist in many real-world situations) or outliers [16–18]. To combat non-Gaussian noises or outliers and improve the generalization ability, the regularized correntropy criterion is used to replace the -norm based loss function in original ELM model to develop the ELM-RCC [16]. In [19], ELM with -norm based loss function (ORELM) was proposed to achieve robust performance.

The kernel risk-sensitive loss (KRSL) is a nonlinear similarity measure firstly proposed in [20], which can reach a more satisfying robust performance. The KRSL is based on the original structure of risk-sensitive loss and is defined in the reproducing kernel Hilbert space (RKHS) [21, 22]:where denotes the mathematical expectation, is the Gaussian kernel with bandwidth , and is the risk-sensitive parameter. In this paper, we propose a KRSL based matching pursuit (KRSLMP) method. The KRSLMP is then embedded to ELM to construct a robust and sparse ELM model.

The rest of the paper is structured as follows. In Section 2, we sketch the related work, including similarity measures in kernel space, kernel risk-sensitive loss, ELM model, and orthogonal matching pursuit algorithm. In Section 3, we develop the KRSLMP-ELM. In Section 4, experiments on regression problem with synthetic and real-world data sets are conducted to verify the effectiveness of the proposed algorithm. The sensitivity of the KRSLMP-ELM to free parameters is also analyzed. Finally, conclusion is given in Section 5.

For convenience of presentation, the following notations used in this paper are introduced. Vectors and matrices are represented with boldface lowercase letters and boldface capital letters, respectively. For any vector , we use to denote its th entry. The notation denotes the subvector of with entries indexed by the set . The complementary set of is denoted as .

2.1. Similarity Measures in Kernel Space

Let and be two random variables; the correntropy between and is defined by [17, 23]where is the joint distribution function of . The Gaussian kernel with bandwidth is given by

Correntropy is a local correlation measure in the kernel space . According to Mercers theorem [24], it can be expressed in terms of the inner product as

It applies a kernel trick that nonlinearly maps the original space to a higher dimensional feature space. It can be shown that correntropy is directly related to the probability of how similar two random variables are in a neighborhood of the joint space controlled by the kernel bandwidth [17, 25, 26].

2.2. Kernel Risk-Sensitive Loss

Similarity measures in kernel space have the ability to extract higher-order statistics of data, which can significantly improve the learning performance in non-Gaussian environments [21]. The optimization problem can be determined by maximizing the correntropy criterion (MCC) or equivalently minimizing the correntropic loss (C-Loss) [27, 28] between the output estimation and the target response. However, highly nonconvex problem may happen in C-Loss performance surface which has steep slopes around the optimal solution but is extremely flat far from the solution. This may lead to slow convergence and poor performance. Choosing a large kernel bandwidth may overcome the above problem. But the robustness will decrease significantly when outliers occur with kernel bandwidth increasing [29]. To achieve a satisfying performance surface, the KRSL was proposed in [20].

The KRSL is defined bywhich can also be expressed in a traditional risk-sensitive loss form as [30]where is the risk-sensitive parameter that controls the shape of performance surface.

In practice, the joint distribution function of and is usually unknown and only a finite number of samples are available. The KRSL can thus be estimated byAs one can see, (6) defines a distance between the vectors and .

2.3. Extreme Learning Machine

Extreme learning machine (ELM) was proposed by Huang et al. for training single hidden layer feedforward neural networks (SLFNs) [2, 31]. The input weights and biases are initialized randomly in ELM and remain unchanged during training. The network learning thus becomes optimizing the output weights, which can be formulated as solving a linear equation. Let be given by training samples, where input and corresponding desired output ; the relationship between and can be represented under the assumption of the model. The network model of ELM with hidden neurons can be modeled and expressed aswhere is hidden nodes number, is the weight connecting the th hidden node and output nodes, is the activation function (in this work, is a sigmoid function without explicit mention), denotes the weight that connects the th hidden node and input nodes, and represents the randomly chosen bias of the th hidden node. Equation (7) can be compactly written as a matrix notationwhereand is the minimal norm least squares solution of (8). The parameter can be obtained bywhere is the Moore Penrose generalized inverse of the hidden layer output matrix .

2.4. Orthogonal Matching Pursuit

Matching pursuit method is one of the effective methods for sparse representation [14, 32, 33]. In general, a sparse representation problem can be formulated aswhere denotes the measurement matrix, is the sparse vector, and represents the noise vector. The main purpose is to recover the sparse vector from the observation and the measurement matrix . The OMP uses the -norm constrained least squares modelwhere counts the number of nonzero coordinates of .

In the following, we briefly describe the OMP method. First, we initialize the residual , the index set , and the iteration . At each iteration, OMP algorithm selects a column of the measurement matrix which is most correlated to the residual aswhere denotes the residual in th iteration and is the th column of . Then collect to index set

We can solve an LS problem to obtain a new estimation supported in :where supp() denotes the support set of . If the stopping criterion is satisfied, we output as the estimate of .

Then one can update the residual

From (8) and (11), we can find that ELM has a similar network model for sparse representation problem. Thus, one can take advantage of the OMP algorithm for selecting the best hidden nodes of the ELM network. The OMP estimates the sparse vector by using the -norm based criterion, which performs well with the Gaussian error distribution. However, the presence of non-Gaussian noise may give rise to performance degradation.

3. Kernel Risk-Sensitive Loss Based Matching Pursuit Extreme Learning Machine

To address the aforementioned issue, we propose a robust kernel risk-sensitive loss based orthogonal matching pursuit extreme learning machine algorithm (KRSLMP-ELM) in this section. In the KRSLMP-ELM, we initialize the residual as and the initial index set as . Then, similar to OMP, a column of most correlated with the residual is selected and the index set is augmented at each iteration. Then we obtain a new estimation by solving the following KRSL minimization problem:

We utilize the half-quadratic (HQ) theory [34] to construct the optimization algorithm. Considering that the measurements may include both large and small noise, we can use HQ optimization to estimate the importance of different samples. The samples severely corrupted will be assigned small weight values in learning procedure to decrease the impact of large noise. Thus, the performance of KRSLMP-ELM can be significantly further improved.

According to the convex optimization theory [35], the dual function for is convex and defined asand thenwhere the infimum is reached at . We point out here that when the parameter , the KRSLMP-ELM can also work well in our simulations. Substituting (18) for (20), the KRSLMP-ELM objective function can be reformulated aswhere diag() represents a diagonal matrix with its primary diagonal element and is the regularization parameter. Inspired by the HQ theory, (21) can be solved by the following alternate technique:where denotes the iteration number. In the proposed algorithm, the bandwidth is adaptively chosen during the iteration. In order to make the scheme robust to outliers, we calculate the value of as follows.

Denote the training error as , . We can then reorder the error in an ascending order, and we get the reordered as . Let , where scalar and outputs the largest integer smaller than . We can select as the bandwidth in accordance with the proportion of outlier. Discussions on the detailed experimental results by choosing different bandwidths are given in the experiment section. A solution for the optimization problem in (21) can be derived as follows:where and denotes the identity matrix.

Since the importance degree of the measurements is employed to adaptively update the output weight vector in the KRSLMP-ELM, we update the residual

It is noted that the sparsity level has to be assigned in advance in the KRSLMP-ELM. The sparsity directly determines the number of the active hidden nodes used in ELM due to the fact that more hidden nodes than necessary are generated. To obtain the best sparsity level , namely, the best number of hidden nodes used in ELM, we utilize the root mean square error (RMSE) as the criterionwhere denotes the target response and the corresponding output estimated by the KRSLMP-ELM.

For different sparsity level , the corresponding RMSE is first calculated. Then the best coefficients associated with the minimum RMSE value are selected.

The iteration is repeated until achieving the stopping criterion. The KRSLMP-ELM is summarized in Algorithm 1.

Input: samples
Output: weight vector
Parameters setting: number of hidden nodes , regularization parameter and sparsity level .
Initialization: randomly initialize ELM parameters: input weights and biases in measurement matrix .
Set the index set , the residual , the iteration counter and .
for do

Find a column of most correlated with the residual

Augment the index set

Solve the KRSLMP minimization problem by the following iterations


The solution is denoted as
Update residual
end for

4. Experimental Results

To validate the effectiveness of the proposed KRSLMP-ELM algorithm, experiments on two synthetic data sets and seven benchmark data sets are conducted in this section. The performance of the new method is compared to five state-of-the-art algorithms, namely, ELM, RELM, ELM-RCC, OMP-ELM, and ORELM. Sigmoid function is used as the activation function for all methods.

4.1. Synthetic Data Sets

In this subsection, experiments on two synthetic regression data sets for nonlinear function approximation problem are carried out. Descriptions of the two data sets are as follows.

Sinc.The synthetic data set is generated by , where andand contains two mutually independent noises that are inner noise and outliers noise . Specifically, is defined as , where is binary distributed with the probability masses and . and are independent of . In this experiment, is set at 0.1. The outlier is generated by using a zero-mean Gaussian distributed noise with standard deviation 4.0. For the inner noise , two different noises are tested, which are (a) uniform distribution over and (b) Sine wave noise , with uniformly distributed over . We uniformly generate the input data from , where 200 data points are used for training and another 200 clean data points which are not contaminated by any noise are used for testing.

Func. This synthetic data set is generated bywhere is a zero-mean Gaussian distributed noise vector with standard deviation 0.4. The input data vectors and are uniformly generated from . Similar to the previous experiments, 200 data samples are used for training and another 200 data samples without noise are used for testing.

Parameters used in the six methods for experiments of the two synthetic data sets are summarized in Table 1, where , , , and represent the number of hidden layer nodes, regularization parameter, sparsity level, and risk-sensitive parameter in KRSLMP-ELM. We set in Sinc synthetic data set experiment and in Func synthetic data set experiment. For the convenient distinguishment of the proposed method with other methods in Sinc function approximation problem, only the estimation results of the original ELM, ORELM, ELM-RCC, and KRSLMP-ELM are illustrated in Figure 1. In Figure 2, we plot the squared training errors obtained by the KRSLMP-ELM, ELM-RCC, ORELM, and the original ELM, respectively. As shown in these figures, the KRSLMP-ELM wins the best approximation performance. The testing RMSEs of six algorithms are presented in Table 2. It is indicated that the KRSLMP-ELM is more robust than the other five methods.

(a)

(b)

(a)

(b)

(c)

(d)

Further, we perform another experiment to compare the performance of KRSLMP-ELM to that of the original ELM with different outliers. We consider the Sinc function approximation problem and set the inner noise as a zero-mean Gaussian distributed noise with standard deviation 0.1, and the outliers noise is zero-mean Gaussian with standard deviation ranging between 0.1 and 10. We run 100 trials for different outliers noises and show the RMSE results in Figure 3. One can see that the original ELM’s performance degrades severely when the outliers get enhanced while the KRSLMP-ELM’s performance is much less influenced by outliers.

4.2. Benchmark Data Sets

In this subsection, seven benchmark regression data sets from UCI machine learning repository [36] are tested to support the superiority of the proposed method. Specifications of the data sets are shown detailedly in Table 3. It should be pointed out that the training and testing data samples are randomly chosen in each data set and all the features are normalized into . The parameters of each method are all chosen by the fivefold cross-validation and are given in Table 4. For all algorithms, 100 independent trials are conducted and the average results are reported. The training and testing RMSEs and their standard deviation of all algorithms are listed in Table 5. As highlighted in boldface, the ELM-KRSLMP achieves the best performance in most regression data sets.

4.3. Sensitivity of Parameters

We analyze the sensitivity of the parameters , , , and of KRSLMP-ELM in this subsection. For illustration, we use the regression results obtained by the Servo data set as an example. For each parameter, its sensitivity is tested by fixing the remaining parameters as the ones used in Table 4. Then, the testing RMSEs are recorded as criteria for performance comparison. The results of the regression performance are demonstrated in Figure 4.

(a)

(b)

(c)

(d)

5. Conclusion

In this paper, a robust matching pursuit based ELM algorithm, called the kernel risk-sensitive loss based matching pursuit extreme learning machine (KRSLMP-ELM), has been developed. Kernel risk-sensitive loss (KRSL) is a nonlinear similarity measure defined in kernel space, and it can achieve better performance than the conventional MSE criterion when dealing with non-Gaussian and nonlinear problems. Incorporating the KRSL into the existing orthogonal matching pursuit algorithm, we developed an improved KRSLMP-ELM algorithm, which is more robust than the OMP-ELM method. Comparisons with several existing state-of-the-art algorithms have also been provided to validate the superiority of the proposed KRSLMP-ELM algorithm.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was partially supported by the National Natural Science Foundation-Shenzhen Joint Research Program (no. U1613219) and National Natural Science Foundation of China (no. 91648208 and no. 61372152).

References

G. Huang, “An insight into extreme learning machines: random neurons, random features and kernels,” Cognitive Computation, 2014.
View at: Publisher Site | Google Scholar
G. B. Huang, Q. Y. Zhu, and C. K. Siew, “Extreme learning machine: theory and applications,” Neurocomputing, vol. 70, no. 1–3, pp. 489–501, 2006.
View at: Publisher Site | Google Scholar
G. B. Huang, Q. Y. Zhu, and C. K. Siew, “Extreme learning machine: a new learning scheme of feedforward neural networks,” in Proceedings of the IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990, July 2004.
View at: Publisher Site | Google Scholar
W. Zong and G.-B. Huang, “Face recognition based on extreme learning machine,” Neurocomputing, vol. 74, no. 16, pp. 2541–2551, 2011.
View at: Publisher Site | Google Scholar
V. Malathi, N. S. Marimuthu, S. Baskar, and K. Ramar, “Application of extreme learning machine for series compensated transmission line protection,” Engineering Applications of Artificial Intelligence, vol. 24, no. 5, pp. 880–887, 2011.
View at: Publisher Site | Google Scholar
R. Singh and S. Balasundaram, “Application of extreme learning machine method for time series analysis,” in Proceedings of the Rampal Singh and S Balasundaram. Application of extreme learning machine method for time series analysis. International Journal of Intelligent Technology, vol. 2, pp. 256–262, 2007.
View at: Google Scholar
J. Deng, K. Li, and G. W. Irwin, “Fast automatic two-stage nonlinear model identification based on the extreme learning machine,” Neurocomputing, vol. 74, no. 16, pp. 2422–2429, 2011.
View at: Publisher Site | Google Scholar
Y. Miche, M. van Heeswijk, P. Bas, O. Simula, and A. Lendasse, “TROP-ELM: a double-regularized ELM using LARS and Tikhonov regularization,” Neurocomputing, vol. 74, no. 16, pp. 2413–2421, 2011.
View at: Publisher Site | Google Scholar
W. Y. Deng, Q. H. Zheng, and L. Chen, “Regularized extreme learning machine,” in Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining (CIDM '09), pp. 389–395, April 2009.
View at: Publisher Site | Google Scholar
H. Wang, “Block principal component analysis with L1-norm for image analysis,” Pattern Recognition Letters, vol. 33, no. 5, pp. 537–542, 2012.
View at: Publisher Site | Google Scholar
J. M. Martínez-Martínez, P. Escandell-Montero, E. Soria-Olivas, J. D. Martín-Guerrero, R. Magdalena-Benedito, and J. Gómez-Sanchis, “Regularized extreme learning machine for regression problems,” Neurocomputing, vol. 74, no. 17, pp. 3716–3721, 2011.
View at: Publisher Site | Google Scholar
L.-C. Shi and B.-L. Lu, “EEG-based vigilance estimation using extreme learning machines,” Neurocomputing, vol. 102, pp. 135–143, 2013.
View at: Publisher Site | Google Scholar
Y. Miche, P. Bas, C. Jutten, O. Simula, and A. Lendasse, “A methodology for building regression models using extreme learning machine: OP-ELM,” in Proceedings of the 16th European Symposium on Artificial Neural Networks—Advances in Computational Intelligence and Learning (ESANN '08), pp. 247–252, April 2008.
View at: Google Scholar
J. A. Tropp and A. C. Gilbert, “Signal recovery from random measurements via orthogonal matching pursuit,” Institute of Electrical and Electronics Engineers Transactions on Information Theory, vol. 53, no. 12, pp. 4655–4666, 2007.
View at: Publisher Site | Google Scholar | MathSciNet
O. F. Alcin, A. Sengur, J. Qian, and M. C. Ince, “OMP-ELM: Orthogonal matching pursuit-based extreme learning machine for regression,” Journal of Intelligent Systems, vol. 24, no. 1, pp. 135–143, 2015.
View at: Publisher Site | Google Scholar
H.-J. Xing and X.-M. Wang, “Training extreme learning machine via regularized correntropy criterion,” Neural Computing and Applications, vol. 23, no. 7-8, pp. 1977–1986, 2013.
View at: Publisher Site | Google Scholar
W. Liu, P. P. Pokharel, and J. C. Principe, “Correntropy: properties and applications in non-Gaussian signal processing,” IEEE Transactions on Signal Processing, vol. 55, no. 11, pp. 5286–5298, 2007.
View at: Publisher Site | Google Scholar | MathSciNet
Y. Feng, X. Huang, L. Shi, Y. Yang, and J. A. Suykens, “Learning with the maximum correntropy criterion induced losses for regression,” Journal of Machine Learning Research (JMLR), vol. 16, pp. 993–1034, 2015.
View at: Google Scholar | MathSciNet
K. Zhang and M. Luo, “Outlier-robust extreme learning machine for regression problems,” Neurocomputing, vol. 151, no. 3, pp. 1519–1527, 2015.
View at: Publisher Site | Google Scholar
B. Chen, L. Xing, B. Xu, H. Zhao, N. Zheng, and J. C. Principe, “Kernel risk-sensitive loss: definition, properties and application to robust adaptive filtering,” IEEE Transactions on Signal Processing, vol. 65, no. 11, pp. 2888–2901, 2017.
View at: Publisher Site | Google Scholar | MathSciNet
J. C. Principe, Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives, Information Science and Statistics, Springer, New York, NY, USA, 2010.
View at: Publisher Site | MathSciNet
X. Luo, J. Deng, J. Liu, W. Wang, X. Ban, and J. Wang, “A quantized kernel least mean square scheme with entropy-guided learning for intelligent data analysis,” China Communications, vol. 14, no. 7, pp. 127–136, 2017.
View at: Publisher Site | Google Scholar
B. Chen and J. C. Príncipe, “Maximum correntropy estimation is a smoothed MAP estimation,” IEEE Signal Processing Letters, vol. 19, no. 8, pp. 491–494, 2012.
View at: Publisher Site | Google Scholar
B. Schölkopf and A. J. Smola, Learning with kernels: support vector machines, regularization, optimization, and beyond, MIT press, Cambridge, Massachusetts, USA, 2002.
W. Ma, J. Duan, B. Chen, G. Gui, and W. Man, “Recursive generalized maximum correntropy criterion algorithm with sparse penalty constraints for system identification,” Asian Journal of Control, vol. 19, no. 3, pp. 1164–1172, 2017.
View at: Publisher Site | Google Scholar | MathSciNet
I. Santamaría, P. P. Pokharel, and J. C. Principe, “Generalized correlation function: definition, properties, and application to blind equalization,” IEEE Transactions on Signal Processing, vol. 54, no. 6 I, pp. 2187–2197, 2006.
View at: Publisher Site | Google Scholar
B. Chen, Y. Zhu, J. Hu, and J. Principe, “System Parameter Identification: Information Criteria and Algorithms,” System Parameter Identification: Information Criteria and Algorithms, pp. 1–249, 2013.
View at: Publisher Site | Google Scholar
B. Chen, J. Wang, H. Zhao, N. Zheng, and J. C. Principe, “Convergence of a fixed-point algorithm under maximum correntropy criterion,” IEEE Signal Processing Letters, vol. 22, no. 10, pp. 1723–1727, 2015.
View at: Publisher Site | Google Scholar
X. Luo, J. Deng, W. Wang, J.-H. Wang, and W. Zhao, “A quantized kernel learning algorithm using a minimum kernel risk-sensitive loss criterion and bilateral gradient technique,” Entropy, vol. 19, no. 7, article no. 365, 2017.
View at: Publisher Site | Google Scholar
R. K. Boel, M. R. James, and I. R. Petersen, “Robustness and risk-sensitive filtering,” Institute of Electrical and Electronics Engineers Transactions on Automatic Control, vol. 47, no. 3, pp. 451–461, 2002.
View at: Publisher Site | Google Scholar | MathSciNet
X. Luo, Y. Xu, W. Wang et al., “Towards enhancing stacked extreme learning machine with sparse autoencoder by correntropy,” Journal of The Franklin Institute, 2017.
View at: Publisher Site | Google Scholar
M. A. Davenport and M. B. Wakin, “Analysis of orthogonal matching pursuit using the restricted isometry property,” Institute of Electrical and Electronics Engineers Transactions on Information Theory, vol. 56, no. 9, pp. 4395–4401, 2010.
View at: Publisher Site | Google Scholar | MathSciNet
Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,” in Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers, vol. 1, pp. 40–44, Pacific Grove, Calif, USA, November 1993.
View at: Publisher Site | Google Scholar
M. Nikolova and M. K. Ng, “Analysis of half-quadratic minimization methods for signal and image recovery,” SIAM Journal on Scientific Computing, vol. 27, no. 3, pp. 937–966, 2005.
View at: Publisher Site | Google Scholar | MathSciNet
R. T. Rockafellar, Convex Analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, NJ, USA, 1970.
View at: MathSciNet
A. Frank, Uci machine learning repository, 2010, http://archive.ics.uci.edu/ml.

Copyright

Copyright © 2018 Zejian Yuan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1677

Downloads

1175

Citations

Scientific Programming

Programming Foundations for Scientific Big Data Analytics

Robust Matching Pursuit Extreme Learning Machines

Abstract

1. Introduction

2. Preliminaries and Related Works

2.1. Similarity Measures in Kernel Space

2.2. Kernel Risk-Sensitive Loss

2.3. Extreme Learning Machine

2.4. Orthogonal Matching Pursuit

3. Kernel Risk-Sensitive Loss Based Matching Pursuit Extreme Learning Machine

4. Experimental Results

4.1. Synthetic Data Sets

4.2. Benchmark Data Sets

4.3. Sensitivity of Parameters

5. Conclusion

Conflicts of Interest

Acknowledgments

References

Copyright