Research Article  Open Access
Shao Jie, Wang Li, Zhao WeiSong, Zhong YaQin, Reza Malekian, "Numerical Analysis of Modeling Based on Improved Elman Neural Network", The Scientific World Journal, vol. 2014, Article ID 271593, 12 pages, 2014. https://doi.org/10.1155/2014/271593
Numerical Analysis of Modeling Based on Improved Elman Neural Network
Abstract
A modeling based on the improved Elman neural network (IENN) is proposed to analyze the nonlinear circuits with the memory effect. The hidden layer neurons are activated by a group of Chebyshev orthogonal basis functions instead of sigmoid functions in this model. The error curves of the sum of squared error (SSE) varying with the number of hidden neurons and the iteration step are studied to determine the number of the hidden layer neurons. Simulation results of the halfbridge classD power amplifier (CDPA) with twotone signal and broadband signals as input have shown that the proposed behavioral modeling can reconstruct the system of CDPAs accurately and depict the memory effect of CDPAs well. Compared with VolterraLaguerre (VL) model, Chebyshev neural network (CNN) model, and basic Elman neural network (BENN) model, the proposed model has better performance.
1. Introduction
The networks, communications, and television systems have entered the digital age. The classD power amplifiers (CDPAs) [1] have become increasingly popular for audio applications because of their high power efficiency. Since the output transistors of CDPAs operate in the ohmic and cutoff regions, there exists nonlinearity in the system. One of the nonlinear phenomena is the intermodulation distortion (IMD) [2]. The CDPAs also have the memory effects, resulting from the node voltage and current depending on not only the current input but also the historical signals due to the existence of parameters with dynamic distribution [3]. The existence of memory effects [4] is often identified by imbalances between the corresponding upper and lower distortion products, such as the same order of IMD.
Behavioral modeling [5, 6] of nonlinear circuits and systems has received much attention in recent years. In behavioral modeling, the nonlinear component is generally considered as a “black box,” which is completely characterized by external responses, that is, in terms of input and output signals, through the use of relatively simple mathematical expressions. Behavioral modeling techniques provide a convenient and efficient means to predict systemlevel performance without the computational complexity of full circuit simulation or physical level analysis of nonlinear systems, thereby significantly speeding up the analysis process. The existing PA’s behavioral models are mainly based on Volterra series or its expanded and simplified forms [7, 8]. However, its large number of coefficients complicates its practical implementation, which makes the standard Volterra series only limited to “weak” nonlinear PAs.
Owing to the fact that the neural networks are provided with available solutions for nonlinear function approximation, system identification, exclusive or and encoder problems, the study of PA’s behavioral models based on neural networks has already been developed in recent years [9–12]. In order to well study the nonlinear characteristics of CDPAs, a new behavioral model based on improved Elman neural network (IENN) is proposed in this paper. In IENN, a selfconnection of context nodes is added in this model, which could make the neurons more sensitive to the history of input data. The Chebyshev orthogonal polynomials [13, 14] instead of sigmoid functions are employed as the activation functions of hidden layer neurons to improve accuracy and the convergence rate of IENN, and the structure of IENN is simpler. The gradient descent (GD) algorithm is used to train the neural network model. Simulation results with twotone signal, linear frequency modulated (LFM) signal, and binary phase shift keying (2PSK) signal as inputs have shown that the proposed model IENN could well depicts the nonlinear distortions of PAs.
The remainder of this paper is organized as follows. The basic Elman neural network (BENN) is introduced in Section 2. In Section 3, the new behavioral model based on IENN and the training algorithm of IENN is presented in detail. Simulation results using twotone signal and broadband signals as input are given in Section 4. The conclusion is shown in Section 5.
2. The Basic Elman Neural Network
The architecture of BENN [15, 16] is illustrated in Figure 1, which is generally divided into four layers: input layer, hidden layer, context layer, and output layer. The feedforward loop consists of input layer, hidden layer, and output layer in which the weights connecting two neighboring layers are variable. There exists a backforward loop between context layer and hidden layer, which makes the neural networks sensitive to the history of input data. In BENN, the context neurons can be treated as the memory units, so the model can manifest the memory effect of nonlinear system theoretically. Furthermore, because the dynamic characteristics of BENN are provided only by internal connections, there is no need to use the state as input or training signal, which makes BENN prior to static feedforward network.
3. The Behavioral Model Based on IENN
3.1. The Architecture of Improved Elman Neural Network
The architecture of IENN is presented in Figure 2; it is similar to BENN. To improve the learning speed and output accuracy, some changes are made. To better deliver the memory effect of the nonlinear system, a selffeedback is added to the context layer neurons with a feedback coefficient gain. This operation increases the memory depth and makes the model’s output more sensitive to the history inputs. The value of Chebyshev orthogonal basis functions can easily be calculated by recursion operation, which is simpler than the sigmoid function. And the Chebyshev orthogonal basis functions have been used as active functions in many neural networks [14, 17, 18] for different applications and proved to be fast and accurate. We consider using the first category Chebyshev orthogonal basis as the activation function in the hidden layer instead of the sigmoid function in BENN. The research results have proved that the IENN model can simplify the computing complexity, reduce the training time, and enhance the convergence precision.
In IENN, the input layer has nodes, the hidden layer and the context layer own nodes, and the output layer possesses nodes. The basic functions in each layer are as follows.
3.1.1. Input Layer
In the input layer, where represents the th iteration step and and denote the input and the output of the input layer, respectively.
3.1.2. Hidden Layer
The input of the th hidden layer neuron is where is the output of the th context layer neuron, represents the weight from the th context layer neuron to the th hidden layer neuron, and represents the weight from the th input layer neuron to the th hidden layer neuron.
Since the input of Chebyshev orthogonal basis functions is defined within the interval , the input of the hidden layer needs to be normalized. The normalization of is defined as
The output of the th hidden layer neuron is
The function indicates the first category Chebyshev orthogonal basis functions given in Figure 2.
3.1.3. Context Layer
In the context layer, the output is represented as where is the selfconnection feedback gain of the context layer. When , this network is reductive into the BENN.
3.1.4. Output Layer
The output of IENN can be expressed as where denotes the weight from th hidden layer neuron to th output layer neuron.
3.2. Training Algorithm
Training of the neural networks has been developed rapidly in recent years [19–21]. The gradient descent (GD) algorithm, as a basic approach for training neural networks in many areas, searches the parameter space of the network in the steepest descent way to minimize the error between the network output and the desired output [22]. In IENN, the gradient descent algorithm is used to update the weights. Assume that the actual system output vector is ( is used for transpose) and the th iteration of IENN model output vector is . The errorfunction, namely, the sum of squared error (SSE), is defined as
By using the partial derivative of errorfunction with respect to the weight parameters, the increments of the weights are as follows:
with where is the first derivative of the normalized input of hidden layer neurons . , , and represent the learning rate of , , and , respectively.
In order to well analyze the error of system output and IENN output, the transient absolute error vector is defined as
The mean error of is
3.3. Training Steps of IENN
By using the GD method, the training steps to determine the optimal number of neurons in hidden layer are as follows. The initial values of , , , and are got by continuous testing.
Step 1. Prepare the training input and output data. Set the initial number for neurons in hidden layer, define the maximum neurons in hidden layer, and the maximum iteration step . The threshold value of SSE is .
Step 2. Set the selfconnection feedback coefficient ; weights of IENN , , and as constant 0 and their learning rate ; the partial derivative . The initial iteration step .
Step 3. Increase the number of iteration step ; if , end the training process. According to formulas (1) to (7), calculate the value of every neuron in every layer and the of th iteration step. If the is less than , end the training process.
Step 4. Calculate the increments of the weights shown in formulas (8); then the weights are updated as , and . Jump to Step 3.
Step 5. Increase the number of neurons in hidden layer; if , end the training process. Jump to Step 2.
4. Simulation Results and Analysis
In order to verify the correctness and reliability of IENN model, the training sample sequences are achieved from input of halfbridge CDPA shown in Figure 3.
As shown in Figure 3, the PWM signal is produced by the comparison between the twotone signal and the triangular signal. The frequency and amplitude of triangular signal are kHz and . The output signal of CDPA is termed as . A group of the training data is extracted by the sampling frequency MHz. The testing data has the same length and sampling frequency with the training data; the difference is the starting time. In the simulation results, the testing data’s starting time is treated as 0 ms.
4.1. Optimal Neurons Number in Hidden Layer
In order to determine the optimal neurons number in hidden layer of two models, by using the training data of twotone signal, with the frequencies of the twotone signal being kHz and kHz, their amplitudes being , and the signal length being 0.5 ms, the relationship between and the number of hidden layer neurons is studied. When the maximum iteration step , the number of hidden neurons increases from 5 to 100 with the interval of 5; do not set the error threshold of ; the error curves of are shown in the left side of Figure 4. The lower side of Figures 4(a) and 4(b) shows the changing with hidden neurons number; when the iteration step , the neurons number in hidden layer increases from 5 to 50 with the interval of 1.
(a) BENN model
(b) IENN model
It can be seen in the top of Figures 4(a) and 4(b) that, with the increase of the iteration step, the error curves of drop rapidly. The larger the number of hidden neurons is, the faster decreases, and the less iteration steps needed to reach the same SSE are. The comparison between BENN and IENN shows that IENN has faster convergence rate than BENN. For example, to reach the same of 50, with the same number of hidden neurons , BENN needs an iteration number of about 55, while IENN needs an iteration number of only 25; the calculation of IENN is reduced to almost half of BENN.
On the lower side of Figures 4(a) and 4(b), when BENN and IENN have the same number of iteration; to achieve the same , IENN needs less neurons in hidden layer than BENN. For instance, to reach the logarithmic of 0 dB, BENN needs 30 neurons in hidden layer while IENN needs only 15 neurons in hidden layer.
In consideration of the convergence rate and the calculation, is chosen as the number of hidden layer neurons in the following discussion.
4.2. Simulation Analysis of Four Models with TwoTone Signal Input
The VolterraLaguerre (VL) model [7] and the Chebyshev neural network (CNN) model [17, 23, 24] are introduced to be compared with the BENN and IENN model. The VL model is proposed in [7]. There are two parameters in this model: the number of Laguerre orthogonal functions and the pole of Laguerre functions (). When , this model cannot reconstruct the output well. Here, we choose and ; there are 605 parameters needed to be estimated. The CNN model in [23] employs a group of Chebyshev orthogonal polynomials to activate the hidden layer neurons, and based on the GD method, the iterative training formula is obtained. For three neural network models, set the number of hidden layer neurons and the iteration step . Using the twotone signal as input, the simulation results of four behavioral models in time domain are shown in Figure 5.
(a) VL model
(b) CNN model
(c) BENN model
(d) IENN model
In Figure 5, the time domain error is the transient error . Values of the mean error and the maximum transient error of four models are listed in Table 1 with twotone signal input.

It can be seen in Figure 5 and Table 1 that IENN is the most accurate model among four models. The VL and CNN model cannot reconstruct the output signal accurately, and the transient error is very large at the beginning of the data. Both the BENN and IENN have stable approximation capability; under the same conditions, IENN is more precise than BENN. The final maximum transient error of BENN is 0.0391 V, while it is only V in IENN.
The twotone signal is often used to study the memory effect of the nonlinear system [4, 25] since the IMD of the signal is easy to measure. When a twotone signal is used as training data, the simulation results of four models in frequency domain are given in Figure 6.
(a) VL model
(b) CNN model
(c) BENN model
(d) IENN model
In Figure 6(d), kHz and kHz are the input twotone signal’s frequencies. and are the second order IMD (IMD2). and are the third order IMD (IMD3). The existence of IMD means the system is nonlinear and the asymmetry of IMD demonstrates the memory effect of the system. The circuit output spectrum and the spectrum error are listed in Table 2.

As shown in Figure 6 and Table 2, the spectrum error at IMD2 and IMD3 of VL model is a little large, the asymmetry between the upper and lower sidebands has been weakened, and some of the memory effect characteristics are lost. The short memory length of VL model is the reason for this. But the number of parameters in this model is already large; if the memory length increases, the parameters will increase rapidly. The spectrum of CNN in Figure 6(b) shows that it has lost almost all the information of the IMD. Since the CNN model is a feedforward neural network, the output of the model is only related to the input at present moment; it cannot express the previous influence of the inputs on the output, namely, that the CNN model cannot demonstrate the memory effect. The spectrum errors of BENN and IENN model are stable; under the same conditions, the spectrum error of BENN is 0.011 dB, and IENN is dB. The IENN is much more accurate than BENN.
4.3. Simulation Analysis of Four Models with LFM Signal Input
For the experimental validation, the LFM signal is used as input of halfbridge CDPA, whose center frequency is 30 kHz, amplitude is 8.5 V, bandwidth is 4 kHz, and training data length is 2.0 ms. Other parameters of the simulation are the same as above. Using the LFM signal as training samples, the simulation results of four behavioral models in time domain are shown in Figure 7; the results in frequency domain are given in Figure 8. The mean error and the maximum transient error of four models in time domain are listed in Table 3, and the average spectrum errors and the maximum spectrum errors are listed in Table 4.


(a) VL model
(b) CNN model
(c) BENN model
(d) IENN model
(a) VL model
(b) CNN model
(c) BENN model
(d) IENN model
As shown in Figure 7 and Table 3, when the LFM signal is used as input of CDPA, the transient error of the VL and CNN model is very huge, and the output signal cannot be reconstructed accurately. In the same conditions of the number of hidden neurons and the iteration step , the time domain errors of both IENN and BENN model are basically the same and the final maximum transient error of BENN is V, while it is V in IENN. Both the BENN and IENN have stable approximation capability.
As shown in Figure 8 and Table 4, using the training data of LFM signal, the spectrum error of the VL and CNN model is similarly very large and has lost the correct information of the memory effect in frequency domain. The spectrum errors of BENN and IENN model are stable; under the same simulation conditions, the maximum spectrum error of BENN is dB, and IENN is dB. The performance of IENN and BENN model in frequency domain is almost the same.
4.4. Simulation Analysis of Four Models with 2PSK Signal Input
In further experiments, a 2PSK signal is used as input of halfbridge CDPA, whose carrier frequency is 20 kHz, amplitude is 8.5 V, digital baseband signal is a 7bit pseudorandom sequence ( sequence), baseband symbol width is 0.25 ms, and testing data length is 1.75 ms. Other parameters of the models are the same as above too. Using the 2PSK signal input, the simulation results of four behavioral models in time domain are shown in Figure 9, and the results in frequency domain are given in Figure 10. The mean error and the maximum transient error of four models in time domain are listed in Table 5, and the average spectrum error and the maximum spectrum error are listed in Table 6.


(a) VL model
(b) CNN model
(c) BENN model
(d) IENN model
(a) VL model
(b) CNN model
(c) BENN model
(d) IENN model
It can be seen in Figure 9 and Table 5 that with a 2PSK signal input, the VL and CNN model cannot reconstruct the CDPA output accurately, and the transient error is still very large. Under the same conditions that the number of hidden neurons is 25 and the iteration step is 50, IENN model is more precise than BENN model. The final maximum transient error of BENN is 0.0398 V, while it is only V in IENN. IENN model is the most accurate model among four models.
As shown in Figure 10 and Table 6, using the training samples of 2PSK signal, the spectrum errors of the VL and CNN model are still very large, and the memory effect of CDPA cannot be demonstrated by these models. The spectrum errors of BENN and IENN model are steady. Under the same conditions, the maximum spectrum error of BENN is 0.011 dB, and IENN is dB. At this point, the IENN model is much more accurate than BENN model.
The comparison among four behavioral models under the condition of different input signals and the same simulation parameters shows that the proposed IENN model is the most accurate model for analyzing the nonlinearity and memory effect of the CDPAs in both time domain and frequency domain.
5. Conclusions
In this paper, a behavioral modeling based on IENN is proposed to describe the nonlinearity and memory effect of CDPAs. In IENN, a group of Chebyshev orthogonal basis functions is employed to activate hidden layer neurons to improve the learning speed and the accuracy and also to simplify the structure of model. A selfconnection of context nodes is added to make the output more sensitive to the history of input data.
According to the simulation results, it can be seen that, to reach the same error threshold, compared to BENN, IENN needs fewer hidden layer neurons and less iteration steps. It means that IENN has fast learning speed and can use simpler network structure to achieve the same requirements than many other neural networks. Using the same number of hidden layer neurons and iteration number, simulation results by using the training data of twotone, LFM and 2PSK signal have shown that the IENN is superior to VL, CNN, and BENN model in accuracy; it has reconstructed the nonlinear CDPA system with almost no transient or spectrum error; the memory effect is also visualized. Above all, the proposed IENN model is an effective, efficient, and simple behavioral model for nonlinear systems.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported in part by the Foundation of Key Laboratory of China’s Education Ministry and A Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.
References
 M. Margaliot and G. Weiss, “The lowfrequency distortion in Dclass amplifiers,” IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 57, no. 10, pp. 772–776, 2010. View at: Publisher Site  Google Scholar
 J. Yu, M. T. Tan, S. M. Cox, and W. L. Goh, “Timedomain analysis of intermodulation distortion of closedloop classD amplifiers,” IEEE Transactions on Power Electronics, vol. 27, no. 5, pp. 2453–2461, 2012. View at: Publisher Site  Google Scholar
 H. Zhou, G. Wan, and L. Chen, “A nonlinear memory power amplifier behavior modeling and identification based on memory polynomial model in softdefined shortwave transmitter,” in Proceedings of the 2010 6th International Conference on Wireless Communications, Networking and Mobile Computing (WiCOM '10), pp. 1–4, September 2010. View at: Publisher Site  Google Scholar
 H. Ku and J. S. Kenney, “Behavioral modeling of nonlinear RF power amplifiers considering memory effects,” IEEE Transactions on Microwave Theory and Techniques, vol. 51, no. 12, pp. 2495–2504, 2003. View at: Publisher Site  Google Scholar
 P. N. Landin, J. Fritzin, W. Van Moer, M. Isaksson, and A. Alvandpour, “Modeling and digital predistortion of classD outphasing RF power amplifiers,” IEEE Transactions on Microwave Theory and Techniques, vol. 60, no. 6, pp. 1907–1915, 2012. View at: Publisher Site  Google Scholar
 T. Liu, Y. Ye, X. Zeng, and F. M. Ghannouchi, “Memory effect modeling of wideband wireless transmitters using neural networks,” in Proceedings of the 2008 4th IEEE International Conference on Circuits and Systems for Communications (ICCSC '08), pp. 703–707, May 2008. View at: Publisher Site  Google Scholar
 A. Zhu and T. J. Brazil, “RF power amplifier behavioral modeling using volterra expansion with laguerre functions,” in Proceedings of the 2005 IEEE MTTS International Microwave Symposium, pp. 963–966, June 2005. View at: Publisher Site  Google Scholar
 H. Ku, M. D. McKinley, and J. S. Kenney, “Quantifying memory effects in RF power amplifiers,” IEEE Transactions on Microwave Theory and Techniques, vol. 50, no. 12, pp. 2843–2849, 2002. View at: Publisher Site  Google Scholar
 J. Li, J. Nan, and J. Zhao, “Study and simulation of RF power amplifier behavioral model based on RBF Neural network,” in Proceedings of the 2010 International Conference on Microwave and Millimeter Wave Technology (ICMMT '10), pp. 1465–1467, May 2010. View at: Publisher Site  Google Scholar
 H. Ning, X. Jing, and L. Cheng, “Online identification of nonlinear spatiotemporal systems using kernel learning approach,” IEEE Transactions on Neural Networks, vol. 22, no. 9, pp. 1381–1394, 2011. View at: Publisher Site  Google Scholar
 H. LiNa and N. JingChang, “Researches on GRNN neural network in RF nonlinear systems modeling,” in Proceedings of the 2011 International Conference on Computational ProblemSolving (ICCP '11), pp. 577–580, October 2011. View at: Publisher Site  Google Scholar
 F. Mkadem and S. Boumaiza, “Physically inspired neural network model for RF power amplifier behavioral modeling and digital predistortion,” IEEE Transactions on Microwave Theory and Techniques, vol. 59, no. 4, pp. 913–923, 2011. View at: Publisher Site  Google Scholar
 H. Zhao and J. Zhang, “Pipelined Chebyshev functional link artificial recurrent neural network for nonlinear adaptive filter,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, no. 1, pp. 162–172, 2010. View at: Publisher Site  Google Scholar
 M. Li, J. Liu, Y. Jiang, and W. Feng, “Complexchebyshev functional link neural network behavioral model for broadband wireless power amplifiers,” IEEE Transactions on Microwave Theory and Techniques, vol. 60, no. 6, pp. 1979–1989, 2012. View at: Publisher Site  Google Scholar
 Y.C. Cheng, W.M. Qi, and J. Zhao, “A new Elman neural network and its dynamic properties,” in Proceedings of the 2008 IEEE International Conference on Cybernetics and Intelligent Systems (CIS '08), pp. 971–975, September 2008. View at: Publisher Site  Google Scholar
 Q. Song, “On the weight convergence of Elman networks,” IEEE Transactions on Neural Networks, vol. 21, no. 3, pp. 463–480, 2010. View at: Publisher Site  Google Scholar
 L. L. Jiang, D. L. Maskell, and J. C. Patra, “Chebyshev functional link neural networkbased modeling and experimental verification for photovoltaic arrays,” in Proceedings of the 2012 Annual International Joint Conference on Neural Networks (IJCNN '12), pp. 1–8, June 2012. View at: Publisher Site  Google Scholar
 M. Li and Y. He, “Nonlinear system identification using adaptive Chebyshev neural networks,” in Proceedings of the 2010 IEEE International Conference on Intelligent Computing and Intelligent Systems (ICIS '10), pp. 243–247, October 2010. View at: Publisher Site  Google Scholar
 A. Slowik, “Application of an adaptive differential evolution algorithm with multiple trial vectors to artificial neural network training,” IEEE Transactions on Industrial Electronics, vol. 58, no. 8, pp. 3160–3167, 2011. View at: Publisher Site  Google Scholar
 X. Jing and L. Cheng, “An optimal PID control algorithm for training feedforward neural networks,” IEEE Transactions on Industrial Electronics, vol. 60, no. 6, pp. 2273–2283, 2013. View at: Publisher Site  Google Scholar
 X. Jing, “Robust adaptive learning of feedforward neural networks via LMI optimizations,” Neural Networks, vol. 31, pp. 33–45, 2012. View at: Publisher Site  Google Scholar
 F.J. Lin, Y.S. Kung, S.Y. Chen, and Y.H. Liu, “Recurrent waveletbased Elman neural network control for multiaxis motion control stage using linear ultrasonic motors,” IET Electric Power Applications, vol. 4, no. 5, Article ID IEPAAN000004000005000314000001, pp. 314–332, 2010. View at: Publisher Site  Google Scholar
 X. Xiao, X. Jiang, and Y. Zhang, “An algorithm for designing Chebyshev neural network,” in Proceedings of the 2009 Second ISECS International Colloquium on Computing, Communication, Control, and Management (CCCM '09), pp. 206–209, August 2009. View at: Publisher Site  Google Scholar
 L. Mu, H. YiGang, T. Wen, and L. Zurun, “Fast learning algorithm for controlling Logistic chaotic system based on Chebyshev neural network,” in Proceedings of the 5th International Conference on Natural Computation (ICNC '09), pp. 149–153, August 2009. View at: Publisher Site  Google Scholar
 D. H. Wisell, B. Rudlund, and D. Rönnow, “Characterization of memory effects in power amplifiers using digital twotone measurements,” IEEE Transactions on Instrumentation and Measurement, vol. 56, no. 6, pp. 2757–2766, 2007. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2014 Shao Jie et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.