Research Article | Open Access
Ahmad Kamal Hassan, Adnan Affandi, "On Modelling and Comparative Study of LMS and RLS Algorithms for Synthesis of MSA", Modelling and Simulation in Engineering, vol. 2016, Article ID 9742483, 8 pages, 2016. https://doi.org/10.1155/2016/9742483
On Modelling and Comparative Study of LMS and RLS Algorithms for Synthesis of MSA
This paper deals with analytical modelling of microstrip patch antenna (MSA) by means of artificial neural network (ANN) using least mean square (LMS) and recursive least square (RLS) algorithms. Our contribution in this work is twofold. We initially provide a tutorial-like exposition for the design aspects of MSA and for the analytical framework of the two algorithms while our second aim is to take advantage of high nonlinearity of MSA to compare the effectiveness of LMS and that of RLS algorithms. We investigate the two algorithms by using gradient decent optimization in the context of radial basis function (RBF) of ANN. The proposed analysis is based on both static and adaptive spread factor. We model the forward side or synthesis of MSA by means of worked examples and simulations. Contour plots, 3D depictions, and Tableau presentations provide a comprehensive comparison of the two algorithms. Our findings point to higher accuracies in approximation for synthesis of MSA using RLS algorithm as compared with that of LMS approach; however the computational complexity increases in the former case.
Microstrip patch antenna (MSA) has been extensively used in wireless transmission links because of its simplicity in design and fabrication, less weight, low cost, and small size. Major parameters in the design consideration of MSA include bandwidth, gain, directivity, polarization, and center frequency. It is well understood that there is a tradeoff in selection of these parameters and design engineers have to assign appropriate weights based on their work objectives [1–3]. Usually MSA finds applications in areas where the bandwidth requirement is narrow and with multiband resonance frequencies to account for diversity issues [4, 5]. There exist many other applications [6–8] where MSA can be integrated with a given automation system to better the existing results.
Terminology of patch is based on the aspects of radiating element “photoetchen” on dielectric constant. Patch can be of different geometries both deterministic and random. Deterministic geometry points to triangular, elliptical, circular, square, and last but not least rectangular (used in this work) shapes of patch. Choice of patch dimension such as length, width, permeability, and thickness of the substrate plays significant role in obtaining the resonance frequency. There exist numerous models in literature that account for the determination of dimension of patch based on artificial neural networks (ANN) for modelling purposes [9, 10] but an algorithmic comparison of such models is unavailable. Nevertheless, ANN in general and radial basis function (RBF) in particular have been extensively used with excellent results in modelling and simulation techniques for other nonlinear mechanical systems [11, 12] and signal processing routines  and in the design aspects of MSA [14–17]. The design of MSA using ANN is subdivided into forward modelling and backward modelling. Forward modelling accounts for the synthesis of MSA and hence it is useful in obtaining both length () and width () of the patch. In backward modelling, the primary task is the extraction of resonance frequency (). Both forward side of the problem (synthesis) and reverse side of the problem (analysis) are basic building blocks for commercially available simulation software such as advanced design system (ADS) and Ansoft high frequency simulation system (HFSS) . In this work, the analysis for the synthesis of MSA using RBF is performed. We present herein an exposition of significant algorithms such as least mean square (LMS) and recursive least square (RLS) and further investigate their deviants based on the gradient decent approach (GDA). The choice of the LMS and RLS algorithm is because they are considered fundamental in many subdisciplines of engineering such as adaptive filtering and signal processing. The parameters that are fed to ANN are resonance frequency (), permeability constant ), and height of the substrate (). Consequently, width () and length () are extracted from ANN. The rationale behind the proposed work is that the MSA design and its pertinent equations are of high nonlinearity; therefore choosing an algorithm that results in minimal error between the estimated data and the desired data can be useful in the development of the aforementioned commercial software. Moreover, computational complexity of algorithms is another area which needs to be looked into before choosing an algorithm; hence we outline the complexity difference in terms of complex multiplications and additions that are performed in an algorithm. To the best of the authors’ knowledge such investigation and comparative analysis have not been done previously for the synthesis of MSA.
Organization of this paper is such that, after introduction in Section 1, an overview of the design aspects of MSA is given in Section 2. Section 3 introduces RBF, LMS, and RLS approaches and mathematical formulation of adaptive spread factor. Complexity analysis is presented in Section 4. Simulation results and comparative analysis of the four algorithms are given in Section 5. Subsequent section deals with concluding remarks of this work and references.
2. Design of the Rectangular Microstrip Patch Antenna
Patch antennas are multilayered components with conductors, that is, patch and ground plane separated by dielectric substrate. Rectangular microstrip patch antenna (R-MSA) consists of width () and length () as presented in Figure 1. The other significant parameters required in designing and fabrication of MSA include substrate thickness () and its permeability (). Choice of substrate determines the size and its relevant application base. Most common substrates on the market are RT/duroid with dielectric constant of 2.2 and liquid crystal polymer (LCP) with the range of dielectric constant between 2.9 and 3.2 . Generally, there are numerous substrates available on the market for usability in MSA and in practice the permeability of dielectric with MSA in perspective is in the range of . Feed is another crucial aspect in design of patch antenna and there exist multiple methods by which a signal can be fed. The simplest method is direct contact method shown in Figure 1 and thereby used in this work. Other methods can be probe feed, microstrip proximity coupling, and microstrip aperture coupling .
In MSA, physical length () and electrical (effective) length () are different because of fringing fields at the edges of the patch. The effective electrical length of the patch antenna is given by [2, 19–22]where is the change in length given byEffective dielectric constant of the dielectric material is expressed as The actual length of the patch under specified conditions isHence the practical width is determined as follows:
Based on the above mathematical modelling, both synthesis and analysis can be designed using soft computing tools. We, however, focus on the synthesis of MSA; where the inputs to ANN are resonance frequency (), substrate thickness (), and dielectric constant () while length () and width () are extracted from the RBF designs. Synthesis aspect of this work is presented in Figure 2. We utilize this model as a test bench to compare LMS and RLS algorithms in the forthcoming sections.
3. Synthesis Using RBF-ANN
In this section we examine the feedforward architecture of the ANN based on RBF. Layout of RBF is simplistic and it consists of hidden layer and output layer neurons. The function of hidden layer is to perform a nonlinear operation on the set of inputs. For the purposes of performing a nonlinear operation, one can resort to plethora of nonlinear functions such as multiquadrics, inverse multiquadrics, and Gaussian function [21–23]. Moreover, Kernels of the RBF have been extended in multiple ways with notable work in [24–26]. In this work, we however use the conventional approach expressed in [21, 22] for our hypothesis testing. The nonlinear function pertinent to th neuron () is considered as Gaussian function which can be expressed by means of the following expressions:where is the time index, is the spread factor of th Gaussian function and it is determined empirically, is the total number of basis functions employed, is the maximum distance between any two bases, is the th input data, and is the th center of basis function.
Once the nonlinear activation function is performed on the input set of data (), the output can be extracted from output layer neuron () aswhere is the total number of basis functions and are the weighted interconnection between the hidden neurons and the output neurons.
Weight update is processed by using recursion principle of RBF-ANN and its optimality is an active research area with the synthesis of MSA in perspective. An effort is made herein to model algorithms with better convergence and stability properties of the objective function which is basically error minimization. The following subsections provide an overview of algorithms utilized in approximating MSA metrics and recursively update weights and the corresponding activation function. Figure 3 presents an RBF neural network where input data is fed to Gaussian function; each nonlinear activation function has a weighted interconnection with the output neuron. Our primary focus here is the minimization of objective function which accounts for the mean square error (MSE) between the desired output () and the estimated output (). We use two benchmark algorithms, namely, the LMS and the RLS independently. Lastly, spread can also be recursively updated based on gradient decent approach (GDA) which has the potential to further minimize the MSE.
3.1. Least Mean Square (LMS)
Least mean square (LMS) algorithm is used widely in the domain of adaptive filtering  and it is also more often than not utilized in RBF for MSA design . The LMS utilizes GDA approach to recursively update the weights of neurons based on the instantaneous MSE. Consider an objective function which is simply based on instantaneous error of all the output neurons given bywhere corresponds to the desired output which is compared with the approximated result of the output neuron . Gradient of the cost function is thus calculated asHence, the corresponding weights update equation can be computed and updated aswhere is the learning rate or step size and it plays an important role in the convergence properties of the given objective function.
3.2. Recursive Least Square (RLS)
Recursive least square (RLS) is another powerful adaptive algorithm where the cost function is minimized by recursively updating the weights. There is a plethora of literature available on the design of RLS algorithm; therefore  and references therein can be insightful for interested readers. The main aim herein is not to redrive the RLS algorithm but to briefly overview its core principles. The linear least square objective function is given bywhere is the forgetting factor of RLS and has values in the range of .
The order of RLS denoted as and the corresponding weights update () expressions are given by
Convergence of RLS is much faster as compared with LMS though computational complexity of RLS substantially increases, and this aspect is shown in the subsequent section.
3.3. Gradient Decent Approach for Adaptive Spread
In this subsection, the spread factor has also been made adaptive using gradient decent approach for both LMS and RLS algorithms. The relevant gradient of objective function and recursive update of spread factor are given by [21, 22]where the term () in (14a) is the first derivative of Green’s function with respect to its argument and is the learning rate for adaptive spread.
Depiction of GDA based spread update is shown using dashed-lines of Figure 3. Recursive update of spread expressed in (14a), (14b), and (14c) can greatly enhance the approximation capabilities of the neural networks since it is tuning the most significant part of RBF, namely, the activation function defined in (6a). It is also to be noted herein that a further extension of the aforementioned model is also possible in which one can update centers of the Gaussian function as well [21–23]; however subtractive clustering method is used in this work.
4. Computational Complexity
Computation complexity of the four algorithms based on matrix and vector dimension denoted as is summarized in Table 1. It can be seen that algorithm 4 has the highest computational complexity and it increases rapidly with increasing the dimension of the system, for example, number of neurons of RBF. In order to calculate the complexity of overall system, input vectors are needed to be considered alongside the inherited routines of a particular algorithm. For the detailed formation of computational complexity, we suggest that the reader follows mechanics of . It is shown in the next section that RLS algorithm which needs more computational power for the present problem benefits in terms of minimizing the MSE which drops from 152 to just over 1 for the testing data. There are certainly some emerging algorithms [30–34] and conventional techniques of algorithm manipulation [35, 36] with varying degree of complexity which can lead to the potential expansions of this work.
5. Experimental Results
In this section, four methods have been investigated and analyzed for synthesis of MSA, namely, LMS, gradient decent approach for updating spread fused with LMS (-GDA-LMS), RLS, and lastly gradient decent approach for updating spread integrated with RLS (-GDA-RLS). Four algorithms mentioned are tested based on the frequency range between 2.2 GHz and 5 GHz with substrate thickness variable between 0.2175 mm and 0.5175 mm and for a scalar value of dielectric constant set as 2.33 which is Rogers RT/duroid, and similar frequency band has been designed using rectangular slot antenna in  and therefore adopted herein. Normalized spread used for LMS algorithm is 0.151. The number of epochs used in training is 100, and subtractive clustering approach is utilized as in the previous work of . Training results for LMS is shown in Figure 4. It can be seen that, during the training phase, the estimated values of length () and width () superimpose the actual output values. The mean square error (MSE) for this algorithm is 156 for the testing batched data. By making the spread adaptive for the LMS algorithm based on gradient decent approach, MSE decreases to 153 which is not a substantive decrease in overall error for this approach. On the other hand, with the similar set of data as in LMS case, RLS is employed with forgetting factor ( set at 0.93 and training of ANN is done. On testing it is found that the MSE drops to 129 which is better than both LMS and LMS with adaptive spread. Further, using the adaptive spread technique in RLS algorithm, performance of ANN is enhanced tremendously and the MSE is reduced to 1.396. Training results with ANN based on fourth approach is shown Figure 5 whereas Table 2 represents the comparative MSE analysis for the 4 algorithms discussed. Please note that the difference between the initial spread and final spread for the second and fourth algorithm in Table 2 is disproportionate which is indicative of the overall performance metric of the two algorithms.
In Figure 6, we present a 3D depiction for the variables involved in the synthesis design of MSA, namely, resonance frequency , substrate thickness (), and length () and width (), using adaptive spread based RLS algorithm. There is excellent match of the algorithm output and the desired output. The flouring (contour) of Figure 6 indicates the degree of match. Building on the contour of Figure 6 and to better analyze the four algorithms, contour plots for the testing phase of all four algorithms are provided in Figure 7. It can be seen in the contour plots that approximated results of and are represented by “red” and “green” lines while the actual or target for both and is shown using “blue” lines. In algorithm , results almost superimpose desired “blue” lines; the least matching contours are for LMS. The degree of match between approximated and desired output is shown to improve from algorithm 1 to algorithm 4 in respective order. Table 3 represents the desired and approximated width and length for the four algorithms, it can be seen that most significant approximation using RBF is of the patch which contributes significantly to MSE for LMS; however, it is better approximated using RLS algorithms. From Table 3, it can also be seen that algorithms 1 and 3 are almost similar in approximation of while, in all four test cases, is better approximated using algorithm 3 as compared with LMS approach of algorithm 1. Mean square error (MSE) among all four cases is minimal while using adaptive spread based RLS algorithm. The accuracy of our fourth algorithm is 99.89% as compared with 99.09% achieved by authors of  which is their highest in the synthesis work of MSA.
6. Concluding Remarks
This paper presented the comparative analysis of four different algorithms for synthesis aspect of MSA using RBF. Estimation based on LMS, LMS with adaptive spread, RLS, and RLS with adaptive spread algorithms is utilized for approximating length and width of the MSA based on the input of resonance frequency, permeability constant, and substrate thickness. It is shown that, by augmenting the adaptive spread scheme with LMS and RLS algorithm, there is possibility of significantly improving the performance of RBF Kernel. It is also shown in this work that the most significant metric that accounts for maximum error is length of patch antenna and hence its estimation is of utmost importance which is accounted in RLS algorithm with good measure. The computational power that is desired by machines has been explicitly expressed in terms of complex multiplication and additions. Lastly, some potential extension of this work is discussed.
The authors declare that there is no conflict of interests regarding the publication of this paper.
- J. R. James and P. S. Hall, Eds., Handbook of Microstrip Antennas, vol. 28, IET, 1989.
- D. M. Pozar and H. D. Schaubert, Eds., Microstrip Antennas: The Analysis and Design of Microstrip Antennas and Arrays, John Wiley & Sons, 1995.
- D. M. Pozar, “Microstrip antennas,” Proceedings of the IEEE, vol. 80, no. 1, pp. 79–91, 1992.
- J. Huang, F. Shan, J. She, and Z. Feng, “A novel multiband and broadband fractal patch antenna,” Microwave and Optical Technology Letters, vol. 48, no. 4, pp. 814–817, 2006.
- S. E. El-Khamy, O. A. Abdel-Alim, A. M. Rushdi, and A. H. Banah, “Code-fed omnidirectional arrays,” IEEE Journal of Oceanic Engineering, vol. 14, no. 4, pp. 384–395, 1989.
- Y. C. Huang and C. E. Lin, “Flying platform tracking for microwave air-bridging in sky-net telecom signal relaying operation,” Journal of Communication and Computer, vol. 11, pp. 323–331, 2014.
- A. K. Hassan, A. Hoque, and A. Moldsvor, “Automated Micro-Wave(MW) antenna alignment of Base Transceiver Stations: time optimal link alignment,” in Proceedings of the Australasian Telecommunication Networks and Applications Conference (ATNAC '11), pp. 1–5, Melbourne, Australia, November 2011.
- M. A. Hoque and A. K. Hassan, “Modeling and performance optimization of automated antenna alignment for telecommunication transceivers,” Engineering Science and Technology, vol. 18, no. 3, pp. 351–360, 2015.
- F. Gozasht, G. R. Dadashzadeh, and S. Nikmehr, “A comprehensive performance study of circular and hexagonal array geometries in the LMS algorithm for smart antenna applications,” Progress in Electromagnetics Research, vol. 68, pp. 281–296, 2007.
- C. I. Kourogiorgas, M. Kvicera, D. Skraparlis et al., “Modeling of first-order statistics of LMS channel under tree shadowing for various elevation angles at L-band,” in Proceedings of the 8th European Conference on Antennas and Propagation (EuCAP '14), pp. 2273–2277, IEEE, The Hague, The Netherlands, April 2014.
- N. I. Galanis and D. E. Manolakos, “Finite element analysis of the cutting forces in turning of femoral heads from AISI 316l stainless steel,” in Proceedings of the World Congress on Engineering (WCE '14), pp. 1232–1237, July 2014.
- A. P. Markopoulos, S. Georgiopoulos, and D. E. Manolakos, “On the use of back propagation and radial basis function neural networks in surface roughness prediction,” Journal of Industrial Engineering International, vol. 12, no. 3, pp. 389–400, 2016.
- W. Aftab, M. Moinuddin, and M. S. Shaikh, “A novel kernel for RBF based neural networks,” Abstract and Applied Analysis, vol. 2014, Article ID 176253, 10 pages, 2014.
- N. Türker, F. Güneş, and T. Yildirim, “Artificial neural networks applied to the design of microstrip antennas,” Mikrotalasna Revija, vol. 12, no. 1, pp. 10–14, 2006.
- K. Guney, S. Sagiroglu, and M. Erler, “Generalized neural method to determine resonant frequencies of various microstrip antennas,” International Journal of RF and Microwave Computer-Aided Engineering, vol. 12, no. 1, pp. 131–139, 2002.
- J. E. Rayas-Sánchez, “EM-based optimization of microwave circuits using artificial neural networks: the state-of-the-art,” IEEE Transactions on Microwave Theory and Techniques, vol. 52, no. 1, pp. 420–435, 2004.
- Q.-J. Zhang, K. C. Gupta, and V. K. Devabhaktuni, “Artificial neural networks for RF and microwave design—from theory to practice,” IEEE Transactions on Microwave Theory and Techniques, vol. 51, no. 4, pp. 1339–1350, 2003.
- J. Virtanen, L. Ukkonen, A. Z. Elsherbeni, V. Demir, and L. Sydänheimo, “Comparison of different electromagnetic solvers for modeling of inkjet printed RFID humidity sensor,” in Proceedings of the ACES Conference The 26th Annual Review of Progress in Applied Computational Electromagnetics, Tampere, Finland, April 2010.
- D. C. Thompson, O. Tantot, H. Jallageas, G. E. Ponchak, M. M. Tentzeris, and J. Papapolymerou, “Characterization of liquid crystal polymer (LCP) material and transmission lines on LCP substrates from 30 to 110 GHz,” IEEE Transactions on Microwave Theory and Techniques, vol. 52, no. 4, pp. 1343–1352, 2004.
- T. C. Edwards and M. B. Steer, Foundations of Interconnect and Microstrip Design, vol. 3, John Wiley & Sons, 2000.
- S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall, Englewood Cliffs, NJ, USA, 2nd edition, 1999.
- C. A. Balanis, Antenna Theory: Analysis and Design, John Wiley & Sons, 2012.
- S. Haykin, Neural Networks and Learning Machines, vol. 3, Pearson Education, Upper Saddle River, NJ, USA, 2009.
- K. B. Cho and B. H. Wang, “Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction,” Fuzzy Sets and Systems, vol. 83, no. 3, pp. 325–339, 1996.
- L. Zhou, G. Yan, and J. Ou, “Response surface method based on radial basis functions for modeling large-scale structures in model updating,” Computer-Aided Civil and Infrastructure Engineering, vol. 28, no. 3, pp. 210–226, 2013.
- R. Martolia, A. Jain, and L. Singla, “Analysis survey on fault tolerance in radial basis function networks,” in Proceedings of the IEEE International Conference on Computing, Communication & Automation (ICCCA '15), pp. 469–473, 2015.
- S. Chen, B. Mulgrew, and P. M. Grant, “A clustering technique for digital communications channel equalization using radial basis function networks,” IEEE Transactions on Neural Networks, vol. 4, no. 4, pp. 570–579, 1993.
- L. Zhang and P. N. Suganthan, “A survey of randomized algorithms for training neural networks,” Information Sciences, vol. 364–365, pp. 146–155, 2016.
- S. Ljung and L. Ljung, “Error propagation properties of recursive least-squares adaptation algorithms,” Automatica, vol. 21, no. 2, pp. 157–167, 1985.
- M. Moinuddin and A. Zerguine, “A unified performance analysis of the family of normalized least mean algorithms,” Arabian Journal for Science and Engineering, vol. 39, no. 10, pp. 7145–7157, 2014.
- A. Timesli, B. Braikat, H. Lahmam, and H. Zahrouni, “An implicit algorithm based on continuous moving least square to simulate material mixing in friction stir welding process,” Modelling and Simulation in Engineering, vol. 2013, Article ID 716383, 14 pages, 2013.
- U. M. Al-Saggaf, M. Moinuddin, M. Arif, and A. Zerguine, “The q-Least Mean Squares algorithm,” Signal Processing, vol. 111, pp. 50–60, 2015.
- H. Ait Abdelali, F. Essannouni, L. Essannouni, and D. Aboutajdine, “An adaptive object tracking using Kalman filter and probability product kernel,” Modelling and Simulation in Engineering, vol. 2016, Article ID 2592368, 8 pages, 2016.
- M. T. Akhtar, M. Abe, and M. Kawamata, “A new variable step size LMS algorithm-based method for improved online secondary path modeling in active noise control systems,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 14, no. 2, pp. 720–726, 2006.
- A. M. Rushdi, “Improved variable-entered Karnaugh map procedures,” Computers & Electrical Engineering, vol. 13, no. 1, pp. 41–52, 1987.
- A. M. Rushdi, “Performance indexes of a telecommunication network,” IEEE Transactions on Reliability, vol. 37, no. 1, pp. 57–64, 1988.
- J.-W. Wu, H.-M. Hsiao, J.-H. Lu, and S.-H. Chang, “Dual broadband design of rectangular slot antenna for 2.4 and 5 GHz wireless communication,” Electronics Letters, vol. 40, no. 23, pp. 1461–1463, 2004.
Copyright © 2016 Ahmad Kamal Hassan and Adnan Affandi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.