Iterative Methods and ApplicationsView this Special Issue
Research Article | Open Access
Gou Hosoya, Hiroyuki Yashima, "Log-Likelihood Ratio Calculation for Iterative Decoding on Rayleigh Fading Channels Using Padé Approximation", Journal of Applied Mathematics, vol. 2013, Article ID 970126, 10 pages, 2013. https://doi.org/10.1155/2013/970126
Log-Likelihood Ratio Calculation for Iterative Decoding on Rayleigh Fading Channels Using Padé Approximation
Approximate calculation of channel log-likelihood ratio (LLR) for wireless channels using Padé approximation is presented. LLR is used as an input of iterative decoding for powerful error-correcting codes such as low-density parity-check (LDPC) codes or turbo codes. Due to the lack of knowledge of the channel state information of a wireless fading channel, such as uncorrelated fiat Rayleigh fading channels, calculations of exact LLR for these channels are quite complicated for a practical implementation. The previous work, an LLR calculation using the Taylor approximation, quickly becomes inaccurate as the channel output leaves some derivative point. This becomes a big problem when higher order modulation scheme is employed. To overcome this problem, a new LLR approximation using Padé approximation, which expresses the original function by a rational form of two polynomials with the same total number of coefficients of the Taylor series and can accelerate the Taylor approximation, is devised. By applying the proposed approximation to the iterative decoding and the LDPC codes with some modulation schemes, we show the effectiveness of the proposed methods by simulation results and analysis based on the density evolution.
In recent years, iterative decoding techniques based on message passing algorithm such as turbo decoding  or belief-propagation (BP) decoding [2–4] have been attracted by their significant performance which attain close to the Shannon limit. The BP decoding algorithm, a well-known iterative decoding algorithm for LDPC codes [2, 3], has been widely studied for the binary erasure channel or the additive white Gaussian noise (AWGN) channel [2–8]. The algorithms firstly derive channel log-likelihood ratios (LLR) where the messages in the decoder are initialized to these LLR values. To exhibit good performance with BP decoding, this channel LLR should be obtained with high accuracy, but it becomes complicated for some channel models such as wireless fading channel .
In this study, we focus on a calculation of channel LLR over the uncorrelated flat Rayleigh fading channels where the discrete-time component transmitted signal is input to a band-limited channel; that is, . Here , , , and denote a channel output, a fading gain, a channel input, and an additive white Gaussian noise (AWGN) with variance at time , respectively. Hereinafter we drop the subscript . If at each received bit position is known to the receiver, we call this case known channel state information (CSI). If at each received bit position is unknown to the receiver, we call this case unknown CSI. For a known CSI case, channel LLR can be easily calculated using the channel outputs , , and . However, for an unknown CSI case which is more practical than known CSI and is our main interest, a calculation of the channel LLR is rather complex due to an integration of .
The studies of wireless fading channels with the LDPC codes or turbo codes were presented in [11–20] with several modulation schemes [9, 10, 21] such as binary modulation (binary phase shift keying (BPSK)) or nonbinary modulations. In  for BPSK, Hou et al. have studied designing irregular LDPC codes  using density evolution [7, 8] and have shown that these codes can approach the Shannon limit. But they have used the following simple linear approximation for a calculation of the channel LLR: where denotes the expectation of the channel gain . Although the previous approximation is simple and is easy to implement, it is inaccurate that degradation in the decoding performance compared with the true LLR  can be seen. Yazdani and Ardakani [19, 20] have also proposed a linear LLR approximation whose performance is almost identical to the true LLR: where is obtained by maximizing Here is given by where and denotes the complementary error function . However, an optimization of using (3) and (4) requires for each channel parameter , so that it needs large complexity to implement.
Recently Asvadi et al.  have applied Taylor approximation of order to the true LLR function such that where denotes the coefficient of Taylor series of order . They have derived both linear and nonlinear approximations with small orders. For a linear approximation (Taylor series of order ), it is given by For a nonlinear approximation (Taylor series of order ), it is given by From the previous approximations, one can obtain accurate LLR without optimizing complicated functions such as in [19, 20].
To move our attentions to nonbinary modulations, which is more practical case, the LLR calculations are performed bitwise [15, 21]. The previous work by Yazdani and Ardakani , which is an extension of , has devised the LLR approximation method, but it becomes complex to evaluate LLR due to the increment of the number of parameters for the optimization. To fit the true LLR functions, the authors in  have modified the approximation functions of Taylor series of order 3. This modification is not easy to replicate and is required for each parameter of the channels. Moreover it is well known that the Taylor approximation quickly becomes inaccurate as the variable leaves the derivative point.
To overcome these problems, we devise a new LLR approximation using Padé approximation  on the uncorrelated flat Rayleigh fading channels with unknown CSI for BPSK and 8-PAM. Padé approximation expresses the original function by rational form of two polynomials with the same total number of coefficients of the Taylor series, and it can accelerate the Taylor approximation. Generally Padé approximation is accurate not only at the derivative point but also at the wide range of intervals of variables. We show by simulation results and analysis based on the density evolution that our method can approximate LLR function with high accuracy and can yield almost the same decoding performance as the true LLR. The Padé approximation is a generalization of the Taylor approximation, and the proposed method exhibits slightly better performance than the method using Taylor approximation . Moreover we design irregular LDPC codes based on our LLR approximation function.
This paper is organized as follows: Section 2 gives the channel model, LLR calculation method, and LDPC codes. In Section 3, we briefly review Taylor and Padé approximations, and then we present the proposed LLR calculation by Padé approximation. Numerical results are shown in Section 4, and Section 5 concludes the paper.
2.1. Channel Model and LLR Calculation
We here consider the following discrete-time channel model: where and represent the channel input, output, and noise, respectively, and , denote a set of transmitted symbols and that of real numbers, respectively. (As mentioned in Section 1, we drop a time subscript for , , , and .) Moreover is the channel gain with an uncorrelated flat Rayleigh distribution by its probability density function (pdf) , and is the white Gaussian noise with mean 0 and variance .
Using the bit-interleaved coded modulation (BICM) scheme [10, 15, 21] for a transmission, an information bit sequence is mapped to the codeword (bit sequence) of length by error-correcting codes, and then it is partitioned into blocks denoted by , , of length . Hereinafter we drop the subscript of both and to simplify discussions. This block is mapped to transmit a signal in a Gray-labeled -ary signal constellation of size . The signal constellation for BSPK and 8-PAM is depicted in Figure 1.
For the fading channel, two cases can be considered which depends on the knowledge of at the receiver.
2.1.1. Known CSI
For a known CSI case, we can use channel fading gain for each bit position . The channel LLR is given by where denotes the th bit of block which is mapped to and is a set of blocks which satisfies for . Moreover a base of logarithm takes a natural number , and is given by For BPSK, (9) is reduced to The calculation of the previous equation is not a difficult task.
2.1.2. Unknown CSI
For an unknown CSI case, we cannot use channel fading gain for each received bit position. The channel LLR is given by where is given by For BPSK, (12) is given by where and . The calculation of (12) is so complicated that several works have tried to reduce the computational complexity by approximations [11, 13, 14, 19, 20].
For the -ary PAM, in (13) becomes where . Using (15), the LLR in (12) can be evaluated. The previous equations are so complicated to implement that several works have tried to reduce the computational complexity by approximations.
Notice that for the fading channels with some modulations, the log-sum approximation was used for bitwise linear approximation. This approximation is only efficient for a known CSI case, since an integration of fading factor for a calculation of LLR is not needed. But for an unknown CSI case, an integration of fading factor is needed. Moreover it is effective only for a high signal-to-noise (SNR) region, where the sum in (9) is dominated by a single large term.
2.2. LDPC Codes
We here consider binary LDPC codes. An LDPC code is represented by the Tanner graph which consists of the variable nodes and the check nodes. These nodes are incident with the edges. Let and denote the maximum number of edges incident to the variable nodes and check nodes, respectively. Let and be variable node degree distribution and check node degree distribution where and denote fractions of the number of edges incident to the variable node and check node of degrees in the Tanner graph of the code, respectively. An LDPC code is specified by , , and . The rate of the codes is given by where denotes the number of check nodes and is given by .
An ensemble of LDPC codes  is denoted by . Combined with the BP decoding algorithm, LDPC codes with optimized by density evolution [5, 7, 8] can attain high performance which is close to the theoretical limit (Shannon limit). The iterative threshold of an ensemble of LDPC codes is defined as the maximum standard deviation on the channel in (8) such that where denotes the message error probability in iteration of the BP decoding algorithm. is calculated recursively by the density evolution [5, 7, 8] which keeps track the message error probability of the BP decoding algorithm from the pdf of channel LLR. Iterative threshold is sometimes measured by or signal-to-noise ratio (SNR) such that (for BPSK) or (for 8-PAM), respectively, where and denote the average energy per information bit and one-sided power spectral density of the additive white Gaussian noise (AWGN).
3. LLR Approximation Based on Padé Approximation
Before approximating the true LLR function in (12), we briefly explain the Taylor approximation and then describe the Padé approximation.
3.1. Brief Review of Taylor and Padé Approximations
Let be the original function, and let be th derivative of . Let and be closed and open intervals between and , respectively.
Definition 1. Suppose that has derivatives at point and has a derivative at where , . Assume that , are required to be continuous on and is required to exist on . The Taylor polynomial of order for at point is then defined by The remainder term, which is a difference between true value of the function and its Taylor series of polynomial, is given by
For some and , one can approximate the original function by in (16). However, this function quickly becomes inaccurate as leaves , even though is large.
The approximation in (16) can often be accelerated by rearranging it into a ratio of two series using Padé approximation. It generalized the Taylor approximation with the same total number of coefficients of two series. Before describing the Padé approximation, we rewrite (16) as follows:
Definition 2. Suppose that is approximated by the Taylor series in (18). The Padé approximation of order , , , , is given by where and are determined, so that the coefficients of the terms of in (19) are equal to those of in (18).
From Definition 2, the polynomials , , and satisfy the equation Equation (20) tells that all the coefficients of of and in left-hand side of (20) are equal. We can express these relations by the following simultaneous equation for : Notice that the terms in left-hand side of (20) are included in in right-hand side of the equation. These terms are not necessary for the evaluation of and .
We have already evaluated the coefficients in (18) by Taylor series. Moreover we assume that and are normalized, so we set . This normalization is valid since the Padé approximation in (19) is of a rational form. Substituting and into (20), we can obtain coefficients and . Note that the Taylor approximation and the Padé approximation are equivalent to each other if . Therefore we can see that the Padé approximation is a generalization of the Taylor series approximation.
3.2. Applying Padé Approximation to LLR Function
3.2.1. LLR Calculation for BPSK
By applying the Padé approximation to the true LLR function for BPSK in (14), we can obtain the approximated function. To fit the true LLR function, we have searched the approximated function for several pairs of and found that Padé approximation of order is more accurate than Taylor series of order around 0 in (7) which is previously known as the best approximated one . The proposed LLR approximation is given as follows: where , , , and , The previous approximation is obtained by the Taylor series of order : where Then (21) becomes Substituting in (24) and into (26), we get , and .
We then compare the accuracy of the approximated LLR functions. Figures 2(a) and 3(a) show LLR values for the uncorrelated flat Rayleigh fading channel with unknown CSI for and , respectively. LLR values in these figures are evaluated by the true LLR in (14), Taylor series of orders and (“Taylor3” and “Taylor5”) in (7) and (24), respectively, the Padé approximation of order (“Pade23”) in (22), and linear approximation (“Ex”) in (1). From these figures, Padé approximation is almost identical to the true LLR for various channel output . This may be contributory to the fact that the order of Pade23 is larger than that of Taylor3 (). However, we can see that Taylor5 () is inaccurate as becomes large. Therefore large is not an answer for an accurate LLR approximation, especially for using the Taylor approximation.
(a) LLR calculation
(b) Absolute difference
(a) LLR calculation
(b) Absolute difference
To compare the LLR approximations in detail, Figures 2(b) and 3(b) show the absolute differences between true LLR and approximated LLRs (Taylor3, Taylor5, and Pade23). We only show the case of , since the true LLR function is odd symmetric; that is, . From Figures 2 and 3, accuracy of Pade23 and Taylor5 are good especially for small . But accuracy of them are quite different for large , that is, Pade23 shows the best accuracy, but Taylor5 shows the worst accuracy.
Next we consider analysis based on the density evolution for different LLR calculation methods. We derive the pdf of LLR assuming that is transmitted. From (10), we have Averaging (27) over by an integration, we obtain Then the density of the LLR function can be expressed in a parametric form . For in (22), this is given by where denotes a derivative of the function with . Equation (29) is a case of the proposed approximation function, but one can obtain the pdf of the other LLR functions. For example, replacing with in (14), we can obtain the pdf of the true LLR function .
3.2.2. LLR Calculation for 8-PAM
We demonstrate Padé approximation for bitwise LLR of 8-PAM constellation with Gray labeling on the Rayleigh fading channel without CSI (SNR = 7.91 (dB) ()) in Figure 4. The coefficients of LLR approximation functions of Taylor3 and Padé approximation are listed in Table 1. The derivative points for each bit LLR are chosen where these functions take for . Notice that we can omit the coefficients for the case (bits 2 and 3), since these LLR functions are even functions; that is, we can derive from for . The orders of Padé approximation for each bit are different since each bit LLR function is distinct.
|(a) Taylor series of order|
|(b) Padé approximation of numerator|
|(c) Padé approximation of denominator|
(a) Bit 1
(b) Bit 2
(c) Bit 3
(d) Bit 3 (detail for Taylor3)
(e) Bit 3 (detail for Pade41 and Pade34)
(f) Bit 3 (Pade45)
For bit 1, this point is . For bit 2, these points are (2 points). So we have two LLR approximation functions for bit 2. We chose the order pairs of Padé approximation and (denoted by “Pade47” and “Pade24”) for and 2, respectively. The true LLR function for bit 2 is an even function, so we switch two LLR approximation functions on . Since these approximation functions intersect on , the resulting function becomes continuous.
For bit 3, these points are , (4 points). For and , we use Padé approximation of orders and (denoted by “Pade41” and “Pade34”), respectively. The true LLR function for bit 3 is also an even function, so we switch two LLR approximation functions, whose derivative points are (bit 3 (a) in Table 1), on the intersection point . But two LLR approximation functions (bit 3 (a) and (b) in Table 1) do not intersect on any . Therefore we searched two switch points between the interval of to minimize the loss of accuracy, and we set these points for Taylor approximation on , . We take the function (bit 3 (a)) for and take the function (bit 3 (b)) for . Between , we take weighted mean of two approximation functions (bit 3 (a) and (b)) which is shown in Figure 4(d). Likewise, we switch LLR functions for Padé approximation on , as shown in Figure 4(e). From Figure 4(c), we can easily see switch points for Taylor3, but for Padé approximation we cannot see these points explicitly. This is because accuracy of Padé approximation is higher than that of Taylor3 for wide range of variables.
Notice that for bit 3, it is not necessary to consider weighted mean of two approximation functions if we use Padé approximation of higher order pairs. In this case, we found that Padé approximation of orders at point provides almost the same accuracy as the combination of and which is shown in Figure 4(f) (denoted by “Pade45”). But the accuracy of Padé approximation of for large is not so good compared with the combination of Padé approximations and . Also note that the previous work in  have modified Taylor3 functions to fit the true LLR for and 3. But this is not easy to replicate, so we only show the original form of Taylor3.
4. Numerical Results and Discussion
In order to compare the proposed LLR approximations, we use LDPC codes with BP decoding and show results by simulations and analysis based on the density evolution. We only show the case where CSI is unknown to the receiver.
4.1. Results for BPSK
By using the density evolution [5, 7, 8], Table 2 shows iterative thresholds of , , and LDPC code ensembles whose rates are, respectively, , , and for different LLR calculation methods. Evaluating preciously, the thresholds of three methods are almost identical especially for true LLR and Pade23, but there exists a little gap between Taylor3 and the other methods.
|(a) LDPC code ensemble|
|(b) LDPC code ensemble|
|(c) LDPC code ensemble|
Table 3 shows degree distribution profiles and their iterative thresholds for (a) threshold optimized and (b) rate optimized irregular LDPC codes. These profiles and their corresponding thresholds are evaluated based on the true LLR, Taylor3, and Pade23, respectively. From Table 3(a) for fixed rate, the threshold of Pade23 is almost the same as that of true LLR and is slightly better than that of Taylor3. From Table 3(b) for fixed threshold, the rate of the code by Pade23 is almost the same as that by true LLR and is slightly higher than that by Taylor3. From these thresholds, they are close to the Shannon limit; that is, for rate half code, it is ( [dB]).
|(a) Threshold optimized|
|(b) Rate optimized|
4.2. Results for 8-PAM
Figure 5 shows bit error rate (BER) of LDPC codes with for the uncorrelated flat Rayleigh fading channel with 8-PAM. The number of transmitted codewords is . BERs of true LLR and two Padé approximations (combination of orders (4, 1) and (3, 4) and orders (4, 5)) are almost the same, and that of Taylor3 is slightly higher than those of three methods.
Table 4 shows iterative thresholds of LDPC code ensemble for different LLR calculation methods with 8-PAM. The threshold of Padé approximation is almost identical to that of true LLR (inferior to 0.02 dB), and it is better than that of Taylor3. Notice that threshold of Taylor3 in  was SNR = 7.86 (dB) and it is better than that of Padé approximation. But the method in  performed several modifications to fit the true functions for the LLR functions of bits 2 and 3, it is not easy to replicate the approximations. Padé approximation is better than linear approximation in  (SNR = 7.88 (dB)). The threshold of Padé approximation of orders for bit 3 LLR function is SNR = 7.88 (dB) which is the same as the linear approximation.
In this paper, we apply the Padé approximation, which is a generalization of the Taylor approximation, to the LLR function on the uncorrelated flat Rayleigh fading channels with unknown CSI. Using Padé approximation, we can accelerate the accuracy of the LLR function that it is accurate not only around the derivative point but also at the other intervals of the variable. From the simulation results and analysis based on the density evolution, our method can yield almost the same decoding performance as the true LLR function and is slightly better than the conventional approximation method (Taylor approximation of order ). Moreover we derive some irregular LDPC code profiles whose iterative thresholds are close to the Shannon limit.
To apply Padé approximation, other modulations (e.g., 16-QAM) or other channels (Rician fading) are remained for further works.
The authors are grateful for anonymous reviewers for their thorough and conscientious reviewing which improves the quality of this paper. One of the authors, Gou Hosoya, would like to thank Dr. M. Kobayashi at the Shonan Institute of Technology and Dr. H. Yagi at the University of Electro-Communications for their valuable comments and discussions. This research is partly supported by JSPS KAKENHI Grant nos. 25820166 and 25420386.
- C. Berrou, A. Glavieux, and P. Thitimajshima, “Near shannon limit error-correcting coding and decoding: turbo-codes,” in Proceedings of the IEEE International Conference on Communications (ICC '93), pp. 1064–1070, Geneva, Switzerland, May 1993.
- R. G. Gallager, Low-Density Parity-Check Codes, MIT Press, 1963.
- D. J. C. MacKay, “Good error-correcting codes based on very sparse matrices,” IEEE Transactions on Information Theory, vol. 45, no. 2, pp. 399–431, 1999.
- T. J. Richardson and R. L. Urbanke, Modern Coding Theory, Cambridge University Press, Cambridge, UK, 2008.
- S. Y. Chung, On the construction of some capacity-approaching coding schemes [Ph.D. thesis], M.I.T., Cambridge, Mass, USA, 2000.
- M. G. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman, “Improved low-density parity-check codes using irregular graphs,” IEEE Transactions on Information Theory, vol. 47, no. 2, pp. 585–598, 2001.
- T. J. Richardson and R. L. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Transactions on Information Theory, vol. 47, no. 2, pp. 599–618, 2001.
- T. J. Richardson, M. A. Shokrollahi, and R. L. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 47, no. 2, pp. 619–637, 2001.
- J. G. Proakis and M. Salehi, Digital Communications, McGraw-Hill, 5th edition, 2005.
- A. Guillén i Fàbregas, A. Martinez, and G. Caire, “Bit-interleaved coded modulation,” Foundations and Trends in Communications and Information Theory, vol. 5, no. 1-2, pp. 1–153, 2008.
- R. Asvadi, A. H. Banihashemi, M. Ahmadian-Attari, and H. Saeedi, “LLR approximation for wireless channels based on Taylor series and its application to BICM with LDPC codes,” IEEE Transactions on Communications, vol. 60, no. 5, pp. 1226–1236, 2012.
- E. K. Hall and S. G. Wilson, “Design and analysis of turbo codes on Rayleigh fading channels,” IEEE Journal on Selected Areas in Communications, vol. 16, no. 2, pp. 160–174, 1998.
- G. Hosoya, M. Hasegawa, and H. Yashima, “LLR calculation for iterative decoding on fading channels using Padé approximation,” in Proceedings of the International Conference on Wireless Communications and Signal Processing (WCSP '12), CTS 1569651991, pp. 1–6, Huanshan, China, October 2012.
- J. Hou, P. H. Siegel, and L. B. Milstein, “Performance analysis and code optimization of low density parity-check codes on Rayleigh fading channels,” IEEE Journal on Selected Areas in Communications, vol. 19, no. 5, pp. 924–934, 2001.
- J. Hou, P. H. Siegel, L. B. Milstein, and H. D. Pfister, “Capacity-approaching bandwidth-efficient coded modulation schemes based on low-density parity-check codes,” IEEE Transactions on Information Theory, vol. 49, no. 9, pp. 2141–2155, 2003.
- N. Inaba, H. Yashima, T. Kuroda, and T. Tsubouchi, “Error performances of convolutional coded modulation using the maximum likelihood metric and an approximated metric over Rayleigh fading channel,” IEICE Transactions on Fundamentals of Electronics, vol. J78-A, no. 10, pp. 1397–1399, 1995 (Japanese).
- N. Iviyani and J. Weber, “Analysis of random regular LDPC codes on Rayleigh fading channels,” in Proceedings of the 27th Symposium on Information Theory in the Benelux, Noordwijk, the Netherlands, June 2006.
- J. Lin and W. Wu, Performance Analysis of LDPC Codes on Rician Fading Channels, Higher Education Press and Springer, 2006.
- R. Yazdani and M. Ardakani, “Linear LLR approximation for iterative decoding on wireless channels,” IEEE Transactions on Communications, vol. 57, no. 11, pp. 3278–3287, 2009.
- R. Yazdani and M. Ardakani, “Efficient LLR calculation for non-binary modulations over fading channels,” IEEE Transactions on Communications, vol. 59, no. 5, pp. 1236–1241, 2011.
- G. Caire, G. Taricco, and E. Biglieri, “Bit-interleaved coded modulation,” IEEE Transactions on Information Theory, vol. 44, no. 3, pp. 927–946, 1998.
- C. Brezinski, History of Continued Fractions and Padé Approximants, vol. 12, Springer, Berlin, Germany, 1991.
Copyright © 2013 Gou Hosoya and Hiroyuki Yashima. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.